A method and medium for recognizing old dialect speech based on a gated convolution

Abstract

The application discloses an old dialect speech recognition method based on a gated convolution, and comprises the following steps: 1) obtaining at least two audio files recorded by a user; 2) processing each audio file respectively to obtain a spectrum vector Vr and a frequency domain feature Fbankr; 3) inputting the spectrum vector Vr and the frequency domain feature Fbankr into a Gate CNN network as input data to obtain a speech recognition result hr of the rth audio file; 4) repeating the step 3) to obtain speech recognition results of R audio files; and 5) processing the speech recognition results of the R audio files to obtain a final speech recognition result. The application uses the Gate CNN to make an acoustic model, that is, the feature processing capability of the CNN is retained, the time characteristics of the model are retained, and meanwhile, the structure is simple and the parameters are controllable.

Description

TECHNICAL FIELD

[0001] The application relates to the field of speech recognition, and particularly relates to an old dialect speech recognition method based on a gating convolution and a medium. BACKGROUND

[0002] Alzheimer's disease (AD) is an age-related syndrome, and its incidence rate rapidly increases with age, becoming the fourth largest "killer" of the elderly after heart disease, cerebrovascular disease and tumor, but for underdeveloped areas in Chongqing and the world, the awareness rate, treatment rate, diagnosis rate and treatment rate are lower than the world average. Therefore, the automatic speech recognition (ASR) technology capable of accurately recognizing Chongqing dialect will help promote the early screening of dementia, especially the early screening of Alzheimer's disease.

[0003] The ASR technology is an important part of artificial intelligence, and realizes that the computer can "understand" the human voice, the development of the ASR technology helps people communicate with more artificial intelligence products, realizes "human-computer interaction", and thus people enjoy the convenience and efficiency brought by the development of science and technology to life, so the speech recognition model for Chongqing dialect provided in the application will help Chongqing dialect to be more accurately "recognized" by the computer, so that the early screening of senile dementia can be promoted in more areas.

[0004] The output of automatic speech recognition (ASR) is a piece of text, and the main algorithm model currently includes a GMM-HMM-based acoustic model, which has the advantages of fast GMM training speed, small acoustic model and easy transplantation to an embedded platform, but the GMM does not utilize the context information of frames, and the GMM cannot learn deep nonlinear feature transformation.

[0005] Therefore, a DNN-HMM model appears, which has the advantages that the DNN can utilize the context information of frames, such as expansion of 5 frames before and after, and the DNN can learn deep nonlinear feature transformation, and the performance is better than that of the GMM, but the disadvantage is that the historical information cannot be utilized to assist the current task.

[0006] With the development of the recurrent neural network based on the time sequence scene, scholars have proposed an RNN-HMM model and a CNN-HMM acoustic model. The RNN can effectively utilize the historical information, and the historical information is persistent, and in many tasks, the performance of the RNN is better than that of the DNN, but with the increase of the number of layers, the RNN will cause gradient explosion or gradient disappearance, and the network based on the RNN has problems of long training time and large parameter quantity; the CNN-HMM acoustic model utilizes the feature extraction capability of the CNN, uses the CNN to construct an acoustic model, and extracts acoustic features, and the advantage is that the CNN can well describe and learn the signal by adopting a time delay convolutional neural network for the speech signal, but the CNN does not conform to the language model.

[0007] In summary, different model recognition units (word pronunciation model, character pronunciation model, semi-syllable model or phoneme model) have a greater impact on the size of the speech training data, the speech recognition rate and flexibility. For a speech recognition system with a medium vocabulary, the smaller the recognition unit, the smaller the calculation amount and the required model storage amount, but the problem is that it is difficult to locate and segment the corresponding speech segment, and the recognition model rule is more complex. With the development of deep learning, its modeling ability is stronger, and it tends to use pinyin, character and word models.

[0008] In summary, the existing automatic speech recognition technology is difficult to capture different pronunciations of characters, and its accuracy is limited in the application of Chongqing dialect speech recognition. Specifically, the automatic speech recognition technology has the following defects:

[0009] 1) The statistical language model is affected by the size of the corpus, and the effect is limited, and the statistical information has limited expression ability at the semantic level.

[0010] 2) In the doctor-patient communication scenario, patients with poor expression ability are usually asked to repeat the answers, and this useful information is rarely used by algorithm models.

[0011] 3) Because the pronunciation of the Chongqing dialect of the elderly is not accurate, and the articulation is unclear, the model fusion is often more accurate than the single model.

[0012] 4) In a specific doctor-patient communication scenario, the decoding word table is limited, so the size of the language model can be compressed, and the model parameters can be further compressed. SUMMARY

[0013] The purpose of the present application is to provide a method for recognizing the speech of the old dialect based on the gating convolution, comprising the following steps:

[0014] 1) Obtain at least two audio files recorded by the user;

[0015] 2) Process each audio file respectively to obtain a spectrum vector Vr and a frequency domain feature Fbankr; r = 1, 2, …, R; R≥2 represents the number of audio files;

[0016] 3) Build a Gate CNN network;

[0017] 4) Input the spectrum vector Vr and the frequency domain feature Fbankr into the Gate CNN network as input data to obtain the speech recognition result hr of the rth audio file;

[0018] 5) Repeat step 4) to obtain the speech recognition results of the R audio files;

[0019] 6) processing the speech recognition results of the R audio files to obtain the final speech recognition result.

[0020] Further, the step of processing the audio files to obtain the spectrum vector Vr includes:

[0021] a) performing Fourier transform on the audio files to obtain the spectrum graph of the single audio file;

[0022] b) using the VGG network to vectorize the spectrum graph of the single audio file to obtain the spectrum vector Vr.

[0023] Further, the spectrum vector Vr is as follows:

[0024] Vr = VGG(DFT(Ar)) (1)

[0025] In the formula, Ar represents the rth audio file; DFT represents the discrete Fourier transform; and VGG represents the VGG network.

[0026] Further, the Gate CNN network includes an input layer, a plurality of convolutional layers with a gating mechanism, a pooling layer, and an output layer.

[0027] Further, the speech recognition result hr of the rth audio file includes the decoding probability of the M words in the rth audio file.

[0028] In the formula, the decoding probability hrm of the mth word in the rth audio file is as follows:

[0029] hrm = Am*σ(Bm) (2)

[0030] In the formula, Am and σ(Bm) represent the threshold value and the convolution activation value of the output of the last convolutional layer, respectively.

[0031] In the Gate CNN network, the threshold value A mi and the convolution activation value σ(B mi ) of the i-th convolutional layer are as follows:

[0032] A mi = X*W + b (3)

[0033] σ(B mi ) = sigmod(X*V + c) (4)

[0034] In the formula, W and V represent the convolution kernel, b and c represent the bias, and X represents the input.

[0035] Further, the Gate CNN network is obtained by training a plurality of historical audio files, and in the training process, the gradient of the Gate CNN network is updated as follows:

[0036]

[0037] wherein, σ(B i )′ represents a gradient factor. σ(B i ) represents a convolutional activation value.

[0038] Further, the training target of the Gate CNN network is that the loss function Loss reaches the minimum;

[0039] The loss function Loss is as follows:

[0040] Loss = Loss nll + α * Loss kl (6)

[0041] wherein, α is a proportional coefficient;

[0042] wherein, the cross-entropy loss function Loss nll , the divergence loss function loss kl are respectively as follows:

[0043]

[0044]

[0045] wherein, c represents a category; Z is the total number of categories; y j，c represents an output; j represents the jth decoded word; n represents the length of the decoded text. h j，c represents a sample distribution; D kl is a divergence loss function; h j，1 , h j，2 represent different vocabulary distributions; h j，1，t , h j，2，t represent distributions.

[0046] Further, the number of audio files R = 2, at this time, the final speech recognition result is as follows:

[0047] h m = λ * h1 + (1- λ) * h2 (9)

[0048] wherein, λ is a weight. h1, h2 represent vocabulary distributions.

[0049] Further, in step 6), after obtaining the speech recognition result, the pinyin of the speech recognition result is segmented, the pinyin is divided into a plurality of individual syllables, and N similar syllables are selected from the similar syllable dictionary according to the syllable;

[0050] Combine from all possible similar syllables to form new pinyins, so as to obtain several candidate results;

[0051] The similar syllable dictionary is constructed by a statistical probability model, and includes a plurality of syllables and a plurality of other syllables similar to the corresponding syllables; the similarity of different syllables is obtained by statistical audio files.

[0052] A computer readable storage medium, the computer readable medium stores a computer program;

[0053] The computer program is executed by the processor to realize the steps of the above method.

[0054] The technical effect of the present application is self-evident. Chongqing dialect has certain differences in initial and final consonants from Mandarin, and most AD patients are old people, and their pronunciation often has unclear pronunciation and other conditions. Doctors face patients with unclear pronunciation, and ask patients to repeat the second time. The application refers to this mechanism as the "confirmation" mechanism.

[0055] The present application analyzes the differences between Chongqing dialect and Mandarin in the application of speech recognition (ASR) technology, and defines the "confirmation" mechanism according to the characteristics of the doctor-patient scene. The application uses this mechanism in the algorithm model development based on word model granularity, thereby improving the accuracy of speech recognition in the doctor-patient communication scene.

[0056] According to the difference between dialect and Mandarin, the present application summarizes the dictionary that is easily confused in acoustics, and uses the dictionary as a supplement for decoding to improve the final accuracy of ASR.

[0057] The present application uses Gate CNN to do acoustic model, that is, it retains the feature processing capability of CNN and the time characteristics of the model, and at the same time the structure is simple and the parameters are controllable. BRIEF DESCRIPTION OF DRAWINGS

[0058] Fig. 1 The flowchart of speech recognition;

[0059] Fig. 2 The flowchart of speech data processing;

[0060] Fig. 3 The model structure. DETAILED DESCRIPTION

[0061] The present application will be further described below in conjunction with the embodiments, but should not be understood as limiting the above-mentioned subject matter of the present application to the following embodiments. According to ordinary technical knowledge and conventional means in the art, various substitutions and modifications can be made without departing from the above-mentioned technical idea of the present application, and all should be included in the protection scope of the present application.

[0062] Example 1:

[0063] Referring to Figs. 1 to 3 A method for recognizing old dialect speech based on a gated convolutional neural network, comprising the following steps:

[0064] 1) obtaining at least two audio files recorded by a user;

[0065] 2) processing each audio file respectively to obtain a spectrum vector Vr and a frequency domain feature Fbankr; r = 1, 2,..., R; R ≥ 2 represents the number of audio files;

[0066] 3) building a Gate CNN network;

[0067] 4) inputting the spectrum vector Vr and the frequency domain feature Fbankr into the Gate CNN network as input data to obtain a speech recognition result hr of the rth audio file;

[0068] 5) repeating step 4) to obtain speech recognition results of R audio files;

[0069] 6) processing the speech recognition results of the R audio files to obtain a final speech recognition result.

[0070] The step of processing the audio file to obtain the spectrum vector Vr comprises:

[0071] a) performing Fourier transform on the audio file to obtain a spectrum graph of the single audio file;

[0072] b) vectorizing the spectrum graph of the single audio file using a VGG network to obtain the spectrum vector Vr.

[0073] The spectrum vector Vr is as follows:

[0074] Vr = VGG(DFT(Ar)) (1)

[0075] wherein Ar represents the rth audio file; DFT represents discrete Fourier transform; and VGG represents a VGG network.

[0076] The Gate CNN network comprises an input layer, a plurality of convolutional layers with a gating mechanism, a pooling layer, and an output layer.

[0077] The speech recognition result hr of the rth audio file comprises decoding probabilities of M words in the rth audio file.

[0078] wherein the decoding probability hrm of the mth word in the rth audio file is as follows:

[0079] hrm = Am * σ(Bm) (2)

[0080] Am, σ(Bm) respectively represent the threshold value and the convolution activation value of the output of the last convolution layer;

[0081] Gate CNN network, the threshold value A mi and the convolution activation value σ(B mi ) of the i-th convolution layer are as follows:

[0082] A mi = X*W+b (3)

[0083] σ(B mi ) = sigmod (X*V+c) (4)

[0084] Wherein, W, V represent the convolution kernel, b, c represent the bias; X represents the input.

[0085] The Gate CNN network is trained by a plurality of historical audio files, and in the training process, the gradient of the Gate CNN network is updated as follows:

[0086]

[0087] Wherein, σ(B i )' represents the gradient factor.

[0088] The training target of the Gate CNN network is to minimize the loss function Loss;

[0089] The loss function Loss is as follows:

[0090] Loss = Loss nll + alpha * Loss kl (6)

[0091] Wherein, alpha is a proportionality coefficient;

[0092] Wherein, the cross-entropy loss function Loss nll , the divergence loss function loss kk are as follows:

[0093]

[0094]

[0095] Wherein, c represents the category; Z is the total number of categories; y j，c represents the output; j represents the j-th decoded word; n represents the length of the decoded text. h j，c represents the sample distribution, that is, the probability that the j-th sample belongs to the c-th category; D kl is the divergence loss function; h j，1, h j，2 represent different vocabulary distribution;h j，1，t , h j，2，t represent distribution.

[0096] The number of audio files R=2, at this time, the final speech recognition result is as follows:

[0097] h m = lambda*h1+ (1-lambda)*h2 (9)

[0098] In the formula, lambda is a weight. h1 and h2 represent vocabulary distribution.

[0099] In step 6), after obtaining the speech recognition result, the pinyin of the speech recognition result is segmented, the pinyin is divided into a plurality of individual syllables, and N similar syllables are selected from the similar syllable dictionary according to the syllable;

[0100] The new pinyin is combined from all possible similar syllables, thereby obtaining a plurality of candidate results.

[0101] The similar syllable dictionary is constructed based on a statistical probability model, and includes a plurality of syllables and a plurality of other syllables similar to the corresponding syllables; the similarity of different syllables is obtained by historical audio file statistics.

[0102] A computer readable storage medium, the computer readable medium stores a computer program;

[0103] The computer program is executed by the processor to realize the steps of the above method.

[0104] Embodiment 2:

[0105] A gating convolution-based old dialect speech recognition method, the content is as follows:

[0106] 1) According to the audio, the frequency spectrum of a single audio file is obtained by using signal processing technology and Fourier transform, and the vector expression of the whole structure diagram is extracted by VGG network structure.

[0107] The formula can be expressed as:

[0108] V=VGG(DFT(A))

[0109] A: audio file; DFT: discrete Fourier transform; VGG: VGG network; V: VGG output vector expression.

[0110] 2) According to the frequency spectrum, the position information of each frequency spectrum unit in the original graph is obtained, and after embedding the vector, it is input into Gate CNN together with Fbank.

[0111] In the CNN layer:

[0112] A i =X*W+b

[0113] σ(B i )=sigmod(X*V+c)

[0114] So when updating the gradient:

[0115]

[0116] It can be seen that, The term does not include the gradient factor σ(B i )′, which can help reduce gradient dispersion during training when the gradient flows through the layer. At the same time, the amount of information passing down the information can be controlled through the σ function, thereby ensuring that the Gate CNN maintains a certain timing during the training process. The core of the Gate CNN is to add a threshold switch to the convolutional activation value to determine how much probability it will pass to the next layer. Convolution is divided into two parts when calculating, one part is the convolutional activation value, B, which is different from the ordinary convolution in that it does not use Tanh, but is directly linear, but will be subjected to a sigmoid operator.

[0117] The confirmation mechanism is based on the dialogue between the old man and the doctor in the clinic. Conceptually, the dialogue content in the first input and the second repeated input is the same, and for the elderly, especially the elderly with cognitive impairment, due to ambiguity, tone, frequency, speed, pause and other factors, the repeated sentence cannot be kept exactly the same twice. This situation is captured through random simulation. Dropout is a technique commonly used in the training phase of neural networks, when it is applied to neural networks, a "thinned" network is created, in which some neurons in the hidden layer are randomly set to zero with a certain probability during training. Each time the gradient of the model is updated, a new thinned neural network is generated, and this random simulation feature can be used to help simulate the two repeated sentences in the doctor-patient scenario. Assuming that the decoding result of the mth word is a probability distribution of a fixed word table, this process can obtain another probability distribution of the same dimension by inputting the same content repeatedly, and the two probability distributions are randomly generated, which simulates the differences in ambiguity, tone, frequency, speed, pause, etc. between the two repetitions. Introduce a parameter λ between (0, 1.0) to weight the two outputs. The final probability distribution of the mth decoding word is:

[0118] h m =λ*h1+(1-λ)*h2

[0119] Since the two vocabulary distributions are actually calculated from the same sample, theoretically, they should be made consistent as much as possible, and the Kullback Leibler divergence loss function is used to constrain the two outputs. Based on the above considerations, in the design of the loss function, since the decoding of each word is a classification problem, cross-entropy is used as one of the two loss functions. The other loss function is the Kullback Leibler divergence function between the two inputs, which plays a role in ensuring that the two outputs of the two inputs are as consistent as possible. The final loss function is the weighted sum of the two loss functions. These loss functions are represented as:

[0120]

[0121]

[0122] Loss=Loss nll +α*Loss kl

[0123] The cross-entropy loss function only focuses on the role of category c in word classification. Generally, category c is the word with the largest probability value in its probability distribution, and the cross-entropy loss function alone cannot consider other similar words and contexts in training, such as "Chongqing" is a common word, when the previous position decoding result is "heavy", the invention should more likely measure that the current decoding is "heavy" "celebration" or "heavy" "repetition" "complexity", etc. Therefore, the addition of the Kullback Leibler divergence loss function can help the model learn better from the perspective of language modeling.

[0124] The purpose of using dropout in the invention is to simulate repeated conversations between doctors and patients, and dropout repeats the same sentence, so the goal of training is to minimize the difference between the two probability distributions generated by dropout, that is, the Kullback Leibler divergence loss function. The following mathematically proves that the Kullback Leibler divergence function of the invention is always greater than or equal to 0, when it is minimized as a loss function, the training process will push it to approach zero, at which point the two probability distributions become identical in theory. h1 and h2 are two vocabulary distributions, and satisfy the constraint conditions ∑h1=1, ∑h2=1, then the divergence loss function of the two distributions is:

[0125]

[0126] According to the inequality of Qian Sheng, if g(x) and are real functions, then:

[0127]

[0128] -ln x is a convex function, then let:

[0129]

[0130]

[0131] From Qin Sheng's inequality, we can obtain:

[0132]

[0133] therefore:

[0134] D kl (h1, h2) ≥ 0

[0135] D is true if and only if h1 and h2 are equal. kl The result is 0. Therefore, the above proof demonstrates the theoretical support for using the Kullback-Leibler divergence function as the loss function in this invention.

[0136] Due to differences in initials and finals between Chongqing dialect and Standard Mandarin, and because most dementia patients are elderly, their pronunciation often suffers from unclear speech. For example, the initials 'sh' and 's' are easily confused in Chongqing dialect, as are the finals 'in' and 'ing'. Therefore, when speech recognition cannot distinguish between 'in' and 'ing' or 'sh' and 's', this invention proposes constructing a dictionary of similar syllables to improve the reliability of speech recognition. When speech recognition encounters any Chinese character, this invention first segments the character's pinyin into initials and finals, i.e., individual syllables. Then, for each syllable, it searches the dictionary for all possible similar syllables and combines them into a new pinyin. Many of these recombinations may result in a non-existent pinyin or a rare character with a very low frequency of occurrence. If a full search and matching were performed for every character, the speech recognition decoding would become inefficient. Therefore, this invention sets a search range, typically with a value of 2, returning the two new pinyin with the highest probability. This principle is similar to beam search in natural language text generation. Regarding how to determine the probability of forming a new pinyin, this invention uses a statistical probability model, that is, to count a large amount of text from the Chongqing area, count the frequency of individual characters and 2-gram words, and thus calculate the probability of forming a new pinyin. The words that are most likely to appear in the actual text are selected as candidate results, thereby ultimately improving the probability of speech recognition in this invention.

[0137] Example 3:

[0138] A method for speech recognition of elderly dialects based on gated convolution includes the following steps:

[0139] 1) obtaining at least two audio files recorded by a user;

[0140] 2) processing each audio file respectively to obtain a spectrum vector Vr and a frequency domain feature Fbankr; r = 1, 2, …, R; R ≥ 2 represents the number of audio files;

[0141] 3) building a Gate CNN network;

[0142] 4) inputting the spectrum vector Vr and the frequency domain feature Fbankr into the Gate CNN network as input data to obtain a speech recognition result hr of the rth audio file;

[0143] 5) repeating step 4) to obtain speech recognition results of R audio files;

[0144] 6) processing the speech recognition results of the R audio files to obtain a final speech recognition result.

[0145] Embodiment 4:

[0146] A method for recognizing old dialect speech based on a gate convolution mainly includes the steps of the embodiment 3, and the step of processing the audio file to obtain the spectrum vector Vr comprises:

[0147] 1) performing Fourier transform on the audio file to obtain a spectrum graph of a single audio file;

[0148] 2) vectorizing the spectrum graph of the single audio file by using a VGG network to obtain the spectrum vector Vr.

[0149] Embodiment 5:

[0150] A method for recognizing old dialect speech based on a gate convolution mainly includes the steps of the embodiment 3, and the spectrum vector Vr is as follows:

[0151] Vr = VGG(DFT(Ar)) (1)

[0152] In the formula, Ar represents the rth audio file; DFT represents discrete Fourier transform; and VGG represents a VGG network.

[0153] Embodiment 6:

[0154] A method for recognizing old dialect speech based on a gate convolution mainly includes the steps of the embodiment 3, and the Gate CNN network comprises an input layer, a plurality of convolution layers with a gating mechanism, a pooling layer and an output layer.

[0155] Embodiment 7:

[0156] A method for recognizing old dialect speech based on a gated convolution, the main content of which is shown in embodiment 3, the speech recognition result hr of the rth audio file includes the decoding probability of M words in the rth audio file;

[0157] Wherein the decoding probability hrm of the mth word in the rth audio file is as follows:

[0158] hrm=Am*σ(Bm) (2)

[0159] In the formula, Am and σ(Bm) represent the threshold value and the convolution activation value of the output of the last convolution layer, respectively;

[0160] In the Gate CNN network, the threshold value A mi and the convolution activation value σ(B mi ) of the i-th convolution layer are as follows:

[0161] A mi =X*W+b (3)

[0162] σ(B mi )=sigmod(X*V+c) (4)

[0163] In the formula, W and V represent convolution kernels, b and c represent bias, and X represents input.

[0164] Embodiment 8:

[0165] A method for recognizing old dialect speech based on a gated convolution, the main content of which is shown in embodiment 3, the Gate CNN network is obtained by training a plurality of historical audio files, and in the training process, the gradient of the Gate CNN network is updated as follows:

[0166]

[0167] In the formula, σ(B i )' represents a gradient factor. σ(B i ) represents a convolution activation value.

[0168] Embodiment 9:

[0169] A method for recognizing old dialect speech based on a gated convolution, the main content of which is shown in embodiment 3, the training target of the Gate CNN network is to minimize the loss function Loss;

[0170] The loss function Loss is as follows:

[0171] Loss=Loss nll +α*Loss kl (6)

[0172] wherein, a is a proportional coefficient;

[0173] wherein, the cross-entropy loss function Loss nll , the divergence loss function loss kl are respectively as follows:

[0174]

[0175]

[0176] wherein, c represents a class; Z is the total number of classes; y j，c represents an output; j represents the jth decoded word; n represents the length of the decoded text. j，c represents the probability that the jth sample belongs to the cth class; D kl is the divergence loss function; h j，1 , h j，2 represents different vocabulary distributions; h j，1，t , h j，2，t represents the distribution probability.

[0177] Example 10:

[0178] A method for recognizing old dialect speech based on gated convolution, the main content of which is seen in Example 3, the number of audio files R = 2, at this time, the final speech recognition result is as follows:

[0179] h m = lambda * h1 + (1 - lambda) * h2 (9)

[0180] wherein, lambda is a weight. h1, h2 represent vocabulary distributions.

[0181] Example 11:

[0182] A method for recognizing old dialect speech based on gated convolution, the main content of which is seen in Example 3, in step 6), after obtaining the speech recognition result, the pinyin of the speech recognition result is segmented, the pinyin is divided into a plurality of individual syllables, and N similar syllables are selected from a similar syllable dictionary according to the syllable;

[0183] The new pinyin is combined from all possible similar syllables, thereby obtaining a plurality of candidate results.

[0184] The similar syllable dictionary is constructed based on a statistical probability model, including a plurality of syllables and a plurality of other syllables similar to the corresponding syllables; the similarity of different syllables is obtained by statistical analysis of historical audio files.

[0185] Example 12:

[0186] A computer readable storage medium, the computer readable medium storing a computer program;

[0187] The computer program, when executed by a processor, implements the steps of the methods of embodiments 1-11.

Claims

1. A method for recognizing an old dialect voice based on a gated convolution, characterized by, The method comprises the following steps: 1) obtaining at least two audio files with the same content recorded by a user; 2) processing each audio file respectively to obtain a spectrum vector Vr and a frequency domain feature Fbankr; r=1, 2, …, R; R≥2 represents the number of audio files; 3) building a Gate CNN network; 4) inputting the spectrum vector Vr and the frequency domain feature Fbankr into the Gate CNN network as input data to obtain a speech recognition result hr of the rth audio file; 5) repeating step 4) to obtain speech recognition results of R audio files; 6) processing the speech recognition results of the R audio files to obtain a final speech recognition result.

2. The method of claim 1, wherein the method is a gated convolution-based method for recognizing an old dialect voice. The step of processing the audio file to obtain the spectrum vector Vr comprises: 1) performing Fourier transform on the audio file to obtain a spectrum graph of a single audio file; 2) vectorizing the spectrum graph of the single audio file by using a VGG network to obtain the spectrum vector Vr; The spectrum vector Vr is as follows: Vr=VGG(DFT(Ar)) (1) In the formula, Ar represents the rth audio file; DFT represents discrete Fourier transform; and VGG represents a VGG network.

3. The method of claim 1, wherein the method is based on a gated convolution. The Gate CNN network comprises an input layer, a plurality of convolutional layers with a gating mechanism, a pooling layer, and an output layer.

4. The method of claim 1, wherein the method is a gated convolution-based method for recognizing an old dialect voice. The speech recognition result hr of the rth audio file comprises decoding probabilities of M words in the rth audio file; In the formula, Am and σ(Bm) represent a threshold value and a convolutional activation value output by the last convolutional layer, respectively; In the formula, W and V represent convolutional kernels, b and c represent biases, and X represents input. The training target of the Gate CNN network is to minimize a loss function Loss; In the Gate CNN network, the threshold value A of the i-th layer of the convolution layer mi and the convolution activation value σ(B mi ) are respectively as follows: A mi = X * W + b (3) σ(B mi ) = sigmoid(X*V + c) (4) The loss function Loss is as follows:

5. The method of claim 1, wherein the method is based on a gated convolution. The Gate CNN network is trained by a plurality of historical audio files. In the training process, the gradient of the Gate CNN network is updated as follows: The update is as follows: In the formula, σ(B i )′ represents a gradient factor; σ(B i ) represents a convolution activation value; A mi is a threshold value of the i-th layer of convolution layer.

6. The method of claim 5, wherein the method is a gated convolution-based method for recognizing an old dialect voice. In the formula, α is a proportionality coefficient; When the number of audio files R=2, the final speech recognition result is as follows: Loss = Loss nll + a * Loss kl (6) In the formula, λ is a weight; and h1 and h2 represent word table distributions. wherein the cross-entropy loss function Loss nll , the divergence loss function loss kl are respectively as follows: where c denotes the class; Z is the total number of classes; y j,c represents the output; j denotes the jth decoded word; n denotes the length of the decoded text; h j,c represents the sample distribution; D kl is the divergence loss function; h j,1 , h j,2 represents the different vocabulary distributions; h j,1,t , h j,2,t represents the distribution.

7. The method of claim 1, wherein the method is a gated convolution-based method for recognizing an old dialect voice. In step 6), after obtaining the speech recognition result, the pinyin of the speech recognition result is segmented, the pinyin is divided into a plurality of individual syllables, and N similar syllables are selected from a similar syllable dictionary according to the syllables; h m = λ * h1+ (1 - λ) * h2(9) The similar syllables are combined from all possible similar syllables to form new pinyins, thereby obtaining a plurality of candidate results; 8. The method of claim 1, wherein the method is a gated convolution-based method for recognizing an old dialect voice. The similar syllable dictionary is constructed by a statistical probability model and comprises a plurality of syllables and a plurality of other syllables similar to the corresponding syllables; the similarity of different syllables is obtained by statistical analysis of historical audio files. The computer readable medium stores a computer program; The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

9. A computer-readable storage medium, characterized in that, ​ ​

Images