A speech recognition method, electronic device, vehicle-mounted speech recognition system and vehicle

Abstract

The present invention relates to a speech recognition method, electronic equipment, an in-vehicle speech recognition system, and a car. According to one embodiment of the present invention, the speech recognition method includes: extracting features from noisy audio data to obtain a noisy feature vector; performing speech enhancement on the noisy feature vector to obtain a speech enhancement feature vector; fusing the noisy feature vector with the speech enhancement feature vector to obtain a noise-resistant feature vector; and performing speech recognition on the noise-resistant feature vector to obtain a recognition result. In the technical solution of the present invention, the original noisy feature vector is fused with the speech enhancement feature vector output after speech enhancement to obtain a noise-resistant feature vector, so that the speech enhancement feature vector that has lost useful audio information regains its original information, and then performs speech recognition, thereby improving the accuracy of speech recognition for users in human-computer interaction.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and specifically provides a speech recognition method, electronic equipment, a vehicle-mounted speech recognition system, and a vehicle. Background Art

[0002] Human-computer voice interaction is accompanied by the process of voice recognition. Due to environmental noise, the accuracy of voice recognition will be affected.

[0003] Currently, there are several solutions for achieving noise-resistant speech recognition, each with its own shortcomings: 1) Adding simulated noise to existing training corpus. This method can only simulate noise and is different from the actual complex noise environment; 2) Processing noisy speech through speech enhancement before speech recognition. This method will cause some useful information to be lost during the speech enhancement process; 3) Using hardware noise reduction methods. This method is only applicable to specific scenarios.

[0004] Accordingly, a new speech recognition solution is needed in the art to solve at least one of the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects, the present invention is proposed to provide a solution or at least a partial solution to the problems existing in the prior art.

[0006] In a first aspect, the present invention provides a speech recognition method, comprising:

[0007] Extract features from noisy frequency data to obtain noisy feature vectors;

[0008] Performing speech enhancement on the noisy feature vector to obtain a speech enhancement feature vector;

[0009] Obtaining a noise-resistant feature vector based on the noisy feature vector and the speech enhancement feature vector;

[0010] The anti-noise feature vector is used for speech recognition to obtain a recognition result.

[0011] Furthermore, performing speech enhancement on the noisy feature vector to obtain a speech enhancement feature vector includes:

[0012] The noisy feature vector is input into a trained speech enhancement model to obtain the speech enhancement feature vector, wherein the speech enhancement model includes an encoder and a decoder, the encoder is used to convert the noisy feature vector into a dense vector, and the decoder is used to convert the dense vector into the speech enhancement feature vector.

[0013] Furthermore, the encoder is composed of an M-layer bidirectional LSTM network; the decoder is composed of an N-layer bidirectional LSTM network, where M and N are both natural numbers.

[0014] Furthermore, obtaining the anti-noise feature vector based on the noisy feature vector and the speech enhancement feature vector includes:

[0015] Fusing the noisy feature vector with the speech enhancement feature vector to obtain a fused feature vector;

[0016] Passing the fused feature vector through the trained attention mechanism model to obtain a first attention feature vector;

[0017] Multiplying the first attention feature vector by the noisy feature vector to obtain a second attention feature vector;

[0018] The second attention feature vector is added to the speech enhancement feature vector to obtain the anti-noise feature vector.

[0019] Furthermore, the fusion is a splicing operation; the attention mechanism model is:

[0020]

[0021] Where P represents the anti-noise feature vector, ATT represents the self-attention model, x represents the noisy feature vector, sp represents the speech enhancement feature vector, softmax is the activation function, d k is the scaling factor.

[0022] Furthermore, performing speech recognition on the anti-noise feature vector to obtain a recognition result includes:

[0023] Inputting the noise-resistant feature vector into a trained speech recognition model to obtain probabilities of multiple texts corresponding to the noise-resistant feature vector;

[0024] The text corresponding to the maximum probability among the probabilities is selected as the recognition result.

[0025] Furthermore, the method further includes the step of training the speech enhancement model, including:

[0026] The loss function L(θ) is used for training, where:

[0027]

[0028]

[0029] Where x represents the noisy feature vector, Represents the speech enhancement feature vector.

[0030] In a second aspect, the present invention provides an electronic device comprising a processor and a storage device, wherein the storage device is adapted to store a plurality of program codes, and the program codes are adapted to be loaded and run by the processor to execute the speech recognition method according to any one of the first aspects.

[0031] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of program codes, wherein the program codes are suitable for being loaded and executed by a processor to execute the speech recognition method according to any one of the first aspects.

[0032] In a fourth aspect, the present invention provides a vehicle-mounted speech recognition system, comprising:

[0033] The electronic device according to the third aspect;

[0034] The voice collection device is used to collect noisy audio data.

[0035] In a fifth aspect, the present invention provides a car comprising the vehicle-mounted speech recognition system described in the fourth aspect.

[0036] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0037] In the technical solution of this invention, the original noisy feature vector is fused with the speech-enhanced feature vector output after speech enhancement to generate a noise-resistant feature vector. This allows the speech-enhanced feature vector, which has lost useful audio information, to regain its original information, thereby performing speech recognition and improving the accuracy of speech recognition in human-computer interaction. Furthermore, this solution avoids the drawbacks of simulated noise addition during training and does not require specific hardware for noise reduction, making it universally applicable to various application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Furthermore, similar numbers in the drawings represent similar components, wherein:

[0039] Figure 1 is a flow chart of the main steps of a speech recognition method according to an embodiment of the present invention;

[0040] Figure 2 1 is a flow chart of a speech recognition method according to an embodiment of the present invention.

[0041] Figure 3 1 is an internal flow chart of a speech enhancement model of a speech recognition method according to an embodiment of the present invention.

[0042] Figure 4 1 is a flow chart of a speech recognition method according to an embodiment of the present invention.

[0043] Figure 5 1 is a flow chart of a speech recognition method according to an embodiment of the present invention.

[0044] Figure 6 1 is a flow chart of a speech recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0046] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory. It may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor may be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or both A and B. The terms "at least one of A or B" or "at least one of A and B" have similar meanings to "A and / or B" and may include only A, only B, or both A and B. The singular forms "one" and "the" may also include the plural forms.

[0047] Here we first explain some terms involved in the present invention.

[0048] Features: A feature is typically a region of an image or other object that has certain properties. For ease of understanding, using images as an example, a feature might be a specific structure within the image, such as a point, edge, or object. Features can also be the result of general neighborhood operations or feature detection applied to an image. Other examples of features, depending on the requirements, might relate to motion within an image sequence or shapes defined by curves or boundaries between different image regions.

[0049] Feature extraction: To better represent target features or simplify subsequent machine learning, feature extraction is often used. When the input data for an algorithm is too large to process and suspected to be redundant (for example, in an image represented by pixels, where pixels are often repeated), it can be converted into feature vectors, representing the input data using mathematical eigenvectors.

[0050] In current speech recognition, traditional physical noise reduction methods are relatively simple, but their effectiveness is poor and limited in many scenarios, such as noise reduction using sound insulation and filters. Using algorithmic models for noise reduction, whether using the commonly used encoder-decoder model architecture or conventional noise reduction methods, will lose some useful speech information during noise removal, reducing speech recognition accuracy.

[0051] In an example of an application scenario of the present invention, for example, human-vehicle voice interaction in a running car. There are many possible sources of noise during the operation of the car, such as tire noise, wind noise, reasonable noises in the car (such as noises caused by wind chimes in the car, music being played, broadcasts, etc.), noises caused by neighboring cars, etc. Due to safety considerations in the design of the car body, the sound outside the car can definitely be transmitted to the car, so in the process of human-vehicle voice interaction, the above noises will hinder the accuracy of voice recognition. The user's voice command will not be accurately recognized, affecting the user's driving experience.

[0052] Refer to the attached figure, Figure 1 FIG. 1 is a flow chart showing the main steps of a speech recognition method according to an embodiment of the present invention. Figure 1 As shown, the speech recognition method in the embodiment of the present invention mainly includes the following steps S100 to S400.

[0053] Step S100: extracting features from noisy frequency data to obtain noisy feature vectors.

[0054] In this embodiment, the noisy audio data refers to audio data with ambient noise. It should be noted that noisy audio data is not limited to real audio with ambient noise collected by a microphone or other audio receiving device. It can also be simulated audio generated by adding simulated noise to training audio and then performing speech synthesis. The training audio is the audio used for neural network training. The original training audio can be natural human voice captured by a microphone, simulated human voice automatically synthesized by a machine, or human voice clips downloaded from the Internet.

[0055] In one embodiment, FBank is used to extract features from noisy audio data to obtain a noisy feature vector. Before performing FBank feature extraction, the noisy audio data can be preprocessed. During the preprocessing, the noisy audio will be framed. After the frequency division, a minimum unit audio is obtained by dividing the noisy audio into a preset frame length, which is called a frame. The subsequent FBank features and other processing are all frame-by-frame processing of the noisy audio. It should be explained that the frames in the preprocessing are different from the frames in the time domain waveform. In the preprocessing, the frame after segmentation is a sample for analyzing and extracting Fbank features, while the frame in the time domain waveform is a sample obtained by sampling the audio on the time domain scale. FBank feature extraction is performed after preprocessing.

[0056] In this embodiment, the steps of extracting FBank features are as follows: Figure 2 Shown, including:

[0057] S1001: Convert noisy frequency data from a time domain signal to a frequency domain signal.

[0058] In this embodiment, the noisy audio data is pre-processed and framed. The independent variable of a time-domain signal is time, and the dependent variable is the signal amplitude. The independent variable of a frequency-domain signal is frequency, and the dependent variable is the amplitude of that frequency signal. In practice, time-domain representation is generally more vivid and intuitive, while frequency-domain analysis is more concise and convenient for analyzing problems. The noisy audio data after frequency division is still a time-domain signal. To facilitate identification of the signals, it is necessary to convert the time-domain signal into a frequency-domain signal.

[0059] In one embodiment, the noisy audio data is converted from a time domain signal to a frequency domain signal through Fourier transform. Since the noisy audio data in this embodiment is digital audio, a discrete Fourier transform (DFT) is used. Preferably, a fast Fourier transform (FFT) is used in this embodiment to reduce the computational complexity of the Fourier transform.

[0060] Step S1002: The noisy frequency data, which is a frequency domain signal, is filtered through a Mel filter to obtain a noisy feature vector.

[0061] In this embodiment, the Mel filter (Mel Filterbank) is a prior art. To facilitate understanding of the role of the Mel filter, the following explanation is first made: human sensitivity to audio is related to the frequency of the audio. Compared with high-frequency signals, humans perceive low-frequency signals more strongly than high-frequency signals. Specifically, below 1kHz, the degree of perception is linearly related to the frequency, and above 1kHz, the degree of perception is logarithmically related to the frequency. The Mel filter behaves as a scaling rule that can simulate the human perception rules for audio signals of different frequencies. It is understandable that the noisy audio data that passes through the Mel filter is more consistent with the information that humans can receive in reality. The noisy audio data that passes through the Mel filter is then converted into a noisy feature vector to obtain the noisy feature vector x = (x1, x2...x T ), where T corresponds to the number of frames, i.e. the sequence length.

[0062] Step S200: performing speech enhancement on the noisy feature vector to obtain a speech enhanced feature vector.

[0063] In this embodiment, speech enhancement is a noise reduction process. During the speech enhancement process, as ambient noise is reduced, some audio containing information elements is also eliminated, reducing the probability of accurate recognition of the user's voice command. In this embodiment, the speech enhancement step is performed using a trained speech enhancement model to obtain a speech enhancement feature vector.

[0064] In one embodiment, the trained speech enhancement model uses an Encoder-Decoder model architecture, such as Figure 3 As shown in Figure 2. The encoder transforms the input sequence into a dense vector of fixed dimension, and the decoder generates the target translation from this activation state. The encoder expression is: W = F(UX), where W is the dense vector, X is the noisy feature vector, U is the M-layer bidirectional LSTM network, and F is the activation function. The decoder expression is: in is the speech enhancement feature vector, W is the dense vector, V is the N-layer bidirectional LSTM network, and F is the activation function. In this embodiment, the encoder preferably uses a 6-layer bidirectional LSTM network, and the decoder preferably uses a 2-layer bidirectional LSTM network. Selecting a bidirectional LSTM network as the basis function can better capture some features of the context, thereby increasing robustness.

[0065] In addition, when training the speech enhancement model, the loss function L(θ) is used for training, where:

[0066]

[0067]

[0068] Where x represents the noisy feature vector, Represents the speech enhancement feature vector.

[0069] It should be noted that the use of the Encoder-Decoder model architecture itself will cause information loss. Specifically, in the process of the encoder converting the noisy feature vector into a dense vector, the information is subjected to a certain degree of lossy compression. The larger the amount of information in a single shot, the greater the final information loss. Therefore, it is not difficult to understand that when framing, adjusting the time interval between two adjacent frames will have a certain impact on the quality of the speech enhancement feature vector finally output in step S200. In this embodiment, the interval between any two frames is preferably 35ms.

[0070] Step S300: obtaining a noise-resistant feature vector based on the noisy feature vector and the speech enhancement feature vector.

[0071] In one embodiment, Figure 4 、 Figure 5 As shown, step S300 includes the following steps:

[0072] Step S3001: Fusing the noisy feature vector with the speech enhancement feature vector to obtain a fused feature vector.

[0073] In this embodiment, fusion refers to fusing the information carried by the noisy feature vector and the information carried by the speech enhancement feature vector to obtain a fused feature vector.

[0074] In one embodiment, the concat function is used for fusion and splicing. It is explained here that whether it is the noisy feature vector x or the speech enhancement feature All are in matrix form. Take the noisy feature vector x=(x1,x2……x T ) as an example, assuming that the audio to be recognized is evenly divided into 200 parts at 35ms intervals during framing, then T is 200. Furthermore, assuming that the embedding dimension is 200, a 200*200 dimensional matrix will eventually be formed. It is not difficult to understand that the number of columns in the matrix is ​​the number of frames, and each column represents the speech information in each frame. Speech Enhancement Features It is also a matrix corresponding to the noisy feature vector x, where the number of columns of the two is the same. When the concat function is used to combine the noisy feature vector x with the speech enhancement feature When splicing, the matrix is ​​operated. It should be noted that the concat function is only one of the splicing methods in this embodiment. Other functions such as add can also be used. Finally, the fused feature vector is obtained. The fused feature vector is processed by the noisy feature vector x and the speech enhancement feature vector x. The concatenation of the noise and speech features is combined.

[0075] Step S3002: Pass the fused feature vector through the trained attention mechanism model to obtain the first attention feature vector.

[0076] In this embodiment, the fused feature vector is processed using an attention model to generate a first attention feature vector. To facilitate understanding, the attention mechanism is explained below. The attention mechanism selects information that is more critical to the current task objective from the large amount of received information (in this embodiment, the fused feature vector). This improves the efficiency and accuracy of information processing by filtering out key information.

[0077] In one implementation, a softmax function is used for normalization, using the following formula:

[0078]

[0079] Where sp represents the speech enhancement feature vector, x represents the noisy feature vector, softmax is the activation function, d k is the scaling factor, x T is the transposed matrix of x. The scaling factor is set to avoid calculation overflow.

[0080] The sequence number of the speech enhancement feature vector sp is consistent with the noisy feature vector x. Assuming that the speech enhancement feature vector sp is an m*n matrix, the noisy feature vector x should also be an m*n matrix, so we know that x T is an n*m matrix, the speech enhancement feature vector sp and x T Multiply to get the similarity, then divide by After softmax normalization, a weight matrix is ​​obtained, in which each value is a weight coefficient greater than 0 and less than 1.

[0081] Step S3003: Multiply the first attention feature vector and the noisy feature vector to obtain a second attention feature vector.

[0082] In this embodiment, the second attention feature vector is calculated by multiplying and weighting the noisy feature vector x and the first attention feature vector.

[0083] Step S3004: Add the second attention feature vector to the speech enhancement feature vector to obtain an anti-noise feature vector.

[0084] In this embodiment, adding the second attention feature vector to the speech enhancement feature vector can avoid assigning too small a weight to individual values ​​in the ATT weighting. Because the speech enhancement feature vector itself carries relatively more effective speech information and less environmental noise, adding the speech enhancement feature vector to the second attention feature vector increases the weight of the effective speech information and the richness of the overall audio information.

[0085] Specifically, it is expressed as the following formula:

[0086] P=(ATT(concat(sp,x))*x)+sp)

[0087] Where P represents the noise-resistant feature vector, ATT represents the self-attention model, and concat represents concatenation.

[0088] Step S400: performing speech recognition on the noise-resistant feature vector to obtain a recognition result.

[0089] In this embodiment, speech recognition is performed by a trained speech recognition model. Specifically, the anti-noise feature vector is input into the trained speech recognition model to obtain a recognition result. Figure 6 Shown, including:

[0090] Step S4001: inputting the anti-noise feature vector into a trained speech recognition model to obtain the probabilities of multiple texts corresponding to the anti-noise feature vector;

[0091] In this embodiment, text recognition is performed on the noise-resistant feature vector corresponding to each frame. Then, by identifying the text probability corresponding to the signal of a single frame and then multiplying the probabilities of various paths, the probability of each path forming a text can be obtained.

[0092] In one embodiment, it is assumed that the noise-resistant feature vector is represented as p=(p1, p2...p T ), where T is the sequence length. This implementation uses the human-computer voice interaction scenario as an example. Assume that the system receives a 1-second voice command in a human-computer voice interaction, where the time width of a single frame after framing is 500ms. The corresponding voice command is divided into two equal parts, so T = 2.

[0093] After identifying the noise-resistant feature vector, the probability of p1 corresponding to each text is obtained (the probability in the table is arranged from high to low):

[0094] <![CDATA[p1]]> you neon inverse Ni Probability 90％ 70％ 30％ 10％

[0095] The probability of P2 corresponding to each text is (the probability in the table is arranged from high to low):

[0096] <![CDATA[P2]]> <![CDATA[Good( hǎo )]]> Hao <![CDATA[Good( hào )]]> Hao Probability 90％ 70％ 30％ 10％

[0097] It can be easily seen from the above table that a total of 16 different combinations can be generated from the corresponding texts recognized by p1 and p2 above. Each combination is a path, and each path corresponds to a probability. For example, for a path with the result of "Hello( hǎo )", the corresponding probability is the product of the probability of 90% that the recognized text corresponding to p1 is "you" and the probability of 90% that the recognized text corresponding to p2 is "good", which is finally 0.81. Similarly, the probabilities corresponding to other paths can also be obtained. For example, "Hello( hào )" is 30% * 90% = 0.27, and "Nihao( hǎo )" is 70% * 90% = 0.63. It can be easily seen from this embodiment that the higher the probability corresponding to the path, the greater the probability that it is the target speech. The calculation probability of the recognized text corresponding to a single path is as follows:

[0098]

[0099] where represents the posterior probability at time t in a path.

[0100]

[0101] p(Y丨p) represents the set of probability data for each path.

[0102] Step S4002: Select the text corresponding to the maximum probability among the probabilities as the recognition result.

[0103] In this embodiment, the text with the maximum probability is selected from the above probabilities p(Y丨p) as the target text. The formula is as follows:

[0104]

[0105] where argmax is used to return the index value of the maximum value in the array p(Y丨p). That is, the final output is the target text with the maximum probability.

[0106] It should be noted that the above embodiment simplifies the numerical conditions and possibilities for the convenience of understanding the technical solution. For example, the time width corresponding to a single frame is 500 ms. In actual applications, the time width of a single frame is usually less than 50 ms, such as setting it to 35 ms. When it is set to 35 ms, T = 100. In this case, the following situation may occur: the recognized text with the highest probability of p1 is "you", and the recognized text with the highest probability of p2 is also "you". When there are consecutive repeated "yous", corresponding de-duplication recognition will occur, and the corresponding output text will be "you", or other reduplicated words according to the corresponding scenario.

[0107] Based on the above steps S100 to S400, a solution or at least a partial solution can be provided to the problem in the prior art that the speech enhancement technology may lose some useful audio information, resulting in low accuracy of speech recognition.

[0108] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0109] Regarding the explanation, it can be understood by those skilled in the art that the present invention can implement all or part of the process of the method of the above embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal, and software distribution medium that can carry the computer program code. It should be noted that the content of the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0110] Furthermore, the present invention also provides an electronic device. In an electronic device embodiment according to the present invention, the electronic device includes a processor and a storage device. The storage device can be configured to store a program for executing the speech recognition method of the above-mentioned method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to a program for executing the speech recognition method of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The electronic device can be a control device formed by various electronic devices.

[0111] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the speech recognition method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned speech recognition method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0112] Furthermore, the present invention also provides a vehicle-mounted speech recognition system, comprising:

[0113] The electronic device mentioned above; the voice collection device is used to collect noisy audio data. For example, the voice collection device is a car microphone.

[0114] Furthermore, it should be understood that since the configuration of each module is merely for the purpose of illustrating the functional units of the apparatus of the present invention, the physical devices corresponding to these modules may be the processor itself, or a portion of the software in the processor, a portion of the hardware, or a combination of software and hardware. Therefore, the number of modules in the figure is merely illustrative.

[0115] Furthermore, the present invention also provides a car, comprising the above-mentioned vehicle-mounted speech recognition system.

[0116] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules does not cause the technical solution to deviate from the principles of the present invention. Therefore, the technical solutions after splitting or merging will fall within the scope of protection of the present invention.

[0117] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A speech recognition method, characterized in that: include: Extract features from noisy frequency data to obtain noisy feature vectors; Performing speech enhancement on the noisy feature vector to obtain a speech enhancement feature vector; Fusing the noisy feature vector with the speech enhancement feature vector to obtain a fused feature vector; Passing the fused feature vector through the trained attention mechanism model to obtain a first attention feature vector; Multiplying the first attention feature vector by the noisy feature vector to obtain a second attention feature vector; Adding the second attention feature vector to the speech enhancement feature vector to obtain an anti-noise feature vector; The anti-noise feature vector is used for speech recognition to obtain a recognition result.

2. The speech recognition method according to claim 1, wherein: The step of performing speech enhancement on the noisy feature vector to obtain a speech enhancement feature vector includes: The noisy feature vector is input into a trained speech enhancement model to obtain the speech enhancement feature vector, wherein the speech enhancement model includes an encoder and a decoder, the encoder is used to convert the noisy feature vector into a dense vector, and the decoder is used to convert the dense vector into the speech enhancement feature vector.

3. The speech recognition method according to claim 2, wherein: The encoder is composed of an M-layer bidirectional LSTM network; the decoder is composed of an N-layer bidirectional LSTM network, where M and N are both natural numbers.

4. The speech recognition method according to claim 1, wherein: The fusion is a splicing operation; The attention mechanism model is: Where ATT represents the attention model, x represents the noisy feature vector, sp represents the speech enhancement feature vector, softmax is the activation function, d k is the scaling factor.

5. The speech recognition method according to claim 1, wherein: The performing speech recognition on the anti-noise feature vector to obtain a recognition result includes: Inputting the noise-resistant feature vector into a trained speech recognition model to obtain probabilities of multiple texts corresponding to the noise-resistant feature vector; The text corresponding to the maximum probability among the probabilities is selected as the recognition result.

6. The speech recognition method according to claim 2, wherein: The method further includes the step of training the speech enhancement model, including: The loss function L(θ) is used for training, where: Where x represents the noisy feature vector, Represents the speech enhancement feature vector.

7. An electronic device comprising a processor and a storage device, wherein the storage device is adapted to store a plurality of program codes, wherein: The program code is suitable for being loaded and executed by the processor to perform the speech recognition method according to any one of claims 1 to 6.

8. A vehicle-mounted speech recognition system, characterized in that: include: The electronic device according to claim 7; The voice collection device is used to collect noisy audio data.

9. An automobile, characterized in that: The vehicle-mounted speech recognition system according to claim 8 is included.

Images