Speech recognition conversion method and system based on artificial intelligence

Abstract

The invention discloses a voice recognition conversion method and system based on artificial intelligence, and relates to the technical field of voice recognition. The method comprises the following steps that a platform obtains dialects of all regions and texts corresponding to the dialects, and an acoustic model is trained; the microphone collects user voice and preprocesses the collected voice data; extracting keywords from the preprocessed voice data, and converting the keywords into a feature vector sequence as the input of an acoustic model; and the acoustic model judges the conversion condition, processes the voice data according to a judgment result, and generates a recognition output result. The dialects of the regions and the texts corresponding to the dialects are obtained through the platform to train the acoustic model, the collected voice data are preprocessed and the keywords are extracted during voice recognition, the keywords are converted into the feature vector sequence to serve as the input of the acoustic model, the recognition output result is generated, and the voice recognition efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the field of speech recognition technology, and in particular relates to a speech recognition conversion method and system based on artificial intelligence. Background Art

[0002] The field of audio data processing technology involves the acquisition, analysis, processing, and recognition of speech signals, with the primary goal of enabling computers to understand and respond to human speech. Audio data processing technologies include, but are not limited to, signal digitization, noise reduction, echo cancellation, feature extraction, pattern recognition, and speech synthesis. It is widely used in various fields such as intelligent assistants, automatic voice response systems, speech translation, and assistive hearing devices. With the development of artificial intelligence technology, this field has made significant progress in natural language processing and machine learning, continuously improving the accuracy and efficiency of speech recognition.

[0003] Speech recognition conversion is a technology that enables computers to recognize and understand human speech and convert it into machine-readable formats, such as text or commands. The main purpose of this method is to improve the naturalness and efficiency of human-computer interaction, allowing computers to communicate with humans more intuitively and conveniently. For example, speech recognition conversion allows users to interact with devices via voice input instead of keyboard input, thereby improving accessibility and reducing the need for physical manipulation.

[0004] Traditional speech recognition methods primarily focus on processing single audio signals, which may lead to insufficient recognition of complex or ambiguous speech commands in practical applications. Due to the lack of video data, traditional methods suffer from low accuracy in environments with significant noise interference or when the speaker's expression is unclear. Furthermore, traditional methods typically rely on pre-set models and algorithms, lacking effective user feedback mechanisms to continuously optimize and adapt to new environments or scenarios, thus limiting their performance in dynamic or diverse application environments. For example, a fixed model may struggle to handle user dialects or unique pronunciation patterns, thereby impacting user experience and system usability. Summary of the Invention

[0005] The purpose of this invention is to provide a speech recognition and conversion method and system based on artificial intelligence. After processing the speech using regional dialects, the system inputs the processed speech into a trained acoustic model for speech recognition and conversion, thus solving the problems of low efficiency and low accuracy in existing speech recognition methods.

[0006] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] The present invention is a speech recognition conversion method based on artificial intelligence, comprising the following steps:

[0008] Step S1: The platform obtains regional dialects and corresponding texts and trains an acoustic model;

[0009] Step S2: The microphone collects the user's voice and pre-processes the collected voice data;

[0010] Step S3: extract keywords from the preprocessed speech data and convert the keywords into feature vector sequences as input to the acoustic model;

[0011] Step S4: The acoustic model judges the conversion condition, processes the speech data according to the judgment result, and generates a recognition output result.

[0012] As a preferred technical solution, in step S1, the specific process of training the acoustic model is as follows:

[0013] Step S11: Acquire the voiceprint image of each frame of speech;

[0014] Step S12: Each frame of voiceprint information samples is processed through Fourier transform, improving the quality of each frame of voiceprint signal to enhance recognition rate and robustness. A filter is then used to project it into the multi-dimensional MFCC space. Based on the frequency domain analysis of the Fourier transform, low-pass, high-pass, band-pass, or band-stop filters can be designed to suppress or extract signals within a specific frequency range. For example, in voice communication, a low-pass filter is typically designed to eliminate high-frequency noise while retaining the main voice frequency components.

[0015] The Fourier series is a method of decomposing a periodic signal into a set of sine and cosine functions. Its formula is:

[0016]

[0017] Where f(t) is a periodic signal, ω is the fundamental frequency, a0, a n and b n is the coefficient, which can be calculated by calculating the projection of the signal on the fundamental frequency. Fourier series is very useful in the periodic analysis of signals and can help understand the periodic characteristics and frequency components of the signal;

[0018] Then Fourier transform is a method of decomposing a non-periodic signal into a set of sine and cosine functions. Its formula is:

[0019]

[0020] Among them, F(ω) represents the complex function in the frequency domain, f(t) represents the function in the time domain, ω represents frequency, and e-jωt is a complex exponential function. Fourier transform is very useful in frequency domain analysis of signals and can help understand the frequency components and spectral structure of signals.

[0021] Step S13: using discrete cosine transform to remove redundant information, noise and irrelevant information in the sample, thereby improving the accuracy of model recognition by improving the reliability of features;

[0022] Step S14: using a linear function normalization method to eliminate the dimensionality effects between data features, ensuring that the scale of feature time is unified and improving the accuracy of the classifier;

[0023] Step S15: Calculate the difference, capture the dynamic information of the audio signal, and output the characteristic information of the sample; calculate the difference using the first and second derivatives, capture the dynamic information of the voiceprint signal, and output a multi-dimensional MFCC feature vector;

[0024] Step S16: Modeling the temporal relationship of acoustic features.

[0025] As a preferred technical solution, in step S15, the processed speech text is used as input, and a feature vector is obtained after feature extraction, which is recorded as o. The feature vector is input into the model for training, and the output vector of the last layer is recorded as z. It is normalized in the output layer, and the output probability of the i-th type is:

[0026]

[0027] In the formula, i represents the i-th data, and β is the number of set output categories;

[0028] After processing, the values ​​of P(i|o) are all between (0,1) and their sum is 1.

[0029] As a preferred technical solution, in step S2, the speech data preprocessing includes the following steps:

[0030] Step S21, voice segmentation: the voice data is uniformly segmented into audio signals with a sampling rate of 16 kHz. The sampling rate of 16 kHz ensures that all audio files have sufficient length for feature extraction, and the first 5 seconds of each audio file are retained.

[0031] Step S22, pre-emphasis: The audio signal is subjected to high-pass filtering. High-pass filtering is performed because when the vocal cords vibrate to generate a speech signal, the high-frequency portion of the sound is weakened after passing through the mouth. Furthermore, when the microphone collects speech information, the high-frequency portion is also weakened due to external environmental factors.

[0032] Step S23, windowing and framing: Use a Hamming window to cut each segment into 25ms as a frame, and retain 10ms overlap between adjacent frames for framing. This can effectively avoid the interference of sudden signal when extracting the time domain and frequency domain features of its phonemes, and increase the smoothness and continuity of the data.

[0033] As a preferred technical solution, in step S3, the feature vector sequence converted from the keyword and the corresponding text sequence are used as the input of the acoustic model, and the dialect keyword is built into a state sequence through the HMM model. Each state corresponds to a dialect text sequence, and there is a transition probability between each state. The above probabilities constitute a state transition matrix, that is, the conditional probability that the input frame belongs to the speech state (keyword).

[0034] The overall framework of the acoustic model consists of a three-layer neural network: input layer, intermediate layer, and output layer. The input layer is a 39-dimensional MFCC feature vector. The intermediate layer consists of two hidden layers consisting of 600 and 400 neurons respectively. The output layer contains 12 neurons in each round of training, corresponding to the recognition of 12 keywords.

[0035] The intermediate layers of the acoustic model above utilize a deep neural network structure constructed with fully connected layers, where each neuron contains a weight vector and a bias. During the modeling process, Singmiod is used as the activation function to implement nonlinear mapping. The output layer is a Softmax layer, which uses the number of states as the number of nodes and outputs the probability distribution of each state. The entire DNN sequence is aligned using an HMM constructed using a mixture of Gaussian models to form the final recognition model.

[0036] The present invention provides a speech recognition conversion system based on artificial intelligence, comprising a preprocessing module, a data processing module, a feature extraction module and an acoustic model. The preprocessing module comprises a pre-emphasis operation, a framing operation and a windowing operation; the pre-emphasis operation is used to perform high-pass filtering on an audio signal; the framing operation is used to process speech frames; the windowing operation is used to cut audio frames using a Hamming window; the data processing module comprises a Fourier transform process, a discrete cosine transform process and a normalization process; the feature extraction module is used to convert data output by the data processing module into a feature vector; and the acoustic model is used to train a speech recognition model based on the feature vector.

[0037] The present invention has the following beneficial effects:

[0038] The present invention obtains regional dialects and corresponding texts of dialects through the platform to train the acoustic model. When performing speech recognition, the collected speech data is preprocessed, keywords are extracted, and the keywords are converted into feature vector sequences as input to the acoustic model to generate recognition output results, thereby improving the efficiency and accuracy of speech recognition.

[0039] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a speech recognition and conversion method based on artificial intelligence according to the present invention;

[0042] Figure 2 This is a schematic diagram of the structure of an artificial intelligence-based speech recognition and conversion system according to the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-2 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0046] Example 1

[0047] See also Figure 1 As shown, this invention is a speech recognition and conversion method based on artificial intelligence, comprising the following steps:

[0048] Step S1: The platform obtains regional dialects and corresponding texts and trains an acoustic model;

[0049] Step S2: The microphone collects the user's voice and pre-processes the collected voice data;

[0050] Step S3: extract keywords from the preprocessed speech data and convert the keywords into feature vector sequences as input to the acoustic model;

[0051] Step S4: The acoustic model judges the conversion condition, processes the speech data according to the judgment result, and generates a recognition output result.

[0052] In step S1, the specific process of training the acoustic model is as follows:

[0053] Step S11: Acquire the voiceprint image of each frame of speech;

[0054] Step S12: Each frame of voiceprint information samples is processed through Fourier transform, improving the quality of each frame of voiceprint signal to enhance recognition rate and robustness. A filter is then used to project it into the multi-dimensional MFCC space. Based on the frequency domain analysis of the Fourier transform, low-pass, high-pass, band-pass, or band-stop filters can be designed to suppress or extract signals within a specific frequency range. For example, in voice communication, a low-pass filter is typically designed to eliminate high-frequency noise while retaining the main voice frequency components.

[0055] The Fourier series is a method of decomposing a periodic signal into a set of sine and cosine functions. Its formula is:

[0056]

[0057] Where f(t) is a periodic signal, ω is the fundamental frequency, a0, a n and b n is the coefficient, which can be calculated by calculating the projection of the signal on the fundamental frequency. Fourier series is very useful in the periodic analysis of signals and can help understand the periodic characteristics and frequency components of the signal;

[0058] Then Fourier transform is a method of decomposing a non-periodic signal into a set of sine and cosine functions. Its formula is:

[0059]

[0060] Among them, F(ω) represents the complex function in the frequency domain, f(t) represents the function in the time domain, ω represents frequency, and e-jωt is a complex exponential function. Fourier transform is very useful in frequency domain analysis of signals and can help understand the frequency components and spectral structure of signals.

[0061] Step S13: using discrete cosine transform to remove redundant information, noise and irrelevant information in the sample, thereby improving the accuracy of model recognition by improving the reliability of features;

[0062] Step S14: using a linear function normalization method to eliminate the dimensionality effects between data features, ensuring that the scale of feature time is unified and improving the accuracy of the classifier;

[0063] Step S15: Calculate the difference, capture the dynamic information of the audio signal, and output the characteristic information of the sample; calculate the difference using the first and second derivatives, capture the dynamic information of the voiceprint signal, and output a multi-dimensional MFCC feature vector;

[0064] Step S16: Modeling the temporal relationship of acoustic features.

[0065] In step S15, the processed speech text is used as input. After feature extraction, a feature vector is obtained and denoted as o. The feature vector is then input into the model for training. The output vector of the last layer is denoted as z. It is normalized in the output layer. Then, the output probability of the i-th type is:

[0066]

[0067] In the formula, i represents the i-th data, and β is the number of set output categories;

[0068] After processing, the values ​​of P(i|o) are all between (0,1) and their sum is 1.

[0069] In step S2, the speech data preprocessing includes the following steps:

[0070] Step S21, voice segmentation: the voice data is uniformly segmented into audio signals with a sampling rate of 16 kHz. The sampling rate of 16 kHz ensures that all audio files have sufficient length for feature extraction, and the first 5 seconds of each audio file are retained.

[0071] Step S22, pre-emphasis: The audio signal is subjected to high-pass filtering. High-pass filtering is performed because when the vocal cords vibrate to generate a speech signal, the high-frequency portion of the sound is weakened after passing through the mouth. Furthermore, when the microphone collects speech information, the high-frequency portion is also weakened due to external environmental factors.

[0072] Step S23, windowing and framing: Use a Hamming window to cut each segment into 25ms as a frame, and retain 10ms overlap between adjacent frames for framing. This can effectively avoid the interference of sudden signal when extracting the time domain and frequency domain features of its phonemes, and increase the smoothness and continuity of the data.

[0073] In step S3, the feature vector sequence converted from the keyword and the corresponding text sequence are used as the input of the acoustic model. The dialect keyword is built into a state sequence through the HMM model. Each state corresponds to a dialect text sequence, and there is a transition probability between each state. The above probabilities constitute the state transition matrix, that is, the conditional probability that the input frame belongs to the speech state (keyword).

[0074] The overall framework of the acoustic model consists of a three-layer neural network: input layer, intermediate layer, and output layer. The input layer is a 39-dimensional MFCC feature vector. The intermediate layer consists of two hidden layers consisting of 600 and 400 neurons respectively. The output layer contains 12 neurons in each round of training, corresponding to the recognition of 12 keywords.

[0075] The intermediate layers of the acoustic model above utilize a deep neural network structure constructed with fully connected layers, where each neuron contains a weight vector and a bias. During the modeling process, Singmiod is used as the activation function to implement nonlinear mapping. The output layer is a Softmax layer, which uses the number of states as the number of nodes and outputs the probability distribution of each state. The entire DNN sequence is aligned using an HMM constructed using a mixture of Gaussian models to form the final recognition model.

[0076] Example 2

[0077] See Figure 2 As shown, the present invention is a speech recognition conversion system based on artificial intelligence, which can be used to execute the method content of Example 1 of the present invention, including: a preprocessing module, a data processing module, a feature extraction module and an acoustic model; the preprocessing module includes a pre-emphasis operation, a framing operation and a windowing operation; the pre-emphasis operation is used to use high-pass filtering to process the audio signal; the framing operation is used to process the speech frame; the windowing operation is used to cut the audio frame using a Hamming window; the data processing module includes Fourier transform processing, discrete cosine transform processing and normalization processing; the feature extraction module is used to convert the data output by the data processing module into a feature vector; the acoustic model is used to train a model for speech recognition based on the feature vector.

[0078] It is worth noting that the various units included in the above system embodiments are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0079] Furthermore, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the corresponding program can be stored in a computer-readable storage medium.

[0080] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A speech recognition conversion method based on artificial intelligence, characterized in that: The steps include: Step S1: The platform obtains the dialects of each region and the texts corresponding to the dialects, and trains the acoustic model; Step S2: The microphone collects the user's voice and pre-processes the collected voice data; Step S3: extract keywords from the preprocessed speech data and convert the keywords into feature vector sequences as input of the acoustic model; Step S4: The acoustic model determines the conversion condition, processes the speech data according to the determination result, and generates a recognition output result.

2. The method for speech recognition conversion based on artificial intelligence according to claim 1, characterized in that: In step S1, the specific process of training the acoustic model is as follows: Step S11: Acquire the voiceprint image of each frame of speech; Step S12: Processing each frame of voiceprint information samples by Fourier transform; Step S13: using discrete cosine transform to remove redundant information, noise and irrelevant information in the sample; Step S14: using a linear function normalization method to eliminate the dimensional influence between data features; Step S15: Calculate the difference, capture the dynamic information of the audio signal, and output the characteristic information of the sample; Step S16: Modeling the temporal relationship of acoustic features.

3. The method for speech recognition conversion based on artificial intelligence according to claim 2, characterized in that: In step S12, the Fourier series decomposes the periodic signal into a set of sine and cosine functions, and the specific expression formula is: Where f(t) is a periodic signal, ω is the fundamental frequency, a0, a n and b n are the coefficients, calculated by taking the projection of the signal onto the fundamental frequency The Fourier transform decomposes the non-periodic signal into a set of sine and cosine functions. The specific formula is: Among them, F(ω) represents the complex function in the frequency domain, f(t) represents the function in the time domain, ω represents frequency, and e-jωt represents a complex exponential function.

4. The method for speech recognition conversion based on artificial intelligence according to claim 2, characterized in that: In step S13, the specific formula of discrete cosine transform is as follows: Where f(x) is the original signal, F(u) is the coefficient of discrete cosine transform, N is the number of points of the original data, and a(u) is the compensation coefficient.

5. The method for speech recognition conversion based on artificial intelligence according to claim 2, characterized in that: In step S15, the processed speech text is used as input, and a feature vector is obtained after feature extraction, which is recorded as o. The feature vector is input into the model for training, and the output vector of the last layer is recorded as z. It is normalized in the output layer, and the output probability of the i-th type is: In the formula, i represents the i-th data, and β is the number of output categories set; After processing, the values ​​of P(i|o) are all between (0,1) and their sum is 1.

6. The method for speech recognition conversion based on artificial intelligence according to claim 1, characterized in that: In step S16, the acoustic model uses HMM to model the temporal relationship of acoustic features. The input data of the model are audio files and corresponding texts. The specific training process is as follows: an initial HMM model is established, the number of hidden states, transition probabilities and GMM parameters of the model are initialized, and the forward-backward algorithm is used to modify the HMM model state sequence and optimize the acoustic model parameters; in the process of acoustic modeling, the Viterbi algorithm is used to align the speech and state sequences, and the label sequence is obtained, so as to obtain the optimal path and make the speech and state sequences correspond one to one.

7. The method for speech recognition conversion based on artificial intelligence according to claim 1, characterized in that: In step S2, the speech data preprocessing includes the following steps: Step S21, voice cutting: cutting the voice data into 16kHz sampling rate audio signals; Step S22, pre-emphasis: high-pass filtering is performed on the audio signal; Step S23, windowing and framing: using a Hamming window to cut the video into 25 ms segments as one frame, and retaining an overlap of 10 ms between two adjacent frames for framing.

8. The method for speech recognition conversion based on artificial intelligence according to claim 1, characterized in that: In step S3, the feature vector sequence converted from the keywords and the corresponding text sequence are used as the input of the acoustic model, and the dialect keywords are built into a state sequence through the HMM model. Each state corresponds to a dialect text sequence, and there is a transition probability between each state. The above probabilities constitute a state transition matrix.

9. A speech recognition conversion system based on artificial intelligence, comprising a preprocessing module, a data processing module, a feature extraction module and an acoustic model, characterized in that: The pre-processing module includes a pre-emphasis operation, a framing operation and a windowing operation; the pre-emphasis operation is used to perform high-pass filtering on the audio signal; the framing operation is used to process the speech frame; the windowing operation is used to cut the audio frame using a Hamming window; The data processing module includes Fourier transform processing, discrete cosine transform processing and normalization processing; The feature extraction module is used to convert the data output by the data processing module into a feature vector; The acoustic model is used to train a model for speech recognition based on the feature vector.

Images