Interactive user voice recognition method for AI intelligent agent

Abstract

The invention relates to the technical field of voice recognition, and discloses an interactive user voice recognition method for an AI agent. The interactive user speech recognition method for the AI agent comprises the following steps: establishing a deep learning system; establishing a cross-language knowledge migration system; establishing a noise suppression system; establishing a voiceprint recognition system; establishing a cloud computing system; according to the invention, by adopting the deep learning system, the voice recognition system can better capture the time sequence characteristics of the voice signals, increase the learning content of semantic information and simplify the processing flow of mapping acoustic characteristics to text output, so that the efficiency and accuracy of voice recognition can be effectively improved; by establishing a cross-language knowledge migration system, the performance and generalization ability of a speech recognition system can be effectively improved by using feature conversion, migration learning and data enhancement methods; by learning the relationship between the voice signal and the noise signal, the voice signal is enhanced by using the acoustic characteristics.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech recognition, and specifically relates to an interactive user speech recognition method for an AI agent. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, speech recognition technology has been widely used in the field of AI agents. As an intelligent device that can be operated through human-computer voice interaction, AI agents have gradually become an indispensable part of people's lives. As the core technology of smart speakers, speech recognition technology undoubtedly plays a leading role in this field. First of all, speech recognition technology provides a convenient user interaction method for AI agents, greatly improving the user experience and operation convenience, and making AI agents more widely used.

[0003] However, in the field of AI agents, speech recognition technology still faces some challenges. First, the accuracy of speech recognition technology is an important issue. At present, although speech recognition technology has made significant progress, there are still some cases of misrecognition. This is mainly due to factors such as environmental noise, accent, and speaking speed, which will affect the quality of voice guidance and thus the accuracy of speech recognition technology. Secondly, the real-time performance of speech recognition technology is also a challenge. AI agents require speech recognition technology to complete the recognition and processing of voice commands in a very short time to achieve instant interaction between users and devices. However, since speech recognition technology requires a lot of calculations and processing, real-time performance has become a challenging issue. How to improve the processing speed of speech recognition technology and ensure the real-time performance of the system; therefore, in view of the current situation, it needs to be improved. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an interactive user speech recognition method for an AI agent, which effectively solves the problems of misrecognition and low real-time performance of speech recognition technology in the field of AI agents.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for interactive user speech recognition for an AI agent, characterized in that it comprises the following steps:

[0006] Step 1: Establish a deep learning system to provide audio enhancement and noise suppression training for the system based on TIMIT, LibriSpeech and Switchboard speech datasets; improve the system's performance in capturing the temporal features of speech signals based on convolutional neural networks, recurrent neural networks and long short-term memory networks; train the system to learn richer semantic information based on recurrent neural network language models and Transformer models; establish an end-to-end model to map acoustic features to text output, thereby improving the efficiency and accuracy of speech recognition;

[0007] Step 2: Establish a cross-language language knowledge transfer system, use feature conversion method to convert the speech features of the source language into feature representations of the target language, reduce language differences in speech recognition, use transfer learning method to transfer model parameters and feature knowledge learned from the original language to the target language, make fine adjustments on the target language, and cooperate with data enhancement technology. In cross-language knowledge transfer, by transforming and expanding the data of the source language, generate training data of the target language, thereby improving the speech recognition performance on the target language and increasing the diversity and coverage of the data;

[0008] Step 3: Establish a noise suppression system, use the inverse spectrum and domain adaptive MFCC algorithm to extract the characteristics of speech signals in a noisy environment, establish a noise model through recurrent neural network and convolutional neural network, remove noise through the noise model, establish an acoustic model and a statistical model to learn the relationship between speech signals and noise signals, use acoustic features to enhance speech signals, and expand the training set by artificially synthesizing data of different noise intensities and types to improve the robustness of the model;

[0009] Step 4: Establish a voiceprint recognition system, collect individual voice samples, convert voice signals into voiceprint feature vectors through voiceprint feature extraction technology, train the collected voiceprint feature data, and establish a voiceprint model through machine learning algorithms and model optimization technology. By comparing the test voice with the established voiceprint model, use the voiceprint recognition algorithm to identify and verify the input voice;

[0010] Step 5: Establish a cloud computing system, deploy the voice input module, voice recognition module and text output module in the interactive user voice recognition system of the AI ​​agent to different cloud servers, and use the API of the cloud service provider for communication and data transmission between modules. Test the recognition accuracy and response speed of the system through simulated and actual voice input, and tune and optimize the system parameters according to the test results. Set up identity authentication and authorization mechanisms to restrict the access of illegal users, use AES and RSA encryption algorithms to encrypt and store data, use data desensitization and de-identification technology to protect user privacy information, monitor the operation records and network traffic information of the voice recognition system in real time, and use intrusion detection system and intrusion prevention system to discover and handle security vulnerabilities and abnormal behaviors.

[0011] Preferably, the end-to-end model proposed in step 1 uses Mel-spectrogram coefficients as acoustic features to describe the spectral distribution of speech signals, adopts a hidden Markov model to process the temporal relationship of modeled speech signals, and adopts a deep neural network to process the nonlinear relationship between features.

[0012] Preferably, the convolutional neural network proposed in step 1 is a type of feedforward neural network that includes convolution calculations and has a deep structure, and is a representative algorithm for deep learning. Convolutional neural networks have representational learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure; the recurrent neural network is a type of recurrent neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain, the recurrent neural network has memory, parameter sharing, and Turing completeness, and the recurrent neural network constructed by introducing the recurrent neural network into the convolutional neural network can handle AI agent vision problems containing sequence input; the long short-term memory network is a time recurrent neural network.

[0013] Preferably, the inverse spectrum algorithm proposed in step three is a secondary analysis technique. The result of inverse Fourier transform of the logarithm of the power spectrum can simplify the sideband spectral lines in the original spectrum diagram into a single spectral line, so as to extract and analyze the periodic signals that are difficult to identify with the naked eye on the original spectrum diagram. After taking the logarithm of the power spectrum with multiple averages, the noise and other signals in the power spectrum that are not related to the modulation sideband are also amplified by larger weight coefficients, thereby reducing the signal-to-noise ratio.

[0014] Preferably, the domain adaptive MFCC algorithm proposed in step three uses FFT to convert the time domain signal into the frequency domain, then convolves its logarithmic energy spectrum with a triangular filter group distributed according to the Mel scale, and finally performs discrete cosine transform DCT on the vector composed of the output of each filter, and takes the first N coefficients.

[0015] Preferably, the API proposed in step 5 allows different applications, services and systems to share information and functions, achieve interconnection and interoperability through agreed API interfaces, and reduce system coupling.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The present invention adopts a deep learning system to enable the speech recognition system to better capture the temporal characteristics of the speech signal, increase the learning content of the semantic information, and simplify the acoustic feature mapping to the text output processing flow, thereby effectively improving the efficiency and accuracy of speech recognition; by establishing a cross-language language knowledge transfer system, the performance and generalization ability of the speech recognition system can be effectively improved by utilizing feature conversion, transfer learning and data enhancement methods; a noise suppression system is established, and by utilizing the inverse spectrum and domain adaptive MFCC algorithm, it can better adapt to the speech signal characteristics in a noisy environment, and by combining the noise model, the acoustic model and the statistical model, the relationship between the speech signal and the noise signal is learned, and the acoustic features are utilized to enhance the speech signal.

[0017] 2. The present invention establishes a voiceprint recognition system, and through voiceprint feature extraction, acoustic model training and recognition steps, the system can achieve accurate recognition and verification of speech. When establishing and improving the system, the key technical points of voiceprint feature selection, data quality and quantity, and model training algorithm should be comprehensively considered, and optimized in combination with the system application scenario to improve the performance and practicality of the system.

[0018] 3. In the present invention, the deployment and security strategy of the cloud computing-based speech recognition system is the key to ensuring system performance and user privacy. The speech recognition system can be effectively deployed by selecting a suitable cloud computing platform, planning and configuring resource deployment and integration systems, and testing and tuning system steps. At the same time, the security and stability of the system can be improved through security strategies such as access control, data encryption and privacy protection, security monitoring and vulnerability repair, and threat detection and prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0020] In the attached picture:

[0021] Figure 1 A block diagram of the interactive user speech recognition method for an AI agent of the present invention;

[0022] Figure 2 A block diagram of step 1 in the interactive user speech recognition method for an AI agent of the present invention;

[0023] Figure 3A block diagram of step 3 of the interactive user speech recognition method for an AI agent of the present invention; DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] See also Figure 1-Figure 3 The present invention provides an interactive user speech recognition method for an AI agent, comprising the following steps:

[0026] Step 1: Establish a deep learning system to provide audio enhancement and noise suppression training for the system based on TIMIT, LibriSpeech and Switchboard speech datasets; improve the system's performance in capturing the temporal features of speech signals based on convolutional neural networks, recurrent neural networks and long short-term memory networks; train the system to learn richer semantic information based on recurrent neural network language models and Transformer models; establish an end-to-end model to map acoustic features to text output, thereby improving the efficiency and accuracy of speech recognition;

[0027] Step 2: Establish a cross-language language knowledge transfer system, use feature conversion method to convert the speech features of the source language into feature representations of the target language, reduce language differences in speech recognition, use transfer learning method to transfer model parameters and feature knowledge learned from the original language to the target language, make fine adjustments on the target language, and cooperate with data enhancement technology. In cross-language knowledge transfer, by transforming and expanding the data of the source language, generate training data of the target language, thereby improving the speech recognition performance on the target language and increasing the diversity and coverage of the data;

[0028] Step 3: Establish a noise suppression system, use the inverse spectrum and domain adaptive MFCC algorithm to extract the characteristics of speech signals in a noisy environment, establish a noise model through recurrent neural network and convolutional neural network, remove noise through the noise model, establish an acoustic model and a statistical model to learn the relationship between speech signals and noise signals, use acoustic features to enhance speech signals, and expand the training set by artificially synthesizing data of different noise intensities and types to improve the robustness of the model;

[0029] Step 4: Establish a voiceprint recognition system, collect individual voice samples, convert voice signals into voiceprint feature vectors through voiceprint feature extraction technology, train the collected voiceprint feature data, and establish a voiceprint model through machine learning algorithms and model optimization technology. By comparing the test voice with the established voiceprint model, use the voiceprint recognition algorithm to identify and verify the input voice;

[0030] Step 5: Establish a cloud computing system, deploy the voice input module, voice recognition module and text output module in the interactive user voice recognition system of the AI ​​agent to different cloud servers, and use the API of the cloud service provider for communication and data transmission between modules. Test the recognition accuracy and response speed of the system through simulated and actual voice input, and tune and optimize the system parameters according to the test results. Set up identity authentication and authorization mechanisms to restrict the access of illegal users, use AES and RSA encryption algorithms to encrypt and store data, use data desensitization and de-identification technology to protect user privacy information, monitor the operation records and network traffic information of the voice recognition system in real time, and use intrusion detection system and intrusion prevention system to discover and handle security vulnerabilities and abnormal behaviors.

[0031] Among them, the end-to-end model proposed in step 1 uses Mel spectrum coefficients as acoustic features to describe the spectral distribution of speech signals, adopts hidden Markov model to process the temporal relationship of modeled speech signals, and adopts deep neural network to process the nonlinear relationship between features. The convolutional neural network proposed in step 1 is a type of feedforward neural network with deep structure and convolution calculation, which is a representative algorithm of deep learning. Convolutional neural network has the ability of representation learning, and can classify input information in a translation-invariant manner according to its hierarchical structure; the recurrent neural network is a type of recurrent neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. The recurrent neural network has memory, parameter sharing and Turing completeness. The recurrent neural network constructed by introducing recurrent neural network into convolutional neural network can handle AI agent vision problems containing sequence input; the long short-term memory network is a time recurrent neural network. The inverse spectrum algorithm proposed in step 3 is a secondary analysis technology. The result of Fourier inverse transform of the logarithm of the power spectrum can simplify the sideband spectrum lines of the original spectrum graph into a single The root spectrum lines are used to extract and analyze the periodic signals that are difficult to identify with the naked eye on the original spectrum diagram. After taking the logarithm of the power spectrum with multiple averages, the noise and other signals in the power spectrum that are not related to the modulation sidebands are also amplified by larger weight coefficients, which reduces the signal-to-noise ratio. The domain adaptive MFCC algorithm proposed in step three uses FFT to convert the time domain signal into the frequency domain, and then convolves its logarithmic energy spectrum with a triangular filter group distributed according to the Mel scale. Finally, the vector composed of the output of each filter is discrete cosine transformed DCT, and the first N coefficients are taken. The API proposed in step five allows different applications, services, and systems to share information and functions, and achieve interconnection and interoperability with agreed API interfaces to reduce system coupling.

[0032] By using the above method, the deep learning system can enable the speech recognition system to better capture the temporal characteristics of the speech signal, increase the learning content of semantic information, and simplify the acoustic feature mapping to text output processing flow, thereby effectively improving the efficiency and accuracy of speech recognition; by establishing a cross-language language knowledge transfer system, the performance and generalization ability of the speech recognition system can be effectively improved by using feature conversion, transfer learning and data enhancement methods; by establishing a noise suppression system, by using the inverse spectrum and domain adaptive MFCC algorithm, it can better adapt to the characteristics of speech signals in a noisy environment, and by combining the noise model, acoustic model and statistical model, the relationship between the speech signal and the noise signal can be learned, and the acoustic features can be used to enhance the speech signal; by establishing a voiceprint recognition system, through voiceprint feature extraction, acoustic model training and recognition steps, the system can achieve accurate recognition and verification of speech. When establishing and improving the system, the selection of voiceprint features, data quality and quantity, and key technical points of the model training algorithm should be comprehensively considered, and the optimization should be combined with the system application scenario to improve the performance and practicality of the system; the deployment and security strategy of the cloud computing-based speech recognition system is the key to ensuring system performance and user privacy. By selecting the appropriate cloud computing platform, planning and configuring resource deployment and integration systems, testing and tuning system steps, the speech recognition system can be effectively deployed. At the same time, security strategies such as access control, data encryption and privacy protection, security monitoring and vulnerability repair, and threat detection and prevention can improve the security and stability of the system.

[0033] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0034] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An interactive user speech recognition method for an AI agent, characterized in that: The following steps are involved: Step 1: Establish a deep learning system to provide audio enhancement and noise suppression training for the system based on TIMIT, LibriSpeech, and Switchboard speech datasets; improve the system's performance in capturing the temporal features of speech signals based on convolutional neural networks, recurrent neural networks, and long short-term memory networks; Based on the recurrent neural network language model and Transformer model training system, it learns richer semantic information; an end-to-end model is established to map acoustic features to text output, which improves the efficiency and accuracy of speech recognition; Step 2: Establish a cross-language language knowledge transfer system, use feature conversion method to convert the speech features of the source language into feature representations of the target language, reduce language differences in speech recognition, use transfer learning method to transfer model parameters and feature knowledge learned from the original language to the target language, make fine adjustments on the target language, and cooperate with data enhancement technology. In cross-language knowledge transfer, by transforming and expanding the data of the source language, generate training data of the target language, thereby improving the speech recognition performance on the target language and increasing the diversity and coverage of the data; Step 3: Establish a noise suppression system, use the inverse spectrum and domain adaptive MFCC algorithm to extract the characteristics of speech signals in a noisy environment, establish a noise model through recurrent neural network and convolutional neural network, remove noise through the noise model, establish an acoustic model and a statistical model to learn the relationship between speech signals and noise signals, use acoustic features to enhance speech signals, and expand the training set by artificially synthesizing data of different noise intensities and types to improve the robustness of the model; Step 4: Establish a voiceprint recognition system, collect individual voice samples, convert voice signals into voiceprint feature vectors through voiceprint feature extraction technology, train the collected voiceprint feature data, and establish a voiceprint model through machine learning algorithms and model optimization technology. By comparing the test voice with the established voiceprint model, use the voiceprint recognition algorithm to identify and verify the input voice; Step 5: Establish a cloud computing system, deploy the voice input module, voice recognition module and text output module in the interactive user voice recognition system of the AI ​​agent to different cloud servers, and use the API of the cloud service provider for communication and data transmission between modules. Test the recognition accuracy and response speed of the system through simulated and actual voice input, and tune and optimize the system parameters according to the test results. Set up identity authentication and authorization mechanisms to restrict the access of illegal users, use AES and RSA encryption algorithms to encrypt and store data, use data desensitization and de-identification technology to protect user privacy information, monitor the operation records and network traffic information of the voice recognition system in real time, and use intrusion detection system and intrusion prevention system to discover and handle security vulnerabilities and abnormal behaviors.

2. The interactive user speech recognition method for an AI agent according to claim 1, characterized in that: The end-to-end model proposed in step 1 uses Mel-spectrogram coefficients as acoustic features to describe the spectral distribution of speech signals, adopts a hidden Markov model to process the temporal relationship of modeled speech signals, and adopts a deep neural network to process the nonlinear relationship between features.

3. The interactive user speech recognition method for an AI agent according to claim 2, characterized in that: The convolutional neural network proposed in step 1 is a type of feedforward neural network that includes convolution calculations and has a deep structure, and is a representative algorithm for deep learning. The convolutional neural network has representation learning capabilities and can perform translation-invariant classification of input information according to its hierarchical structure; The recurrent neural network is a type of recurrent neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. The recurrent neural network has memory, parameter sharing, and is Turing complete. The recurrent neural network constructed by introducing the recurrent neural network into the convolutional neural network can handle AI agent vision problems involving sequence input; the long short-term memory network is a temporal recurrent neural network.

4. The interactive user speech recognition method for an AI agent according to claim 1, characterized in that: The inverse spectrum algorithm proposed in step three is a secondary analysis technique. The result of inverse Fourier transform on the logarithm of the power spectrum can simplify the clustered sideband spectral lines on the original spectrum graph into a single spectral line, so as to extract and analyze the periodic signals that are difficult to identify with the naked eye on the original spectrum graph. After taking the logarithm of the power spectrum with multiple averages, the noise and other signals in the power spectrum that are not related to the modulation sidebands are also amplified by larger weight coefficients, thereby reducing the signal-to-noise ratio.

5. The interactive user speech recognition method for an AI agent according to claim 1, characterized in that: The domain adaptive MFCC algorithm proposed in step three uses FFT to convert the time domain signal into the frequency domain, then convolves its logarithmic energy spectrum with a triangular filter group distributed according to the Mel scale, and finally performs discrete cosine transform DCT on the vector composed of the output of each filter to take the first N coefficients.

6. The interactive user speech recognition method for an AI agent according to claim 1, characterized in that: The API proposed in step 5 allows different applications, services and systems to share information and functions, achieve interconnection and interoperability through agreed API interfaces, and reduce system coupling.

Images