Self-learning method and device based on offline speech recognition

Abstract

The embodiment of the invention provides a self-learning method and device based on offline voice recognition, and the method comprises the following steps: responding to a preset voice instruction, and entering a command word self-learning mode; acquiring and storing a first voice command; obtaining a second voice command, comparing the second voice command with the first voice command, and entering a current recognition model optimization mode when a comparison result meets a preset requirement; and fusing the first voice command and the second voice command, and optimizing a current recognition model through a preset algorithm to complete command word learning. In combination with a voiceprint recognition processing mode, the current recognition model is optimized through simple user voice, and the effects of replacing existing command word recognition, optimizing the recognition effect for existing users or customizing command words are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech recognition, and in particular relates to a self-learning method, device, equipment and storage medium based on offline speech recognition. Background Art

[0002] With the technological advancement of smart speakers, they are increasingly being used in different products to achieve convenient interaction. Currently, voice recognition technology mainly uses voiceprint recognition and command word recognition.

[0003] Voiceprint recognition is divided into two main processes: voiceprint registration and voiceprint verification. Voiceprint registration requires obtaining the user's registered audio, performing noise reduction and enhancement on the voiceprint audio, performing quality inspection, extracting valid voice, extracting voiceprint features and saving them in the voiceprint database. Voiceprint verification is to perform noise reduction and enhancement on the user's voice, extracting valid voice, comparing the voiceprint features with the voiceprint database, and displaying the recognition results after the comparison is passed. Voiceprint recognition requires a built-in voiceprint model and a local voiceprint library for comparison, which requires a lot of computing power for the local device. After voiceprint registration, only the recognition results for the registrant are improved, and the results for others are not optimized.

[0004] Command word recognition includes training models and using models. The training model needs to collect voice data for training, perform preprocessing (after noise reduction, framing, etc.) to extract voice signal features, and save the extracted features in a recognition model. When using the model, the user's audio is noise-reduced and enhanced, and then compared with the deployed model to identify the corresponding command words. The command word is fixed and the user cannot change the command word instruction. The recognition result depends on the basic model effect and cannot be further optimized for actual user use. Summary of the invention

[0005] To solve the above problems, the purpose of the present invention is to provide a self-learning method, device, equipment and storage medium based on offline speech recognition, combined with the voiceprint recognition processing mode, through simple user voice, to optimize the current recognition model, to achieve replacement of existing command word recognition, to optimize the recognition effect for existing users or to customize the effect of command words.

[0006] The technical solution provided by the present invention is: a self-learning method based on offline speech recognition, comprising the following steps:

[0007] In response to a preset voice command, enter a command word self-learning mode;

[0008] Obtaining and storing a first voice command;

[0009] Obtain a second voice command, compare it with the first voice command, and when the comparison result meets the preset requirements, enter the current recognition model optimization mode;

[0010] The first voice command and the second voice command are integrated, and the current recognition model is optimized through a preset algorithm to complete command word learning.

[0011] Preferably, fusing the first voice command and the second voice command further comprises:

[0012] Preprocessing and extracting features from the first voice command and the second voice command;

[0013] The feature fusion algorithm based on deep learning uses frequency domain mixing or time-frequency mixing to fuse the extracted feature vectors of the first voice command and the second voice command.

[0014] Preferably, optimizing the current recognition model by a preset algorithm to complete command word learning further includes:

[0015] Inputting the feature vector of the fusion of the first voice command and the second voice command into the current recognition model, retraining the current recognition model to recognize these new command words, wherein in order to save storage space, a composite state node group is used to reuse state nodes;

[0016] The current recognition model is retrained regularly by recording new speech data for continuous optimization.

[0017] Preferably, the method further comprises:

[0018] The second voice command is compared with the first voice command. When the comparison result does not meet the preset requirement, the second voice command is waited for again or the current process is terminated.

[0019] Preferably, obtaining the second voice command and comparing it with the first voice command further comprises:

[0020] Preprocessing and extracting features of the first voice command and the second voice command respectively;

[0021] Calculate the distance between two time series based on the feature vectors extracted from the first voice command and the second voice command using a dynamic time warping algorithm;

[0022] When the calculated distance is less than the preset threshold, the current recognition model optimization mode is entered.

[0023] Preferably, the method further comprises:

[0024] The first voice command is obtained and compared with the corresponding instruction in the current recognition model. When the comparison result meets the preset requirements, a second voice command is prepared to be obtained.

[0025] Preferably, the method further comprises: after acquiring the voice instruction or voice command, performing preprocessing including noise removal, echo cancellation, and noise reduction.

[0026] Based on the same concept, the present invention also provides a self-learning device based on offline speech recognition, comprising:

[0027] A response module, used for responding to a preset voice command and entering a command word self-learning mode;

[0028] A voice acquisition module, used to acquire and store a first voice command; and acquire a second voice command, compare it with the first voice command, and enter a current recognition model optimization mode when the comparison result meets a preset requirement;

[0029] The optimization module is used to fuse the first voice command and the second voice command, and optimize the current recognition model through a preset algorithm to complete command word learning.

[0030] Based on the same concept, the present invention also provides an electronic device, comprising:

[0031] A memory, the memory being used to store a processing program;

[0032] A processor, wherein when executing the processing program, the processor implements any one of the self-learning methods based on offline speech recognition described above.

[0033] Based on the same concept, the present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, any one of the above-mentioned self-learning methods based on offline speech recognition is implemented.

[0034] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art:

[0035] 1. The present invention combines the voiceprint recognition processing mode, optimizes the current recognition model through simple user voice, realizes the replacement of existing command word recognition, optimizes the recognition effect for existing users, or customizes the effect of command words. Under the actual usage scenario, after the system initializes the relevant parameters and settings of command word self-learning, the user first reads out a specific command word, and enters the command word self-learning mode through the specific command word. The user reads the customized command word once according to the voice prompt to complete the learning. After the learning is completed, the successful learning is announced. For the second learning, the user reads the customized command word again according to the prompt. It will be compared with the first self-learning. If the feature comparison passes, the learning is successful, otherwise the learning fails and re-learning is required. After successful learning, the learning success is announced. This process can be used for wake-up words as well as command words.

[0036] 2. The present invention inputs the fused feature vector into the current recognition model and retrains the model so that the model can learn and adapt to new voice command words. This self-learning mechanism enables the model to continuously update its knowledge base over time, thereby better adapting to changes in the user's voice commands. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings, wherein:

[0038] Figure 1 This is a flow chart of the self-learning method based on offline speech recognition of the present invention. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0040] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0041] First embodiment

[0042] like Figure 1 As shown, this embodiment provides a self-learning method based on offline speech recognition, comprising the following steps:

[0043] In response to a preset voice command, enter a command word self-learning mode;

[0044] Obtaining and storing a first voice command;

[0045] Obtain a second voice command, compare it with the first voice command, and when the comparison result meets the preset requirements, enter the current recognition model optimization mode;

[0046] The first voice command and the second voice command are integrated, and the current recognition model is optimized through a preset algorithm to complete command word learning.

[0047] In the command word recognition scenario, users often change command words, or the recognition effect of some users is not good. However, the traditional way of retraining the model is too long and the cost is too high, so users will not accept it. This embodiment combines the voiceprint recognition processing mode, optimizes the current recognition model through simple user voice, realizes the replacement of existing command word recognition, optimizes the recognition effect for existing users or customizes the effect of command words. In the actual usage scenario, after the system initializes the relevant parameters and settings of command word self-learning, the user first reads out a specific command word, and enters the command word self-learning mode through the specific command word. The user reads the customized command word once according to the voice prompt to complete the learning. After the learning is completed, the learning is successful. For the second learning, the user reads the customized command word again according to the prompt. It will be compared with the first self-learning. If the feature comparison passes, the learning is successful, otherwise the learning fails and re-learning is performed. After the learning is successful, the learning is successful. This process can be used for wake-up words and command words. In addition, since all processing is completed on the local device, no network connection is required, and the system can respond to the user's voice input instantly to achieve fast command word learning and recognition. Users can quickly change command words through self-learning mode without waiting for the model to be retrained, reducing the time and economic costs of retraining the model. Since all voice processing and recognition are completed locally, there is no need to upload data to the cloud, thus reducing the risk of data leakage and protecting user privacy.

[0048] Preferably, fusing the first voice command and the second voice command further comprises:

[0049] Preprocessing and extracting features from the first voice command and the second voice command;

[0050] The feature fusion algorithm based on deep learning uses frequency domain mixing or time-frequency mixing to fuse the extracted feature vectors of the first voice command and the second voice command.

[0051] The system provides instant feedback after the user completes learning of custom command words, and optimizes the recognition model by integrating the user's features, allowing the user to feel the intelligence and responsiveness of the system, thereby enhancing the user experience.

[0052] Preferably, optimizing the current recognition model by a preset algorithm to complete command word learning further includes:

[0053] Inputting the feature vector of the fusion of the first voice command and the second voice command into the current recognition model, retraining the current recognition model to recognize these new command words, wherein in order to save storage space, a composite state node group is used to reuse state nodes;

[0054] The current recognition model is retrained regularly by recording new speech data for continuous optimization.

[0055] By inputting the fused feature vector into the current recognition model and retraining it, the model is able to learn and adapt to new voice command words. This self-learning mechanism enables the model to continuously update its knowledge base over time, thereby better adapting to changes in user voice commands. Using composite state node groups to reuse state nodes is an efficient storage management strategy. This approach can reduce redundant storage of model parameters, thereby saving a lot of storage space without sacrificing performance. This is particularly important for resource-constrained devices because it allows more model data to be saved in limited storage capacity. Regularly retraining the model by entering new voice data ensures that the model is always up to date and can adapt to changes in user voice and new command words. This continuous learning process helps to stabilize and improve model performance in the long term. With the improvement of model performance and rapid adaptation to new command words, users will experience higher recognition accuracy and faster response speed when using offline speech recognition systems. This will directly improve user satisfaction and frequency of use.

[0056] Preferably, the method further comprises:

[0057] The second voice command is compared with the first voice command. When the comparison result does not meet the preset requirement, the second voice command is waited for again or the current process is terminated.

[0058] Preferably, obtaining the second voice command and comparing it with the first voice command further comprises:

[0059] Preprocessing and extracting features of the first voice command and the second voice command respectively;

[0060] Calculate the distance between two time series based on the feature vectors extracted from the first voice command and the second voice command using a dynamic time warping algorithm;

[0061] When the calculated distance is less than the preset threshold, the current recognition model optimization mode is entered.

[0062] Specifically, the user reads out the customized command word again, and the system compares this voice with the features extracted from the first learning. If the comparison is successful, the system confirms that the second learning is successful and announces the success of the second learning. The first and second recognition features are merged to optimize the current recognition model. If the comparison fails, the system prompts the user to relearn. Announce that the second learning failed, please relearn. If the second learning is successful, the system merges the features of the first and second learning, optimizes the current recognition model, and completes the command word learning process. By extracting representative feature vectors (such as pitch, timbre, and syllables) and matching them using the dynamic time warping (DTW) algorithm, the user's voice command words can be more accurately captured and recognized, thereby improving the recognition accuracy.

[0063] Preferably, the method further comprises:

[0064] The first voice command is obtained and compared with the corresponding instruction in the current recognition model. When the comparison result meets the preset requirements, a second voice command is prepared to be obtained.

[0065] Specifically, the user reads out a custom command word in the self-learning mode, and the system compares the voice with the basic recognition model. If the comparison is successful, the system confirms that the first learning is successful. The system announces that the first learning is successful and extracts the first learning features. If the comparison is not successful, the system prompts the user to learn again. The system announces that the first learning failed and asks the user to learn again.

[0066] Preferably, the method further comprises: after acquiring the voice instruction or voice command, performing preprocessing including noise removal, echo cancellation, and noise reduction.

[0067] Through preprocessing steps such as noise removal, echo cancellation and noise reduction, the system can improve the quality of speech signals, maintain a high recognition accuracy in different environments, and improve the robustness of the system.

[0068] Based on the same concept, the present invention also provides a self-learning device based on offline speech recognition, comprising:

[0069] A response module, used for responding to a preset voice command and entering a command word self-learning mode;

[0070] A voice acquisition module, used to acquire and store a first voice command; and acquire a second voice command, compare it with the first voice command, and enter a current recognition model optimization mode when the comparison result meets a preset requirement;

[0071] The optimization module is used to fuse the first voice command and the second voice command, and optimize the current recognition model through a preset algorithm to complete command word learning.

[0072] Based on the same concept, the present invention also provides an electronic device, comprising: a memory, the memory being used to store a processing program; and a processor, the processor implementing any one of the above-mentioned self-learning methods based on offline speech recognition when executing the processing program.

[0073] Based on the same concept, the present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, any one of the above-mentioned self-learning methods based on offline speech recognition is implemented.

[0074] If the self-learning method based on offline speech recognition is implemented in the form of program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or part of the prior art that contributes to the technical solution or all or part of the technical solution can be embodied in the form of software, and the computer software is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0075] Those skilled in the art can clearly understand that, for the sake of convenience and brevity in description, the identification content specifically executed by the above-described system and device can refer to the corresponding process in the aforementioned method embodiment.

[0076] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the protection scope of the present invention.

Claims

1. A self-learning method based on offline speech recognition, characterized in that: The following steps are involved: In response to a preset voice command, enter a command word self-learning mode; Obtaining and storing a first voice command; Obtain a second voice command, compare it with the first voice command, and when the comparison result meets the preset requirements, enter the current recognition model optimization mode; The first voice command and the second voice command are integrated, and the current recognition model is optimized through a preset algorithm to complete command word learning.

2. The self-learning method based on offline speech recognition according to claim 1, characterized in that: The fusing of the first voice command and the second voice command further includes: Preprocessing and extracting features from the first voice command and the second voice command; The feature fusion algorithm based on deep learning uses frequency domain mixing or time-frequency mixing to fuse the extracted feature vectors of the first voice command and the second voice command.

3. The self-learning method based on offline speech recognition according to claim 2, characterized in that: Optimizing the current recognition model through a preset algorithm to complete command word learning further includes: Inputting the feature vector of the fusion of the first voice command and the second voice command into the current recognition model, retraining the current recognition model to recognize these new command words, wherein in order to save storage space, a composite state node group is used to reuse state nodes; The current recognition model is retrained regularly by recording new speech data for continuous optimization.

4. The self-learning method based on offline speech recognition according to claim 1, characterized in that: The method further comprises: The second voice command is compared with the first voice command. When the comparison result does not meet the preset requirement, the second voice command is waited for again or the current process is terminated.

5. The self-learning method based on offline speech recognition according to claim 1, characterized in that: Acquiring a second voice command and comparing it with the first voice command further includes: Preprocessing and extracting features of the first voice command and the second voice command respectively; Calculate the distance between two time series based on the feature vectors extracted from the first voice command and the second voice command using a dynamic time warping algorithm; When the calculated distance is less than the preset threshold, the current recognition model optimization mode is entered.

6. The self-learning method based on offline speech recognition according to claim 1, characterized in that: The method further comprises: The first voice command is obtained and compared with the corresponding instruction in the current recognition model. When the comparison result meets the preset requirements, a second voice command is prepared to be obtained.

7. The self-learning method based on offline speech recognition according to claim 1, characterized in that: The method further includes: after acquiring the voice instruction or voice command, performing preprocessing including noise removal, echo cancellation, and noise reduction.

8. A self-learning device based on offline speech recognition, characterized in that: include: A response module, used for responding to a preset voice command and entering a command word self-learning mode; A voice acquisition module, used to acquire and store the first voice command; and obtaining a second voice command, comparing it with the first voice command, and entering a current recognition model optimization mode when the comparison result meets a preset requirement; The optimization module is used to fuse the first voice command and the second voice command, and optimize the current recognition model through a preset algorithm to complete command word learning.

9. An electronic device, characterized in that: include: A memory, the memory being used to store a processing program; A processor, wherein when executing the processing program, the processor implements the self-learning method based on offline speech recognition as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a processing program, and when the processing program is executed by the processor, the self-learning method based on offline speech recognition as described in any one of claims 1 to 7 is implemented.

Images