Intelligent voice assistant optimization method and system

By collecting, cleaning and updating data in the speech recognition system, using incremental learning and GPU acceleration, combining multimodal interaction interfaces and user feedback, the system's self-optimization is realized, and the recognition of Chinese speech recognition system in complex environments is solved. The accuracy and response speed issues have improved the system's recognition capabilities and user experience.

CN119091865APending Publication Date: 2024-12-06BEIYIN FINANCIAL TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411200844.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-06

Smart Images

  • Figure CN119091865A_ABST
    Figure CN119091865A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent voice assistant optimization method and system. The voice assistant optimization method comprises the following steps: collecting, cleaning, labeling and updating training data; model parameters are automatically adjusted or retraining is triggered according to user feedback, and self-optimization of the system is achieved; the interface of the intelligent device is obtained, and receiving and processing of multi-mode interaction data are supported. And through continuous optimization and adjustment, a Chinese speech recognition system with efficient recognition capability is trained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent voice technology, and in particular to an intelligent voice assistant optimization method and system. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent voice assistants have become an indispensable part of people's daily lives. However, when facing the complex and ever-changing Chinese language environment, existing voice recognition systems still have problems such as low recognition accuracy and slow response speed.

[0003] Most users of existing solutions need to control devices based on buttons or remote controls. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide an intelligent voice assistant optimization method and system that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, an intelligent voice assistant optimization method is provided, and the voice assistant optimization method comprises:

[0006] Collection, cleaning, labeling and updating of training data;

[0007] Automatically adjust model parameters or trigger retraining based on user feedback to achieve self-optimization of the system;

[0008] Obtain the interface of smart devices to support the reception and processing of multimodal interaction data.

[0009] Optionally, the collection, cleaning, labeling and updating of the training data specifically include:

[0010] Collect data sets, including Chinese speech data sets of speech data over a period of time;

[0011] Preprocessing the Chinese speech data set;

[0012] Features of the audio file are extracted and used as input to the model.

[0013] Optionally, the preprocessing of the Chinese speech data set specifically includes: format conversion of the audio file, noise removal, and volume normalization.

[0014] Optionally, the extracted audio file features specifically include: Mel frequency cepstral coefficients.

[0015] Optionally, the preprocessing of the Chinese speech data set specifically includes:

[0016] Automated processing: using advanced speech recognition technology to automatically process the collected speech data, including denoising, segmentation, and feature extraction;

[0017] Manual verification and labeling: Manual verification and labeling are performed on data judged by automated processing.

[0018] Optionally, the updating of the training data specifically includes:

[0019] Regular updates: Based on actual needs and data availability, new voice data is regularly collected from various channels and updated to the dataset after cleaning and annotation;

[0020] Incremental learning: The Eesen framework supports incremental learning, which allows the model to be fine-tuned using new data without retraining the entire model.

[0021] GPU acceleration uses GPU to accelerate the training process, allowing the model to be retrained after the data is updated to adapt to the new data distribution and characteristics.

[0022] Optionally, the automatic adjustment of model parameters or triggering of retraining according to user feedback to achieve self-optimization of the system specifically includes: formulation of personalized learning paths; real-time feedback and intervention.

[0023] Optionally, the personalized learning path formulation specifically includes:

[0024] Intelligent recommendation: based on the results of data analysis, develop personalized learning goals and learning plans for each learner;

[0025] Dynamic adjustment: During the learning process, the learning path is dynamically adjusted based on the learner's learning progress and feedback.

[0026] Optionally, the real-time feedback and intervention specifically include:

[0027] Instant feedback: Provide learners with real-time feedback on their learning progress and performance, helping them understand their learning status in a timely manner;

[0028] Manual intervention: supports manual intervention by teachers or system administrators.

[0029] The present invention also provides an intelligent voice assistant optimization system, which applies the intelligent voice assistant optimization method described above, and the optimization system specifically includes:

[0030] Data management and update module, used for collecting, cleaning, labeling and updating training data;

[0031] Adaptive learning module, which is used to automatically adjust model parameters or trigger retraining based on user feedback to achieve self-optimization of the system;

[0032] The multimodal interface module is used to obtain the interface of the smart device and support the reception and processing of multimodal interaction data.

[0033] The present invention provides an intelligent voice assistant optimization method and system, the voice assistant optimization method includes: collecting, cleaning, labeling and updating training data; automatically adjusting model parameters or triggering retraining according to user feedback to achieve self-optimization of the system; obtaining the interface of the intelligent device to support the reception and processing of multimodal interactive data. Through continuous optimization and adjustment, a Chinese speech recognition system with efficient recognition ability is trained.

[0034] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0036] Figure 1 A flowchart of an intelligent voice assistant optimization method provided by an embodiment of the present invention;

[0037] Figure 2 A block diagram of the composition of an intelligent voice assistant optimization system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0039] The terms "comprises" and "having" and any variations thereof in the description embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0040] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0041] like Figure 1As shown, an intelligent voice assistant system based on the Eesen forTHCHS30 model includes: a data management and update module, an adaptive learning module and a multimodal interface module.

[0042] Data management and update module: responsible for the collection, cleaning, labeling and updating of training data to ensure that the data source for model training remains valid.

[0043] Multimodal interface module: provides interfaces with other smart devices and supports the reception and processing of multimodal interactive data.

[0044] Adaptive learning module: automatically adjusts model parameters or triggers retraining based on user feedback to achieve self-optimization of the system.

[0045] like Figure 2 As shown, an intelligent voice assistant method based on the Eesen forTHCHS30 model includes:

[0046] Data collection and preprocessing include:

[0047] Data source: The core data source is the THCHS-30 dataset, which is a high-quality dataset provided by Fudan University that contains 30 hours of Chinese spoken recordings. This dataset covers a wide range of language environments and speakers, which helps train models with high generalization capabilities.

[0048] Real-time collection: In addition to the THCHS-30 dataset, the system also collects real-time voice data from users during use for continuous optimization and updating of the model. These real-time data may come from various application scenarios, such as smart homes, voice assistants, etc.

[0049] Preprocessing includes:

[0050] Automated processing: Use advanced speech recognition technology (such as the sequence-to-sequence model in the Eesen framework) to automatically process the collected speech data, including denoising, segmentation, feature extraction and other steps.

[0051] Manual verification and labeling: For data that is difficult to accurately judge through automated processing, manual verification and labeling will be performed to ensure the accuracy and consistency of the data.

[0052] Data storage and version control, including:

[0053] Secure storage: The cleaned and labeled data will be securely stored in the system, using encryption and access control to ensure data security. This avoids the risk of data leakage and illegal access.

[0054] Version control: Perform version control on each updated dataset and record the updated content and time. This helps to roll back to the previous version when necessary, ensuring the traceability and manageability of the data. For example, record the size of the dataset, the amount of new data, and the modified data point information for each version.

[0055] Data update strategies include:

[0056] Regular updates: The system will regularly collect new voice data from various channels based on actual needs and data availability, and update it to the dataset after cleaning and annotation. This regular update strategy helps maintain the timeliness and diversity of the dataset.

[0057] Incremental learning: The Eesen framework supports incremental learning, which means that the model can be fine-tuned using newly added data without retraining the entire model. This approach improves the adaptability and update efficiency of the model while reducing the consumption of computing resources. In the Eesen forTHCHS30 model, the acoustic model and language model are continuously optimized through incremental learning to improve speech recognition performance.

[0058] GPU acceleration: GPU is used to accelerate the training process, which greatly shortens the training time. This enables the model to be quickly retrained after the data is updated to adapt to the new data distribution and characteristics.

[0059] Specific scenario optimization and exception handling include:

[0060] Optimization for specific scenarios: For specific application scenarios (such as noisy environments, different dialects, etc.), the system may introduce additional processing steps to optimize data quality. For example, in a noisy environment, more advanced noise reduction and speech enhancement techniques can be used to improve the clarity of voice data.

[0061] Abnormal processing: During the data update process, if abnormal data (such as extremely low-quality recordings, unrecognizable voices, etc.) is encountered, the system will perform special processing. These abnormal data may be marked, excluded, or further analyzed and processed to ensure the overall quality of the data set.

[0062] Practical application cases include:

[0063] Smart home scenario: In a smart home environment, the data management and update module will regularly collect control commands issued by users through the voice assistant, such as "turn on the living room light", "turn off the air conditioner", etc. After cleaning and labeling, this data will be used to train and optimize the voice assistant model to improve its recognition accuracy and response speed in this scenario.

[0064] In-car navigation scenario: In the in-car navigation system, the module collects navigation instructions input by users through voice, such as "go to the nearest gas station", "check tomorrow's weather", etc. At the same time, based on user feedback and real-time traffic information, the model is continuously updated and optimized to provide more accurate and personalized navigation services.

[0065] Adaptive learning module: automatically adjusts model parameters or triggers retraining based on user feedback to achieve self-optimization of the system.

[0066] 1. Personalized learning path development

[0067] Intelligent recommendation: Based on the results of data analysis, the adaptive learning module will develop personalized learning goals and learning plans for each learner. Through the intelligent recommendation algorithm, the system can recommend appropriate learning content and resources to learners based on their learning needs and interests.

[0068] Dynamic adjustment: During the learning process, the adaptive learning module will dynamically adjust the learning path according to the learner's learning progress and feedback. If the learner shows difficulty in a certain knowledge point, the system will reduce the difficulty and provide more auxiliary materials; if the learner has a good grasp of the knowledge, the difficulty will be increased to keep the learning challenging.

[0069] 2. Real-time feedback and intervention

[0070] Instant feedback: The adaptive learning module can provide learners with real-time feedback on their learning progress and performance, helping them to understand their learning status in a timely manner. Through instant feedback, learners can adjust their learning strategies in a timely manner to improve their learning effects.

[0071] Manual intervention: In addition to automatic adjustments, the adaptive learning module also supports manual intervention by teachers or system administrators. When the system detects that learners encounter problems that are difficult to solve, it proactively sends alerts to teachers or administrators so that they can intervene and provide help in a timely manner.

[0072] Multimodal interface module: provides interfaces with other smart devices and supports the reception and processing of multimodal interactive data.

[0073] The training process of the Eesen for THCHS-30 model is a complex but systematic process that aims to optimize and train an efficient Chinese speech recognition system using a specific dataset (i.e. THCHS-30). The following is a detailed description of the training process, including the necessary points and summaries:

[0074] 1. Data Preparation

[0075] Dataset selection: The THCHS-30 dataset is a Chinese speech dataset containing about 30 hours of speech data, open sourced by Tsinghua University. This dataset is widely used in the training and evaluation of Chinese speech recognition systems.

[0076] Data preprocessing: The THCHS-30 dataset is preprocessed, including audio file format conversion, noise removal, volume normalization and other steps to ensure data consistency and high quality.

[0077] Extract features of audio files, such as MFCC (Mel Frequency Cepstral Coefficients), which will be used as input to the model.

[0078] 2. Model Construction

[0079] Model selection: Choose a suitable speech recognition model architecture based on task requirements, such as convolutional neural network (CNN), recurrent neural network (RNN) or transformer. The Eesen framework usually supports multiple model architectures, which can be selected based on specific circumstances.

[0080] Model configuration: configure the model parameters, including the number of network layers, number of nodes, activation function, optimizer, etc. The setting of parameters will directly affect the performance and training efficiency of the model.

[0081] 3. Training Process

[0082] Initialize parameters: Initialize the parameters in the model using random initialization or pre-trained weights.

[0083] Forward propagation: The preprocessed feature data is input into the model, and the output of each layer is calculated through forward propagation until the final prediction result is obtained.

[0084] Calculate loss: Use an appropriate loss function (such as cross entropy loss function or CTC loss function) to calculate the difference between the predicted result and the true label.

[0085] Back propagation: The gradient of the loss function for each parameter is calculated through the back propagation algorithm.

[0086] Parameter update: Use an optimization algorithm (such as gradient descent or its variants) to update the parameters in the model based on the gradient information to reduce the value of the loss function.

[0087] Iterative training: Repeat the process of forward propagation, loss calculation, backpropagation and parameter update until the set stopping condition is reached (such as reaching the maximum number of iterations, loss function convergence, etc.).

[0088] 4. Dialect Optimization (Dialect Expansion for THCHS-30)

[0089] Although the THCHS-30 dataset is mainly oriented towards standard Mandarin, it is optimized for dialects through the following strategies:

[0090] Dialect data collection: Collect speech data containing specific dialects and annotate them.

[0091] Data enhancement: Enhance the dialect data, such as adding noise, changing the speaking speed, etc., to increase data diversity.

[0092] Model adjustment: Adjust the parameters and structure of the model according to the characteristics of the dialect, such as adding dialect-specific phoneme modeling, adjusting the vocabulary library of the language model, etc.

[0093] Incremental training: Use dialect data to incrementally train the model to gradually improve the model's ability to recognize dialects.

[0094] 5. Evaluation and Tuning

[0095] Model evaluation: Use the test set to evaluate the trained model. The evaluation indicators can be accuracy and word error rate (WER).

[0096] Model tuning: Tune the model based on the evaluation results, including adjusting hyperparameters, changing the model architecture, and increasing the amount of data.

[0097] Beneficial effects: The training process of the Eesen forTHCHS-30 model is a systematic process, including data preparation, model building, training process, dialect optimization, evaluation and tuning. Through continuous optimization and adjustment, a Chinese speech recognition system with efficient recognition capabilities is trained. For dialect optimization, additional dialect data needs to be collected and targeted model adjustments need to be made.

[0098] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent voice assistant optimization method, characterized in that: The voice assistant optimization method comprises: Collection, cleaning, labeling and updating of training data; Automatically adjust model parameters or trigger retraining based on user feedback to achieve self-optimization of the system; Obtain the interface of smart devices to support the reception and processing of multimodal interaction data.

2. The intelligent voice assistant optimization method according to claim 1, characterized in that: The collection, cleaning, labeling and updating of the training data specifically include: Collect data sets, including Chinese speech data sets of speech data over a period of time; Preprocessing the Chinese speech data set; Features of the audio file are extracted and used as input to the model.

3. The intelligent voice assistant optimization method according to claim 2, characterized in that: The preprocessing of the Chinese speech data set specifically includes: format conversion of audio files, noise removal, and volume normalization.

4. The intelligent voice assistant optimization method according to claim 2, characterized in that: The features of the extracted audio file specifically include: Mel frequency cepstral coefficients.

5. The intelligent voice assistant optimization method according to claim 2, characterized in that: The preprocessing of the Chinese speech data set specifically includes: Automated processing: using advanced speech recognition technology to automatically process the collected speech data, including denoising, segmentation, and feature extraction; Manual verification and labeling: Manual verification and labeling are performed on data judged by automated processing.

6. The intelligent voice assistant optimization method according to claim 1, characterized in that: The updating of the training data specifically includes: Regular updates: Based on actual needs and data availability, new voice data is regularly collected from various channels and updated to the dataset after cleaning and annotation; Incremental learning: The Eesen framework supports incremental learning, which allows the model to be fine-tuned using new data without retraining the entire model. GPU acceleration uses GPU to accelerate the training process, allowing the model to be retrained after the data is updated to adapt to the new data distribution and characteristics.

7. The intelligent voice assistant optimization method according to claim 1, characterized in that: The automatic adjustment of model parameters or triggering of retraining according to user feedback to achieve self-optimization of the system specifically includes: formulation of personalized learning paths; real-time feedback and intervention.

8. The intelligent voice assistant optimization method according to claim 7, characterized in that: The personalized learning path formulation specifically includes: Intelligent recommendation: based on the results of data analysis, develop personalized learning goals and learning plans for each learner; Dynamic adjustment: During the learning process, the learning path is dynamically adjusted based on the learner's learning progress and feedback.

9. The intelligent voice assistant optimization method according to claim 7, characterized in that: The real-time feedback and intervention specifically include: Instant feedback: Provide learners with real-time feedback on their learning progress and performance, helping them understand their learning status in a timely manner; Manual intervention: supports manual intervention by teachers or system administrators.

10. An intelligent voice assistant optimization system, applying an intelligent voice assistant optimization method according to any one of claims 1 to 9, characterized in that: The optimization system specifically includes: Data management and update module, used for collecting, cleaning, labeling and updating training data; Adaptive learning module, which is used to automatically adjust model parameters or trigger retraining based on user feedback to achieve self-optimization of the system; The multimodal interface module is used to obtain the interface of the smart device and support the reception and processing of multimodal interaction data.

Citation Information

Cited By

  • Personalized language model construction method of intelligent voice assistant

    CN120220660A