An adaptive voice wake-up interaction method and an interaction system
By combining multi-class speech recognition and voiceprint feature clustering to train a cloud wake-up model, the problem of decreased voice wake-up accuracy in multi-user and complex acoustic environments is solved, and an adaptive voice wake-up system with high accuracy and stability in multi-user scenarios is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AISPEECH CO LTD
- Filing Date
- 2026-05-26
- Publication Date
- 2026-07-21
AI Technical Summary
Existing voice wake-up systems suffer from decreased wake-up accuracy in multi-user or complex acoustic environments, struggle to adapt to changes in wake words, and are prone to false or missed wake-ups.
By combining multi-class speech recognition processing, voiceprint feature clustering, and training a cloud-based wake-up model, an adaptive wake-up model is generated to distinguish the speech features of different users. The model is then deeply optimized and trained in the cloud, and a multi-head neural network structure is used to optimize wake-up word recognition.
It improves the accuracy of voice wake-up in multi-user scenarios and the adaptability of custom wake-up words for terminal devices, thereby enhancing the recognition accuracy and stability of the voice wake-up system.
Smart Images
Figure CN122435928A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of speech processing technology, and in particular relates to an adaptive voice wake-up interaction method and interaction system. Background Technology
[0002] With the development of artificial intelligence and speech recognition technology, voice interaction has gradually become one of the important interaction methods for smart terminal devices. Through voice wake-up technology, users can wake up the device and perform subsequent interactions via voice commands without manual operation, thereby improving the ease of use and interactive experience. Currently, voice wake-up technology is widely used in various application scenarios such as smart speakers, smart home devices, mobile terminals, and in-vehicle devices.
[0003] Existing voice wake-up systems typically use preset wake-up words or user-defined wake-up words to wake up devices. Some solutions collect user voice samples and extract acoustic features to establish a comparison threshold; wake-up word recognition is achieved when the real-time matching value exceeds the threshold. Alternatively, user-defined wake-up words are converted into recognizable speech units using pinyin or phoneme modeling and then recognized using existing wake-up models.
[0004] However, existing technologies are prone to a decrease in wake-up accuracy when the wake word changes; at the same time, existing wake-up models are difficult to continuously optimize based on the voice characteristics of different users, and are prone to false wake-ups or missed wake-ups in multi-user or complex environments. Summary of the Invention
[0005] This invention provides an adaptive voice wake-up interaction system to solve the problems of decreased wake-up accuracy and difficulty in waking up when using custom wake-up words in multi-user scenarios and complex acoustic environments.
[0006] One objective of this invention is to achieve adaptive wake-up recognition for different user voice characteristics by combining multi-class speech recognition processing, voiceprint feature clustering and training a cloud wake-up model, thereby improving the accuracy of voice wake-up and the ability to adapt to custom wake-up words in multi-user scenarios.
[0007] According to a first aspect of the present invention, an adaptive voice wake-up interaction method is provided for waking up a terminal device, the method comprising: Obtain user voice data; The user voice data is processed by speech recognition to generate first voice interaction data, wherein the first voice interaction data is text information and voice information corresponding to the user voice data; The user voice data is subjected to voiceprint feature extraction and the voiceprint features are clustered to obtain different user data sets; Based on the first voice interaction data and the different user data sets, corresponding second voice interaction data is generated, where the second voice interaction data is the synthesized voice data of the corresponding user. The cloud-based wake-up model is trained based on the second voice interaction data to generate a first wake-up model. A first wake-up recognition result is obtained based on the first wake-up model to wake up the terminal device.
[0008] In some possible embodiments, the user voice data is subjected to voiceprint feature extraction and the voiceprint features are clustered to obtain different user data sets, including: User identity information is determined based on the voiceprint features to establish a correspondence between the first voice interaction data and the corresponding user identity.
[0009] In some possible embodiments, voiceprint feature extraction includes: Extract the original voiceprint features from the user's voice data and add the original voiceprint features as a reference feature vector to the voiceprint comparison library to update the voiceprint comparison benchmark. Historical voice data is acquired, and user identity information is obtained by performing identity recognition on the historical voice data. A clustering model is then trained based on the user identity information.
[0010] In some possible embodiments, the user voice data is subjected to voiceprint feature extraction and the voiceprint features are clustered to obtain different user data sets, including the following steps: Extract the voice data of the current user who has set a custom wake word, perform voiceprint feature extraction and clustering processing to obtain the first user data set corresponding to the current user; Extract the voice data of other users in the current user's family group, perform voiceprint feature extraction and clustering processing to obtain a second data set corresponding to other users in each family group; Based on the voice data of external users, voiceprint feature extraction and clustering are performed to obtain a third dataset corresponding to the external users.
[0011] In some possible embodiments, the second voice interaction data includes audio features corresponding to the user, the audio features including at least one of timbre features, pitch features, tone features, and volume features.
[0012] In some possible embodiments, training the cloud-based wake-up model based on the second voice interaction data includes: The second voice interaction data is subjected to acoustic environment simulation processing to generate third voice interaction data under different acoustic environments, and the cloud wake-up model is trained based on the second voice interaction data and the third voice interaction data.
[0013] In some possible embodiments, training the cloud-based wake-up model based on the second voice interaction data and the third voice interaction data includes: The cloud-based wake-up model is trained based on the second and third voice interaction data corresponding to the current user and other users in the user's family group to generate the second wake-up model. After generating the second wake-up model, the cloud-based wake-up model is trained based on the second and third voice interaction data corresponding to the external user to generate the third wake-up model.
[0014] In some possible embodiments, training the cloud wake-up model includes: When training the cloud-based wake-up model, the parameters of the default wake-up word recognition branch are locked, and the parameters of the custom wake-up word recognition branch are updated. The cloud wake-up model adopts a multi-head neural network structure, which includes a shared feature extraction layer, a default wake-up word recognition branch, and a custom wake-up word recognition branch. The shared feature extraction layer is used to extract acoustic features shared by the default wake-up word recognition branch and the custom wake-up word recognition branch.
[0015] In some possible embodiments, it also includes: The local wake-up model of the terminal device is updated based on the first wake-up model, the second wake-up model, and the third wake-up model.
[0016] According to a second aspect of the present invention, an interactive system is provided for waking up a terminal device, comprising: The acquisition module is used to acquire user voice data; The first voice interaction module is used to perform voice recognition processing on the user voice data to generate first voice interaction data, wherein the first voice interaction data is text information and voice information corresponding to the user voice data. The voiceprint module is used to extract voiceprint features from the user's voice data and cluster the voiceprint features to obtain different sets of user data. The second voice interaction module is used to generate corresponding second voice interaction data based on the first voice interaction data and the different user data sets, wherein the second voice interaction data is the synthesized voice data of the corresponding user. An adaptive training module is used to train a cloud-based wake-up model based on the second voice interaction data to generate a first wake-up model, and to obtain a first wake-up recognition result based on the first wake-up model to wake up the terminal device.
[0017] According to the present invention, user voice data is processed by speech recognition to generate text information and voice information, and user voice data is processed by voiceprint feature extraction and clustering to obtain different user data sets. This enables the system to distinguish the voice features of different users, thereby generating synthetic voice data for the corresponding user based on the text information and different user data sets. Then, the synthetic voice data is used to train the cloud wake-up model to improve the custom wake-up recognition capability of the terminal device in multi-user scenarios.
[0018] Furthermore, by establishing a correspondence between the first voice interaction data and the user's identity, the generated synthetic voice data can be matched with different user identities, thereby achieving voice data generation tailored to different users. Simultaneously, by introducing a voiceprint comparison library and training a clustering model with historical voice data, the voiceprint recognition results can be continuously optimized. In addition, by simulating the acoustic environment of the synthetic voice data and employing staged training and a multi-head neural network structure, the wake-up model can adapt to multiple users and complex acoustic environments, thereby improving the recognition accuracy and stability of the voice wake-up system.
[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below. Attached Figure Description
[0020] Figure 1 A schematic flowchart of an adaptive voice wake-up interaction method according to an embodiment of the present invention is shown; Figure 2 A schematic flowchart of the method for step S300 is shown; Figure 3 A schematic flowchart of the method for step S501 is shown; Figure 4 A block diagram of an interactive system according to an embodiment of the present invention is shown; In the diagram: 601 - Terminal device, 602 - Acquisition module, 603 - First voice interaction module, 604 - Voiceprint module, 605 - Second voice interaction module, 606 - Adaptive training module. Detailed Implementation
[0021] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, it should be noted that, for ease of description, only the parts relevant to this application are shown in the accompanying drawings, not the entire structure. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.
[0022] The terms “comprising” and “having”, and any variations thereof, used in this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.
[0023] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0024] Figure 1 A schematic flowchart of an adaptive voice wake-up interaction method according to an embodiment of the present invention is shown. This adaptive voice wake-up interaction method is used to wake up a terminal device, which collects user voice data and uploads it to the cloud. A voice program, system, or service deployed in the cloud processes the user voice data, such as... Figure 1 As shown, the method includes the following steps S100 to S500.
[0025] Step S100: Obtain user voice data.
[0026] The user voice data includes target voice data generated by the user during the process of setting a custom wake word, as well as continuous voice data generated by the user during daily voice interactions. The voice content can be complete everyday expressions or short wake words or variations thereof. Furthermore, the user voice data covers different groups of people, including but not limited to the user themselves, family members, and other interacting individuals. Simultaneously, the voice data retains individual user characteristics such as timbre, tone, emotion, intonation, and volume.
[0027] This step provides a data foundation for subsequent adaptive voice wake-up interaction methods by acquiring voice data from multiple sources and of multiple types.
[0028] Step S200: Perform speech recognition processing on the user's voice data to generate first voice interaction data, which consists of text information and voice information corresponding to the user's voice data.
[0029] The process involves performing speech recognition and speech-to-text processing on user voice data to obtain the first voice interaction data. This first voice interaction data can be annotated with necessary information, including timestamps and voice quality details, for subsequent processing and analysis. The generated first voice interaction data can be stored in the cloud to support other data management, annotation, and training applications.
[0030] This step converts user voice data into text and voice information, enabling structured expression of voice content, thereby improving the readability and processability of voice information, and providing a foundation for unified management and analysis of voice data through annotation and storage.
[0031] Step S300: Extract voiceprint features from user voice data and cluster the voiceprint features to obtain different user data sets.
[0032] Clustering involves grouping different voice data based on the similarity between voiceprint features, thus forming multiple user datasets. Each user dataset corresponds to a category of voice data, representing the voice features of the same user or users of the same category, achieving effective differentiation and aggregation of multi-user voice data. User datasets include, but are not limited to, datasets of the user themselves, family members, and other interacting individuals, and each dataset is distinct from the others.
[0033] In one specific embodiment, it includes: User identity information is determined based on voiceprint features to establish a correspondence between the first voice interaction data and the corresponding user identity.
[0034] It should be noted that a unique user identifier can be assigned to each voice data point based on the similarity of voiceprint features. This allows multiple voice inputs from the same user to be attributed to the same identity, and the first voice interaction data is associated with and stored with this user identifier. This establishes a one-to-one or many-to-one mapping relationship between voice data and user identity, ensuring that data from different users is distinguishable and data from the same user is consistently aggregated. This correspondence or mapping relationship can be stored in the user information system to improve the clarity and consistency of voice data organization.
[0035] In another embodiment, voiceprint feature extraction includes the following steps: Extract the original voiceprint features from the user's voice data and add them as the baseline feature vector to the voiceprint comparison library to update the voiceprint comparison baseline.
[0036] Historical voice data is acquired, and user identity information is obtained by performing identity recognition on the historical voice data. A clustering model is then trained based on the user identity information.
[0037] It should be noted that the original voiceprint features are used to characterize the individual voice differences of the initial speech users. Continuously updating these original voiceprint features into the voiceprint comparison database allows the database to continuously reflect the latest voice feature distribution, thereby improving the consistency and stability of subsequent voiceprint comparisons. Furthermore, a clustering model is used for clustering processing. Training the clustering model with historical speech data containing user identifiers enables it to learn the distribution patterns of voiceprint features among different users, thus achieving more accurate speech data clustering and user differentiation in subsequent processing.
[0038] Figure 2 It shows Figure 1 The schematic flowchart for step S300 is shown. Figure 2 As shown, step S300 may include steps S301 to S303.
[0039] Step S301: Extract the voice data of the current user who has set a custom wake word, and perform voiceprint feature extraction and clustering processing to obtain the first user data set corresponding to the current user.
[0040] Specifically, the system determines the current user's identity information based on the currently set custom wake word, and identifies the current user's voice data from the user's voice data based on this identity information for subsequent processing. The first user data set is the voice data set corresponding to the current user's identity information, used to characterize the user's voice features.
[0041] Step S302: Extract the voice data of other users in the current user's family group, perform voiceprint feature extraction and clustering processing to obtain a second data set corresponding to each other user in each family group.
[0042] The second user dataset is a unified set of the individual datasets of other users within each household group.
[0043] It should be noted that other users within the current user's family group can be identified based on historical correlations in the user's voice data. Specifically, by statistically analyzing other voice data that appeared in the same interaction scenario, time period, or continuous interaction process as the current user in historical voice data, and calculating the correlation ratio between corresponding voice data, when the correlation ratio between a user's voice data and the current user's voice data is higher than a first preset threshold, that user can be identified as another user belonging to the same family group as the current user. The solution of this invention does not specifically set the first preset threshold; it can be adaptively determined or flexibly adjusted according to the actual application scenario, user scale, and voice data distribution to ensure the rationality and applicability of the family group division.
[0044] Step S303: Based on the voice data of external users, perform voiceprint feature extraction and clustering processing to obtain a third data set corresponding to the external users.
[0045] Among them, external users are all other users excluding the current user and other users in the current user's family group. The voice data of external users can be extracted from the user's language data or generated by the cloud-based voice processing program or the system itself.
[0046] This step, through voiceprint feature extraction and clustering of user voice data, effectively distinguishes and aggregates multi-user voice data, and constructs a multi-level dataset based on user identity, thereby improving the orderliness and accuracy of voice data organization. Furthermore, by differentiating between the datasets of current users, family group users, and external users, the distribution of voice features among different user groups can be precisely determined, thus enhancing the adaptability of voice processing in multi-user scenarios.
[0047] Step S400: Generate corresponding second voice interaction data based on the first voice interaction data and different user data sets. The second voice interaction data is the synthesized voice data of the corresponding user.
[0048] The second voice interaction data includes the corresponding user's audio characteristics, which include at least one of timbre, pitch, tone, and volume characteristics. This second voice interaction data is voice data generated based on the unique audio characteristics of the corresponding user, capable of being accurately recognized by the voice processing program.
[0049] It should be noted that the second voice interaction data is synthesized voice data. This synthesized voice data includes voice content directly corresponding to the text information in the first voice interaction data, including derived expanded sentences generated based on this voice content, thereby enriching the voice expression. In this way, a large amount of synthesized voice data with a clear correspondence to the corresponding users can be constructed, thus fully representing the user's personalized audio characteristics and providing rich and accurate training data for subsequent voice processing.
[0050] This step generates a large amount of synthetic speech data corresponding to the user, achieving accurate representation of the user's personalized audio characteristics and improving the adaptability of the voice wake-up interaction method.
[0051] Step S500: Train the cloud wake-up model based on the second voice interaction data to generate the first wake-up model, and obtain the first wake-up recognition result based on the first wake-up model to wake up the terminal device.
[0052] The first wake-up model is an initial model generated in real-time for the user's initial voice interaction. It is used to quickly respond to the user's first voice input, prioritizing processing of the second voice interaction data corresponding to the text information in the first voice interaction data. Based on the first wake-up model, a first wake-up recognition result can be obtained, which reflects the first wake-up model's real-time wake-up judgment of the initial voice interaction. After generation, the first wake-up model and the first wake-up recognition result can be immediately sent to the terminal device, enabling a rapid response to the initial voice interaction and thus improving the user's real-time voice wake-up experience.
[0053] Furthermore, the second voice interaction data corresponding to other derived extended sentences is stored in the cloud and continuously used for training the wake-up model. Cloud training allows for deep adaptive optimization of the wake-up model using larger-scale and more diverse synthesized voice data corresponding to the current user, thereby improving the overall performance of the model. After training, the updated wake-up model can be deployed to the terminal device in a timely manner, enabling the terminal device to achieve higher-precision wake-up capabilities. This approach achieves a two-layer operating mechanism combining rapid on-device response and deep cloud training, ensuring an immediate voice wake-up experience for users while also improving long-term wake-up performance and adaptability.
[0054] In one specific embodiment, the cloud wake-up model is trained based on the second voice interaction data, including step S501: The second voice interaction data is processed through acoustic environment simulation to generate the third voice interaction data under different acoustic environments, and the cloud wake-up model is trained based on the second and third voice interaction data.
[0055] The process involves generating third-party voice interaction data under different acoustic environments to simulate the voice changes of corresponding users in various usage scenarios. This includes, but is not limited to, voice changes when a user has a cold or is sick, voice environments in specific situations, background noise interference, changes in the acoustic characteristics of different rooms or devices, and differences in pronunciation, speech rate, tone, or volume. By performing acoustic environment simulation processing on the second-party voice interaction data, a large amount of diverse third-party voice interaction data can be constructed to reflect various voice changes that users may experience in real-world usage environments. Subsequently, a cloud-based wake-up model is trained using both the second-party and third-party voice interaction data. This significantly improves the model's responsiveness and robustness to different environments and voice states, ensuring that the wake-up system can accurately and reliably recognize user voice input in complex and ever-changing real-world usage scenarios.
[0056] Figure 3 A schematic flowchart of the method for step S501 is shown. Figure 3 As shown, it may include steps S5011 to S5012.
[0057] When training the cloud-based wake-up model based on the second and third voice interaction data, a batch training strategy can be adopted. Specifically, data from the current user and other users in their household group are used first for model training to improve the response time of high-frequency users. On this basis, data from external users are then used for subsequent training to further optimize the overall performance and generalization ability of the model, thereby balancing the model's real-time performance and long-term performance improvement.
[0058] Step S5011: Based on the second and third voice interaction data corresponding to the current user and other users in the family group, train the cloud wake-up model to generate the second wake-up model.
[0059] Step S5012: After generating the second wake-up model, train the cloud wake-up model based on the second and third voice interaction data corresponding to the external user to generate the third wake-up model.
[0060] It should be noted that after generating the second or third wake-up model, the corresponding wake-up model needs to be tested and verified. Specifically, the wake-up model can be tested using the simulated third voice interaction data of the corresponding user. When the wake-up accuracy reaches the second preset threshold, it indicates that the wake-up model meets the performance requirements, and it can then be deployed to the terminal device for application. This invention does not limit the specific value of the second preset threshold; it can be adaptively determined or flexibly adjusted according to the actual application scenario, user scale, and voice data distribution to ensure the rationality and applicability of the wake-up model performance evaluation.
[0061] This step uses the second voice interaction data and the third voice interaction data generated by its simulation to perform hierarchical and adaptive training on the cloud wake-up model, realizing the gradual improvement of the wake-up model from fast response to deep optimization. While ensuring the real-time performance of the initial voice interaction, it further improves the recognition accuracy and robustness of the model in multi-user, multi-scenario and complex acoustic environments, thereby improving the overall performance and user experience of the voice wake-up system.
[0062] In one embodiment, training a cloud wake-up model includes: locking the parameters of the default wake-up word recognition branch and updating the parameters of the custom wake-up word recognition branch when training the cloud wake-up model.
[0063] The cloud wake-up model adopts a multi-head neural network structure, which includes a shared feature extraction layer, a default wake-up word recognition branch, and a custom wake-up word recognition branch. The shared feature extraction layer is used to extract the acoustic features shared by the default wake-up word recognition branch and the custom wake-up word recognition branch.
[0064] By employing a multi-head neural network structure and locking the parameters of the default wake word recognition branch during training while only updating the parameters of the custom wake word recognition branch, custom wake words can be specifically optimized based on shared acoustic features. This improves the recognition performance of custom wake words while avoiding impacting the performance of the default wake word recognition, thereby achieving effective isolation and collaborative optimization of the performance between different wake words.
[0065] In one embodiment, the first wake-up model, the second wake-up model, and the third wake-up model are used to update the local wake-up model of the terminal device.
[0066] The local wake-up model is a pre-configured wake-up model based on a default wake-up word on the terminal device. After the first wake-up model is updated, the local wake-up model can make wake-up judgments based on the voice data of the current user with a set custom wake-up word; after the second wake-up model is updated, the local wake-up model can further make wake-up judgments based on the voice data of the current user and other users in their household group; after the third wake-up model is updated, the local wake-up model can also make wake-up judgments based on the voice data of external users.
[0067] By updating the local wake-up model in stages, it can gradually acquire the ability to recognize voice data from different user groups, thereby improving the adaptability and accuracy of voice wake-up in multi-user scenarios.
[0068] According to the present invention, user voice data is processed by speech recognition to generate text and voice information. Voiceprint feature extraction and clustering are then performed on the user voice data to obtain different user data sets. This enables the system to distinguish the voice features of different users, thereby generating synthetic voice data for the corresponding user based on the text information and different user data sets. The synthetic voice data is then used to train a cloud-based wake-up model, improving the custom wake-up recognition capability of terminal devices in multi-user scenarios. Furthermore, by establishing a correspondence between the first voice interaction data and the user's identity, the generated synthetic voice data can be matched with different user identities, thus achieving voice data generation for different users. Simultaneously, by introducing a voiceprint comparison library and historical voice data to train the clustering model, the voiceprint recognition results can be continuously optimized. In addition, by simulating the acoustic environment of the synthetic voice data and employing staged training and a multi-head neural network structure, the wake-up model can adapt to multi-user and complex acoustic environments, thereby improving the recognition accuracy and stability of the voice wake-up system.
[0069] Figure 4 A block diagram of an interactive system according to an embodiment of the present invention is shown. Figure 4 As shown, the interactive system, used to wake up terminal device 601, includes: The acquisition module 602 is used to acquire user voice data.
[0070] The first voice interaction module 603 is used to perform voice recognition processing on user voice data to generate first voice interaction data, wherein the first voice interaction data consists of text information and voice information corresponding to the user voice data.
[0071] The voiceprint module 604 is used to extract voiceprint features from user voice data and perform clustering processing on the voiceprint features to obtain different user data sets.
[0072] The second voice interaction module 605 is used to generate corresponding second voice interaction data based on the first voice interaction data and different user data sets. The second voice interaction data is the synthesized voice data of the corresponding user.
[0073] The adaptive training module 606 is used to train the cloud wake-up model based on the second voice interaction data to generate the first wake-up model, and obtain the first wake-up recognition result based on the first wake-up model to wake up the terminal device 601.
[0074] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in the details; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An adaptive voice wake-up interaction method for waking up a terminal device, characterized in that, The method includes: Obtain user voice data; The user voice data is processed by speech recognition to generate first voice interaction data, wherein the first voice interaction data is text information and voice information corresponding to the user voice data; The user voice data is subjected to voiceprint feature extraction and the voiceprint features are clustered to obtain different user data sets; Based on the first voice interaction data and the different user data sets, corresponding second voice interaction data is generated, where the second voice interaction data is the synthesized voice data of the corresponding user. The cloud-based wake-up model is trained based on the second voice interaction data to generate a first wake-up model. A first wake-up recognition result is obtained based on the first wake-up model to wake up the terminal device.
2. The interaction method according to claim 1, characterized in that, The user voice data is subjected to voiceprint feature extraction, and the voiceprint features are clustered to obtain different user data sets, including: User identity information is determined based on the voiceprint features to establish a correspondence between the first voice interaction data and the corresponding user identity.
3. The interaction method according to claim 1, characterized in that, Voiceprint feature extraction includes: Extract the original voiceprint features from the user's voice data and add the original voiceprint features as a reference feature vector to the voiceprint comparison library to update the voiceprint comparison benchmark. Historical voice data is acquired, and user identity information is obtained by performing identity recognition on the historical voice data. A clustering model is then trained based on the user identity information.
4. The interaction method according to claim 1, characterized in that, The process of extracting voiceprint features from the user voice data and clustering these features to obtain different user data sets includes the following steps: Extract the voice data of the current user who has set a custom wake word, perform voiceprint feature extraction and clustering processing to obtain the first user data set corresponding to the current user; Extract the voice data of other users in the current user's family group, perform voiceprint feature extraction and clustering processing to obtain a second data set corresponding to other users in each family group; Based on the voice data of external users, voiceprint feature extraction and clustering are performed to obtain a third dataset corresponding to the external users.
5. The interaction method according to claim 4, characterized in that, The second voice interaction data includes the audio features of the corresponding user, and the audio features include at least one of timbre features, pitch features, tone features, and volume features.
6. The interaction method according to claim 5, characterized in that, The cloud-based wake-up model is trained based on the second voice interaction data, including: The second voice interaction data is subjected to acoustic environment simulation processing to generate third voice interaction data under different acoustic environments, and the cloud wake-up model is trained based on the second voice interaction data and the third voice interaction data.
7. The interaction method according to claim 6, characterized in that, Training the cloud-based wake-up model based on the second and third voice interaction data includes: The cloud-based wake-up model is trained based on the second and third voice interaction data corresponding to the current user and other users in the user's family group to generate the second wake-up model. After generating the second wake-up model, the cloud-based wake-up model is trained based on the second and third voice interaction data corresponding to the external user to generate the third wake-up model.
8. The interaction method according to claim 1, characterized in that, Training the cloud wake-up model includes: When training the cloud-based wake-up model, the parameters of the default wake-up word recognition branch are locked, and the parameters of the custom wake-up word recognition branch are updated. The cloud wake-up model adopts a multi-head neural network structure, which includes a shared feature extraction layer, a default wake-up word recognition branch, and a custom wake-up word recognition branch. The shared feature extraction layer is used to extract acoustic features shared by the default wake-up word recognition branch and the custom wake-up word recognition branch.
9. The interaction method according to claim 1, characterized in that, Also includes: The local wake-up model of the terminal device is updated based on the first wake-up model, the second wake-up model, and the third wake-up model.
10. An interactive system for waking up a terminal device, characterized in that, include: The acquisition module is used to acquire user voice data; The first voice interaction module is used to perform voice recognition processing on the user voice data to generate first voice interaction data, wherein the first voice interaction data is text information and voice information corresponding to the user voice data. The voiceprint module is used to extract voiceprint features from the user's voice data and cluster the voiceprint features to obtain different sets of user data. The second voice interaction module is used to generate corresponding second voice interaction data based on the first voice interaction data and the different user data sets, wherein the second voice interaction data is the synthesized voice data of the corresponding user. An adaptive training module is used to train a cloud-based wake-up model based on the second voice interaction data to generate a first wake-up model, and to obtain a first wake-up recognition result based on the first wake-up model to wake up the terminal device.