Voice reminding method and device and wearable intelligent equipment

Abstract

The invention relates to the technical field of wearable intelligent devices, in particular to a voice reminding method and device and a wearable intelligent device. The method comprises the following steps: acquiring original audio data of a target role, and preprocessing the original audio data; processing each frame of audio data after preprocessing to obtain a normalized MFCC feature vector; processing all the normalized MFCC feature vectors through a Gaussian mixture model to obtain a voiceprint feature model; loading a corresponding voiceprint feature model according to a target role ID carried in reminding information input by a user; performing format conversion on a voiceprint feature vector in a voiceprint feature model corresponding to the target role ID; inputting a text ID corresponding to the reminding information and the voiceprint feature vector after format conversion into a TTS model to obtain a Mel spectrum; and converting the Mel spectrum into time domain waveform data through a vocoder, and playing the time domain waveform data. According to the method, the voice reminding of simulating a real person is realized, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of wearable smart device technology, and further to a voice reminder method and device, and a wearable smart device. Background Technology

[0002] Currently, wearable smart devices typically include voice reminder functions, covering scenarios such as alarm clocks, phone calls, and text message notifications. However, existing voice reminder modes are rather mechanical, lacking personalization and realism, and struggle to simulate the voice characteristics of real people such as parents, friends, and teachers, thus reducing the user experience. Summary of the Invention

[0003] To address the aforementioned technical issues, this application provides a voice reminder method and apparatus, as well as a wearable smart device, which improves the user experience.

[0004] In a first aspect, this application provides a voice reminder method applied to wearable smart devices, comprising: acquiring raw audio data of a target character; preprocessing the raw audio data; processing each frame of preprocessed audio data to obtain a normalized MFCC feature vector; processing all normalized MFCC feature vectors using a Gaussian mixture model to obtain a voiceprint feature model; loading the corresponding voiceprint feature model based on the target character ID carried in the reminder information input by the user; performing format conversion on the voiceprint feature vector in the voiceprint feature model corresponding to the target character ID; inputting the text ID corresponding to the reminder information and the format-converted voiceprint feature vector into a TTS model to obtain a Mel spectrum; converting the Mel spectrum into time-domain waveform data using a vocoder and playing the time-domain waveform data.

[0005] The above voice reminder method acquires and preprocesses the target character's original audio data, extracts normalized MFCC feature vectors, constructs a voiceprint feature model using a Gaussian mixture model, loads the corresponding model based on user input and converts its format, and combines a TTS model and a vocoder to generate time-domain waveform data for playback. This achieves voice reminders that simulate real people, effectively improving the realism and interactivity of voice reminders and enhancing the user experience.

[0006] In one implementation, a voiceprint feature model is obtained by processing all normalized MFCC feature vectors using a Gaussian mixture model. Specifically, this includes: initializing the model parameters of the Gaussian mixture model; clustering all normalized MFCC feature vectors using a clustering algorithm and setting the cluster centers as the initial mean of the model parameters; iterating the set model parameters a predetermined number of times using an expectation-maximization algorithm to obtain a trained Gaussian mixture model; and compressing the trained Gaussian mixture model to obtain the voiceprint feature model.

[0007] The above voice prompt method improves the training efficiency and stability of the Gaussian mixture model by initializing the parameters of the Gaussian mixture model through cluster centers; iterative optimization using the expectation-maximization algorithm enhances the model's accuracy and robustness; and finally, model compression reduces storage and computational resource consumption, making the voiceprint feature model more suitable for resource-constrained wearable smart devices.

[0008] In one implementation, the raw audio data is preprocessed, specifically including: filtering the raw audio data; detecting human voices in the filtered raw audio data; and dividing the raw audio data after human voice detection into continuous frame audio data according to a fixed duration.

[0009] In one implementation, each frame of preprocessed audio data is processed to obtain a normalized MFCC feature vector. Specifically, this includes: normalizing each frame of preprocessed audio data; inputting all normalized frame audio data into a pre-loaded deep learning model, performing model inference to obtain the MFCC feature vector; and normalizing the MFCC feature vector to obtain a normalized MFCC feature vector.

[0010] In one implementation, the method further includes: loading any frame of audio data into a register; accelerating Fourier transform calculations using a multimedia processing library; calculating the energy of the Mel filter bank in parallel according to vector instructions from the multimedia processing library; and accelerating the discrete cosine transform using a lookup table method.

[0011] In one implementation, it also includes periodically controlling the wearable smart device to perform resource-intensive operations when the wearable smart device is charging or connected to a non-metered network.

[0012] In one implementation, the method further includes: encrypting the original audio data before preprocessing using an encryption algorithm, and constructing a first request object based on the encrypted original audio data; sending the first request object to the cloud, wherein the first request object is used to instruct the cloud to generate a voiceprint feature vector based on the encrypted original audio data; responding to a second request object sent by the cloud, and verifying the status code in the second request object; and when the verification passes, saving the voiceprint feature vector carried in the second request object.

[0013] The above voice prompt method ensures data security during transmission by encrypting the original audio data, preventing the risk of data leakage and tampering. The encrypted data is then sent to the cloud to generate a voiceprint feature vector. Verifying the status code in the second request object ensures the integrity and reliability of data processing; the voiceprint feature vector is only saved when the status code verification passes, further guaranteeing data quality and security. Simultaneously, retrieving the voiceprint feature vector from the cloud eliminates the need for wearable smart devices to perform the voiceprint feature vector generation operation, effectively reducing power consumption and resource usage, and improving the user experience.

[0014] In one implementation, the method further includes adjusting the fundamental frequency, speech rate factor, and energy in the format-converted voiceprint feature vector according to the emotion type specified by the user.

[0015] Secondly, this application also provides a voice reminder device for wearable smart devices, comprising: an audio acquisition module configured to acquire raw audio data of a target character and preprocess the raw audio data; a voiceprint modeling module configured to: process each frame of preprocessed audio data to obtain a normalized MFCC feature vector; process all normalized MFCC feature vectors using a Gaussian mixture model to obtain a voiceprint feature model; a speech synthesis module configured to: load the corresponding voiceprint feature model based on the target character ID carried in the reminder information input by the user; perform format conversion on the voiceprint feature vector in the voiceprint feature model corresponding to the target character ID; input the text ID corresponding to the reminder information and the format-converted voiceprint feature vector into a TTS model to obtain a Mel spectrum; convert the Mel spectrum into time-domain waveform data using a vocoder; and a voice output module configured to play the time-domain waveform data.

[0016] Thirdly, this application also provides a wearable smart device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above-described voice reminder methods.

[0017] In one implementation, wearable smart devices include smartwatches.

[0018] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0019] 1. By acquiring and preprocessing the original audio data of the target character, extracting the normalized MFCC feature vector, constructing a voiceprint feature model using a Gaussian mixture model, loading the corresponding model according to user input and converting the format, and combining the TTS model and vocoder to generate time-domain waveform data and play it, the system realizes the voice reminder that simulates a real person, effectively improving the realism and interactivity of the voice reminder and enhancing the user experience.

[0020] 2. Initializing Gaussian mixture model parameters through cluster centers can improve the training efficiency and stability of the Gaussian mixture model; iterative optimization using the expectation-maximization algorithm can enhance the model's accuracy and robustness; finally, compressing the model reduces storage and computational resource consumption, making the voiceprint feature model more suitable for resource-constrained wearable smart devices.

[0021] 3. Encrypting the original audio data ensures data security during transmission, preventing the risk of data leakage and tampering. The encrypted data is sent to the cloud to generate a voiceprint feature vector. Verifying the status code in the second request object ensures the integrity and reliability of data processing; the voiceprint feature vector is only saved when the status code verification passes, further guaranteeing data quality and security. Simultaneously, retrieving the voiceprint feature vector from the cloud eliminates the need for wearable smart devices to perform the voiceprint feature vector generation operation, effectively reducing power consumption and resource usage, and improving the user experience. Attached Figure Description

[0022] The preferred embodiments will now be described in a clear and easy-to-understand manner, in conjunction with the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of the present invention.

[0023] Figure 1 A flowchart of a voice reminder method provided in an embodiment of this application is shown;

[0024] Figure 2 This document illustrates a flowchart of a voiceprint feature model generation method provided in an embodiment of this application.

[0025] Figure 3 The flowchart illustrating an embodiment of this application for obtaining a voiceprint feature vector from the cloud is shown. Detailed Implementation

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0027] To keep the drawings concise, each figure only schematically shows the parts relevant to the invention, and these do not represent the actual structure of the product. Furthermore, to facilitate understanding, in some figures, only one of components with the same structure or function is schematically depicted, or only one is labeled. In this document, "one" not only means "only one," but can also mean "more than one."

[0028] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] In this document, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0030] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] It should be noted that the above embodiments can be freely combined as needed. The above are merely preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0032] In the field of wearable smart devices, traditional reminder methods such as alarm clocks, phone calls, or SMS voice prompts suffer from problems such as mechanicalness, lack of emotion, and poor interactivity, which can easily bore users. Although existing speech synthesis technology can generate natural speech, it is mostly limited to functional voice assistants, lacking personalization and realism, and it is difficult to simulate the voice characteristics of real people such as parents, teachers, and friends, which can easily reduce the user experience.

[0033] Therefore, this application provides a voice reminder method for wearable smart devices. By performing audio acquisition, voiceprint feature extraction, voiceprint modeling, and voiceprint synthesis output on the target character, the method can simulate the voice reminder of a real person, thereby improving the user experience.

[0034] The following explanation is based on the accompanying diagram:

[0035] Reference Appendix Figure 1 The diagram illustrates a flowchart of a voice reminder method provided in an embodiment of this application. This method is applied to wearable smart devices, such as… Figure 1 As shown, it includes:

[0036] S100: Obtain the raw audio data of the target character and preprocess the raw audio data.

[0037] S110 processes each frame of preprocessed audio data to obtain a normalized MFCC feature vector.

[0038] S120 uses a Gaussian mixture model to process all normalized MFCC feature vectors to obtain a voiceprint feature model.

[0039] S130: Load the corresponding voiceprint feature model based on the target role ID carried in the reminder information input by the user.

[0040] S140: Convert the format of the voiceprint feature vector in the voiceprint feature model corresponding to the target character ID.

[0041] S150, input the text ID corresponding to the reminder information and the voiceprint feature vector after format conversion into the TTS model to obtain the Mel spectrum.

[0042] S160 converts the Mel spectrum into time-domain waveform data using a vocoder and then plays the time-domain waveform data.

[0043] Raw audio data of a target person can be acquired through microphones, audio acquisition circuits, or other modules or units with audio acquisition capabilities installed on wearable smart devices. Microphones can include condenser microphones, dynamic microphones, aluminum microphones, and high-sensitivity microphone arrays, either built into or external to the wearable smart device. This application uses a high-sensitivity microphone as an example. High-sensitivity microphone arrays have functions such as bone conduction detection and resistance to environmental noise, which can improve the accuracy of audio acquisition. The target person can be selected according to user needs, such as parents, doctors, teachers, and friends.

[0044] Configure the microphone's audio parameters, including but not limited to sampling rate, channels, audio encoding format (or audio data encoding format), and calculate the minimum buffer size. For example, the sampling rate can be set to 16000Hz, the channels can be configured as mono, and the audio encoding format can be configured as 16-bit linear pulse code modulation (PCM). Then, create an AudioRecord instance to specify the optimized audio source. After passing the configured audio parameters to the microphone, begin acquiring the target character's raw audio data.

[0045] After obtaining the raw audio data of the target character, the raw audio data can be preprocessed sequentially. The purpose of preprocessing is to convert the raw audio data into frame-by-frame audio data. A pre-compiled deep learning model is loaded; this model can be a TFLite (TensorFlow Lite) model. Then, each frame of preprocessed audio data is processed using this deep learning model to obtain the Mel-Frequency Cepstral Coefficients (MFCC) feature vector (hereinafter referred to as MFCC feature vector) corresponding to each frame of audio data. The MFCC feature vector corresponding to each frame of audio data is normalized, and then a Gaussian mixture model is trained based on all the MFCC feature vectors. The trained Gaussian mixture model is then compressed to obtain the voiceprint feature model, which can finally be saved locally. The voiceprint feature model corresponds to the target character; if raw audio data of multiple target characters is collected, multiple corresponding voiceprint feature models will be obtained.

[0046] Based on the user-inputted reminder information, the target role ID is determined. This reminder information can be a voice prompt or text content provided by the user. Then, based on the target role ID, the corresponding voiceprint feature model is loaded locally. Further, a Text-to-Speech (TTS) model (hereinafter referred to as the TTS model) and a voiceprint adapter are loaded. The voiceprint adapter converts the voiceprint feature vector in the voiceprint feature model corresponding to the target role ID into a format suitable for driving the TTS model. The text content corresponding to the reminder information is converted into the ID sequence required by the TTS model, i.e., converted into the corresponding text ID. Then, the text ID and the format-converted voiceprint feature vector are input into the TTS model, and model inference is performed to obtain the Mel spectrum. Further, a vocoder can be called to convert the Mel spectrum into a linear spectrum (melToLinear), followed by multiple (e.g., 50) Griffin-Lim iterative phase reconstructions, and finally, time-domain waveform data (time-domain audio waveform data) is obtained through inverse short-time Fourier transform (inverseSTFT). Finally, the time-domain waveform data can be played through voice output devices such as speakers and bone conduction headphones. The specific text content in the played time-domain waveform data can also be customized according to needs. For example, if a user wants the target character to remind them with "It's already 10 PM, time to sleep," this content can be added to the reminder message entered by the user. Thus, during the generation of the time-domain waveform data, the customized content "It's already 10 PM, time to sleep" in the reminder message can be automatically converted into a text ID and input into the TTS model. Therefore, the content played in the final generated time-domain waveform data is the customized content "It's already 10 PM, time to sleep."

[0047] This application embodiment acquires and preprocesses the original audio data of the target character, extracts the normalized MFCC feature vector, constructs a voiceprint feature model using a Gaussian mixture model, loads the corresponding model according to user input and converts the format, combines a TTS model and a vocoder to generate time-domain waveform data and plays it, thereby realizing a voice reminder that simulates a real person, effectively improving the realism and interactivity of the voice reminder and enhancing the user experience.

[0048] In some embodiments of this application, the original audio data is preprocessed, specifically including: filtering the original audio data; detecting human voices in the filtered original audio data; and dividing the original audio data after human voice detection into continuous frame audio data according to a fixed duration.

[0049] After obtaining the raw audio data, it can be filtered, specifically pre-emphasis filtering, to boost high-frequency components and compensate for the attenuation of high-frequency parts in the audio data. Then, using the WebRTC VAD library integrated with native libraries, human voice detection is performed on the filtered raw audio data for noise reduction, ensuring that only human voice data remains in the original audio data. Finally, the raw audio data after human voice detection is segmented into continuous frames of audio data at fixed intervals (e.g., 20ms).

[0050] In some embodiments of this application, each frame of preprocessed audio data is processed to obtain a normalized MFCC feature vector. Specifically, this includes: normalizing each frame of preprocessed audio data; inputting all normalized frame audio data into a pre-loaded deep learning model, performing model inference, and obtaining an MFCC feature vector; and normalizing the MFCC feature vector to obtain a normalized MFCC feature vector.

[0051] After obtaining the preprocessed audio data, a pre-compiled deep learning model is loaded. Then, each frame of preprocessed audio data is input into a buffer and normalized. This normalized frame is then input into the deep learning model for inference, yielding MFCC feature vectors. Normalizing these MFCC feature vectors produces normalized MFCC feature vectors. Finally, the normalized MFCC feature vectors undergo Gaussian mixture model processing, speaker signature model loading, format conversion, Mel spectrum generation, and time-domain waveform data generation. The resulting time-domain waveform data is then played back.

[0052] Reference Appendix Figure 2 This illustrates a flowchart of a voiceprint feature model generation method provided in an embodiment of this application. Figure 2 As shown, it includes:

[0053] S200 initializes the model parameters of the Gaussian mixture model.

[0054] S210 uses a clustering algorithm to cluster all normalized MFCC feature vectors and sets the cluster centers as the initial mean of the model parameters.

[0055] S220: The model parameters are iterated a preset number of times using the expectation-maximization algorithm to obtain the trained Gaussian mixture model.

[0056] S230 compresses the trained Gaussian mixture model to obtain the voiceprint feature model.

[0057] First, determine the number of Gaussian components in the Gaussian mixture model; for example, 16 Gaussian components can be set. Then, initialize the model parameters, such as the weight array, mean array, and covariance matrix array. Perform initial clustering on all normalized MFCC feature vectors using the K-means clustering algorithm, and then set the cluster centers to the initial mean of the Gaussian mixture model parameters. Iterate the set model parameters using the Expectation Maximization (EM) algorithm a preset number of times to obtain the trained Gaussian mixture model. The specific training process can include an E-step and an M-step. The E-step calculates the posterior probability of each MFCC feature vector belonging to the weight array, mean array, and covariance matrix array, respectively. The M-step updates the weight array, mean array, and covariance matrix array based on the calculated posterior probabilities. Finally, compress the trained Gaussian mixture model to obtain the voiceprint feature model. Further, perform the processes of loading the voiceprint feature model, format conversion, generating Mel spectrum, and generating time-domain waveform data, and then play the obtained time-domain waveform data.

[0058] This application's embodiments initialize Gaussian mixture model parameters through cluster centers, which can improve the training efficiency and stability of the Gaussian mixture model; iterative optimization using the expectation-maximization algorithm can enhance the model's accuracy and robustness; finally, model compression reduces storage and computing resource consumption, making the voiceprint feature model more suitable for resource-constrained wearable smart devices.

[0059] In some embodiments of this application, the method further includes: loading any frame of audio data into a register; accelerating Fourier transform calculations using a multimedia processing library; calculating the energy of the Mel filter bank in parallel according to vector instructions from the multimedia processing library; and accelerating the discrete cosine transform using a lookup table method.

[0060] Any frame of audio data can be loaded into the NEON register, and then the Fourier transform can be accelerated using the multimedia processing library (or Ne10 library). The energy of the Mel filter bank is calculated in parallel using vector instructions (vmulq_f32, vld1q_f32, vaddvq_f32) from the multimedia processing library, and the discrete cosine transform (or DCT transform) is accelerated using a lookup table method. Through these processes, the extraction of normalized MFCC eigenvectors can be accelerated.

[0061] In some embodiments of this application, the method further includes: when the wearable smart device is charging or connected to a non-metered network, periodically controlling the wearable smart device to perform resource-intensive operations, thereby reducing the power consumption of the wearable smart device and extending its lifespan. The specific implementation process is as follows: First, obtain a system task scheduling service instance using `getSystemService(JOB_SCHEDULER_SERVICE)`. Then, use `JobInfo.Builder` to create system task configuration information, specifically setting the system task identifier, execution component, trigger condition (the trigger condition is that the wearable smart device is charging or connected to a non-metered network), and execution period. Finally, call `scheduler.schedule(job)` to submit the system task to the functional scenario service, which automatically schedules and executes the resource-intensive operation when the trigger condition is met. That is, the system task is used to instruct the functional scenario service, which can periodically check whether the functional scenario service meets the trigger condition based on the system task configuration information, and automatically schedule and execute the resource-intensive operation when the trigger condition is met. The functional scenario service is part of the wearable smart device's system. Therefore, it can be concluded that when a functional scenario service automatically schedules and executes resource-intensive operations when the triggering conditions are met, it is equivalent to when a wearable smart device automatically schedules and executes resource-intensive operations when the triggering conditions are met.

[0062] In some embodiments of this application, the method further includes adjusting the fundamental frequency, speech rate factor, and energy in the format-converted voiceprint feature vector according to the emotion type specified by the user. For example, when the emotion type specified by the user is "happy," the corresponding fundamental frequency, speech rate factor, and energy can be set to 30f, 1.2f, and 0.2f, respectively. Other emotion types can also be set according to user needs, and this application does not limit them.

[0063] Reference Appendix Figure 3 This illustrates a flowchart of an embodiment of this application for obtaining voiceprint feature vectors from the cloud. Figure 3 As shown, it includes:

[0064] S300 encrypts the original audio data before preprocessing using an encryption algorithm, and constructs a first request object based on the encrypted original audio data.

[0065] S310, the first request object is sent to the cloud. The first request object is used to instruct the cloud to generate a voiceprint feature vector based on the encrypted original audio data.

[0066] S320 responds to the second request object sent from the cloud and verifies the status code in the second request object.

[0067] S330, when the verification passes, save the voiceprint feature vector carried in the second request object.

[0068] A gRPC channel (gRPC is an RPC (Remote Procedure Call) framework developed by Google) is established between the wearable smart device and the cloud. Then, a key derived from the wearable smart device's ID is used to encrypt the raw audio data before preprocessing using Advanced Encryption Standard (AES). The encrypted raw audio data, a pre-set session ID, and the sampling rate are encapsulated into a first request object (VoiceprintRequest), which is sent to the cloud via the gRPC channel. The cloud decrypts the first request object and generates a voiceprint feature vector based on the encrypted raw audio data. The process of generating the MFCC feature vector, voiceprint feature model, and voiceprint feature vector in the cloud is the same as that in the wearable smart device, and will not be elaborated further. The cloud can then encapsulate the generated voiceprint feature vector into a second request object (VoiceprintResponse). The wearable smart device responds to the second request object and verifies the status code (status_code) within it. If the verification is successful, the voiceprint feature vector carried in the second request object is extracted.

[0069] Furthermore, a Text-to-Speech (TTS) model (hereinafter referred to as the TTS model) and a voiceprint adapter are loaded. The voiceprint adapter converts the voiceprint feature vector into a format suitable for driving the TTS model. The text content corresponding to the reminder message is converted into the ID sequence required by the TTS model, that is, into the corresponding text ID. Then, the text ID and the format-converted voiceprint feature vector are input into the TTS model and model inference is performed to obtain the Mel spectrum. Further, a vocoder can be called to convert the Mel spectrum into a linear spectrum (melToLinear), and then multiple (e.g., 50 times) Griffin-Lim iterative phase reconstructions are performed. Finally, time-domain waveform data (time-domain audio waveform data) is obtained through inverse short-time Fourier transform (inverseSTFT). Ultimately, the time-domain waveform data can be played through voice output devices such as speakers and bone conduction headphones.

[0070] This application embodiment ensures data security during transmission by encrypting the original audio data, preventing the risk of data leakage and tampering. The encrypted data is sent to the cloud to generate a voiceprint feature vector. By verifying the status code in the second request object, the integrity and reliability of data processing are ensured; the voiceprint feature vector is only saved when the status code verification passes, further guaranteeing data quality and security. Simultaneously, obtaining the voiceprint feature vector from the cloud eliminates the need for wearable smart devices to perform the voiceprint feature vector generation operation, effectively reducing power consumption and resource usage, and improving the user experience.

[0071] This application also provides a voice reminder device for wearable smart devices, including: an audio acquisition module configured to acquire raw audio data of a target character and preprocess the raw audio data; a voiceprint modeling module configured to: process each frame of preprocessed audio data to obtain a normalized MFCC feature vector; process all normalized MFCC feature vectors using a Gaussian mixture model to obtain a voiceprint feature model; a speech synthesis module configured to: load the corresponding voiceprint feature model based on the target character ID carried in the reminder information input by the user; perform format conversion on the voiceprint feature vector in the voiceprint feature model corresponding to the target character ID; input the text ID corresponding to the reminder information and the format-converted voiceprint feature vector into a TTS model to obtain a Mel spectrum; convert the Mel spectrum into time-domain waveform data using a vocoder; and a voice output module configured to play the time-domain waveform data.

[0072] This application embodiment acquires and preprocesses the original audio data of the target character, extracts the normalized MFCC feature vector, constructs a voiceprint feature model using a Gaussian mixture model, loads the corresponding model according to user input and converts the format, combines a TTS model and a vocoder to generate time-domain waveform data and plays it, thereby realizing a voice reminder that simulates a real person, effectively improving the realism and interactivity of the voice reminder and enhancing the user experience.

[0073] This application also provides a wearable smart device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the voice reminder method of any of the above embodiments.

[0074] In some embodiments of this application, wearable smart devices include smartwatches.

[0075] The wearable smart device used in this application embodiment is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0076] It should be noted that the above embodiments can be freely combined as needed. The above are merely preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A voice reminder method, applied to wearable smart devices, characterized in that, include: Obtain the raw audio data of the target character and preprocess the raw audio data; Each frame of preprocessed audio data is processed to obtain a normalized MFCC feature vector. The voiceprint feature model is obtained by processing all normalized MFCC feature vectors using a Gaussian mixture model. Based on the target role ID carried in the reminder information entered by the user, load the corresponding voiceprint feature model; The voiceprint feature vector in the voiceprint feature model corresponding to the target role ID is converted into a new format. The text ID corresponding to the reminder information and the format-converted voiceprint feature vector are input into the TTS model to obtain the Mel spectrum; The Mel spectrum is converted into time-domain waveform data using a vocoder, and the time-domain waveform data is then played back.

2. The voice reminder method according to claim 1, characterized in that, The process of processing all normalized MFCC feature vectors using a Gaussian mixture model to obtain the voiceprint feature model specifically includes: The model parameters of the Gaussian mixture model are initialized; All normalized MFCC feature vectors are clustered using a clustering algorithm, and the cluster centers are set as the initial mean of the model parameters. The Gaussian mixture model is obtained by iterating the set model parameters a preset number of times using the expectation-maximization algorithm. The trained Gaussian mixture model is compressed to obtain the voiceprint feature model.

3. The voice reminder method according to claim 1, characterized in that, The preprocessing of the raw audio data specifically includes: The original audio data is then filtered. Human voice detection is performed on the filtered original audio data; The original audio data, after human voice detection, is divided into continuous frame audio data according to a fixed duration.

4. The voice reminder method according to claim 1, characterized in that, The process of processing each frame of preprocessed audio data to obtain a normalized MFCC feature vector specifically includes: Normalize each frame of preprocessed audio data; The normalized audio data of all frames is input into a pre-loaded deep learning model, and model inference is performed to obtain MFCC feature vectors. The MFCC eigenvectors are normalized to obtain normalized MFCC eigenvectors.

5. The voice reminder method according to claim 4, characterized in that, Also includes: Load any frame of audio data into the register; Accelerate Fourier transform calculations using a multimedia processing library; The energy of the Mel filter bank is calculated in parallel according to the vector instructions of the multimedia processing library; The discrete cosine transform is accelerated by using a lookup table method.

6. The voice reminder method according to claim 1, characterized in that, Also includes: When the wearable smart device is charging or connected to a non-metered network, it is periodically controlled to perform resource-intensive operations.

7. The voice reminder method according to any one of claims 1-6, characterized in that, Also includes: The original audio data before preprocessing is encrypted using an encryption algorithm, and a first request object is constructed based on the encrypted original audio data. The first request object is sent to the cloud, and the first request object is used to instruct the cloud to generate a voiceprint feature vector based on the encrypted original audio data; Respond to the second request object sent from the cloud and verify the status code in the second request object; When the verification passes, the voiceprint feature vector carried in the second request object is saved.

8. The voice reminder method according to any one of claims 1-6, characterized in that, Also includes: Based on the emotion type specified by the user, the fundamental frequency, speech rate factor, and energy in the converted voiceprint feature vector are adjusted.

9. A voice reminder device, applied to wearable smart devices, characterized in that, include: The audio acquisition module is configured to acquire the raw audio data of the target character and preprocess the raw audio data. The voiceprint modeling module is configured to process each frame of preprocessed audio data to obtain a normalized MFCC feature vector. The voiceprint feature model is obtained by processing all normalized MFCC feature vectors using a Gaussian mixture model. The speech synthesis module is configured to: load the corresponding voiceprint feature model based on the target role ID carried in the reminder information input by the user; perform format conversion on the voiceprint feature vector in the voiceprint feature model corresponding to the target role ID; input the text ID corresponding to the reminder information and the format-converted voiceprint feature vector into the TTS model to obtain the Mel spectrum; and convert the Mel spectrum into time-domain waveform data through a vocoder. The voice output module is configured to play the time-domain waveform data.

10. A wearable smart device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the voice reminder method according to any one of claims 1-8.

Images