Method for generating wake-up voice, generating device and electronic equipment

Abstract

The application provides a method and device for generating a wake-up voice and an electronic device, wherein the method for generating a wake-up voice comprises the following steps: obtaining a wake-up word text and a reference voice; inputting the wake-up word text and the reference voice into a target model to generate a target voice corresponding to the wake-up word text; and performing feature enhancement on the target voice according to a first preset algorithm to generate a wake-up voice.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, and electronic device for generating wake-up voice. Background Technology

[0002] In related technologies, during voice interaction between electronic devices and users, the electronic device is woken up by a wake-up voice. Typically, this requires first training a wake-up voice model to improve the recognition accuracy of wake-up voices. Then, the wake-up voice model is deployed into the electronic device to enable its voice wake-up function. However, the training process for the wake-up voice model requires collecting a large number of diverse wake-up voices to ensure that the trained model can recognize wake-up voices from different users. This, in turn, ensures the flexible response of the electronic device during voice interaction after the model is deployed. Therefore, improving the diversity of wake-up voices has become a pressing technical problem to be solved. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art or related technologies.

[0004] Therefore, the first aspect of this application proposes a method for generating wake-up speech.

[0005] The second aspect of this application proposes a device for generating wake-up speech.

[0006] A third aspect of this application proposes a wake-up voice generation device.

[0007] The fourth aspect of this application proposes an electronic device.

[0008] The fifth aspect of this application proposes a readable storage medium.

[0009] The sixth aspect of this application proposes a computer program product.

[0010] In view of this, the first aspect of this application proposes a method for generating wake-up speech, comprising: acquiring wake-up word text and reference speech; inputting the wake-up word text and reference speech into a target model to generate target speech corresponding to the wake-up word text; and performing feature enhancement on the target speech according to a first preset algorithm to generate wake-up speech.

[0011] The wake-up voice generation method provided in this application can generate diverse wake-up voices, which can be used to wake up electronic devices. Furthermore, the generated diverse wake-up voices can be used to train a wake-up voice model, ensuring that the trained wake-up voice model can accurately recognize wake-up voices issued by different users. After the wake-up voice model is deployed in an electronic device, it ensures the flexible response of the electronic device during voice interaction. Specifically, during user interaction with an electronic device via voice, the user first speaks a sentence. After receiving the speech, the electronic device recognizes the speech. If it confirms that the received speech is a wake-up voice capable of waking up the electronic device, the electronic device can then enter the interaction process with the user and further operate based on other speech issued by the user.

[0012] Understandably, since different users have different timbres and rhythms, electronic devices need to be able to accurately identify whether the voice emitted by the user is a wake-up voice. This requires the wake-up voice model deployed on the electronic device to collect a large number of wake-up voice samples with different timbres and rhythms during the training process. In other words, it is necessary to maximize the diversity of wake-up voice samples collected during model training. In this way, the resulting wake-up voice model can accurately identify whether the voice corresponds to the wake-up voice when it receives voices with different timbres and rhythms emitted by different users, thereby ensuring the flexible response of the electronic device in the process of interacting with the user.

[0013] Furthermore, the wake-up speech generation method provided in this application can generate a large number of wake-up speech samples with different timbres and tones for collection during the training of the wake-up speech model. This ensures both the diversity and efficiency of wake-up speech generation.

[0014] Specifically, the process begins with acquiring the wake-up word text and reference speech. The wake-up word text is the text data corresponding to the wake-up speech. The reference speech can be any audio clip; it doesn't need to correspond to the wake-up word text. Therefore, acquiring the reference speech is relatively simple. It doesn't require recording a large amount of audio for a specific wake-up word; any audio information can be retrieved from an existing database as a reference, effectively reducing the cost of acquiring it. Furthermore, since the existing database contains a large amount of different audio information from different people, compared to finding staff to record wake-up speech for a specific wake-up word, it also solves the problem of reference speech diversity, thereby increasing the diversity of the generated wake-up speech.

[0015] Furthermore, after obtaining the wake-up word text and reference speech, these can be input into the target model for synthesis, thereby generating the target speech corresponding to the wake-up word text. It is understandable that by inputting the wake-up word text and a large number of reference speeches into the target model, the target speech output by the model can express the content of the wake-up word text with a large number of different timbres and rhythms, thus effectively improving the diversity of the target speech.

[0016] It should be noted that the target speech output by the target model can express the content of the wake word text with a large number of different timbres and rhythms. Therefore, this target speech can also be used as wake-up speech for the wake-up speech model to collect during the training process. Compared with finding staff to record wake words, this can effectively improve the diversity of wake-up speech and ensure the flexible response of electronic devices.

[0017] Furthermore, after generating the target speech corresponding to the wake-up word text using the target model, feature enhancement can be performed on the target speech according to the first preset algorithm. The feature-enhanced target speech is the desired wake-up speech. By enhancing the features of the target speech, further diversification processing can be performed on the target speech, thereby further improving the diversity of the generated wake-up speech. For example, the target speech can be enhanced in terms of vocal tract characteristics, that is, adjusting the vocal tract characteristics of the target speech, or enhancing the speed characteristics of the target speech, or enhancing the frequency domain masking characteristics of the target speech, or enhancing the noise characteristics of the target speech.

[0018] The wake-up speech generation method provided in this application first inputs the wake-up word text and reference speech into a target model for speech synthesis to generate target speech corresponding to the wake-up word text. Then, the target speech is enhanced using a first preset algorithm to finally generate the required wake-up speech. Since the reference speech does not need to correspond to the content of the wake-up word text, it can be obtained from an existing database without the need for recording. This reduces the cost of wake-up speech generation and ensures the diversity of the obtained reference speech, thereby effectively improving the diversity of the generated wake-up speech. Furthermore, by performing feature enhancement on the target speech, it can be further diversified, thus further improving the diversity of the generated wake-up speech. This ensures the flexible response of the electronic device.

[0019] According to a second aspect of this application, a wake-up speech generation apparatus is proposed, comprising: an acquisition unit for acquiring a wake-up word text and a reference speech; a generation unit for inputting the wake-up word text and the reference speech into a target model to generate a target speech corresponding to the wake-up word text; and a feature enhancement method for performing feature enhancement on the target speech according to a first preset algorithm to generate the wake-up speech.

[0020] According to a third aspect of this application, a wake-up speech generation apparatus is proposed, comprising: a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, perform the following steps: acquiring a wake-up word text and a reference speech; inputting the wake-up word text and the reference speech into a target model to generate a target speech corresponding to the wake-up word text; and performing feature enhancement on the target speech according to a first preset algorithm to generate the wake-up speech.

[0021] According to a fourth aspect of this application, an electronic device is proposed, including a wake-up voice generation device according to any of the above-described technical solutions.

[0022] According to the fifth aspect of this application, a readable storage medium is proposed, on which a program or instruction is stored, which, when executed by a processor, performs the following steps: acquiring a wake-up word text and a reference speech; inputting the wake-up word text and the reference speech into a target model to generate a target speech corresponding to the wake-up word text; and performing feature enhancement on the target speech according to a first preset algorithm to generate a wake-up speech.

[0023] According to a sixth aspect of this application, a computer program product is proposed, comprising a computer program or instructions, which, when executed by a processor, perform the following steps: acquiring a wake-up word text and a reference speech; inputting the wake-up word text and the reference speech into a target model to generate a target speech corresponding to the wake-up word text; and performing feature enhancement on the target speech according to a first preset algorithm to generate a wake-up speech.

[0024] Additional aspects and advantages of this application will become apparent in the following description or may be learned by practice of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0026] Figure 1 A flowchart illustrating a method for generating wake-up voice according to an embodiment of this application is shown;

[0027] Figure 2 A flowchart illustrating a method for generating wake-up voice according to another embodiment of this application is shown;

[0028] Figure 3 A flowchart illustrating a method for generating wake-up voice according to another embodiment of this application is shown;

[0029] Figure 4 A flowchart illustrating a method for generating wake-up voice according to another embodiment of this application is shown;

[0030] Figure 5 A flowchart illustrating the preset model training in a wake-up voice generation method provided in one embodiment of this application is shown.

[0031] Figure 6 A flowchart illustrating a method for generating target speech according to an embodiment of this application is shown;

[0032] Figure 7 A flowchart illustrating the target speech feature enhancement in a wake-up speech generation method provided in one embodiment of this application is shown. Detailed Implementation

[0033] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0034] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.

[0035] The following reference Figures 1 to 7 This application describes a method for generating wake-up voice, an apparatus for generating wake-up voice, a readable storage medium, an electronic device, and a computer program product provided according to some embodiments of the present application.

[0036] like Figure 1 As shown, according to one embodiment of this application, a method for generating wake-up speech is proposed, comprising:

[0037] Step S102: Obtain the wake word text and reference speech;

[0038] Step S104: Input the wake-up word text and reference speech into the target model to generate target speech corresponding to the wake-up word text;

[0039] Step S106: According to the first preset algorithm, feature enhancement is performed on the target speech to generate wake-up speech.

[0040] The wake-up voice generation method provided in this application can generate diverse wake-up voices, which can be used to wake up electronic devices. Furthermore, the generated diverse wake-up voices can be used to train a wake-up voice model, ensuring that the trained wake-up voice model can accurately recognize wake-up voices issued by different users. After the wake-up voice model is deployed in an electronic device, it ensures the flexible response of the electronic device during voice interaction. Specifically, during user interaction with an electronic device via voice, the user first speaks a sentence. After receiving the speech, the electronic device recognizes the speech. If it confirms that the received speech is a wake-up voice capable of waking up the electronic device, the electronic device can then enter the interaction process with the user and further operate based on other speech issued by the user.

[0041] Understandably, since different users have different timbres and rhythms, electronic devices need to be able to accurately identify whether the voice emitted by the user is a wake-up voice. This requires the wake-up voice model deployed on the electronic device to collect a large number of wake-up voice samples with different timbres and rhythms during the training process. In other words, it is necessary to maximize the diversity of wake-up voice samples collected during model training. In this way, the resulting wake-up voice model can accurately identify whether the voice corresponds to the wake-up voice when it receives voices with different timbres and rhythms emitted by different users, thereby ensuring the flexible response of the electronic device in the process of interacting with the user.

[0042] Furthermore, the wake-up speech generation method provided in this application can generate a large number of wake-up speech samples with different timbres and tones for collection during the training of the wake-up speech model. This ensures both the diversity and efficiency of wake-up speech generation.

[0043] Specifically, the process begins with acquiring the wake-up word text and reference speech. The wake-up word text is the text data corresponding to the wake-up speech. The reference speech can be any audio clip; it doesn't need to correspond to the wake-up word text. Therefore, acquiring the reference speech is relatively simple. It doesn't require recording a large amount of audio for a specific wake-up word; any audio information can be retrieved from an existing database as a reference, effectively reducing the cost of acquiring it. Furthermore, since the existing database contains a large amount of different audio information from different people, compared to finding staff to record wake-up speech for a specific wake-up word, it also solves the problem of reference speech diversity, thereby increasing the diversity of the generated wake-up speech.

[0044] Furthermore, after obtaining the wake-up word text and reference speech, these can be input into the target model for synthesis, thereby generating the target speech corresponding to the wake-up word text. It is understandable that by inputting the wake-up word text and a large number of reference speeches into the target model, the target speech output by the model can express the content of the wake-up word text with a large number of different timbres and rhythms, thus effectively improving the diversity of the target speech.

[0045] It should be noted that the target speech output by the target model can express the content of the wake word text with a large number of different timbres and rhythms. Therefore, the target speech can also be used as wake-up speech for electronic devices to collect. Compared with having staff record wake words, this can effectively improve the diversity of wake-up speech and ensure the flexible response of electronic devices.

[0046] Furthermore, after generating the target speech corresponding to the wake-up word text using the target model, feature enhancement can be performed on the target speech according to the first preset algorithm. The feature-enhanced target speech is the desired wake-up speech. By enhancing the features of the target speech, further diversification processing can be performed on the target speech, thereby further improving the diversity of the generated wake-up speech. For example, such as... Figure 7 As shown, the target speech can be enhanced in terms of its vocal tract characteristics, that is, by adjusting the vocal tract characteristics of the target speech, enhancing the speed characteristics of the target speech, enhancing the frequency domain masking characteristics of the target speech, or enhancing the noise characteristics of the target speech.

[0047] The wake-up speech generation method provided in this application first inputs the wake-up word text and reference speech into a target model for speech synthesis to generate target speech corresponding to the wake-up word text. Then, the target speech is enhanced using a first preset algorithm to finally generate the required wake-up speech. Since the reference speech does not need to correspond to the content of the wake-up word text, it can be obtained from an existing database without the need for recording. This reduces the cost of wake-up speech generation and ensures the diversity of the obtained reference speech, thereby effectively improving the diversity of the generated wake-up speech. Furthermore, by performing feature enhancement on the target speech, it can be further diversified, thus further improving the diversity of the generated wake-up speech. This ensures the flexible response of the electronic device.

[0048] According to one embodiment of this application, such as Figure 2 As shown, a method for generating wake-up speech is proposed, including:

[0049] Step S202: Obtain the wake-up word text and reference speech;

[0050] Step S204: Input the wake-up word text and reference speech into the target model to generate the target speech corresponding to the wake-up word text;

[0051] Step S206: Based on multiple first preset algorithms, feature enhancement is performed on different features of the target speech to generate multiple feature-enhanced target speech;

[0052] Step S208: Mix the multiple target speech samples after feature enhancement to generate a wake-up speech.

[0053] In this embodiment, such as Figure 7 As shown, there can be multiple first preset algorithms, and each first preset algorithm can enhance different features of the target speech. Specifically, the first preset algorithms may include the VTLP (Vocal Tract Length Perturbation) algorithm and the SpecAugment algorithm, etc. By using multiple first preset algorithms, feature enhancement of different features of the target speech can be achieved, thereby further improving the diversity of the target speech.

[0054] Furthermore, after generating multiple feature-enhanced target speech through multiple first preset algorithms, the multiple feature-enhanced target speech are mixed to generate a wake-up speech, which has multiple enhanced speech features, thereby further improving the diversity of the wake-up speech.

[0055] According to one embodiment of this application, such as Figure 3 As shown, a method for generating wake-up speech is proposed, including:

[0056] Step S302: Obtain text data and corresponding voice data;

[0057] Step S304: Train the preset model based on text data and speech data to generate the target model;

[0058] Step S306: Obtain the wake-up word text and reference speech;

[0059] Step S308: Input the wake-up word text and reference speech into the target model to generate target speech corresponding to the wake-up word text;

[0060] Step S310: Based on multiple first preset algorithms, feature enhancement is performed on different features of the target speech to generate multiple feature-enhanced target speech;

[0061] Step S312: Mix the multiple target speech samples after feature enhancement to generate a wake-up speech.

[0062] In this embodiment, before acquiring the wake word text and reference speech, model training is first required to obtain the target model, thereby ensuring that the target model can accurately fuse the wake word text and reference speech, that is, to express the wake word text through different timbre and rhythm features.

[0063] Specifically, first, text data is acquired, which can be the wake word text or other text data. Then, corresponding speech data is acquired, meaning the content expressed by the speech data corresponds to the content expressed by the text data. It can be understood that the correspondence between text and speech data can also include a timeline alignment between the two, further ensuring that the output speech during training clearly expresses the content corresponding to the text data.

[0064] Furthermore, text data and corresponding speech data are input into a preset model to train the preset model. The preset model after training is the target model required for synthesizing speech.

[0065] The preset model can be an end-to-end (tacotron) speech synthesis model, or the target model can be an acoustic model (fastspeech).

[0066] According to one embodiment of this application, such as Figure 4 As shown, a method for generating wake-up speech is proposed, including:

[0067] Step S402: Obtain text data and corresponding voice data;

[0068] Step S404: Input text data and voice data into a preset model to generate voice output data;

[0069] Step S406: Determine the training loss value based on the speech output data and speech data;

[0070] Step S408: Update the parameters of the preset model based on the training loss value;

[0071] Step S410: Obtain the wake word text and reference speech;

[0072] Step S412: Input the wake-up word text and reference speech into the target model to generate the target speech corresponding to the wake-up word text;

[0073] Step S414: Based on multiple first preset algorithms, feature enhancement is performed on different features of the target speech to generate multiple feature-enhanced target speech.

[0074] Step S416: Mix the multiple target speech samples after feature enhancement to generate a wake-up speech.

[0075] In this embodiment, for training the preset model, firstly, text data and corresponding speech data can be input into the preset model to generate speech output data. This is speech reconstruction, which involves fusing the text data and speech data to reconstruct the speech.

[0076] Furthermore, the training loss value is determined based on the speech data and the speech output data output by the preset model. This involves comparing the speech output data with the speech data to obtain the training loss value for the speech data during model training. Next, based on this training loss value, the parameters of the preset model are updated. Then, the text data and corresponding speech data are input into the parameter-updated preset model again to generate speech output data. The training loss value is then determined again, and the parameters of the preset model are updated again based on the training loss value. This process is repeated until the obtained training loss value is less than the preset loss value, indicating that training is complete. The preset model after training is the target model required for speech fusion.

[0077] In some embodiments, optionally, such as Figure 5 As shown, text data and speech data are input into a preset model to generate speech output data, including: the text encoder of the preset model encodes the text data to generate text feature vectors; the speech encoder of the preset model encodes the speech data to generate speech feature vectors; and the decoder of the preset model decodes the text feature vectors and speech feature vectors to generate speech output data.

[0078] Specifically, the preset model may include a text encoder and a speech encoder. The process of inputting text data and speech data involves inputting the text data into the text encoder of the preset model, whereby the text encoder encodes the text data to generate a text feature vector. Correspondingly, the speech data is input into the speech encoder of the preset model to generate a speech feature vector.

[0079] Furthermore, the preset model also includes a decoder. After the text feature vectors and speech feature vectors are generated, they can be simultaneously input into the decoder of the preset model for decoding. This fuses the text feature vectors and speech feature vectors and decodes them into speech output data.

[0080] In some embodiments, the speech encoder may optionally include a speaker encoder and a prosody encoder. The speech encoder, using a preset model, encodes the speech data to generate a speech feature vector, including: encoding the speech data using the speaker encoder to generate a speaker feature vector; encoding the speech data using the prosody encoder to generate a prosodic feature vector; and concatenating the speaker feature vector and the prosodic feature vector to generate a speech feature vector.

[0081] In this embodiment, the speech encoder of the preset model may include a speaker encoder and a prosodic encoder. After inputting speech data into the speaker encoder, a speaker feature vector is generated, representing the speaker characteristics of the speech data, i.e., the characteristics of the speaker. Based on the speaker feature vector, speech output data with different speaker characteristics can be obtained after speech fusion. Correspondingly, after inputting speech data into the prosodic encoder, a prosodic feature vector is generated, and based on the prosodic feature vector, speech output data with different prosodic features can be obtained after speech fusion.

[0082] During the training phase of the preset model, by setting the encoder to a speaker encoder and a prosodic encoder, the generated speech output data can have different speaker features and prosodic features included in the speech data, thereby further improving the feature diversity of the generated speech output data. As a result, after the preset model is trained, the target model generates a variety of wake-up voices.

[0083] Furthermore, after generating the speaker feature vector and prosodic feature vector, the speaker feature vector and prosodic feature vector can be concatenated to generate a speech feature vector.

[0084] In some embodiments, optionally, before the step of the decoder of the preset model decoding the text feature vector and the speech feature vector to generate speech output data, the method includes: the normalization layer of the preset model normalizing the speech feature vector.

[0085] In this embodiment, the preset model further includes a normalization layer connected between the encoder and decoder. After generating text feature vectors and speech feature vectors, the speaker feature vector and prosodic feature vector are first concatenated to generate speech feature vectors. Then, the speech feature vectors are input into the normalization layer for normalization, thereby making the data distribution of the speech feature vectors more stable and ensuring the stability and generalization ability of the preset model during training.

[0086] Furthermore, before the step of decoding the text feature vector and speech feature vector through the decoder of the preset model to generate speech output data, the generation method also includes: adjusting the speech feature vector through the regulator of the preset model.

[0087] Specifically, before inputting the text feature vector and speech feature vector into the decoder, the speech feature vector can be adjusted using a pre-defined model regulator. By adjusting the speech feature vector, the diversity of the generated speech output data can be further improved, thereby enhancing the diversity of the target speech generated by the target model after model training is complete.

[0088] Furthermore, the step of adjusting the speech feature vector through the regulator of the preset model includes: the regulator adjusts the reference speech features according to the preset formula X=s×x+β; where X is the adjusted speech feature vector, x is the speech feature vector before adjustment, s is the scaling factor of the speech feature vector, and β is the offset factor of the speech feature vector. During the training phase of the preset model, s=1 and β=0.

[0089] Specifically, the regulator in the preset model adjusts the reference speech feature vector using scaling and offset factors. The reference speech features can be adjusted according to the preset formula X = s × x + β. Here, X is the adjusted speech feature vector, x is the original speech feature vector, s is the scaling factor, and β is the offset factor. During the adjustment process, by inputting the values ​​of the scaling factor s and the offset factor β, the speech feature vector is scaled and offset, resulting in more diverse speech feature vectors and thus improving the diversity of the decoded speech output data.

[0090] After the speech feature vector is normalized, its mean and variance are obtained. The normalized speech feature vector can be expressed by the following formula: x = x' - μ / σ. Here, x' is the speech feature vector before normalization, and x is the normalized speech feature vector, i.e., the speech feature vector before adjustment. Furthermore, the above-mentioned formula can also be expressed as X = s × (x' - μ) / σ + β. Here, μ is the mean of the speech feature vector, and σ is the variance of the speech feature vector.

[0091] It is understandable that, such as Figure 5 As shown, during the training of the preset model, the speech feature vector is normalized in the normalization layer and then enters the regulator of the preset model. At this point, the scaling factor can be set to 1 and the offset factor can be set to 0. This ensures that the speech output data generated during training will not be offset, thus ensuring the accuracy of the preset model training.

[0092] In some embodiments, such as Figure 6As shown, further, the wake-up word text and reference speech are input into the target model to generate target speech corresponding to the wake-up word text, including: the text encoder of the target model encodes the wake-up word text to generate a wake-up word text feature vector; the speech encoder of the target model encodes the reference speech to generate a reference speech feature vector; the regulator of the target model adjusts the reference speech feature vector; and the decoder of the preset model decodes the wake-up word text feature vector and the adjusted reference speech feature vector to generate the target speech.

[0093] In this embodiment, after the target model is trained, the wake word text and reference speech can be input into the target model for speech fusion to obtain target speech with different timbres and rhythms corresponding to the wake word text.

[0094] Specifically, similar to the training process of the preset model, the process of inputting the wake-up word text and the reference speech involves feeding the wake-up word text into the text encoder of the preset model, whereby the text encoder encodes the wake-up word text to generate a wake-up word text feature vector. Correspondingly, the reference speech is input into the speech encoder of the preset model to generate a reference speech feature vector.

[0095] Furthermore, before inputting the wake-up word text feature vector and the reference speech feature vector into the decoder, the reference speech feature vector can be adjusted by the regulator of the target model. By adjusting the reference speech feature vector, the diversity of the reference speech can be further improved, thereby further enhancing the diversity of the wake-up speech after it is generated.

[0096] Furthermore, the wake-up word text feature vector and the reference speech feature vector are decoded using the decoder of the target model to obtain the fused target speech.

[0097] Furthermore, the regulator of the target model adjusts the reference speech feature vector, including: the regulator of the target model adjusts the reference speech features according to a preset formula X1=s1×x1+β1; wherein, X1 is the adjusted reference speech feature vector, x1 is the reference speech feature vector before adjustment, s1 is the scaling factor of the reference speech feature vector, and β1 is the offset factor of the reference speech feature vector.

[0098] Specifically, the regulator of the target model adjusts the reference speech feature vector through scaling and offset factors. Specifically, the reference speech features can be adjusted according to the preset formula X1 = s1 × x1 + β1. Here, X1 is the adjusted reference speech feature vector, x1 is the original reference speech feature vector, s1 is the scaling factor, and β1 is the offset factor. During the adjustment process, by inputting the values ​​of the scaling factor s1 and the offset factor β1, the reference speech feature vector is scaled and offset, thereby obtaining reference speech feature vectors with more diverse features, thus improving the diversity of the decoded target speech.

[0099] After the reference speech feature vector is normalized, its mean and variance can be obtained. The normalized reference speech feature vector can be expressed by the following formula: x = x' - μ / σ. Here, x' is the reference speech feature vector before normalization, and x is the reference speech feature vector after normalization, i.e., the reference speech feature vector before adjustment. Furthermore, the above preset formula can also be expressed as X = s × (x' - μ) / σ + β. Here, μ is the mean of the reference speech feature vector, and σ is the variance of the reference speech feature vector.

[0100] Furthermore, the speech encoder of the target model encodes the reference speech to generate a reference speech feature vector, including: the speaker encoder of the target model encodes the reference speech to generate a speaker feature vector; the prosody encoder of the target model encodes the reference speech to generate a prosodic feature vector; and the speaker feature vector and the prosodic feature vector are concatenated to generate a reference speech feature vector.

[0101] Specifically, the speech encoder of the target model can include a speaker encoder and a prosodic encoder. After inputting the reference speech into the speaker encoder of the target model, a speaker feature vector of the reference speech can be generated. This speaker feature vector represents the speaker features of the reference speech, that is, the characteristics of the speaker. Based on the speaker feature vector, after speech fusion, target speech with different speaker features can be obtained. Correspondingly, after inputting the reference speech into the prosodic encoder of the target model, a prosodic feature vector of the reference speech can be generated. Based on the prosodic feature vector, after speech fusion, target speech with different prosodic features can be obtained.

[0102] By setting the encoder of the target model to a speaker encoder and a prosodic encoder, the generated target speech can have different speaker features and prosodic features included in the reference speech, thereby further improving the feature diversity of the generated target speech and thus improving the diversity of the generated wake-up speech.

[0103] Furthermore, after generating the speaker feature vector and prosodic feature vector, the speaker feature vector and prosodic feature vector can be concatenated to generate a reference speech feature vector.

[0104] According to a second aspect of this application, a wake-up speech generation apparatus is proposed, comprising: an acquisition unit for acquiring a wake-up word text and a reference speech; a generation unit for inputting the wake-up word text and the reference speech into a target model to generate a target speech corresponding to the wake-up word text; and a feature enhancement method for performing feature enhancement on the target speech according to a first preset algorithm to generate the wake-up speech.

[0105] The wake-up speech generation device provided in this application first inputs the wake-up word text and reference speech into a target model for speech synthesis to generate target speech corresponding to the wake-up word text. Then, it further enhances the target speech using a first preset algorithm to finally generate the required wake-up speech. Since the reference speech does not need to correspond to the content of the wake-up word text, it can be obtained from an existing database without the need for recording. This reduces the cost of wake-up speech generation and ensures the diversity of the obtained reference speech, thereby effectively improving the diversity of the generated wake-up speech. Furthermore, by performing feature enhancement on the target speech, it can be further diversified, thus further improving the diversity of the generated wake-up speech. This ensures the flexible response of the electronic device.

[0106] Furthermore, the generation unit is specifically used to perform feature enhancement on different features of the target speech according to multiple first preset algorithms to generate multiple feature-enhanced target speech; and to mix the multiple feature-enhanced target speech according to a second preset algorithm to generate wake-up speech.

[0107] Furthermore, the acquisition unit is also used to acquire text data and speech data corresponding to the text data before the steps of acquiring the wake-up word text and the reference speech.

[0108] The wake-up voice generation device also includes a training unit, which is used to train a preset model based on text data and voice data to generate a target model.

[0109] Furthermore, the training unit is specifically used to input text data and speech data into a preset model to generate speech output data; determine the training loss value based on the speech output data and speech data; and update the parameters of the preset model based on the training loss value.

[0110] Furthermore, the training unit is specifically used to encode text data using a text encoder of a preset model to generate text feature vectors; to encode speech data using a speech encoder of a preset model to generate speech feature vectors; and to decode the text feature vectors and speech feature vectors using a decoder of a preset model to generate speech output data.

[0111] Furthermore, the speech encoder includes a speaker encoder and a prosodic encoder. The training unit is specifically used to encode the speech data through the speaker encoder to generate a speaker feature vector; to encode the speech data through the prosodic encoder to generate a prosodic feature vector; and to connect the speaker feature vector and the prosodic feature vector to generate a speech feature vector.

[0112] Furthermore, before the step of decoding the text feature vector and speech feature vector by the decoder of the preset model to generate speech output data, the training unit is also used to normalize the speech feature vector through the normalization layer of the preset model.

[0113] Furthermore, before the step of decoding the text feature vector and speech feature vector through the decoder of the preset model to generate speech output data, the training unit is also used to adjust the speech feature vector through the regulator of the preset model.

[0114] Furthermore, the training unit is specifically used to adjust the reference speech features according to the preset formula X = s × x + β through the regulator; where X is the adjusted speech feature vector, x is the speech feature vector before adjustment, s is the scaling factor of the speech feature vector, and β is the offset factor of the speech feature vector. During the training phase of the preset model, s = 1 and β = 0.

[0115] Furthermore, the generation unit is specifically used for the text encoder of the target model to encode the wake-up word text and generate a wake-up word text feature vector; the speech encoder of the target model to encode the reference speech and generate a reference speech feature vector; the regulator of the target model to regulate the reference speech feature vector; and the decoder of the preset model to decode the wake-up word text feature vector and the regulated reference speech feature vector to generate the target speech.

[0116] Furthermore, the generation unit is specifically used for: the regulator of the target model to adjust the reference speech features according to the preset formula X1=s1×x1+β1; where X1 is the adjusted reference speech feature vector, x1 is the reference speech feature vector before adjustment, s1 is the scaling factor of the reference speech feature vector, and β1 is the offset factor of the reference speech feature vector.

[0117] Furthermore, the generation unit is specifically used for: the speaker encoder of the target model encoding the reference speech to generate a speaker feature vector; the prosodic encoder of the target model encoding the reference speech to generate a prosodic feature vector; and concatenating the speaker feature vector and the prosodic feature vector to generate a reference speech feature vector.

[0118] According to a third aspect of this application, a wake-up voice generation apparatus is proposed, comprising: a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and when the program or instructions are executed by the processor, implement the steps of the wake-up voice generation method as described in any of the above technical solutions.

[0119] The wake-up voice generation apparatus provided in this application includes a memory and a processor, and also includes a program or instructions stored in the memory. When the program or instructions are executed by the processor, they can implement the steps of the wake-up voice generation method of any of the above-mentioned technical solutions. Therefore, the wake-up voice generation method has all the beneficial effects of the above-mentioned wake-up voice generation method, which will not be repeated here.

[0120] According to a fourth aspect of this application, an electronic device is proposed, including a wake-up voice generation device according to any of the above-described technical solutions.

[0121] The electronic device provided in this application includes a wake-up voice generation device as described in any of the above technical solutions. Therefore, the electronic device has all the beneficial effects of the wake-up voice generation device, which will not be repeated here.

[0122] According to the fifth aspect of this application, a readable storage medium is proposed, on which a program or instructions are stored, which, when executed by a processor, implement a method for generating wake-up voice as described in any of the above technical solutions.

[0123] The storage medium provided in this application stores a program or instructions. When the program or instructions are executed by a processor, they can realize the wake-up voice generation method as described in any of the above technical solutions. Therefore, the storage medium has all the beneficial effects of the above wake-up voice generation method, which will not be elaborated here.

[0124] According to a sixth aspect of this application, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the steps of the wake-up voice generation method as described in any of the above embodiments. Therefore, this computer program product possesses all the beneficial effects of the aforementioned wake-up voice generation method, which will not be elaborated further here.

[0125] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified and limited. The terms "connection," "installation," and "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0126] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0127] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating wake-up speech, characterized in that, include: Obtain the wake word text and reference speech; Input the wake word text and the reference speech into the target model to generate the target speech corresponding to the wake word text; According to the first preset algorithm, the target speech is enhanced with features to generate the wake-up speech; The step of inputting the wake-up word text and the reference speech into the target model to generate target speech corresponding to the wake-up word text includes: The text encoder of the target model encodes the wake word text to generate a wake word text feature vector; The speech encoder of the target model encodes the reference speech to generate a reference speech feature vector; The modulator of the target model modulates the reference speech feature vector; The decoder of the target model decodes the wake word text feature vector and the adjusted reference speech feature vector to generate the target speech; The regulator of the target model adjusts the reference speech feature vector, including: The regulator of the target model adjusts the reference speech feature vector according to the preset formula X1=s1×x1+β1; Wherein, X1 is the adjusted reference speech feature vector, x1 is the unadjusted reference speech feature vector, s1 is the scaling factor of the reference speech feature vector, and β1 is the offset factor of the reference speech feature vector.

2. The generation method according to claim 1, characterized in that, The first preset algorithm has multiple components. The step of performing feature enhancement on the target speech according to the first preset algorithm to generate the wake-up speech includes: Based on multiple first preset algorithms, feature enhancement is performed on different features of the target speech to generate multiple feature-enhanced target speech; The multiple target speech samples with enhanced features are mixed to generate the wake-up speech.

3. The generation method according to claim 1 or 2, characterized in that, Before the step of obtaining the wake-up word text and the reference speech, the generation method further includes: Acquire text data and corresponding voice data; The preset model is trained based on the text data and the speech data to generate the target model.

4. The generation method according to claim 3, characterized in that, The step of training a preset model based on the text data and the speech data to generate a target model includes: The text data and the voice data are input into the preset model to generate voice output data; The training loss value is determined based on the speech output data and the speech data; The parameters of the preset model are updated based on the training loss value.

5. The generation method according to claim 4, characterized in that, The step of inputting the text data and the voice data into the preset model to generate voice output data includes: The text encoder of the preset model encodes the text data to generate a text feature vector; The speech encoder of the preset model encodes the speech data to generate a speech feature vector; The decoder of the preset model decodes the text feature vector and the speech feature vector to generate the speech output data.

6. The generation method according to claim 5, characterized in that, The speech encoder includes a speaker encoder and a prosodic encoder. The speech encoder of the preset model encodes the speech data to generate a speech feature vector, including: The speaker encoder encodes the speech data to generate a speaker feature vector; The prosody encoder encodes the speech data to generate a prosody feature vector; The speaker feature vector and the prosodic feature vector are concatenated to generate the speech feature vector.

7. The generation method according to claim 5, characterized in that, Before the step of the decoder of the preset model decoding the text feature vector and the speech feature vector to generate the speech output data, the following steps are included: The normalization layer of the preset model normalizes the speech feature vector.

8. The generation method according to claim 5, characterized in that, Before the step of the decoder of the preset model decoding the text feature vector and the speech feature vector to generate the speech output data, the following steps are included: The regulator of the preset model adjusts the speech feature vector.

9. The generation method according to claim 8, characterized in that, The step of adjusting the speech feature vector by the regulator of the preset model includes: The regulator adjusts the speech feature vector according to a preset formula X=s×x+β; Where X is the adjusted speech feature vector, x is the unadjusted speech feature vector, s is the scaling factor of the speech feature vector, and β is the offset factor of the speech feature vector. During the training phase of the preset model, s=1 and β=0.

10. The generation method according to claim 1 or 2, characterized in that, The speech encoder of the target model encodes the reference speech to generate a reference speech feature vector, including: The speaker encoder of the target model encodes the reference speech to generate a speaker feature vector. The prosodic encoder of the target model encodes the reference speech to generate a prosodic feature vector. The speaker feature vector and the prosodic feature vector are concatenated to generate the reference speech feature vector.

11. A device for generating wake-up voice, characterized in that, include: The acquisition unit is used to acquire the wake word text and reference speech; The generation unit is used to input the wake word text and the reference speech into the target model to generate target speech corresponding to the wake word text; as well as According to the first preset algorithm, the target speech is enhanced with features to generate the wake-up speech; The generation unit is specifically used for the text encoder of the target model to encode the wake-up word text and generate a wake-up word text feature vector; the speech encoder of the target model to encode the reference speech and generate a reference speech feature vector. The modulator of the target model adjusts the reference speech feature vector; the decoder of the target model decodes the wake word text feature vector and the adjusted reference speech feature vector to generate the target speech. The generation unit is specifically used for: the regulator of the target model to adjust the reference speech features according to the preset formula X1=s1×x1+β1; Wherein, X1 is the adjusted reference speech feature vector, x1 is the unadjusted reference speech feature vector, s1 is the scaling factor of the reference speech feature vector, and β1 is the offset factor of the reference speech feature vector.

12. A device for generating wake-up voice, characterized in that, include: A processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the wake-up voice generation method as described in any one of claims 1 to 10.

13. An electronic device, characterized in that, include: The wake-up voice generation apparatus as described in claim 11 or 12.

14. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the wake-up voice generation method as described in any one of claims 1 to 10.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the wake-up voice generation method as described in any one of claims 1 to 10.

Images