Voice playing method, device, equipment, storage medium and computer program product

Abstract

The present disclosure relates to a speech playing method, device, equipment, storage medium and computer program product. The speech playing method comprises: determining text corresponding to a speech to be played, and determining a current scene style feature based on the text and a first model, the first model being used for generating a scene style feature based on a text, wherein the scene style feature is used for representing a pronunciation style corresponding to a speech playing scene; and synthesizing the current scene style feature and the text to obtain a played speech. The method provided in the embodiments of the present disclosure can improve the auditory experience of a user.

Description

Technical Field

[0001] This disclosure relates to the field of information processing, and in particular to a voice playback method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] Speech synthesis technology, also known as text-to-speech (TTS) technology, is a technique that converts text into sound (usually human voice). Among related technologies, the application of speech synthesis technology is mainly reflected in multiple fields such as intelligent assistants, accessibility devices, and education and training.

[0003] With the development of speech synthesis technology and the increasing demands of users, users have more and more requirements for speech synthesized based on speech technology. For example, users hope that the voice played (speech synthesized based on speech synthesis technology) can be more human-like, thereby improving the auditory experience. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a voice playback method, apparatus, device, storage medium, and computer program product.

[0005] According to a first aspect of the present disclosure, a voice playback method is provided, comprising: determining the text corresponding to the voice to be played, and determining a current scene style feature based on the text and a first model, wherein the first model is used to generate scene style features based on the text, and the scene style features are used to represent the pronunciation style corresponding to the voice playback scene; and synthesizing the current scene style features and the text to obtain the voice to be played.

[0006] In one embodiment, synthesizing the current scene style features and the text to obtain the played speech includes: identifying text keywords in the text and determining the stress features corresponding to the text keywords, wherein the stress features characterize the intensity of syllable pronunciation emphasis; and synthesizing the stress features, the current scene style features, and the text to obtain the played speech.

[0007] In one embodiment, identifying text keywords in the text and determining the stress features corresponding to the text keywords includes: identifying text keywords based on the text and a second model, and determining the stress features corresponding to the text keywords, wherein the second model is used to determine the text keywords and the stress features corresponding to the text keywords based on the text.

[0008] In one embodiment, the second model is trained as follows: A first training sample is obtained, comprising a training sample pair consisting of a speech training sample and a corresponding text training sample; acoustic features are extracted from the first training sample to obtain acoustic features corresponding to the text training sample; the acoustic features are clustered, and a target acoustic feature corresponding to the text training sample is determined based on the clustering results, and the target acoustic feature is identified as the accent feature corresponding to the text training sample, wherein the value of the target acoustic feature is greater than or equal to; a second label is constructed using the target acoustic feature, and the second model is trained based on the second label.

[0009] In one embodiment, determining the current scene style features includes: determining the speech playback scene corresponding to the text based on the first model and the text, and determining the scene style features that match the current speech playback scene based on the speech playback scene.

[0010] In one implementation, the first model is trained as follows: the first initial model is called to classify the scene style features of the second training sample to obtain the scene style feature classification result corresponding to the second training sample; based on the scene style feature classification result and the corresponding loss value, the parameters of the first initial model are adjusted to obtain the first model.

[0011] In one embodiment, synthesizing the accent feature, the current scene style feature, and the text to obtain the played speech includes: performing speech synthesis based on a third model, the accent feature, the style feature, and the text to obtain the played speech, wherein the memory usage of the third model is less than or equal to a memory usage threshold.

[0012] In one embodiment, the third model is trained as follows: speech synthesis is performed based on the second initial model and the third training samples, and the synthesized speech is used as the fourth training sample, wherein the memory usage of the second initial model is greater than the memory threshold; the second initial model is used as the teacher model, and distillation learning is performed based on the fourth training samples to obtain the student model, and the student model is confirmed as the third model, wherein the memory usage of the student model is less than or equal to the memory usage threshold.

[0013] According to a second aspect of the present disclosure, a voice playback device is provided, comprising: a determining unit, configured to determine the text corresponding to the voice to be played, and to determine a current scene style feature based on the text and a first model, wherein the first model is configured to generate scene style features based on the text, and the scene style features are used to represent the pronunciation style corresponding to the voice playback scene; and a processing unit, configured to synthesize the current scene style feature and the text to obtain the voice to be played.

[0014] In one embodiment, the processing unit synthesizes the current scene style features and the text to obtain the played speech in the following manner: identifying text keywords in the text and determining the stress features corresponding to the text keywords, wherein the stress features characterize the intensity of syllable pronunciation emphasis; synthesizing the stress features, the current scene style features, and the text to obtain the played speech.

[0015] In one embodiment, the processing unit identifies text keywords in the text and determines the stress features corresponding to the text keywords in the following manner: based on the text and a second model, it identifies text keywords and determines the stress features corresponding to the text keywords, wherein the second model is used to determine text keywords and the stress features corresponding to the text keywords based on the text.

[0016] In one embodiment, the second model is trained by the processing unit in the following manner: obtaining a first training sample, the first training sample comprising a training sample pair consisting of a speech training sample and a text training sample corresponding to the speech training sample; extracting acoustic features from the first training sample to obtain acoustic features corresponding to the text training sample; clustering the acoustic features, determining the target acoustic features corresponding to the text training sample based on the clustering results, and determining the target acoustic features as the accent features corresponding to the text training sample, wherein the value of the target acoustic features is greater than or equal to; constructing a second label using the target acoustic features, and training the second model based on the second label.

[0017] In one embodiment, the determining unit determines the current scene style features in the following manner: based on the first model and the text, it determines the voice playback scene corresponding to the text, and based on the voice playback scene, it determines the scene style features that match the current voice playback scene.

[0018] In one implementation, the first model is trained by the determining unit in the following manner: calling the first initial model to classify the scene style features of the second training sample to obtain the scene style feature classification result corresponding to the second training sample; and adjusting the parameters of the first initial model based on the scene style feature classification result and the corresponding loss value to obtain the first model.

[0019] In one embodiment, the synthesis unit synthesizes the accent feature, the current scene style feature, and the text in the following manner to obtain the played speech: speech synthesis is performed based on a third model, the accent feature, the style feature, and the text to obtain the played speech, wherein the memory usage of the third model is less than or equal to a memory usage threshold.

[0020] In one embodiment, the third model is trained by the processing unit in the following manner: speech synthesis is performed based on the second initial model and the third training samples, and the synthesized speech is used as the fourth training sample, wherein the memory usage of the second initial model is greater than the memory threshold; the second initial model is used as the teacher model, and distillation learning is performed based on the fourth training sample to obtain the student model, and the student model is identified as the third model, wherein the memory usage of the student model is less than or equal to the memory usage threshold.

[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: execute the voice playback method described in the first aspect or any embodiment of the first aspect.

[0022] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the processor to perform the voice playback method described in the first aspect or any embodiment of the first aspect.

[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, implements the voice playback method described in the first aspect or any embodiment of the first aspect.

[0024] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: By determining the scene style features of the current scene based on the text corresponding to the speech to be played and a first model. Since scene style features can be used to represent the pronunciation features of the speech playback scene, it is possible to determine the pronunciation features corresponding to the current speech playback scene using text and the first model. Because the speech played in the current playback scene is synthesized based on the current scene style features, and the pronunciation features of the played sound correspond to the current speech playback scene, the played speech is more suitable for the current speech playback scene compared to speech synthesized only based on emotional features, thus making the played speech more vivid and improving the user's auditory experience.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0027] Figure 1 This is a flowchart illustrating a voice playback method according to an exemplary embodiment.

[0028] Figure 2 This is a flowchart illustrating a method for determining current scene style features according to an exemplary embodiment.

[0029] Figure 3A This is a flowchart illustrating a first model training method according to an exemplary embodiment.

[0030] Figure 3B This is a schematic diagram of a first model training architecture according to an exemplary embodiment.

[0031] Figure 4 This is a flowchart illustrating a voice playback method according to an exemplary embodiment.

[0032] Figure 5 This is a flowchart illustrating an accent feature determination method according to an exemplary embodiment.

[0033] Figure 6A This is a flowchart illustrating a second model training method according to an exemplary embodiment.

[0034] Figure 6B This is a schematic diagram of a scenario constructed according to an exemplary embodiment of an accent feature label.

[0035] Figure 7AThis is a flowchart illustrating a third model training method according to an exemplary embodiment.

[0036] Figure 7B This is a schematic diagram illustrating a third model training method scenario according to an exemplary embodiment.

[0037] Figure 8 This is a schematic diagram illustrating a voice playback method scenario according to an exemplary embodiment.

[0038] Figure 9 This is a block diagram illustrating a voice playback device according to an exemplary embodiment.

[0039] Figure 10 This is a block diagram of an apparatus for voice playback according to an exemplary embodiment. Figure 1 .

[0040] Figure 11 This is a block diagram of an apparatus for voice playback according to an exemplary embodiment. Figure 2 . Detailed Implementation

[0041] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0042] Speech synthesis technology, also known as text-to-speech (TTS) technology, is a technique that converts text into sound (usually human voice). Among related technologies, the application of speech synthesis technology is mainly reflected in multiple fields such as intelligent assistants, accessibility devices, and education and training.

[0043] With the development of speech synthesis technology and the increasing demands of users, users have more and more requirements for speech synthesized based on speech technology. For example, users hope that the voice played (speech synthesized based on speech synthesis technology) can be more human-like, thereby improving the auditory experience.

[0044] To address the aforementioned needs, related technologies typically assign relevant emotional features to the played speech, thereby imbuing it with corresponding emotions and making the speech more vivid. Emotional features can be understood as characteristics in speech that can be used to express emotions. For example, common emotions include happiness, sadness, and anger. During speech synthesis, relevant emotional features can be added to imbue the played speech with the corresponding emotional characteristics. For instance, during speech synthesis, emotional features representing happiness can be added (e.g., by adding corresponding modal particles to the text or adjusting the tone of parts of the text) to give the played speech a happy feeling.

[0045] It's understandable that speech synthesis technology can be applied to different playback scenarios, and setting different pronunciation styles for the voice in different scenarios can make the voice sound more human, providing users with a better auditory experience. However, related technologies can only change the emotion assigned to the synthesized voice; they cannot fully integrate the scene with the style characteristics of the voice (e.g., the style characteristics of the synthesized voice). This can lead to situations where the style of the voice being played doesn't match the current playback environment. Take intelligent voice assistants playing voice messages as an example. For instance, in some scenarios, users might be chatting with a voice assistant. In this scenario, users might prefer the voice assistant to play the voice in a more relaxed style (e.g., a faster speech rate combined with corresponding happy emotional characteristics), but the voice assistant might play the voice in a more somber style (e.g., a slower speech rate combined with corresponding sad emotional characteristics). Thus, in some application scenarios, users might hear voice messages with emotional characteristics that don't match the scene, thereby reducing their auditory experience.

[0046] In order to solve the above-mentioned technical problems, this disclosure proposes a voice playback method, which determines the style features matching the current scene based on the text corresponding to the voice to be played, and synthesizes a sound for playback based on the determined style features and the text.

[0047] For ease of understanding, the technical terms that may appear in the embodiments of this disclosure will be explained by way of example below.

[0048] Emotional features refer to the speech parameters and attributes used to express specific emotions or feelings. These features enable synthesized speech to not only contain linguistic information but also convey the speaker's emotional state, thus making human-computer interaction richer and more natural.

[0049] Style features refer to the various parameters and attributes that influence the style or characteristics of synthesized speech. These features can include pitch, speech rate, intonation, accent, emotional features, and pronunciation characteristics, which together determine the personalization, naturalness, and expressiveness of synthesized speech.

[0050] The following are acoustic features (or characteristics) that can characterize style, and can also be called acoustic parameters (or parameters):

[0051] Pitch: Pitch refers to the highness or lowness of a sound, determined by the frequency of vocal cord vibration. In speech synthesis, adjusting the pitch can make the synthesized speech sound more natural and can also be used to express different emotions or tones.

[0052] Speech rate: Speech rate refers to the speed at which speech is delivered, and it affects the rhythm of speech. In speech synthesis, the speech rate can be adjusted to suit different scenarios and the needs of different audiences.

[0053] Intonation: Intonation refers to the rise and fall of the voice during speech, and it is very important for expressing questions, commands, exclamations, and other emotional states. The intonation of synthesized speech needs to conform to the prosodic rules of language to enhance naturalness.

[0054] Accent: An accent refers to the pronunciation characteristics of a specific region or social group. In speech synthesis, different accents can be simulated to cater to the preferences of different listeners.

[0055] Emotional features: Emotional features refer to the emotional expression in speech, such as happiness, sadness, and anger. Adding emotional features to synthesized speech can make the speech more vivid and engaging.

[0056] Articulation characteristics include the manner of pronunciation, clarity, and articulation of vowels and consonants. These characteristics affect the recognizability and naturalness of speech.

[0057] Loudness: Loudness refers to the intensity of a sound, which can be changed by adjusting the volume. In speech synthesis, changes in loudness can be used to express emphasis or contrast.

[0058] Formants: Formants are key factors affecting the quality of sound and distinguishing different vowels. Adjusting formants can change the timbre of a speech.

[0059] Syllable: The smallest unit of pronunciation in a language. It consists of one or more phonemes, usually containing a main vowel phoneme (sometimes called the nucleus), and usually surrounded by one or more consonant phonemes.

[0060] Duration alignment refers to the process of adjusting the duration of each phoneme (or syllable) in synthesized speech so that the synthesized speech matches the target speech (usually a reference human speech) in time.

[0061] Knowledge Distillation is a model compression technique that aims to transfer the knowledge of a large, complex teacher model to a smaller student model, thereby enabling the student model to maintain high performance while having a smaller model size and faster inference speed.

[0062] Self-supervised learning is a learning paradigm in the field of machine learning that allows machines to learn by generating their own supervisory signals using large amounts of unlabeled data. In self-supervised learning, manually labeled data is not required; instead, pretext tasks are designed to allow the model to learn meaningful representations from the data.

[0063] Figure 1 This is a flowchart illustrating a voice playback method according to an exemplary embodiment, such as... Figure 1 As shown, the voice playback method used in the terminal includes the following steps.

[0064] In step S11, the text corresponding to the speech to be played is determined, and the current scene style features are determined based on the text and the first model.

[0065] In step S12, the current scene style features and text are synthesized to obtain the played speech.

[0066] The first model is used to generate scene style features based on text, and the scene style features are used to represent the pronunciation style corresponding to the speech playback scene.

[0067] In this embodiment, the scene style features of the current scene are determined based on the text corresponding to the speech to be played and a first model. Since scene style features can represent the pronunciation features of the speech playback scene, it is possible to determine the pronunciation features corresponding to the current speech playback scene using the text and the first model. Because the speech played in the current playback scene is synthesized based on the current scene style features, and the pronunciation features of the played sound correspond to the current speech playback scene, the played speech is more suitable for the current speech playback scene compared to speech synthesized solely based on emotional features, thus making the played speech more vivid and improving the user's auditory experience.

[0068] It should be noted that the terminals in this disclosure embodiment include, for example, at least one of the following: mobile phone, wearable device, Internet of Things device, car with communication function, smart car, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal device in industrial control, wireless terminal device in self-driving, wireless terminal device in remote medical surgery, wireless terminal device in smart grid, wireless terminal device in transportation safety, wireless terminal device in smart city, and wireless terminal device in smart home, but are not limited thereto.

[0069] For ease of understanding, the embodiments disclosed herein can be illustrated by taking deployment on a terminal as the main example.

[0070] In some scenarios, such as when a user interacts with a voice assistant deployed on a terminal, the terminal (e.g., by calling the voice assistant) can generate corresponding dialogue text based on the dialogue content with the user. This text can then be used as the text corresponding to the audio to be played.

[0071] For example, the text content corresponding to the voice to be played generated by the terminal is "Today is a sunny day". After the terminal determines the text, it performs corresponding processing operations to obtain the voice to be played (for example, it generates the corresponding scene style features based on the text content, and performs speech synthesis based on the features to obtain the voice to be played), and plays the voice. The content played could be, for example, "Today is such a nice day, it's a sunny day".

[0072] In some embodiments, the text corresponding to the speech to be played, as determined by the terminal, may be obtained, for example, based on a Natural Language Processing (NLP) model and the acquired dialogue context.

[0073] Scene style features can be understood as the style features of the speech playback in the corresponding playback scene. Based on the foregoing description of the embodiments of this disclosure, style features can be determined based on relevant acoustic parameters (e.g., pitch, emotional features, etc.). Therefore, scene style features can be understood as the relevant acoustic parameter features of the speech playback in the corresponding playback scene (e.g., what emotional features the speech is played with, and / or what tone the speech is played with, and / or what speech rate the speech is played with, etc.).

[0074] In some embodiments, the correspondence between scene style features and acoustic parameters can be a predetermined relationship. For example, the user can predetermine that a first scene style corresponds to acoustic parameter A, and a second scene style corresponds to acoustic parameter B, wherein the first scene style and the second scene style are different, and acoustic parameter A and acoustic parameter B are different. (For example, acoustic parameter A and acoustic parameter B may correspond to the same acoustic parameter category, but with different acoustic parameter values, or acoustic parameter A and acoustic parameter B may correspond to different acoustic parameter categories).

[0075] It should be noted that in the above embodiments, acoustic parameters may include one or more parameters. For example, acoustic parameter A may refer to a set of acoustic parameters, which may include one or more parameters, etc.

[0076] As for how to determine the current scene style features based on the text corresponding to the speech to be played (for ease of understanding, it will be referred to as text in the following embodiments) and the first model, it can be achieved in the following ways.

[0077] Figure 2 This is a flowchart illustrating a method for determining current scene style features according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following steps.

[0078] In step S21, the voice playback scenario corresponding to the text is determined based on the first model and the text.

[0079] In step S22, scene style features that match the current voice playback scene are determined based on the voice playback scene.

[0080] In this embodiment of the disclosure, a first model is used to predict the speech playback scene corresponding to the text based on the text, and then the scene style features that match the speech playback scene are predicted based on the predicted speech playback scene. Since the scene style features are obtained based on the speech playback scene prediction, the scene style features predicted by the first model can be more closely matched with the current speech playback scene.

[0081] Understandably, the first model can determine the corresponding speech playback scenario based on textual information (such as semantics) in the text.

[0082] For example, if the text is "Speed ​​limit 60 ahead, please slow down," then it can be inferred that the current voice playback scenario is a "navigation scenario." Based on this, style features applicable to the "navigation scenario" can be determined, and the determined style features (e.g., voice broadcast style) can be used as the scenario style features that match the current voice playback scenario.

[0083] For example, if the text is "Please keep a happy mood," then it can be inferred that the current voice playback scenario is an "entertainment scenario" (e.g., a relaxed conversation between the user and the terminal). Based on this, the style characteristics of the "entertainment scenario" can be determined, and the determined style (e.g., entertainment style) can be used as the scene style characteristics to match the current voice playback scenario.

[0084] In some embodiments, the first model can determine the current voice playback scenario based on the context of the text, and can periodically determine the current voice playback scenario based on the context of the text.

[0085] In response to the identification of a new voice playback scenario, the corresponding scenario style features are determined based on the new voice playback scenario.

[0086] For example, consider a car as the terminal. The terminal receives a user's command (e.g., a voice command), such as "Navigate to location A." Based on this command, the terminal can generate the text "Navigating for you." Based on this text and the first model, it can determine the current voice playback scenario as a "navigation scenario" and play the voice using the scene style features corresponding to the "navigation scenario." After playing the voice corresponding to the "navigation scenario," if the user's command is "Let's chat," this command serves as context information. The first model can then redetermine the voice playback scenario based on this context information, identifying it as an "entertainment scenario," and play the voice using the scene style features corresponding to the "entertainment scenario."

[0087] Based on this, it is possible to determine the style of the current scene based on the text and the first model.

[0088] The training process for the first model can be, for example, through... Figure 3A This is achieved in the following way. Figure 3A This is a flowchart illustrating a first model training method according to an exemplary embodiment, such as... Figure 3A As shown, the method includes the following steps.

[0089] In step S31, the first initial model is invoked to classify the scene style features of the second training sample, and the scene style feature classification result corresponding to the second training sample is obtained.

[0090] In step S32, based on the scene style feature classification results and the corresponding loss values, the parameters of the first initial model are adjusted to obtain the first model.

[0091] In this embodiment of the disclosure, by combining a self-supervised learning mechanism, a first initial model is invoked to perform scene style feature classification on the second training samples, and the parameters of the first initial model are adjusted based on the task processing results and the corresponding loss values, thereby training the first model.

[0092] Understandably, training the first initial model using a self-supervised learning approach enables the first initial model to learn representations of scene style features from the input data itself.

[0093] For example, the first model is obtained by training the first initial model, and the implementation process can be as follows: A) to D):

[0094] A) Pre-train the first initial model based on the second training samples.

[0095] B) Fine-tune the first initial model after pre-training for downstream tasks.

[0096] C) The first initial model is trained based on the contrastive learning method.

[0097] D) The first initial model is trained based on a generative adversarial network.

[0098] In some embodiments, for A), a language model may be selected as the first initial model, such as a pre-trained language representation model of bidirectional encoders (BERT) or a generative pre-trained speech representation model (GPT).

[0099] In some embodiments, B) could be, for example, further adjustments to the pre-trained first initial model for a specified task.

[0100] For example, suppose the model is applied to a specific downstream task, such as in-vehicle text style classification. The model's parameters are tuned on labeled style data to suit the in-vehicle text style classification task. This process enables the model to better understand the differences between different text styles.

[0101] Understandably, fine-tuning can be understood as a step in transfer learning, which utilizes the knowledge already learned by the pre-trained model to further optimize the model using a small amount of task-specific data.

[0102] In some embodiments, C) could be, for example, a method for learning effective features by comparing differences between different samples.

[0103] For example, in text style classification, contrastive learning might involve constructing positive sample pairs (sample pairs whose scene style features match the playback scene) and negative sample pairs (sample pairs whose scene style features do not match the playback scene), and then training the model to correctly classify these samples. This method helps the model learn more robust feature representations, enabling it to better distinguish between different styles of text.

[0104] In some embodiments, for example, D) can be self-supervised learning using adversarial generative networks (e.g., Siamese networks, etc.).

[0105] For example, in text style classification, the generator in an adversarial generative network generates fake text samples, and the discriminator's task is to distinguish these fake samples from real samples.

[0106] This adversarial training approach enables the model to learn richer and higher-quality text representations because it requires the generator to continuously improve in order to deceive the discriminator, while the discriminator also continuously learns how to better distinguish between real and fake samples, thus achieving the goal of training the initial model.

[0107] It is understood that embodiments A) to D) above can be implemented individually to complete the training of the first model or can be implemented in combination to complete the training of the first model.

[0108] To facilitate understanding, it will be explained through Figure 3B The above training methods are illustrated by examples. Figure 3B This is a schematic diagram illustrating a first model training architecture according to an exemplary embodiment. For example... Figure 3B As shown, for the training architecture of the first initial model, the first initial model performs the training process described in A) above based on the second training samples (i.e., pre-training the first initial model based on the second training samples). After performing the training process in A), one or more learning paths can be selected from transfer learning (e.g., as described in B above), contrastive learning (e.g., as described in C above), and adversarial learning (e.g., as described in D above) to train the first model.

[0109] It should be noted that Loss1 represents the loss value obtained based on transfer learning, used to adjust the parameters of the first initial model based on transfer learning to obtain the first model. Loss2 represents the loss value obtained based on contrastive learning, used to adjust the parameters of the first initial model based on contrastive learning to obtain the first model. Loss3 represents the loss value obtained based on adversarial learning, used to adjust the parameters of the first initial model based on adversarial learning to obtain the first model.

[0110] It should also be noted that if multiple learning paths are chosen to train the first model, the initial model can be adjusted based on the loss value corresponding to each learning path.

[0111] Therefore, by training the first model through the above embodiments, a first model that can be used to determine scene style features based on text can be obtained.

[0112] However, it's understandable that in some scenarios, the voice prompts played to users may contain crucial information, and users often prefer to be reminded of this information through the voice (e.g., by emphasizing the pronunciation). For example, in a "navigation scenario," the voice prompt might say, "Speed ​​limit ahead 60, please slow down." From this voice prompt, the user might want to know the "speed limit." Therefore, in this scenario, to improve the user's auditory experience, the "60" could be processed to serve as a reminder.

[0113] Based on this, it can be done as follows Figure 4 The method shown is used to play audio, thereby reminding users of key information in the audio. Figure 4 This is a flowchart illustrating a voice playback method according to an exemplary embodiment, such as... Figure 4 As shown, the method includes the following steps.

[0114] In step S41, text keywords in the text are identified, and the stress features corresponding to the text keywords are determined.

[0115] In step S42, the accent features, current scene style features, and text are synthesized to obtain the played speech.

[0116] Among them, stress features characterize the intensity of syllable pronunciation being emphasized.

[0117] In this embodiment of the disclosure, by identifying text keywords in the text and determining the stress features corresponding to the text keywords, it is possible to achieve the effect of reminding the user of keywords by accurately marking the stress features of keywords during the speech synthesis stage.

[0118] It should be noted that stress features typically refer to the intensity or pitch change of a syllable or word during pronunciation. The stress features corresponding to keywords can be understood as the stress features of the syllables within the keyword, that is, the intensity or pitch change of the syllables within the keyword that are emphasized.

[0119] It's understandable that keywords in a text may correspond to different characters, and different characters correspond to different syllables. That is, the pronunciation of the words is different. Therefore, the stress characteristics corresponding to keywords that correspond to different syllables are also different. For example, the keywords "start" and "stop" correspond to different syllables, and their respective stress characteristics are also different.

[0120] Based on this, the above method can be used to make the played audio contain the stress features corresponding to keywords, thereby reminding the user of keywords.

[0121] The acquisition of stress features for the aforementioned keywords is typically achieved through manual annotation in relevant technologies. Understandably, the process of determining stress features through manual annotation is quite complex. Therefore, as... Figure 5 This is achieved as shown. Figure 5 This is a flowchart illustrating an accent feature determination method according to an exemplary embodiment, the method comprising the following steps.

[0122] In step S51, text keywords are identified based on the text and the second model.

[0123] In step S52, the stress features corresponding to the text keywords are determined, wherein the second model is used to determine the text keywords and the stress features corresponding to the text keywords based on the text.

[0124] In this embodiment, keywords in the text are determined based on the text itself, and a second model is used to determine the corresponding stress features based on the keywords. By using an artificial intelligence model to identify stress features, the efficiency of stress feature identification can be improved compared to stress identification methods in related technologies.

[0125] In some embodiments, identifying text keywords in text can be achieved, for example, through a second model.

[0126] It is understandable that the location of keywords in different texts will vary, and keyword identification can be achieved, for example, through the following methods 1) to 4):

[0127] 1) Provided directly from external sources.

[0128] 2) Based on markers or indicator symbols.

[0129] 3) Automatic prediction based on context.

[0130] 4) Personalized settings based on user preferences.

[0131] In some embodiments, for 1), location information of control keywords can be entered, for example, through a specific command or interface.

[0132] For example, in the voice interaction interface of a smart car, the user can directly tell the system the location of a certain type of keyword, and the system can make corresponding adjustments in speech synthesis after receiving this information.

[0133] In some embodiments, for 2), for example, specific markers or indicators in the text are used to indicate which parts of the speech synthesis system require special processing.

[0134] For example, special markers or tags can be inserted into the text to instruct the system to emphasize or adjust certain keywords when synthesizing speech.

[0135] In some embodiments, for 3), for example, the location information of keywords can be automatically predicted based on the context of the text using a second model (or other models).

[0136] For example, keyword location information can be predicted based on the context of the dialogue text, or it can be predicted based on preceding and following tokens (e.g., tokens) in the text.

[0137] In some embodiments, for 4), users can pre-set their preferences, specifying the position and representation of certain keywords or phrases in speech synthesis. These preferences can be managed by the user or automatically optimized by the system learning user behavior and feedback.

[0138] Therefore, it is possible to determine the stress features corresponding to keywords based on the second model.

[0139] The training methods for models used to determine syllable stress features require not only recording text but also recording audio, and the text also needs to be labeled, making the training process quite cumbersome.

[0140] Based on this, in some embodiments, the training method for the model used to determine syllable stress features can be, for example, by means of... Figure 6A The training method for the second model shown is implemented.

[0141] Figure 6A This is a flowchart illustrating a second model training method according to an exemplary embodiment. For example... Figure 6A As shown, the method includes the following steps.

[0142] In step S61, a first training sample is obtained, which includes a training sample pair consisting of a speech training sample and a text training sample corresponding to the speech training sample.

[0143] In step S62, acoustic features are extracted from the first training sample to obtain the acoustic features corresponding to the text training sample.

[0144] In step S63, the acoustic features are clustered, and the target acoustic features corresponding to the text training samples are determined based on the clustering results. The target acoustic features are then identified as the accent features corresponding to the text training samples, wherein the feature value of the target acoustic features is greater than or equal to the feature value threshold.

[0145] In step S64, a second label is constructed using the target acoustic features, and a second model is trained based on the second label.

[0146] In this embodiment, the first training sample includes training sample pairs consisting of a speech training sample and a corresponding text training sample. Therefore, by extracting acoustic features from the first training sample, acoustic features corresponding to the text training sample can be obtained. Clustering is then performed based on the acoustic features of the text training sample, and acoustic features with feature values ​​greater than or equal to a feature value threshold are used as the stress features of the text training sample. A second model is obtained by constructing a second label based on these stress features. This allows for the construction of labels for the stress features corresponding to the text using a smaller number of training samples, and the training of the second model is achieved through these labels. This enables the second model to determine its corresponding stress features based on the text. Compared to the model training process of related technologies, the second model can be trained using fewer training samples.

[0147] It should be noted that since the first training sample includes the text training sample corresponding to the speech training sample, the acoustic features corresponding to each character in the text training sample can be obtained by extracting acoustic features from the speech training sample.

[0148] As for the extraction of acoustic features, they can be extracted based on parameters of different categories, such as the relevant parameters described in the foregoing content of the embodiments of this disclosure, which will not be repeated here.

[0149] It is understandable that this is done in order to ensure that the results of acoustic feature extraction from the first training sample correspond one-to-one with the text features.

[0150] In some embodiments, speech training samples and text training samples may be temporally aligned before acoustic feature extraction.

[0151] Understandably, after acoustic extraction of the training samples is completed, the extracted acoustic features (e.g., energy values, frequency values, etc.) can be obtained. The acoustic features extracted for each syllable are clustered, and a corresponding threshold is set based on the clustering results to determine the stress features corresponding to each syllable.

[0152] For example, taking the syllable corresponding to "stop" as an example, assuming that the acoustic features extracted for this sample include energy and frequency, the feature values ​​corresponding to the acoustic features also include energy and frequency values. These values ​​may come from different speech training samples. Based on the clustering results corresponding to the acoustic features of this sample, the feature values ​​of acoustic features with frequencies greater than or equal to a frequency threshold and energy greater than or equal to an energy threshold are set as the stress features of the syllable corresponding to "stop".

[0153] Therefore, after determining the accent features corresponding to the text training samples, accent feature labels can be constructed based on the accent features for training the second model.

[0154] For example, to facilitate understanding, accent feature labels can be expressed as follows: Figure 6B It is constructed in this way. Figure 6B This is a schematic diagram of a scenario constructed according to an exemplary embodiment of an accent feature label.

[0155] like Figure 6B As shown, the first training sample can be temporally aligned using a duration alignment tool to obtain temporal feature parameters such as duration or speech rate. Relevant acoustic features are then extracted from the first training sample to obtain related acoustic features (such as...). Figure 6B The features related to the parameters shown include, for example, fundamental frequency, energy, and spectral parameters. After extracting acoustic features from the first training sample, the extracted features are clustered. Based on the clustering statistical analysis, reasonable frequency thresholds for types such as speech rate, fundamental frequency, and energy are determined. Weighted summation is then used to generate a stress information score for each syllable, which serves as a reference standard for stress labeling.

[0156] In some embodiments, the second model trained on the second label can be obtained using the following related machine learning methods, including: based on Conditional Random Field (CRF) or Long Short-Term Memory (LSTM), etc.

[0157] Therefore, the second model can be trained, and then the stress features corresponding to the keywords can be identified based on the second model.

[0158] To synthesize the current scene style features and text, the resulting spoken language can be obtained, for example, through a relevant speech synthesis model.

[0159] Understandably, while deep-structured models can achieve highly natural-sounding speech synthesis, their computationally intensive nature may be impractical in certain application scenarios (such as in-vehicle environments) due to limitations in computing resources and network connectivity. Therefore, deploying lightweight models on the terminal can reduce its memory footprint.

[0160] For example, in some embodiments, speech synthesis can be performed based on a third model, accent features, style features, and text to obtain the played speech, wherein the memory usage of the third model is less than or equal to a memory usage threshold.

[0161] In some embodiments, if it is not necessary to obtain stress features during the playback of speech, speech synthesis can be performed based on the third model, style features, and text to obtain the played speech, wherein the memory usage of the third model is less than or equal to the memory usage threshold.

[0162] Understandably, since the memory usage of the third model is less than or equal to the memory usage threshold, the third model can be deployed on the terminal to obtain the audio to be played, thereby reducing the terminal's memory usage.

[0163] The training process for the third model can be achieved, for example, by combining distillation learning. Figure 7A This is a flowchart illustrating a third model training method according to an exemplary embodiment, the method comprising the following steps.

[0164] In step S71, speech synthesis is performed based on the second initial model and the third training sample, and the synthesized speech is used as the fourth training sample. The memory usage of the second initial model is greater than the memory threshold.

[0165] In step S72, the second initial model is used as the teacher model, and distillation learning is performed based on the fourth training sample to obtain the student model. The student model is then confirmed as the third model, wherein the memory usage of the student model is less than or equal to the memory usage threshold.

[0166] In this embodiment, by using a second initial model with a larger memory footprint to synthesize speech based on a third training sample, the second initial model can learn a larger language representation, thereby enabling it to have higher performance and speech synthesis effect. Since the second initial model has higher performance and synthesis effect, using it as a teacher model and the synthesized speech as a fourth training sample for distillation learning allows the student model to maintain high performance and speech synthesis effect while remaining lightweight.

[0167] To facilitate understanding, the following will be explained through... Figure 7B The process of implementing the above embodiments is described by way of example. Figure 7B This is a schematic diagram illustrating a third model training method scenario according to an exemplary embodiment.

[0168] exist Figure 7B In this process, a third initial model is trained using a third training sample to obtain a second initial model. Then, the second initial model (e.g., a Tacotron or FastSpeech structure model) is used to synthesize speech from the third training sample. Since the third training sample may contain a relatively complex set of training samples, a large amount of distillation data with mixed styles can be obtained. During this process, training samples with a specific style can also be introduced (e.g., training samples from in-vehicle scenarios for targeted distillation data acquisition).

[0169] In some embodiments, the construction of the third model can be determined by distillation learning based on a large amount of mixed-style distillation data (i.e., the fourth training sample).

[0170] In some embodiments, the construction of the third model can be determined by distillation learning based on newly introduced small-scale target-style distillation data.

[0171] In some embodiments, a first intermediate model can be constructed based on large-scale mixed-style distillation data, and a second intermediate model can be constructed based on small-scale target-style distillation data through distillation learning. The two models are then fused, and the resulting fused model (i.e., the intermediate model) is subjected to distillation learning to obtain a third model (as shown by the dashed lines in the figure).

[0172] Based on this, the training of the third model can be completed.

[0173] For the sake of clarity regarding the overall scheme of the embodiments disclosed herein, the following will be used... Figure 8 The complete model structure used in the scheme is described. Figure 8 This is a schematic diagram illustrating a voice playback method scenario according to an exemplary embodiment. For example... Figure 8 As shown, the models used in the implementation of the voice playback method are the first model, the second model, and the third model.

[0174] The acquired text of the speech to be played is input into the first model and the second model respectively. The input "stress features corresponding to keywords" to the first model and the output "style features" to the second model are obtained. The outputs of the first and second models are then input into the third model to synthesize the target speech (i.e., the speech to be played).

[0175] Based on this, the voice playback method provided in the embodiments of this disclosure can realize voice synthesis in various scenarios and styles.

[0176] Based on the same concept, this disclosure also provides a voice playback device 100.

[0177] It is understood that the voice playback device provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0178] Figure 9 This is a block diagram illustrating a voice playback device according to an exemplary embodiment. (Refer to...) Figure 9 The device 100 includes a determining unit 101 and a processing unit 102.

[0179] The determining unit 101 is used to determine the text corresponding to the speech to be played, and to determine the current scene style features based on the text and the first model. The first model is used to generate scene style features based on the text, wherein the scene style features are used to represent the pronunciation style corresponding to the speech playback scene.

[0180] The processing unit 102 is used to synthesize the current scene style features and text to obtain the played speech.

[0181] In one embodiment, the processing unit 102 synthesizes the current scene style features and text to obtain the played speech in the following manner: It identifies text keywords in the text and determines the stress features corresponding to the text keywords, whereby the stress features characterize the intensity of syllable pronunciation emphasis. The stress features, the current scene style features, and the text are then synthesized to obtain the played speech.

[0182] In one embodiment, the processing unit 102 identifies text keywords in the text and determines the stress features corresponding to the text keywords in the following manner: based on the text and the second model, it identifies text keywords and determines the stress features corresponding to the text keywords. The second model is used to determine the text keywords and the stress features corresponding to the text keywords based on the text.

[0183] In one embodiment, the second model is trained by the processing unit 102 in the following manner: acquiring a first training sample, the first training sample including a training sample pair consisting of a speech training sample and a text training sample corresponding to the speech training sample; extracting acoustic features from the first training sample to obtain acoustic features corresponding to the text training sample; clustering the acoustic features, determining the target acoustic features corresponding to the text training sample based on the clustering results, and determining the target acoustic features as the accent features corresponding to the text training sample, wherein the feature value of the target acoustic features is greater than or equal to a feature value threshold; constructing a second label using the target acoustic features, and training the second model based on the second label.

[0184] In one embodiment, the determining unit 101 determines the current scene style features in the following manner: based on the first model and the text, it determines the speech playback scene corresponding to the text, and based on the speech playback scene, it determines the scene style features that match the current speech playback scene.

[0185] In one implementation, the first model is trained by the determining unit 101 in the following manner: The first initial model is invoked to classify the scene style features of the second training samples, resulting in a scene style feature classification result corresponding to the second training samples. Based on the scene style feature classification result and the corresponding loss value, the parameters of the first initial model are adjusted to obtain the first model.

[0186] In one embodiment, the synthesis unit synthesizes accent features, current scene style features, and text in the following manner to obtain the played speech: speech synthesis is performed based on a third model, accent features, style features, and text to obtain the played speech, wherein the memory usage of the third model is less than or equal to a memory usage threshold.

[0187] In one embodiment, the third model is trained by the processing unit 102 in the following manner: speech synthesis is performed based on the second initial model and the third training samples, and the synthesized speech is used as the fourth training sample, wherein the memory usage of the second initial model is greater than a memory threshold. The second initial model is used as the teacher model, and distillation learning is performed based on the fourth training samples to obtain the student model, which is then identified as the third model, wherein the memory usage of the student model is less than or equal to the memory usage threshold.

[0188] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0189] Figure 10 This is a block diagram of an apparatus for voice playback according to an exemplary embodiment. Figure 1 Device 200 can be provided as a terminal for executing the voice playback method. For example, device 200 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0190] Reference Figure 10 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0191] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0192] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0193] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0194] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0195] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0196] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0197] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0198] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0199] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0200] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0201] Figure 11 This is a block diagram of an apparatus for voice playback according to an exemplary embodiment. Figure 2 For example, device 300 can be provided as a server. (See reference...) Figure 11 The device 300 includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in the memory 332 may include one or more modules, each corresponding to a set of instructions.

[0202] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 358. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0203] Based on the same concept, this disclosure also provides a computer program product, wherein the computer program product includes a computer program. This computer program can be executed by a processor, and when executed by the processor, it can perform any of the voice playback methods described above.

[0204] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0205] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0206] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.

[0207] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0208] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0209] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0210] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A voice playback method, characterized in that, include: The text corresponding to the speech to be played is determined, and based on the text and the first model, the current scene style features are determined. The first model is used to generate scene style features based on the text, wherein the scene style features are used to represent the pronunciation style corresponding to the speech playback scene. The current scene style features and the text are synthesized to obtain the played speech.

2. The method according to claim 1, characterized in that, The process of synthesizing the current scene style features and the text to obtain the played speech includes: Identify text keywords in the text and determine the stress features corresponding to the text keywords, wherein the stress features characterize the intensity of syllable pronunciation being emphasized; The accent features, the current scene style features, and the text are synthesized to obtain the played speech.

3. The method according to claim 2, characterized in that, The process of identifying text keywords in the text and determining the stress features corresponding to the text keywords includes: Based on the text and the second model, text keywords are identified, and the stress features corresponding to the text keywords are determined. The second model is used to determine text keywords and their corresponding stress features based on the text.

4. The method according to claim 3, characterized in that, The second model was trained in the following manner: Obtain a first training sample, which includes a training sample pair consisting of a speech training sample and a text training sample corresponding to the speech training sample; Acoustic features are extracted from the first training sample to obtain the acoustic features corresponding to the text training sample; The acoustic features are clustered, and the target acoustic features corresponding to the text training samples are determined based on the clustering results. The target acoustic features are then identified as the accent features corresponding to the text training samples, wherein the feature value of the target acoustic features is greater than or equal to a feature value threshold. The second label is constructed using the target acoustic features, and the second model is trained based on the second label.

5. The method according to claim 1, characterized in that, Determining the current scene style features includes: Based on the first model and the text, the corresponding voice playback scene is determined, and based on the voice playback scene, scene style features that match the current voice playback scene are determined.

6. The method according to claim 5, characterized in that, The first model was trained in the following manner: The first initial model is invoked to classify the scene style features of the second training sample, and the scene style feature classification result corresponding to the second training sample is obtained. Based on the scene style feature classification results and the corresponding loss values, the parameters of the first initial model are adjusted to obtain the first model.

7. The method according to claim 2, characterized in that, The process of synthesizing the accent features, the current scene style features, and the text to obtain the played speech includes: Speech synthesis is performed based on the third model, the accent features, the style features, and the text to obtain the played speech, wherein the memory usage of the third model is less than or equal to a memory usage threshold.

8. The method according to claim 7, characterized in that, The third model is trained in the following manner: Speech synthesis is performed based on the second initial model and the third training sample, and the synthesized speech is used as the fourth training sample, wherein the memory usage of the second initial model is greater than the memory threshold. The second initial model is used as the teacher model, and distillation learning is performed based on the fourth training sample to obtain the student model. The student model is then identified as the third model, wherein the memory usage of the student model is less than or equal to a memory usage threshold.

9. A voice playback device, characterized in that, include: The determining unit is used to determine the text corresponding to the speech to be played, and to determine the current scene style features based on the text and the first model. The first model is used to generate scene style features based on the text, wherein the scene style features are used to represent the pronunciation style corresponding to the speech playback scene. The processing unit is used to synthesize the current scene style features and the text to obtain the played speech.

10. An electronic device, characterized in that, include: processor: Memory used to store processor-executable instructions; The processor is configured to execute the voice playback method according to any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores instructions that, when executed by a processor, enable the processor to perform the voice playback method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the voice playback method as described in any one of claims 1 to 8.

Images