Voice generation method, device, equipment, program product and robot

Abstract

The invention provides a voice generation method, device and equipment, a program product and a robot, and relates to the technical field of voice interaction. The voice generation method comprises the following steps: acquiring target emotional characteristic data from a target voice signal, wherein the target emotional characteristic data is constructed by acoustic characteristic parameters of the target voice signal; a target morpheme sequence corresponding to the target emotion feature data is generated through an emotion morpheme model, the target morpheme sequence comprises a target morpheme unit, and the target morpheme unit is associated with emotion attribute information; and generating a response voice signal corresponding to the target voice signal based on the target morpheme sequence.

Description

Technical Field

[0001] This specification relates to the field of voice interaction technology, and in particular to a voice generation method, apparatus, device, computer program product, and robot. Background Technology

[0002] Companion robots can achieve voice interaction through text-to-speech technology. For example, the robot can collect the user's voice signal and convert it into text data. Based on the text data, it performs natural language understanding to determine the user's intent and engages in dialogue based on that intent. The dialogue can be task-oriented, such as executing corresponding instructions based on the user's intent. It can also be conversational, such as generating system responses based on user questions, generating natural text from the system responses, and then converting the text to speech (e.g., using text-to-speech technology). However, traditional robot voice generation has significant problems in emotional expression, handling complex emotions, expressing multiple emotions, and regulating emotional intensity. For example, while traditional text-to-speech technology can synthesize clear speech, it has significant deficiencies in emotional subtlety, cuteness, and friendliness, making it difficult to meet the needs of companionship scenarios. Dialogue systems based on large language models often output formulaic expressions and patterns, lacking precise emotional control, which undermines the immersion and authenticity of the companionship experience.

[0003] Therefore, some embodiments of this specification provide a speech generation method that can maintain a cute and soft voice quality, accurately control emotional expression, and also has basic semantic carrying capacity. Summary of the Invention

[0004] This specification provides one or more embodiments of a speech generation method. The method includes: acquiring target emotion feature data from a target speech signal, the target emotion feature data being constructed from acoustic feature parameters of the target speech signal; generating a target morpheme sequence corresponding to the target emotion feature data using an emotion morpheme model, the target morpheme sequence including target morpheme units, the target morpheme units being associated with emotion attribute information; and generating a response speech signal corresponding to the target speech signal based on the target morpheme sequence.

[0005] In some embodiments, the emotion morpheme model includes an emotion morpheme library, which includes preset emotion morpheme units, each of which is associated with emotion attribute information; generating a target morpheme sequence corresponding to the target emotion feature data using the emotion morpheme model includes: determining the target morpheme unit from the emotion morpheme library based on the target emotion feature data; and generating the target morpheme sequence based on the target morpheme unit.

[0006] In some embodiments, generating a target morpheme sequence based on target morpheme units includes: determining a morpheme modulation method based on target emotion feature data; and generating a target morpheme sequence after modulating the target morpheme units based on the morpheme modulation method.

[0007] In some embodiments, generating a target morpheme sequence based on a target morpheme unit includes: determining the emotional intensity attribute of the target morpheme unit based on target emotional feature data; and generating a target morpheme sequence by modulating the target morpheme unit based on the emotional intensity attribute of the target morpheme unit.

[0008] In some embodiments, determining a target morpheme unit from an emotion morpheme library based on target emotion feature data includes: determining the target emotion polarity corresponding to the target emotion feature data based on the target emotion feature data; determining a reference emotion polarity that matches the target emotion polarity based on the target emotion polarity; and determining a morpheme unit with the reference emotion polarity from the emotion morpheme library as the target morpheme unit based on the reference emotion polarity.

[0009] In some embodiments, the emotion morpheme library contains multiple morpheme families, and different morpheme families are used to express emotions with different emotional directions.

[0010] In some embodiments, the emotion morpheme model is a trained machine learning model; generating a target morpheme sequence corresponding to the target emotion feature data using the emotion morpheme model includes: inputting the target emotion feature data into the emotion morpheme model to output the target morpheme sequence.

[0011] In some embodiments, the emotion morpheme model is trained by a training method, which includes: acquiring multiple training samples and training labels corresponding to each training sample, wherein the training samples include sample emotion feature data and the training labels include sample morpheme sequences; and training an initial model based on the multiple training samples and the training labels corresponding to the training samples to obtain the emotion morpheme model.

[0012] In some embodiments, generating a response speech signal corresponding to a target speech signal based on a target morpheme sequence includes: determining prosodic parameters corresponding to the target morpheme sequence; generating acoustic features based on the prosodic parameters and the target morpheme sequence; and generating a response speech signal based on the acoustic features.

[0013] In some embodiments, obtaining target emotion feature data includes: extracting acoustic feature parameters from the target speech signal; and converting the acoustic feature parameters into an acoustic feature vector, wherein the target emotion feature data includes the acoustic feature vector.

[0014] One or more embodiments of the specification also provide a speech generation apparatus, comprising: an acquisition module for acquiring target emotion feature data from a target speech signal, the target emotion feature data being constructed from acoustic feature parameters of the target speech signal; a morpheme generation module for generating a target morpheme sequence corresponding to the target emotion feature data using an emotion morpheme model, the target morpheme sequence including target morpheme units, the target morpheme units being associated with emotion attribute information; and a speech generation module for generating a response speech signal corresponding to the target speech signal based on the target morpheme sequence.

[0015] One or more embodiments of this specification also provide a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions, the computer executes the speech generation method of this specification.

[0016] This specification also provides a computer device for implementing a speech generation method. The computer device includes a processor and a memory. The memory stores a program for implementing the speech generation method. After the computer device is powered on and the processor runs the program for the speech generation method, the speech generation method of this specification can be implemented.

[0017] Some embodiments of this specification also provide a computer program product, including a computer program that, when at least a portion of the computer instructions are executed by a processor, can implement the speech generation method of this specification.

[0018] Some embodiments of this specification also provide a robot, which includes: a voice acquisition device for acquiring user voice data to generate a target voice signal; a processing device for generating a response voice signal for the target voice signal according to the voice generation method of this specification; and a voice output device for playing the response voice signal.

[0019] In some embodiments, the robot further includes a non-voice module; the processing device is also used to control the non-voice module based on the target emotion feature data or the emotion attribute information corresponding to the target morpheme sequence. Attached Figure Description

[0020] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. The same numbers in the drawings denote the same structures or steps.

[0021] Figure 1 This is a flowchart illustrating a speech generation method according to some embodiments of this specification.

[0022] Figure 2 This is a flowchart illustrating the process of determining target morpheme units according to some embodiments of this specification.

[0023] Figure 3 This is a schematic diagram illustrating the process of generating a response speech signal based on a target morpheme sequence, according to some embodiments of this specification.

[0024] Figure 4 This is a schematic diagram of a speech generation device according to some embodiments of this specification.

[0025] Figure 5 This is a schematic diagram of a computer device for implementing a speech generation method, according to some embodiments of this specification.

[0026] Figure 6 This is a schematic diagram illustrating an application scenario of voice interaction based on a speech generation method, according to some embodiments of this specification. Detailed Implementation

[0027] To more clearly illustrate the technical solutions of the embodiments in this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the content described below are some examples or embodiments of this specification. For those skilled in the art, without creative effort, the technical solutions or means disclosed in this specification can be applied to other scenarios based on this technical content.

[0028] It should be understood that the terms "system," "device," "unit," and / or "module" used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0029] Unless otherwise specified, the technical terms used to describe components, elements, etc. in this specification are not singular but may include plural. Generally speaking, terms such as "comprising" or "including" only indicate that explicitly identified steps, elements, or components are included, and these steps, elements, and components do not constitute an exclusive list, as the described method or apparatus may also include other steps or components.

[0030] This specification uses flowcharts to illustrate the operational steps performed by the apparatus or system of related embodiments. However, unless otherwise specified, the order in which these steps are described should not be construed as a limitation on the order of execution. Those skilled in the art can adjust the order of these steps based on the knowledge and information conveyed by the embodiments in this specification. Such adjustments include, but are not limited to, reversing the order of steps, merging multiple steps, and splitting a step.

[0031] The following are the main ways to achieve voice interaction in companion robots: (1) Text-to-speech technology: Although the existing text-to-speech (TTS) technology can synthesize clear speech, it has obvious defects in terms of emotional subtlety, cuteness, and affinity, and it is difficult to meet the needs of companion scenarios. (2) Large model technology: The output of dialogue systems based on large language models often has obvious formulaic expressions and routines, which destroys the immersiveness and authenticity of the companion experience. (3) Onomatopoeia system: Cuteness is created by using simple onomatopoeia (such as "hmm", "wow", "woo woo"), but these methods lack a systematic structure, making it difficult to express complex emotions and simple semantics, thus limiting the richness of interaction. (4) Emotion-acoustic feature mapping technology: This technology can preset different acoustic features corresponding to different emotions. However, this lack of precise control over the acoustic expression of emotions makes it impossible to achieve continuous adjustment of emotional intensity and combined expression of multiple emotions.

[0032] To overcome the problems of traditional voice interaction technology in terms of emotional expression, some embodiments of this specification propose a voice generation method. This method includes: obtaining target emotional feature data from a target speech signal, the target emotional feature data being constructed from the acoustic feature parameters of the target speech signal; generating a target morpheme sequence corresponding to the target emotional feature data using an emotional morpheme model, the target morpheme sequence including target morpheme units, the target morpheme units being associated with emotional attribute information; and generating a response speech signal corresponding to the target speech signal based on the target morpheme sequence.

[0033] This application constructs a structured system of emotional morphemes to express emotional information, attitudes, and interactive intentions without relying on natural language text. The system uses morpheme units as the smallest unit of expression, achieving continuity, controllability, and scalability of emotional expression through morpheme combinations and prosodic modulation. Furthermore, by associating each morpheme unit with emotional attributes such as emotional polarity, emotional intensity, morpheme modulation method, and morpheme family, the system can express not only discrete emotional types but also variations in emotional intensity and fine-grained emotional differences, thus conveying emotional states of varying intensities.

[0034] In some embodiments, the emotion morpheme system can be constructed based on a consonant-vowel (CV) structure to build an emotion morpheme library. The emotion morpheme library contains multiple morpheme families, and different morpheme families are used to express emotions with different emotional directions. Different morpheme families can be distinguished by different phoneme prefixes or phoneme combinations, for example, by forming an expandable set of morphemes through different vowel combinations, consonant extensions, or repetition structures.

[0035] Figure 1This is a schematic flowchart illustrating a speech generation method according to some embodiments of this specification. In some embodiments, process 100 may be executed by speech generation device 400 or computer device 500. In some embodiments, such as Figure 1 As shown, process 100 may include the following steps.

[0036] The speech generation method described in this manual can be applied to devices with audio output capabilities, such as companion robots, toys, wearable devices, and smart home terminals.

[0037] Step 110: Acquire target emotional feature data. In some embodiments, step 110 may be performed by the acquisition module 410. In some embodiments, step 110 may be performed by the processor 502 in the computer device 500.

[0038] Emotional feature data refers to acoustic feature data extracted from speech signals that can quantify and distinguish different emotions. Target emotional feature data can be acoustic feature data extracted from a target speech signal sent by a user. For example, during a user-robot voice interaction, the user's voice can be captured by the robot's own microphone to generate a target speech signal, which is then sent to the speech generation device 400 for extraction of target emotional feature data.

[0039] Target emotion feature data can be constructed based on acoustic feature parameters. Exemplary acoustic feature parameters may include pitch parameters, duration parameters, energy parameters, and phonological parameters. Pitch parameters may include the fundamental frequency mean, fundamental frequency range, and fundamental frequency dynamic profile. Duration parameters may include syllable duration, pause distribution, and speech rate coefficient. Energy parameters may include amplitude envelope, loudness curve, accent intensity, and short-duration energy. Phonological parameters may include spectral centroid (whether energy is concentrated in high or low frequencies), spectral flux (rate of spectral change), harmonic noise ratio, Mel frequency cepstral coefficients, formant position, formant bandwidth, formant trajectory, harmonic noise ratio, perturbations (small irregular changes in the fundamental frequency), amplitude perturbations, and glottal wave parameters.

[0040] In some embodiments, the processor can perform acoustic feature extraction on the target speech signal to obtain acoustic feature parameters, and then encode the acoustic feature parameters to generate target emotion feature data. The target emotion feature data can be represented in different forms; for example, it can be represented as an acoustic feature vector. Alternatively, it can be represented at different intensity scales (e.g., continuous numerical scale, ordinal scale, context-based dynamic scale, etc.). Furthermore, the target emotion feature data can also include context parameters. Context parameters refer to acoustic feature data extracted from previous or subsequent speech signals, excluding the acoustic feature parameters extracted based on the current target speech signal, during emotion feature data analysis.

[0041] The processor can extract acoustic feature parameters based on acoustic feature extraction algorithms. For example, acoustic feature extraction algorithms may include time-domain feature algorithms, frequency-domain feature algorithms, and cepstral domain feature algorithms. The processor can encode the acoustic feature parameters based on acoustic feature encoding algorithms to generate target emotion feature data. For example, acoustic feature encoding algorithms may include scalar numerical encoding algorithms, time-series sequence encoding algorithms, statistical feature encoding algorithms, and continuous spatial encoding algorithms.

[0042] In some embodiments, the processor 502 can directly use a trained machine learning model (also known as an emotion feature extraction model) to process the target speech signal and generate target emotion feature data. For example, the processor can input the target speech signal into the emotion feature extraction model, and the emotion feature extraction model can output target emotion feature data by processing the target speech signal. The emotion feature extraction model can be obtained by training an initial machine learning model (also known as a first initial model) based on multiple training samples.

[0043] Initial machine learning models can include multilayer perceptrons (MLP), convolutional neural networks (CNN), sequence models based on self-attention mechanisms (Transformer), long short-term memory networks (LSTM), recurrent neural networks (RNN), gated recurrent units (GRU), and other models.

[0044] Each training sample may include a speech sample. In some embodiments, each training sample may have a training label, which is a reference emotion feature vector extracted from the speech sample. Each training sample is used as input during the initial training of the machine learning model, and the training label may be used as a reference output (theoretical output) for the initial training of the machine learning model.

[0045] In some embodiments, the initial machine learning model training process can be an iterative optimization process to learn the mapping from input to output, or an iterative process. The iterative process can include multiple iterations. Each iteration can include generating predictions based on training samples, calculating the error (i.e., loss) between the predictions and training labels based on a loss function, calculating the gradient through backpropagation based on the loss function value, and updating the parameters of the initial machine learning model using an optimizer based on the gradient to obtain an initial machine learning model with updated parameters (also referred to as an intermediate machine learning model). In response to the iteration termination condition being met, the intermediate machine learning model with updated parameters generated in the current iteration is determined as the sentiment feature extraction model. In response to the iteration termination condition not being met, the next iteration continues. The intermediate machine learning model used in the next iteration is the intermediate machine learning model with updated parameters from the previous iteration.

[0046] In some embodiments, the target voice signal may be the user's voice information collected by the robot's voice acquisition device (e.g., a microphone). It is understood that in the specific embodiments of this specification, the collection, use, or processing of data (e.g., voice, etc.) requires the data subject's permission or consent when one or more embodiments of this specification are applied to specific products or technologies. Furthermore, the collection, use, or processing of related data must strictly comply with the relevant laws, regulations, and standards of the data source country, implementation country, and other relevant countries and regions. De-identification technology ensures that the final data used is securely processed and de-identified data, protecting the data subject's rights and data security.

[0047] Step 120: Generate a target morpheme sequence corresponding to the target emotion feature data using an emotion morpheme model. In some embodiments, step 120 may be performed by the morpheme generation module 420. In some embodiments, step 120 may be performed by the processor 502 in the computer device 500.

[0048] The target morpheme sequence is a sequence of morphemes from non-natural language. A sequence of morphemes from non-natural language refers to a sequence that is not expressed in the traditional form of natural language but has structured semantic units.

[0049] The target morpheme sequence comprises target morpheme units. Each target morpheme unit in the target morpheme sequence can be associated with one or more emotional attribute information. A morpheme unit is the smallest unit of information with semantic function.

[0050] The correspondence between a target morpheme sequence and target emotion feature data means that the emotion attribute information associated with the target morpheme unit in the target morpheme sequence matches the emotion attribute information reflected in the target emotion feature data. For example, the emotion type in the emotion attribute information associated with the target morpheme unit is consistent with the emotion type reflected in the target emotion feature data. Another example is that the emotion intensity in the emotion attribute information associated with the target morpheme unit is consistent with the emotion intensity reflected in the target emotion feature data.

[0051] Emotional attribute information can include emotional polarity attributes, emotional intensity attributes, morpheme modulation mode attributes, morpheme family attributes, etc.

[0052] The emotion polarity attribute indicates the direction and / or type of emotion expressed by the morpheme unit. Emotional direction can include positive, negative, neutral, etc. Each emotional direction can include multiple types of emotions. For example, positive emotions can include liking, closeness, positivity, happiness, joy, and affection. Negative emotions can include dislike, rejection, anger, fear, resentment, disgust, and sadness. Neutral emotions can include reassurance, comfort, hugging, attachment, calmness, gentleness, peace, and composure. In some embodiments, the emotional direction can also include complex emotions, such as pride, embarrassment, guilt, shame, and jealousy.

[0053] The emotion intensity attribute describes the strength or intensity range of the emotion expressed by a morpheme unit. In some embodiments, the strength of the emotion can be divided into different intensity levels, such as very weak, weak, medium, strong, and very strong. Each target morpheme unit can correspond to a different intensity.

[0054] The morpheme modulation attribute indicates how a morpheme changes during pronunciation. For example, modulation can include sustained vowel modulation, stressed vowel modulation, and repetition modulation. Sustained vowel modulation lengthens the phoneme (e.g., vowel or consonant) within a morpheme, and can be used to intensify emotion. Stress modulation increases the intensity of the phoneme's pronunciation within a morpheme, and can be used to express emphasis or strong intent. Repetition modulation repeats the phoneme within a morpheme, and can be used to express high-intensity emotional fluctuations or emphasis.

[0055] The morpheme family attribute is used to categorize morpheme units into sets of morphemes with similar emotional characteristics (i.e., the same emotional direction). For example, the morpheme family attribute can indicate the morpheme family to which a morpheme unit belongs. For instance, morpheme families can include the first morpheme family, the second morpheme family, the third morpheme family, etc. The first morpheme family is used to express positive or friendly emotions, and its morphemes have soft, rounded acoustic characteristics. The second morpheme family is used to express negative or rejecting emotions, and its morphemes have relatively tense or restrained acoustic characteristics. The third morpheme family is used to express comforting, dependent, or gentle emotions, and its morphemes have low, gentle acoustic characteristics.

[0056] In some embodiments, the emotional direction corresponding to the emotional polarity attribute associated with the same morpheme unit is the same as the emotional direction corresponding to the morpheme family attribute. For example, if the emotional polarity attribute of a morpheme unit is positive, then the morpheme unit's morpheme attribute is the first morpheme family, and the corresponding emotional direction is expressing positive or friendly emotions.

[0057] In some embodiments, the emotion morpheme model may include an emotion morpheme library. The emotion morpheme library may include multiple morpheme units. Each morpheme unit may be constructed from one or more phonemes. A morpheme unit may also be called a morpheme. A morpheme unit is the smallest unit that carries semantic information. A phoneme refers to the smallest unit that distinguishes phonetic forms.

[0058] In some embodiments, morpheme units in the emotional morpheme library can be constructed based on consonants, vowels, repeated phonemes, repeated syllables, etc. For example, morpheme units can be constructed based on a consonant-vowel structure. Another example is the vowel-vowel structure. Yet another example is the vowel-consonant-vowel structure. Yet another example is the consonant-vowel-consonant structure. In some embodiments, morpheme units can be constructed based on a repeated syllable structure. It should be noted that the specific spelling, syllable combination, or pronunciation features of the morpheme units do not constitute a limitation of this application; as long as the morpheme unit is used to carry emotional attributes and participate in the emotional speech generation process, it falls within the protection scope of this application.

[0059] For example, vowels in Mandarin Chinese can include a, o, e, i, u, ü, ai, ou, ao, ei, uo, ia, ie, iou, etc.; consonants can include b, p, m, n, d, t, l, z, c, s, etc. Based on the aforementioned vowels and consonants, morpheme units can be constructed, for example, mo (indicating positive emotions, liking, closeness, and positivity); oi (indicating negative emotions, rejection, discomfort, or anger); mu / mou (indicating soothing, gentleness, dependence, or emotional regulation).

[0060] In an emotion morpheme library, the morpheme units used to express different emotion types can be called emotion morpheme units. Each emotion morpheme unit can correspond to a specific emotion polarity attribute, representing the emotion type (also known as emotion polarity) corresponding to that emotion morpheme unit. Emotion morpheme units can be constructed into verbs used to express different emotion types (or polarities), such as like, happy, coquettish, dislike, angry, afraid, wronged, hug, etc.

[0061] In some embodiments, the emotion morpheme library may include multiple morpheme families, with different morpheme families used to express emotions of different emotional directions. Morpheme units within the same morpheme family correspond to the same or similar emotional features.

[0062] For example, as mentioned above, the morpheme families in the emotion morpheme library can include: a first morpheme family: used to express positive or friendly emotions, whose morpheme units have soft and rounded acoustic features; a second morpheme family: used to express negative or rejecting emotions, whose morpheme units have relatively tense or convergent acoustic features; and a third morpheme family: used to express comforting, dependent, or gentle emotions, whose morpheme units have low and mild acoustic features. It should be noted that the number, classification method, and naming method of the morpheme families in the emotion morpheme library can all be adjusted. For example, based on different emotional dimensions, application scenarios, or interaction needs, morpheme families can be divided into sets of different numbers or different emotional types, or different distinction rules can be used to classify emotion morpheme families; these variations are all within the scope of this application.

[0063] In some embodiments, different morpheme families can be distinguished by different phoneme prefixes or phoneme combinations, for example, by forming an expandable set of morphemes through different vowel combinations, consonant extensions, or repetition structures. Further, for example, morpheme units in the first morpheme family may share a common consonant prefix *m* and the vowel *oi*; morpheme units in the second morpheme family may share a common vowel *o*; and morpheme units in the third morpheme family may share a common consonant prefix *m* and the vowel *ou*.

[0064] As shown in Table 1 below, different phoneme combinations can be used to construct morpheme units representing different emotional polarities, thereby generating a mapping relationship between morpheme units in the emotional morpheme library and emotional polarities. .

[0065] In some embodiments, the emotion morpheme library may include morpheme units for representing referential relationships, also referred to as referential morpheme units. Referential relationships include "I," "you," "she," etc.

[0066] In some embodiments, referential relations can be constructed based on different phonemes or phoneme combinations to generate a mapping relationship between morpheme units in the emotion morpheme library and referential relations, as shown in Table 2 below. .

[0067] In some embodiments, the emotion corpus may also include basic sentence patterns composed of multiple morpheme units. Basic sentence patterns in the emotion corpus can be constructed based on combinations of different morpheme units. For example, the mapping between morpheme units and basic sentence patterns can be constructed based on the mapping between morpheme units and their referents, and the mapping between morpheme units and emotion verbs, as shown in Table 3 below. .

[0068] In some embodiments, generating a target morpheme sequence using an emotion morpheme model includes determining one or more target morpheme units from an emotion morpheme library based on target emotion features. The target morpheme sequence is formed by arranging the determined target morpheme units in a specific order. For example, the target emotion polarity corresponding to the target emotion feature data can be determined based on the target emotion feature data; a reference emotion polarity matching the target emotion polarity can be determined based on the target emotion polarity; and morpheme units possessing the reference emotion polarity can be determined as target morpheme units from the emotion morpheme library based on the reference emotion polarity. Determining one or more target morpheme units from the emotion morpheme library based on target emotion features can refer to… Figure 2 The detailed description in the text is as follows. In this article, morpheme units with emotional polarity can also be called emotional morpheme units, and morpheme units with reference emotional polarity can also be called target emotional morpheme units.

[0069] In some embodiments, the target morpheme units in the target morpheme sequence further include target referential morpheme units. The processor can determine a reference referential relationship based on the target speech signal corresponding to the target emotion feature data, and determine the target referential morpheme unit based on the reference referential relationship. The target emotion feature data is emotion feature data extracted from the target speech signal. The processor can perform semantic recognition on the target speech signal and determine the reference referential relationship based on the semantic recognition result. For example, if the reference referential relationship can be determined to be "I" based on the semantic recognition result, then the processor can determine the target referential morpheme unit corresponding to "I" from the emotion morpheme library.

[0070] In some embodiments, generating a target morpheme sequence based on a target morpheme unit further includes determining other emotional attribute information of the target morpheme unit, such as morpheme modulation method, emotional intensity attribute, etc. The target morpheme unit can be modulated based on this other emotional attribute information to generate the target morpheme sequence.

[0071] For example, generating a target morpheme sequence based on a target morpheme unit may include determining the morpheme modulation method based on target emotion features; and generating the target morpheme sequence after modulating the target morpheme unit based on the morpheme modulation method. In some embodiments, the morpheme adjustment method corresponding to the target morpheme unit may be determined based on a preset mapping relationship between emotion features and morpheme adjustment methods.

[0072] For example, generating a target morpheme sequence based on a target morpheme unit may include determining the emotional intensity attribute of the target morpheme unit based on the target emotional features; and generating a target morpheme sequence by modulating the target morpheme unit based on the emotional intensity attribute of the target morpheme unit.

[0073] In some embodiments, the emotion intensity attribute of the target morpheme unit may have a mapping relationship with the emotion intensity corresponding to the target emotion feature data. This mapping relationship may be preset. For example, when the emotion intensity corresponding to the target emotion feature data is extremely strong, the emotion intensity attribute of the target morpheme unit is extremely low.

[0074] In some embodiments, the mapping relationship between the emotion intensity attribute of the target morpheme unit corresponding to different emotion types and the emotion intensity corresponding to the target emotion feature data can be different. For example, when the emotion type is anger, the emotion intensity corresponding to the target emotion feature data and the emotion intensity of the target morpheme unit can be opposite. Further, for example, when the emotion type is anger, if the emotion intensity corresponding to the target emotion feature data is relatively strong, then the emotion intensity of the target morpheme unit can be relatively low. When the emotion type is happiness, the emotion intensity corresponding to the target emotion feature data and the emotion intensity of the target morpheme unit can be the same. Further, for example, when the emotion type is happiness, if the emotion intensity corresponding to the target emotion feature data is relatively strong, then the emotion intensity of the target morpheme unit can be relatively strong.

[0075] In some embodiments, the modulation method corresponding to the target morpheme unit can be further determined based on the emotional intensity attribute of the target morpheme unit. The processor can determine the modulation method of the morpheme unit corresponding to the target morpheme unit based on preset rules. For example, when the emotional intensity corresponding to the target morpheme unit is a first preset intensity, a long tone modulation method can be selected, which can express the change in emotional intensity through morpheme duration or extension. As another example, when the emotional intensity corresponding to the target morpheme unit is a second preset intensity, a repetition modulation method can be selected, which can express the change in emotional intensity through morpheme repetition or combination. As yet another example, when the emotional intensity corresponding to the target morpheme unit is a third preset intensity, an accent modulation method can be selected, which can express the change in emotional intensity through accent or energy change. Still another example, when the emotional intensity corresponding to the target morpheme unit is a fourth preset intensity, a pitch modulation method can be selected, which can express the change in emotional intensity through pitch contour change. The first, second, third, and fourth preset intensities can be system default settings. Different modulation methods are applied to the selected morpheme unit according to the emotional intensity in the target emotional parameters to express emotional states of different intensities. Some embodiments in this specification can express emotional intensity and fine-grained differences through morpheme families and modulation rules: emotional intensity is encoded by morpheme family division and modulation methods (including prolongation, repetition, stress, pitch change, etc.), so that the same emotional type can present different intensities and subtle differences, thereby enhancing the expressiveness and naturalness of emotional speech.

[0076] In some embodiments, an emotion morpheme system that is not a natural language can be constructed through an emotion morpheme library. Emotional expression is decomposed into morpheme units that can be described in a structured way. By associating emotion polarity, emotion intensity and modulation mode attributes with the morpheme units, they can carry emotional information without relying on natural language text, thereby forming an emotion expression language system that is independent of natural language.

[0077] In some embodiments, the emotion morpheme model can be a trained machine learning model (also known as a morpheme determination model). Generating a target morpheme sequence using the emotion morpheme model based on target emotion feature data includes inputting the target emotion features into the emotion morpheme model to output a target morpheme sequence.

[0078] Emotional morpheme models can be built based on models such as multilayer perceptron (MLP), convolutional neural network (CNN), sequence model based on self-attention mechanism (Transformer), long short-term memory network (LSTM), recurrent neural network (RNN), and gated recurrent unit (GRU).

[0079] In some embodiments, the sentiment morpheme model can be obtained by training an initial machine learning model (also known as a second initial model) based on training samples and their corresponding training labels. Training samples may include sample sentiment feature data. The sample label corresponding to each training sample may be a sequence of sample morphemes corresponding to the sample sentiment feature data. The training method for the sentiment morpheme model is similar to or the same as that for the sentiment feature extraction model, and will not be described again here.

[0080] In some embodiments, an emotion morpheme library and target emotion feature data can be input into an emotion morpheme model to output a target morpheme sequence.

[0081] Step 130: Generate a response speech signal corresponding to the target speech signal based on the target morpheme sequence. In some embodiments, step 130 may be performed by the speech generation module 430. In some embodiments, step 130 may be performed by the processor 502 in the computing device 500.

[0082] In some embodiments, the processor can predict the acoustic features corresponding to the response speech signal based on the target morpheme sequence, and generate the response speech signal based on the acoustic features. Further, the processor can determine the acoustic features based on the target morpheme sequence and the emotional attribute information (e.g., emotional polarity, emotional intensity, morpheme modulation mode, etc.) corresponding to the target morpheme units in the target morpheme sequence. For example, the processor can input the target morpheme sequence and the emotional attribute information corresponding to the target morpheme units in the target morpheme sequence into a deep learning model to generate acoustic features.

[0083] In some embodiments, the processor can determine prosodic parameters based on the target morpheme sequence and the emotion attribute information corresponding to the target morpheme units in the target morpheme sequence, generate acoustic features based on the prosodic parameters, and generate a response speech signal based on the acoustic features. For further description of generating a response speech signal based on the target morpheme sequence, please refer to... Figure 3 Detailed explanation.

[0084] In some embodiments, the processor can input the target morpheme sequence and the corresponding emotional attribute information into the trained machine learning model (i.e., the speech generation model) to generate a speech signal.

[0085] In some embodiments, the processor can acquire the user's personalized data and generate a speech signal based on the personalized data and a target morpheme sequence. Personalized data may include the user's voice style, timbre preferences, and expression habits. The processor can input the user's personalized data, the target morpheme sequence, and the corresponding emotional attribute information into a trained machine learning model (i.e., a speech generation model) to generate a speech signal.

[0086] In some embodiments, the processor may output the response voice signal. For example, a robot may output the response voice signal based on a built-in voice playback device (e.g., a speaker).

[0087] In some embodiments, the processor can control the operation of the robot's non-voice modules, such as the motion module, lighting module, and tactile module, based on target emotion feature data or emotion attribute information corresponding to the target morpheme sequence, to perform operations matching the target emotion feature data. For example, the processor can control the lighting module to emit lights of different intensities and colors based on the emotion type and intensity in the emotion attribute information corresponding to the target morpheme sequence. As another example, the processor can control the motion module to perform actions of different amplitudes and speeds based on the emotion type and intensity in the emotion attribute information corresponding to the target morpheme sequence.

[0088] The speech generation method provided in this manual can achieve controllable emotion expression through the generation path of "emotion → morpheme → prosody → speech". It drives the generation of morpheme sequences through emotion feature vectors and further combines prosodic parameters to generate speech signals, transforming emotion expression from a discrete label to a continuous and adjustable process. This overcomes the problem of unclear mapping between emotion and acoustic features in traditional emotion speech generation, and achieves continuity, controllability and scalability of emotion expression through morpheme combination and prosodic modulation.

[0089] Figure 2 This is a flowchart illustrating the determination of target morpheme units according to some embodiments of this specification. In some embodiments, the processor can determine the target morpheme unit according to... Figure 2 The process 200 involves determining one or more target morpheme units from an emotion morpheme library based on target emotion features. In some embodiments, process 300 may be executed by a speech generation device 400 or a computer device 500. For example, process 300 may be executed by a morpheme generation module 420. In some embodiments, process 200 includes the following steps.

[0090] Step 210: Determine the target emotion polarity corresponding to the target emotion feature data based on the target emotion feature data. The target emotion polarity corresponding to the target emotion feature data can represent the emotion type and / or emotion direction reflected by the target emotion feature data. In some embodiments, the target emotion polarity corresponding to the target emotion feature data can be determined based on a trained machine learning model (also called an emotion determination model). For example, the emotion determination model can be an emotion classifier. The processor can input the target emotion feature data into the emotion determination model, and the emotion determination model can output the target emotion polarity corresponding to the target emotion feature data. The emotion determination model can be obtained by training an initial machine learning model (also called a third initial model) based on training samples with emotion labels. The training samples can be sample emotion feature data, and the emotion label corresponding to each training sample is the sample emotion polarity corresponding to that sample emotion feature data. The training method of the emotion determination model is similar to or the same as the training method of the emotion feature extraction model, and will not be described again here.

[0091] In some embodiments, the target emotional polarity corresponding to the target emotional feature data can be determined based on rule matching. For example, a mapping relationship between emotional feature data and different emotional polarities can be preset, and the target emotional polarity corresponding to the target emotional feature data can be determined based on this mapping relationship.

[0092] In some embodiments, the target emotion polarity corresponding to the target emotion feature data can be determined based on an emotion template matching method. For example, multiple emotion templates can be preset, with different emotion templates corresponding to different emotion polarities. Each emotion template is composed of multiple different emotion feature data. The matching emotion template can be determined by calculating the similarity between the target emotion feature data and the emotion template, and the emotion polarity corresponding to the matching emotion template can be determined as the target emotion polarity.

[0093] Step 220: Determine a reference emotional polarity that matches the target emotional polarity based on the target emotional polarity.

[0094] A reference emotional polarity that matches the target emotional polarity can be the emotional polarity in which the target emotional polarity responds. For example, when a user interacts with a robot via voice, the robot needs to respond emotionally to the user's current expressed emotion; in this case, the reference emotional polarity is the emotional polarity in the robot's response. Further, for example, if the target emotional polarity is happiness, the reference emotional polarity could be happiness. Or, if the target emotional polarity is anger, the reference emotional polarity could be soothing, peaceful, or fearful. And, if the target emotional polarity is liking, the reference emotional polarity could be happiness or playful affection.

[0095] In some embodiments, a reference emotional polarity matching the target emotional polarity can be determined based on a preset mapping rule. The preset mapping rule can define reference emotional polarities corresponding to different target emotional polarities.

[0096] The target emotional polarity and the reference emotional polarity can be the same or different.

[0097] Step 230: Based on the reference emotional polarity, determine the target morpheme unit from the emotional morpheme library that has the reference emotional polarity.

[0098] Determining morpheme units with reference emotional polarity from an emotional morpheme database involves selecting morpheme units with the same emotional polarity as the reference emotional polarity based on the emotional polarity attributes of the morpheme units in the emotional morpheme database; these are the target morpheme units.

[0099] In some embodiments, determining morpheme units with reference emotional polarity from an emotional morpheme library based on reference emotional polarity includes determining a target morpheme family with the same reference emotional polarity from the emotional morpheme library, and further selecting target morpheme units from the target morpheme family based on reference emotional polarity.

[0100] Figure 3 This is a flowchart illustrating a method for generating a response speech signal based on a target morpheme sequence according to some embodiments of this specification. In some embodiments, process 300 may be executed by a speech generation device 400 or a computer device 500. For example, process 300 may be executed by a speech generation module 430. In some embodiments, such as Figure 3 As shown, process 300 may include the following steps.

[0101] Step 310: Determine the prosodic parameters corresponding to the target morpheme sequence.

[0102] Prosodic parameters refer to linguistic parameters that describe auditory perception patterns such as rhythm, intonation, and stress in speech. Prosodic parameters can include duration, energy, and pitch parameters. Duration parameters can include syllable / phoneme duration, pause duration, and speech rate. Syllable / phoneme duration indicates how long a syllable should last. Pause duration indicates how long the pause should be between words, sentences, or phonemes. Speech rate indicates how fast the speech signal is played. Pitch parameters include fundamental frequency profile (also called pitch contour), stress position, and intonation boundaries. Fundamental frequency profile indicates the overall shape of pitch fluctuations in a sentence (e.g., rising, falling, level). Stress position indicates which word, syllable, or phoneme needs emphasis (manifested as a higher pitch and longer duration). Intonation boundaries indicate the boundary pitch pattern of a phrase or sentence. Energy parameters can include stress intensity. Stress intensity indicates the degree of stress (strong, medium, weak).

[0103] In some embodiments, the processor can perform textual linguistic analysis on the target morpheme sequence to predict the prosodic parameters corresponding to the target morpheme sequence. For example, prosodic structure can be predicted based on textual linguistic analysis, such as prosodic hierarchy division, stress position prediction, and intonation contour prediction; the prosodic structure is then converted into specific numerical parameters. Furthermore, since different emotion types, emotion intensities, and morpheme modulation methods correspond to different prosodic parameters, the numerical parameters can be adjusted based on the emotion attribute information (e.g., emotion polarity, emotion intensity, morpheme modulation method) corresponding to the target morpheme sequence, according to preset rules, to determine the prosodic parameters. For example, when the emotion polarity corresponding to the target morpheme sequence is "happy," the pitch parameter can be adjusted to be higher, the speech rate to be faster, and the energy to be stronger.

[0104] In some embodiments, the processor can use a trained machine learning model (also known as a prosody determination model) to process the target morpheme sequence and predict the prosodic parameters corresponding to the target morpheme sequence. For example, the target morpheme sequence and / or the emotional attribute information corresponding to the target morpheme sequence can be input into the prosody determination model to output prosodic parameters. The prosody determination model can be obtained by training an initial machine learning model (also known as a fourth initial model) based on training samples. Each training sample may include a sample morpheme sequence. Each training sample may include a training label, which may be a reference prosodic feature corresponding to the sample morpheme sequence. The training method of the prosody determination model is similar to that of the emotional feature extraction model, and will not be repeated here.

[0105] Step 320: Generate acoustic features based on prosodic parameters and target morpheme sequence.

[0106] The processor can generate acoustic features based on acoustic feature generation algorithms. These algorithms can include statistical parametric speech synthesis algorithms, end-to-end generation algorithms based on neural network models, generation algorithms based on the FastSpeech2 architecture, and generation algorithms based on diffusion models. For example, in an end-to-end generation algorithm based on a neural network model, the processor can input the target morpheme sequence and prosodic parameters into a trained machine learning model (also known as an acoustic feature generation model), which can then directly output acoustic features (or acoustic feature sequences).

[0107] Step 330: Generate a response speech signal based on acoustic features.

[0108] Processors can convert acoustic features into speech signals based on speech synthesis models. Speech synthesis models can include Griffin-Lim models, natural regression models, streaming models, GAN models, etc.

[0109] This specification also provides a speech generation device in its embodiments. Figure 4This is a schematic diagram of a speech generation apparatus according to some embodiments of this specification. The speech generation apparatus 400 may include an acquisition module 410, a morpheme generation module 420, and a speech generation module 430.

[0110] The acquisition module 410 can be used to acquire target emotion feature data from the target speech signal, the target emotion feature data being constructed from the acoustic feature parameters of the target speech signal. For example, the acquisition module 410 can process the target speech signal based on an emotion feature extraction model to determine the target emotion feature data.

[0111] The morpheme generation module 420 can be used to generate a target morpheme sequence corresponding to target emotion feature data using an emotion morpheme model. The target morpheme sequence includes target morpheme units, and each target morpheme unit is associated with emotion attribute information. In some embodiments, the emotion morpheme model may include an emotion morpheme library. The morpheme generation module 420 generates the target morpheme sequence using the emotion morpheme model by determining one or more target morpheme units from the emotion morpheme library based on the target emotion features. The target morpheme sequence is formed by arranging the determined target morpheme units in a certain order. For example, the morpheme generation module 420 can determine the target emotion polarity corresponding to the target emotion feature data based on the target emotion feature data; determine a reference emotion polarity matching the target emotion polarity based on the target emotion polarity; and determine morpheme units possessing the reference emotion polarity from the emotion morpheme library as target morpheme units based on the reference emotion polarity. Determining one or more target morpheme units from the emotion morpheme library based on the target emotion features can refer to… Figure 2 The detailed description is in the text.

[0112] In some embodiments, the emotion morpheme model can be a trained machine learning model. The morpheme generation module 420 can input target emotion feature data into the emotion morpheme model to output a target morpheme sequence.

[0113] The speech generation module 430 can be used to generate a response speech signal corresponding to the target speech signal based on the target morpheme sequence. In some embodiments, the speech generation module 430 can predict the acoustic features corresponding to the response speech signal based on the target morpheme sequence, and generate the response speech signal based on the acoustic features. Further, the speech generation module 430 can determine the acoustic features based on the target morpheme sequence and the emotional attribute information (e.g., emotional polarity, emotional intensity, morpheme modulation mode, etc.) corresponding to the target morpheme units in the target morpheme sequence. For example, the speech generation module 430 can input the target morpheme sequence and the emotional attribute information corresponding to the target morpheme units in the target morpheme sequence into a deep learning model to generate acoustic features.

[0114] In some embodiments, the speech generation module 430 can determine prosodic parameters based on the target morpheme sequence and the emotion attribute information corresponding to the target morpheme units in the target morpheme sequence, generate acoustic features based on the prosodic parameters, and generate a response speech signal based on the acoustic features. For further description of generating a response speech signal based on the target morpheme sequence, please refer to... Figure 3 Detailed explanation.

[0115] This specification also provides an embodiment of a computer device for implementing a speech generation method. Figure 5 This is a schematic diagram of a computer device for implementing a speech generation method, according to some embodiments of this specification. Figure 5 The computer equipment shown may be Figure 4 Another specific embodiment of the speech generation device shown. For example... Figure 5 As shown, in some embodiments, the computer device 500 includes a processor 502 and a memory 504. The memory stores a program for implementing the speech generation method. After the computer device is powered on and the program for the speech generation method is run by the processor, it can implement the functions described in this specification. Figure 1-3 The speech generation method shown.

[0116] Some embodiments of this specification also provide a computer program product, including a computer program that, when at least a portion of the computer instructions are executed by a processor, can implement this specification. Figure 1-3 The speech generation method is illustrated. In some embodiments, the computer program product may involve only computer instructions, which may be carried on a storage medium or processing device. In other embodiments, the computer program product may also be a storage medium or processing device containing the aforementioned computer instructions. The processing device may include one or more processors, and the storage medium.

[0117] In some embodiments, the processor may be a combination of one or more of the following processors: central processing unit (CPU), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), graphics processing unit (GPU), physical processing unit (PPU), digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic device (PLD), programmable logic controller (PLC), reduced instruction set computer (RISC), and microprocessor.

[0118] In some embodiments, the storage medium may include one or more combinations of the following: mass storage, removable storage, volatile read-write memory, and read-only memory (ROM). Exemplary mass storage may include disks, optical disks, solid-state drives, etc. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, compressed hard disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), dual data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), silicon controlled retrieval memory (T-RAM), and zero-capacitance memory (Z-RAM), etc. Exemplary read-only memory may include masked read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compressed hard disk read-only memory (CD-ROM), and digital multifunction hard disk read-only memory, etc.

[0119] Some embodiments of this specification also provide a computer-readable storage medium for storing computer instructions, which, when read by a computer, enable the computer to implement this specification. Figure 1-3 The speech generation method shown.

[0120] Some embodiments of this specification also provide a robot, which may include a speech generation device 400 provided in some embodiments of this specification or a computer device 500 for implementing a speech generation method.

[0121] Figure 6 These are schematic diagrams illustrating application scenarios of voice interaction based on the voice generation method according to some embodiments of this specification. The voice generation method in this specification can be applied to devices with audio output capabilities, such as robots, toys, wearable devices, and smart home terminals.

[0122] The following description uses a robot as an example and is not intended to limit the scope of this specification. Application scenario 600 may include robot 610, processing device 620, storage device 630, terminal 640, and network 650.

[0123] Robot 610 may include a voice acquisition device (e.g., a microphone), a voice output device (e.g., a speaker), etc. The voice acquisition device can be used to acquire user voice data to generate a target voice signal. The voice output device can be used to play a response voice signal generated based on the voice generation method described in this specification in response to the target voice signal. In some embodiments, robot 610 may also include non-voice modules, such as motion modules, lighting modules, tactile modules, etc.

[0124] Processing device 620 can process data and / or information obtained from other devices or system components. In some embodiments, processing device 620 can perform the speech generation methods shown in some embodiments of this specification to generate a response speech signal for a target speech signal. In some embodiments, processing device 620 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-chip processing device). Specific implementations of processing device 620 may include, for example... Figure 4 The speech generation device 400 described herein or as such Figure 5 The aforementioned computer device 500 is used to implement a speech generation method. For example, the processing device 620 can acquire target emotion feature data from a target speech signal, the target emotion feature data being constructed from the acoustic feature parameters of the target speech signal; generate a target morpheme sequence corresponding to the target emotion feature data using an emotion morpheme model; and generate a response speech signal corresponding to the target speech signal based on the target morpheme sequence.

[0125] In some embodiments, the processing device 620 can control the robot 610 to perform corresponding operations. For example, the processing device 620 can control the non-voice modules of the robot 610, such as the motion module, light module, and tactile module, based on the target emotion feature data or the emotion attribute information corresponding to the target morpheme sequence, to perform operations matching the target emotion feature data. Further, for example, the processing device 620 can control the light module to emit lights of different intensities and colors based on the emotion type and intensity in the emotion attribute information corresponding to the target morpheme sequence. As another example, the processing device 620 can control the motion module to perform actions of different amplitudes and speeds based on the emotion type and intensity in the emotion attribute information corresponding to the target morpheme sequence.

[0126] In some embodiments, the processing device 620 may be integrated into the robot 610.

[0127] Storage device 630 can store data or information generated by other devices. In some embodiments, storage device 630 can store data and / or information collected by the robot's voice acquisition device, such as sound data. In some embodiments, storage device 630 can store data and / or information processed by processing device 620, such as response voice signals. Storage device 630 may include one or more storage components, each of which may be a separate device or part of other devices. The storage device may be local or implemented via the cloud.

[0128] In some embodiments, the storage device 630 may be integrated into the robot 610.

[0129] Terminal 640 can control the operation of robot 610. Users can issue operation commands to robot 610 through terminal 640 to cause robot 610 to perform specified operations, such as turning on robot 610 to interact with it. In some embodiments, terminal 640 can instruct processing device 620 to execute a speech generation method as shown in some embodiments of this specification.

[0130] Network 650 can connect the various components of the system and / or connect the system to external resources. Network 650 enables communication between the components and with other parts outside the system, facilitating the exchange of data and / or information. In some embodiments, one or more components in application scenario 600 (e.g., robot 610, processing device 620, storage device 630, terminal 640) can send data and / or information to other components via network 650. In some embodiments, network 650 can be any one or more of a wired network or a wireless network.

[0131] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the content of this specification. Features, structures, methods, and other features of the exemplary embodiments described herein can be combined in various ways to obtain other and / or alternative exemplary embodiments. For example, the processing device 620 may be based on a cloud computing platform, such as a public cloud, private cloud, community cloud, and hybrid cloud. However, these changes and modifications will not depart from the scope of this specification.

[0132] For more information on each module, please refer to [link / reference]. Figure 1-3 The relevant explanations will not be repeated here. It should be understood that... Figure 4The apparatus and modules shown can be implemented in various ways. For example, in some embodiments, the system and modules can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the methods and systems described above can be implemented using computer-executable instructions and / or included in the control code of a processor, such as on a media such as a disk, CD, or DVD-ROM, or in the memory of a programmable device. The systems and modules of this specification can be implemented not only with hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips or transistors, or programmable hardware devices such as field-programmable gate arrays or programmable logic devices, but also with software, for example, executed by various types of processors, or with a combination of the aforementioned hardware circuits and software (e.g., firmware).

[0133] It should be noted that the above description of the system and its modules is for convenience only and should not be construed as limiting this specification to the embodiments described. It is understood that those skilled in the art, after understanding the principles of this system, may arbitrarily combine the various modules without departing from these principles to form subsystems connected to other modules. Alternatively, some modules may be split to obtain more modules or multiple units under a single module. Such modifications are all within the scope of this specification.

[0134] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) proposing a non-natural language emotion morpheme system: for the first time, emotion expression is decomposed into morpheme units that can be described in a structured way, and by associating emotion polarity, emotion intensity and modulation mode attributes with morpheme units, it can carry emotion information without relying on natural language text, thereby forming an emotion expression language system independent of natural language; (2) using "emotion → morpheme → rhythm → The generation path of “speech” realizes controllable expression of emotion: the generation of morpheme sequence is driven by emotion feature vector, and further combined with prosodic parameters to generate speech signal, so that the expression of emotion is transformed from discrete label to continuous adjustable process, breaking through the problem of unclear mapping of emotion and acoustic features in traditional emotional speech generation. The continuity, controllability and scalability of emotional expression are realized through morpheme combination and prosodic modulation; (3) The expression of emotional intensity and fine-grained difference is realized through morpheme family and modulation rules: the encoding of emotional intensity is realized through morpheme family division and modulation method (including extension, repetition, stress, pitch change, etc.), so that the same emotion type can present different intensities and subtle differences, enhancing the expressiveness and naturalness of emotional speech; (4) The emotional morpheme system has scalability independent of specific pronunciation form: this application focuses on the structural and functional attributes of morpheme unit in emotional expression, rather than its specific syllable or pronunciation form, so that different vocal units are used in different embodiments (e.g. different syllable structures or different pronunciation combinations), ensuring the universality of the technical solution under different product forms and design styles. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

[0135] The basic concepts have been described above. It is obvious that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this specification by those skilled in the art. Such modifications, improvements, and corrections are taught in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

Claims

1. A speech generation method, characterized in that, The method includes: Acquire target emotion feature data, which is constructed from the acoustic feature parameters of the target speech signal; A target morpheme sequence corresponding to the target emotion feature data is generated using an emotion morpheme model. The target morpheme sequence includes target morpheme units, and each target morpheme unit is associated with emotion attribute information. A response speech signal corresponding to the target speech signal is generated based on the target morpheme sequence.

2. The method according to claim 1, characterized in that, The emotion morpheme model includes an emotion morpheme library, which includes preset emotion morpheme units. Each emotion morpheme unit in the preset emotion morpheme units is associated with emotion attribute information. The step of generating the target morpheme sequence corresponding to the target emotion feature data using an emotion morpheme model includes: The target morpheme unit is determined from the emotion morpheme library based on the target emotion feature data; as well as The target morpheme sequence is generated based on the target morpheme unit.

3. The method according to claim 2, characterized in that, The step of generating the target morpheme sequence based on the target morpheme unit includes: Determine the morpheme modulation method based on the target emotion feature data; and The target morpheme sequence is generated by modulating the target morpheme unit based on the morpheme modulation method.

4. The method according to claim 2, characterized in that, The step of generating the target morpheme sequence based on the target morpheme unit includes: Based on the target emotion feature data, determine the emotion intensity attribute of the target morpheme unit; and The target morpheme sequence is generated by modulating the target morpheme unit based on the emotion intensity attribute of the target morpheme unit.

5. The method according to any one of claims 2-4, characterized in that, The step of determining the target morpheme unit from the emotion morpheme library based on the target emotion feature data includes: Determine the target emotion polarity corresponding to the target emotion feature data based on the target emotion feature data; Based on the target emotional polarity, a reference emotional polarity matching the target emotional polarity is determined; and Based on the reference emotional polarity, the morpheme unit possessing the reference emotional polarity is determined from the emotional morpheme library as the target morpheme unit.

6. The method according to any one of claims 2-5, characterized in that, The emotion morpheme library contains multiple morpheme families, and different morpheme families are used to express emotions with different emotional directions.

7. The method according to claim 1, characterized in that, The emotion morpheme model is a trained machine learning model; the step of generating a target morpheme sequence corresponding to the target emotion feature data using the emotion morpheme model includes: The target emotion feature data is input into the emotion morpheme model to output the target morpheme sequence.

8. The method according to claim 7, wherein the emotion morpheme model is trained by a training method comprising: Obtain multiple training samples and training labels corresponding to each training sample. The training samples include sample sentiment feature data, and the training labels include sample morpheme sequences. as well as The emotion morpheme model is obtained by training an initial model based on the multiple training samples and the training labels corresponding to the training samples.

9. The method according to any one of claims 1-8, characterized in that, The step of generating a response speech signal corresponding to the target speech signal based on the target morpheme sequence includes: Determine the prosodic parameters corresponding to the target morpheme sequence; Generate acoustic features based on the prosodic parameters and the target morpheme sequence; and The response speech signal is generated based on the acoustic features.

10. The method according to any one of claims 1-9, wherein acquiring the target emotion feature data comprises: Extract the acoustic feature parameters from the target speech signal; as well as The acoustic feature parameters are converted into acoustic feature vectors, and the target emotion feature data includes the acoustic feature vectors.

11. A speech generation device, characterized in that, The device includes: The acquisition module is used to acquire target emotion feature data, which is constructed from the acoustic feature parameters of the target speech signal; A morpheme generation module is used to generate a target morpheme sequence corresponding to the target emotion feature data using an emotion morpheme model. The target morpheme sequence includes target morpheme units, and each target morpheme unit is associated with emotion attribute information. The speech generation module generates a response speech signal corresponding to the target speech signal based on the target morpheme sequence.

12. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it is able to implement the method as described in any one of claims 1 to 10.

13. A computer program product, characterized in that, It includes a computer program that, when at least a portion of the computer program is executed by a processor, enables the implementation of the method as described in any one of claims 1 to 10.

14. A robot, characterized in that, include: A voice acquisition device is used to collect users' voice data to generate target voice signals; A processing device for generating the response speech signal for the target speech signal according to any one of claims 1 to 10; A voice output device for playing the response voice signal.

15. The robot according to claim 14, characterized in that, Also includes: Non-voice module; The processing device is also used to control the non-speech module based on the target emotion feature data or the emotion attribute information corresponding to the target morpheme sequence.

Images