Text sequence playback method, device, electronic device, and storage medium

Abstract

The embodiments of the present application provide a method, device, electronic device, and storage medium for playing a text sequence. The method includes: identifying first emotion features corresponding to multiple speech segments in a speech library, identifying second emotion features corresponding to a text sequence to be played; selecting a target emotion feature that matches the second emotion feature from multiple first emotion features; determining the rhythmic features of all speech segments corresponding to the target emotion feature, wherein the rhythmic features include at least: pitch features, sound length features, and sound intensity features; synthesizing the synthesized audio corresponding to the text to be played according to the indication of the rhythmic features, and playing the synthesized audio. The method is used to achieve the effect of improving the playback effect of the synthesized audio corresponding to the text sequence, so that the emotion of the synthesized audio is consistent with the emotion of the text sequence, and thus closer to the effect of real human pronunciation.

Description

Technical Field

[0001] The present application relates to the field of audio technology, and in particular to a method, device, electronic device, and storage medium for playing a text sequence. Background Art

[0002] With the continuous development of speech synthesis technology, models trained based on existing corpora have made significant progress in generating naturalness and pronunciation accuracy. However, existing corpora usually have the problem of single speakers and lack of rhythm and emotional expression, resulting in insufficient naturalness and pronunciation accuracy of augmented pronunciation data, which limits the performance of the model in actual application scenarios. The lack of rich rhythmic changes and emotional expressions in speech samples makes the synthesized speech appear stiff and unnatural when expressing emotions and intonation. Due to the above problems, the naturalness and pronunciation accuracy of the synthesized speech are affected, especially in high-demand application scenarios, and the user experience is poor.

[0003] Therefore, there is an urgent need for a method for playing text that combines emotional features to at least solve the problems of low naturalness and low pronunciation accuracy of current speech. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, and storage medium for playing a text sequence, so as to achieve the effect of improving the naturalness and pronunciation accuracy of synthesized speech.

[0005] In a first aspect, an embodiment of the present application provides a method for playing a text sequence, comprising: identifying first emotion features corresponding to a plurality of sample speech segments in a speech library, respectively, and identifying a second emotion feature corresponding to a text sequence to be played; selecting a target emotion feature matching the second emotion feature from a plurality of first emotion features; determining a first rhythm feature of all speech segments corresponding to the target emotion feature, wherein the first rhythm is used to characterize the pronunciation method corresponding to the sample speech segment; synthesizing a synthesized audio corresponding to the text sequence to be played according to an indication of the rhythm feature, and playing the synthesized audio.

[0006] In a second aspect, an embodiment of the present application provides a device for playing a text sequence, comprising: an identification module, used to identify first emotion features corresponding to a plurality of sample speech segments in a speech library, and identify second emotion features corresponding to a text sequence to be played; a matching module, used to screen out a target emotion feature that matches the second emotion feature from a plurality of first emotion features; a determination module, used to determine a first rhythm feature of all speech segments corresponding to the target emotion feature, wherein the first rhythm is used to characterize the pronunciation mode corresponding to the sample speech segment; a synthesis module, used to synthesize a synthesized audio corresponding to the text to be played according to an indication of the rhythm feature, and play the synthesized audio.

[0007] In a third aspect, an embodiment of the present application provides a text sequence playback device, including: a memory, a processor;

[0008] The memory stores computer-executable instructions;

[0009] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0011] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0013] Figure 1 Schematic diagram of the process of playing the text sequence provided in this application Figure 1 ;

[0014] Figure 2 Schematic diagram of the process of playing the text sequence provided in this application Figure 2 ;

[0015] Figure 3 Schematic diagram of the process of generating emotional features provided for this application;

[0016] Figure 4 A schematic diagram of the optimization process provided for this application;

[0017] Figure 5 A schematic diagram of the structure of a device for playing a text sequence provided in this application;

[0018] Figure 6 A schematic diagram of the structure of a text sequence playback device provided in this application.

[0019] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0021] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0022] Figure 1 Schematic diagram of the process of playing the text sequence provided in this application Figure 1 ,like Figure 1 As shown, the method includes:

[0023] S101, identifying first emotion features corresponding to a plurality of sample speech segments in a speech library, and identifying second emotion features corresponding to a text sequence to be played;

[0024] Analyze the existing voice samples in the voice library and identify the emotional characteristics expressed by each sample. These emotional characteristics may include happiness, sadness, anger, surprise, etc. Through technical means such as acoustic feature analysis and emotion recognition algorithms, each voice sample can be automatically labeled with an emotion.

[0025] The system needs to analyze the text content of the speech to be synthesized and determine the emotional characteristics it should express. This may require natural language processing technology, such as sentiment analysis, to understand the emotional tendency of the text.

[0026] S102, selecting a target emotion feature that matches the second emotion feature from a plurality of first emotion features;

[0027] Using machine learning algorithms, the emotional features in the speech signal, such as intonation, rhythm, and intensity, are analyzed and matched with the target emotional features. Common algorithms include support vector machines (SVM) and neural networks. Key parameters such as frequency, amplitude, and duration are extracted from multiple first emotional features, and the most representative feature subset is selected through feature selection algorithms to improve matching accuracy and efficiency.

[0028] Optionally, the similarity calculation and comparison uses methods such as cosine similarity or Euclidean distance to calculate the similarity between various emotion features, so as to find the target emotion feature closest to the second emotion feature. This step is the core of the matching process.

[0029] Optionally, use the labeled emotion data to train the matching model and validate it on an independent test dataset to ensure the generalization and accuracy of the model. Continuously optimize the model parameters to improve the matching effect.

[0030] S103, determining a first prosody feature of all speech segments corresponding to the target emotion feature, wherein the first prosody is used to characterize a pronunciation mode corresponding to the sample speech segment;

[0031] Optionally, prosodic features play an important role in speech emotion recognition, including changes in pitch, duration, speed, and weight, etc. These features not only affect the naturalness and fluency of speech, but also reflect the emotional state of the speaker.

[0032] The extraction of prosodic features includes parameters such as duration, fundamental frequency and energy. By calculating the mean, variance and other statistics of these parameters, the emotional changes of speech can be effectively characterized. Commonly used methods include short-time Fourier transform (STFT) and Mel-frequency cepstral coefficient (MFCC).

[0033] There are significant differences in the prosodic features under different emotional states. For example, the energy of speech is higher when angry, while the fundamental frequency is lower when sad. These features provide an important basis for emotion recognition and help improve the naturalness and emotional expression ability of speech synthesis.

[0034] S104: synthesize a synthesized audio corresponding to the to-be-played text sequence according to the indication of the rhythmic features, and play the synthesized audio.

[0035] Optionally, the input speech text to be synthesized is normalized to convert it into standardized text, including basic encoding conversion (such as conversion of traditional Chinese characters into simplified Chinese characters), standardization of date and time formats, etc.

[0036] The regularized text is processed through a pre-trained prosodic model, which can predict prosodic information such as pause levels and intonation changes based on the text content and generate prosodic text containing this prosodic information.

[0037] The prosodic text is further converted into a phoneme sequence and a prosodic sequence at the phoneme level. The phoneme sequence is the basic pronunciation unit of the text, while the prosodic sequence contains the prosodic features such as the duration and fundamental frequency of each phoneme.

[0038] The phoneme sequence and the phoneme-level prosody sequence are input into the acoustic model to predict the mel-spectrogram. In this process, the acoustic model takes the prosodic features into account to ensure that the generated spectrogram reflects the natural rhythm of the text.

[0039] Finally, a vocoder is used to convert the Mel-spectrogram into a final speech signal. This process involves converting the frequency information in the spectrogram into waveform data in the time domain, thereby generating audible audio.

[0040] Figure 2 Schematic diagram of the process of playing the text sequence provided in this application Figure 2 , Figure 2 As shown, optionally, the pronunciation rhythm control structure of the fusion emotion is mainly composed of a fine-grained emotion feature generation module E_gen, a text encoder E_T and a variable information adaptation unit V_a. The rhythm feature encoder E_r extracts and encodes the rhythm features of the out-of-domain speech S_out. On this basis, the emotion feature generator E_g generates the corresponding emotion feature E_1 by adjusting the emotion contained in the fundamental tone. The unit divides the target emotion into five different emotion vectors: neutral, happy, fearful, sad and angry, and uses a 512-dimensional learnable vector to represent these different emotions. When generating the input Mongolian text sequence T, the text encoder E_T first encodes it, and then generates a context vector sequence E_2 by introducing an attention mechanism. E_1 and E_2 are spliced ​​to obtain the fused feature E_3. The variable information adaptation unit V_a can predict and adjust the subtle rhythm features that affect the expression of emotions, such as parameters such as pitch, length and intensity. By predicting and adjusting the fusion feature E_3, the rhythmic representation and emotional expression of the feature are improved, and the Mel spectrum is generated after decoding by the decoder D. Finally, the Waveglow vocoder is used to generate the pronunciation data S_new with diverse emotional expressions, thereby improving the natural fluency and pronunciation accuracy of the augmented pronunciation data.

[0041] Figure 3 The flowchart of the emotion feature generation provided for this application is as follows: Figure 3 The fine-grained emotion feature generation module E_gen shown in the figure consists of a prosodic feature encoder E_r and an emotion feature generator E_g. Among them, E_r captures the prosodic feature representations of different speakers from the input out-of-domain speech data S_out. E_g generates corresponding emotion features based on these prosodic feature representations, thereby ensuring that the model can more accurately fit the acoustic features in the out-of-domain speech data, thereby improving the naturalness of the generated pronunciation.

[0042] The out-of-domain speech is used as the initial input, and the prosodic feature encoder E rExtract the prosodic features related to emotional expression, including changes in pitch, duration, and intensity. Combine prosodic features with emotional embedding to form an embedding vector that integrates emotional and prosodic information. Use Conformer Block to analyze the intrinsic correlation between signal frames in the Mel spectrum, and then extract the prosodic features of the out-of-domain speaker. The calculation is shown in formula (1-1). Where F prosody Represents the extracted prosodic feature representation.

[0043] F prosody =ConformerEncoder(X mel ) (1-1)

[0044] These rhythmic features are then passed to the emotion feature generation module E g , to generate diverse emotional features. The emotional feature generator contains three layers of linear structure, each layer uses ReLU as the activation function to enhance the expressiveness of the feature. The calculation of each layer is shown in formulas (1-2) to (1-4).

[0045] h1=ReLU(Linear1(Fprosody)) (1-2)

[0046] h2=ReLU(Linear2(h1)) (1-3)

[0047] h3=ReLU(Linear3(h2)) (1-4)

[0048] The prosodic feature encoder extracts fine-grained prosodic features from out-of-domain speech. These features are then input into the emotion feature generator. The emotion feature generator contains a three-layer linear structure, each layer uses ReLU as the activation function. In addition, a Softmax layer is introduced to refine the emotion representation under different prosodic features. Its calculation process is listed in formula (1-5). Where E emotion Represents the generated emotion. The emotion feature representation enables the decoder to generate emotion-related prosodic features, thereby ensuring that the synthesized pronunciation data is consistent with the fine-grained prosodic features in the out-of-domain speech.

[0049] E emotion =Softmax(Linear softmax (h3)) (1-5)

[0050] Figure 4 The schematic diagram of the optimization adjustment process provided for this application is as follows: Figure 4As shown in the figure, the variable information adaptation module V_a is composed of a pitch predictor P_h, a length predictor P_d, and a strength predictor P_e. Fine-grained acoustic features such as pitch, length, and strength can be extracted from the input speech. These acoustic features constitute the rhythmic representation of the pronunciation, and the adjustment of the fine-grained rhythmic features further affects the emotional expression. By adjusting different rhythmic features and combining the generated emotional representation, the pronunciation data with natural rhythm and rich emotions is generated, further improving the naturalness of the generated pronunciation data. h Responsible for predicting the output frame-level baseband information F0, P e Responsible for outputting the energy information E of the Mel spectrum mel , P d Responsible for outputting the duration information D of each phoneme in logarithmic scale log The calculation process is as shown in equations (1-6) to (1-8). Where x represents the feature vector input to the pitch predictor, y represents the feature vector input to the intensity predictor, and z represents the feature vector input to the length predictor.

[0051] F0=P h (X) (1-6)

[0052] E mel =P e (y) (1-7)

[0053] D log =P d (z) (1-8)

[0054] Finally, the outputs of these predictors are combined into a fused feature vector H combined , It provides rich rhythmic feature information for subsequent processing. The calculation process is shown in formula (1-9).

[0055] H combined =NeuralNetwork([F0,D log ,E mel ,E emotion ]) (1-9)

[0056] In the feature processing stage, the prosodic feature vector is input, which contains the acoustic features of the speech. Then, the local information of the prosodic feature is extracted through a one-dimensional convolutional layer. After ReLU activation and Dropout regularization, the linear layer outputs the deviation and scale parameters, and then the features are normalized and fused. By repeating this process, the feature H is gradually extracted and fused. combined , and finally output a stable and consistent rhythmic representation of the fused emotions.

[0057] The method for playing a text sequence provided in an embodiment of the present application first requires extracting a variety of sample speech segments from a speech library and analyzing their emotional features. Next, an emotional analysis is performed on the text sequence to be played to determine the second emotional feature that it should express. Once the target emotional feature is determined, it is necessary to find the first prosodic feature of all speech segments corresponding thereto. Finally, based on the determined prosodic feature, speech synthesis technology is used to convert the text sequence to be played into synthesized audio. During the synthesis process, parameters such as pitch, length, and energy are adjusted to make the synthesized speech more naturally reflect the target emotional means, so as to achieve the playback of text combined with emotional features, so as to at least solve the problem of low naturalness and pronunciation accuracy of the current speech efficiency.

[0058] In a possible embodiment, second prosodic features corresponding to the multiple speech segments are determined respectively, and the proportion of different sub-features in each second prosodic feature is determined, wherein the sub-features include at least one of the following: a first pitch feature, a first sound length feature, and a first sound intensity feature; and a first emotion feature of the speech segment corresponding to the second prosodic feature is determined based on the proportion of the different sub-features.

[0059] First, multiple speech segments need to be collected and analyzed to determine their respective prosodic features, which may include sub-features such as pitch, duration, and intensity.

[0060] These prosodic features can be quantified through acoustic analysis techniques such as fundamental frequency extraction, duration measurement, and energy calculation. Next, it is necessary to analyze the proportion of different sub-features in each prosodic feature. This can be achieved through statistical methods, such as calculating the frequency of occurrence or duration ratio of each sub-feature in all prosodic features.

[0061] These percentage data will be used for subsequent sentiment feature analysis.

[0062] Finally, the emotional characteristics of each speech segment can be determined based on the proportion of different sub-features. For example, if the pitch feature of a speech segment accounts for a high proportion, while the length and intensity features account for a low proportion, then this speech segment may express anger or surprise; conversely, if the length and intensity features account for a high proportion, while the pitch feature accounts for a low proportion, then this speech segment may express sadness or contemplation.

[0063] This sentiment analysis method based on prosodic features can help us understand and identify emotional information in speech more accurately.

[0064] In a possible embodiment, proportion information corresponding to the second rhythmic feature is determined according to the proportions of different sub-features; the proportion information is matched with a preset emotion library, and the first emotion feature corresponding to the second rhythmic feature is determined according to the matching result, wherein the emotion library at least includes: multiple sample rhythmic features, emotion features corresponding to the multiple sample rhythmic features respectively, and proportion information corresponding to the multiple sample rhythmic features.

[0065] First, it is necessary to analyze the proportion of different sub-features (such as pitch, duration, and intensity) in each second prosodic feature. This can be achieved through statistical methods, such as calculating the frequency of occurrence or duration ratio of each sub-feature in all prosodic features.

[0066] Next, the above ratio information is matched with a preset emotion library, which at least includes a plurality of sample prosodic features, emotion features corresponding to the sample prosodic features, and ratio information corresponding to the plurality of sample prosodic features.

[0067] By comparing the proportion information of the speech segment to be analyzed with the proportion information of the samples in the emotion library, the most matching emotion feature can be found.

[0068] According to the matching results, the first emotion feature corresponding to the second prosodic feature can be determined. For example, if the pitch feature of a speech segment accounts for a high proportion, while the length and intensity features account for a low proportion, then the speech segment may express anger or surprise; conversely, if the length and intensity features account for a high proportion, while the pitch feature accounts for a low proportion, then the speech segment may express sadness or contemplation.

[0069] This sentiment analysis method based on prosodic features can help us understand and identify emotional information in speech more accurately.

[0070] In summary, by analyzing the proportion of different sub-features in the prosodic features and matching them with the preset emotion library, the emotional features of the speech segment can be determined more accurately. This method has important application value in the fields of speech processing and emotion recognition.

[0071] In a possible embodiment, the text sequence to be played is input into an emotion model to obtain an output result of the emotion model, wherein the emotion model is obtained by computer learning multiple sets of emotion data, and each set of training data in the multiple sets of emotion data includes: a sample text sequence, and an emotion label corresponding to the sample text sequence; and the second emotion feature is determined according to the emotion label in the output result.

[0072] First, the text sequence to be played needs to be preprocessed, including text normalization, encoding conversion and other steps, so as to convert it into a format suitable for emotion model processing.

[0073] The sentiment model is obtained by computer learning multiple sets of sentiment data. Each set of these training data includes sample text sequences and their corresponding sentiment labels.

[0074] Sentiment models are typically based on machine learning algorithms such as support vector machines (SVM), random forests, or deep learning neural networks, which are able to learn the complex relationship between text and sentiment from large amounts of data.

[0075] The preprocessed text sequence is input into the sentiment model, and the model predicts the sentiment label of the text sequence based on its internally learned weights and parameters.

[0076] The output of the emotion model includes an emotion label, which directly reflects the emotion type expressed by the text sequence. Based on this emotion label, the second emotion feature of the text sequence can be determined, that is, the specific emotion state it represents in the emotion model.

[0077] In a possible embodiment, the first prosodic feature is optimized to obtain an optimized first prosodic feature; and it is determined to synthesize the synthesized audio corresponding to the to-be-played text sequence according to an indication of the optimized first prosodic feature.

[0078] In a possible embodiment, the optimization operation includes: inputting the sub-features corresponding to the first prosodic feature into the optimization model corresponding to the sub-features respectively to obtain an optimization result, wherein the optimization model is obtained by computer learning multiple groups of prosodic data, and each group of prosodic data in the multiple groups of prosodic data includes: a sample prosodic feature and an optimization result after the sample prosodic feature is optimized.

[0079] The first prosodic feature needs to be optimized, which usually involves inputting each sub-feature of the first prosodic feature (such as pitch, duration, and intensity) into the corresponding optimization model.

[0080] These optimization models are obtained by computers through learning a large amount of prosodic data. Each set of prosodic data includes sample prosodic features and the results of these features after optimization. When the sub-features of the first prosodic feature are input into the optimization model, the model will predict the optimization results of these sub-features based on its internally learned weights and parameters. Based on the prediction results of the optimization model, the optimized first prosodic feature can be determined. These optimized prosodic features are more consistent with the prosodic pattern of natural language and can better express the emotion and intention of the text.

[0081] The optimized prosodic features will be used in the subsequent speech synthesis process to improve the quality of the synthesized audio.

[0082] In a possible embodiment, synthesizing the synthetic audio corresponding to the text sequence to be played according to the indication of the first prosodic feature includes: synthesizing a digital sound signal corresponding to the text sequence to be played according to the indication of the first prosodic feature; decoding and converting the digital sound signal through a decoder to obtain an analog sound signal corresponding to the digital sound signal; and determining the analog sound signal as the synthetic audio corresponding to the text sequence to be played.

[0083] Figure 5 A schematic diagram of the structure of the text sequence playback device provided in this application, such as Figure 5 As shown, the text sequence playback device 50 provided in this embodiment includes:

[0084] Recognition module 501, used to recognize first emotion features corresponding to a plurality of sample speech segments in the speech library, and to recognize second emotion features corresponding to the text sequence to be played;

[0085] A matching module 502, configured to select a target emotion feature that matches the second emotion feature from a plurality of first emotion features;

[0086] A determination module 503 is used to determine the first prosody feature of all speech segments corresponding to the target emotion feature, wherein the first prosody is used to characterize the pronunciation mode corresponding to the sample speech segment;

[0087] The synthesis module 504 is used to synthesize the synthesized audio corresponding to the text to be played according to the indication of the rhythmic features, and play the synthesized audio.

[0088] In one possible implementation, the recognition module 501 is used to determine the second prosodic features corresponding to the multiple speech segments respectively, and determine the proportion of different sub-features in each second prosodic feature, wherein the sub-features include at least one of the following: a first pitch feature, a first sound length feature, and a first sound intensity feature; and determine the first emotion feature of the speech segment corresponding to the second prosodic feature based on the proportion of the different sub-features.

[0089] In a possible implementation, the identification module 501 is used to determine the proportion information corresponding to the second rhythmic feature according to the proportion of different sub-features; match the proportion information with a preset emotion library, and determine the first emotion feature corresponding to the second rhythmic feature according to the matching result, wherein the emotion library at least includes: multiple sample rhythmic features, emotion features corresponding to the multiple sample rhythmic features respectively, and proportion information corresponding to the multiple sample rhythmic features.

[0090] In one possible implementation, the recognition module 501 is used to input the text sequence to be played into the emotion model to obtain the output result of the emotion model, wherein the emotion model is obtained by computer learning multiple sets of emotion data, and each set of training data in the multiple sets of emotion data includes: a sample text sequence, and an emotion label corresponding to the sample text sequence; and the second emotion feature is determined according to the emotion label in the output result.

[0091] In a possible implementation, the device also includes: an optimization module (not shown in the figure), which is used to optimize the first prosodic feature to obtain an optimized first prosodic feature; determine to synthesize the synthetic audio corresponding to the text sequence to be played according to the indication of the optimized first prosodic feature.

[0092] In a possible implementation, the optimization module (not shown in the figure) is also used to input the sub-features corresponding to the first rhythmic feature into the optimization model corresponding to the sub-features, respectively, to obtain an optimization result, wherein the optimization model is obtained by computer learning multiple groups of rhythmic data, and each group of rhythmic data in the multiple groups of rhythmic data includes: a sample rhythmic feature and an optimization result after the sample rhythmic feature is optimized.

[0093] In a possible implementation, the device further includes: a conversion module (not shown in the figure), configured to synthesize a digital sound signal corresponding to the to-be-played text sequence according to an indication of the first prosodic feature;

[0094] The digital sound signal is decoded and converted by a decoder to obtain an analog sound signal corresponding to the digital sound signal; and the analog sound signal is determined as the synthesized audio corresponding to the text sequence to be played.

[0095] The text sequence playback device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0096] Figure 6 This is a schematic diagram of the structure of the text sequence playback device provided by this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus.

[0097] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0098] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0099] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0100] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0101] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0102] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0103] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0104] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0105] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0106] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0109] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0110] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0111] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for playing a text sequence, characterized in that: include: Identify first emotion features corresponding to a plurality of sample speech segments in the speech library, and identify second emotion features corresponding to the text sequence to be played; Selecting a target emotion feature matching the second emotion feature from a plurality of first emotion features; Determine a first prosody feature of all speech segments corresponding to the target emotion feature, wherein the first prosody is used to characterize a pronunciation mode corresponding to the sample speech segment; Synthesize the synthesized audio corresponding to the to-be-played text sequence according to the indication of the first prosodic feature, and play the synthesized audio.

2. The method according to claim 1, characterized in that Identify the first emotion features corresponding to various speech segments in the speech library, including: Determine the second prosodic features corresponding to the multiple speech segments respectively, and determine the proportion of different sub-features in each second prosodic feature, wherein the sub-features include at least one of the following: a first pitch feature, a first sound length feature, and a first sound intensity feature; The first emotion feature of the speech segment corresponding to the second prosodic feature is determined according to the proportions of the different sub-features.

3. The method according to claim 2, characterized in that Determining a first emotion feature of the speech segment corresponding to the second prosodic feature according to the proportions of the different sub-features includes: Determining proportion information corresponding to the second prosodic feature according to proportions of different sub-features; The ratio information is matched with a preset emotion library, and the first emotion feature corresponding to the second rhythmic feature is determined according to the matching result, wherein the emotion library at least includes: multiple sample rhythmic features, emotion features corresponding to the multiple sample rhythmic features, and ratio information corresponding to the multiple sample rhythmic features.

4. The method according to claim 1, characterized in that: Identifying the second emotion feature corresponding to the text sequence to be played includes: Inputting the to-be-played text sequence into an emotion model to obtain an output result of the emotion model, wherein the emotion model is obtained by computer learning multiple sets of emotion data, and each set of training data in the multiple sets of emotion data includes: a sample text sequence and an emotion label corresponding to the sample text sequence; The second emotion feature is determined according to the emotion label in the output result.

5. The method according to claim 1, characterized in that Before synthesizing the synthesized audio corresponding to the to-be-played text sequence according to the indication of the first prosodic feature, the method further includes: performing an optimization operation on the first prosodic feature to obtain an optimized first prosodic feature; Determine to synthesize the synthesized audio corresponding to the to-be-played text sequence according to the indication of the optimized first prosodic feature.

6. The method according to claim 5, characterized in that The optimization operation includes: The sub-features corresponding to the first prosodic feature are respectively input into the optimization model corresponding to the sub-features to obtain optimization results, wherein the optimization model is obtained by computer learning multiple groups of prosodic data, and each group of prosodic data in the multiple groups of prosodic data includes: sample prosodic features and optimization results after the sample prosodic features are optimized.

7. The method according to claim 1, characterized in that Synthesizing the synthesized audio corresponding to the to-be-played text sequence according to the indication of the first prosodic feature includes: synthesizing a digital sound signal corresponding to the to-be-played text sequence according to the indication of the first prosodic feature; Decoding the digital sound signal through a decoder to obtain an analog sound signal corresponding to the digital sound signal; The simulated sound signal is determined as the synthesized audio corresponding to the text sequence to be played.

8. A device for playing a text sequence, characterized in that: include: A recognition module, used to recognize first emotion features corresponding to a plurality of sample speech segments in a speech library, and to recognize second emotion features corresponding to a text sequence to be played; A matching module, configured to select a target emotion feature that matches the second emotion feature from a plurality of first emotion features; A determination module, configured to determine a first prosody feature of all speech segments corresponding to the target emotion feature, wherein the first prosody is used to characterize a pronunciation mode corresponding to the sample speech segment; A synthesis module is used to synthesize the synthesized audio corresponding to the text to be played according to the indication of the rhythmic features, and play the synthesized audio.

9. A device for playing a text sequence, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images