A method and system for efficiently converting text to speech

Abstract

The present application provides an efficient text-to-speech method and system. Among them, receiving text information input by a user, identifying and analyzing the emotional color and tone characteristics in the text information; selecting a matching data set of basic voice segments from a pre-built emotional speech library according to the emotional color and the tone characteristics; using a dynamic time warping algorithm to adjust the time axis of the basic voice segment so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color; using spectrum fusion technology, the adjusted time axis is fused with the phoneme sequence of the text information to generate speech output, wherein the spectrum fusion technology is used to ensure that basic voice segments with different emotional colors have a smooth transition at the connection. The technical solution provided by the present application improves the emotional expression ability, naturalness and coherence of speech synthesis, thereby greatly improving the user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of information technology, and in particular to a method and system for efficiently converting text to speech. Background Art

[0002] With the rapid development of artificial intelligence technology, text-to-speech technology has been widely used in many fields, such as intelligent customer service, audiobooks, voice assistants, navigation systems, etc. In these application scenarios, users not only want to hear clear and natural speech, but also expect the speech to accurately convey the emotional color and tone characteristics in the text to enhance the affinity and credibility of communication. Therefore, developing a technology that can efficiently and accurately convert text into emotionally colored speech has become a current research hotspot and technical demand.

[0003] At present, mainstream text-to-speech technologies are mainly divided into two categories: rule-based methods and statistical methods. Rule-based methods generate speech through a predefined rule base. Although they can guarantee a certain level of speech quality, they are unable to cope with complex emotional expressions. Statistical methods use a large amount of training data to learn speech generation models, which can generate more natural speech, but they are still insufficient in terms of emotional expression. In recent years, the application of deep learning technology in the field of speech synthesis has made significant progress. Neural network models can better capture the emotional characteristics of text and generate emotional speech.

[0004] Although existing text-to-speech technologies have made some progress, they still have the following major defects:

[0005] Most existing technologies can only generate relatively simple speech when processing emotional color and tone features, which makes it difficult to accurately convey the rich emotions in the text, resulting in the generated speech lacking appeal and authenticity.

[0006] When synthesizing speech, the connection between basic speech segments with different emotional colors is often not smooth enough, and abrupt switches are prone to occur, affecting the naturalness and coherence of the speech.

[0007] When adjusting the speech speed, the existing technology often ignores the consistency of emotional color, resulting in distortion of emotional expression when changing the speech speed, affecting the overall quality of the speech and user experience. Summary of the invention

[0008] The embodiments of the present application provide an efficient text-to-speech method and system to solve the problems in the prior art of speech synthesis such as insufficient emotional expression ability, poor naturalness and coherence, and reduced user experience.

[0009] In a first aspect, an embodiment of the present application provides an efficient text-to-speech method, comprising:

[0010] Receiving text information input by a user, identifying and analyzing the emotional color and tone characteristics in the text information;

[0011] According to the emotional color and the tone characteristics, selecting a matching basic speech segment from a pre-built emotional speech library;

[0012] Using a dynamic time warping algorithm to adjust the time axis of the basic voice segment so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color;

[0013] The adjusted time axis is fused with the phoneme sequence of the text information by using spectrum fusion technology to generate speech output, wherein the spectrum fusion technology is used to ensure that basic speech segments with different emotional colors transition smoothly at the connection points.

[0014] In a second aspect, an embodiment of the present application provides an efficient text-to-speech system, including:

[0015] The receiving and identifying module is used to receive text information input by the user, identify and analyze the emotional color and tone characteristics in the text information;

[0016] A selection module, for selecting a matching basic speech segment from a pre-built emotional speech library according to the emotional color and the tone feature;

[0017] An adjustment module, used for adjusting the time axis of the basic voice segment by using a dynamic time warping algorithm, so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color;

[0018] The fusion module is used to fuse the adjusted time axis with the phoneme sequence of the text information using spectrum fusion technology to generate speech output, wherein the spectrum fusion technology is used to ensure that basic speech segments with different emotional colors transition smoothly at the connection points.

[0019] In a third aspect, an embodiment of the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement an efficient text-to-speech method as described in any one of the first aspects.

[0020] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, wherein when the computer program is executed by a computer, an efficient text-to-speech method as described in any one of the first aspects is implemented.

[0021] In an embodiment of the present application, text information input by a user is received, and the emotional color and tone characteristics in the text information are identified and analyzed; according to the emotional color and the tone characteristics, a matching basic voice segment is selected from a pre-constructed emotional speech library; the time axis of the basic voice segment is adjusted using a dynamic time warping algorithm so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color; the spectral fusion technology is used to fuse the adjusted time axis with the phoneme sequence of the text information to generate a speech output, wherein the spectral fusion technology is used to ensure that basic voice segments with different emotional colors smoothly transition at the connection. The technical solution provided by the present application improves the emotional expression ability, naturalness and coherence of speech synthesis, thereby greatly improving the user experience.

[0022] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A flowchart of an efficient text-to-speech method provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of the structure of an efficient text-to-speech system provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0028] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0030] Figure 1 A flowchart of an efficient text-to-speech method is provided for an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0031] 101. Receive text information input by a user, and identify and analyze emotional color and tone characteristics in the text information;

[0032] In this step, the system first receives the text information provided by the user through the text input interface. Then, the text information is parsed using natural language processing technology to extract keywords, phrases, and sentence structures to understand the basic meaning of the text. Subsequently, the system applies a sentiment analysis algorithm to perform sentiment analysis on the text, determine the type of emotion expressed in the text, and evaluate the intensity or score of the emotion. In addition, the system also uses a tone recognition model to identify the tone characteristics of the text, such as imperative, interrogative, declarative, etc. The recognition and analysis of these emotion and tone characteristics provide an important basis for the subsequent selection of appropriate basic voice segments, ensuring that the generated speech can accurately convey the emotion and tone of the original text.

[0033] This application takes into account that in the prior art, text-to-speech systems often rely on simple rules or predefined emotional tags when processing emotional color and tone features, which results in obvious deficiencies in the generated speech in terms of emotional expression and tone diversity. In order to solve the above technical problems, the embodiment of the present invention proposes a more detailed optional solution, which ensures that the generated speech can more accurately convey the emotional and tone characteristics of the original text through more sophisticated emotional analysis and tone recognition.

[0034] The options include:

[0035] Optionally, the receiving of text information input by the user in step 101, identifying and analyzing the emotional color and tone features in the text information, includes: receiving text information provided by the user through a text input interface; parsing the text information using natural language processing technology, extracting keywords, phrases and sentence structures of the text to understand the basic meaning of the text; applying a sentiment analysis algorithm to perform sentiment analysis on the text information, determining the type of emotion expressed in the text, and evaluating the sentiment score; using a tone recognition model to identify the tone features of the text information, including imperative, interrogative, and declarative, wherein the tone recognition model is trained based on a large amount of text data with tone labels.

[0036] In this step, the text input interface is an interface for users to interact with the system, used to receive text information input by the user.

[0037] Natural language processing technology: a method that enables computers to understand, parse and generate human natural language through computer science and artificial intelligence technology.

[0038] Keywords, Phrases, and Sentence Structure: Core vocabulary, common phrases, and syntactic structures in the text are used to understand the basic meaning of the text.

[0039] Sentiment analysis algorithm: an algorithm used to identify and evaluate the type and intensity of sentiment in text. Common methods include rule-based methods, statistical methods, and deep learning-based methods.

[0040] Emotion type: The category of emotion expressed in the text, such as happiness, sadness, anger, etc.

[0041] Sentiment score: A numerical value used to quantify the intensity of sentiment, usually ranging between 0 and 1.

[0042] Tone recognition model: A model trained through machine learning that is used to identify tone features in text, such as imperative, interrogative, declarative, etc.

[0043] Text data with tone labels: Labeled data used to train the tone recognition model. Each piece of data has a clear tone label.

[0044] The user inputs the text information to be converted through the text input interface, and the system receives and stores the text information.

[0045] The system uses natural language processing technology to deeply analyze the text, extracting keywords, phrases, and sentence structures in order to more accurately understand the basic meaning of the text.

[0046] The system applies sentiment analysis algorithms to perform sentiment analysis on the text, determine the type of sentiment expressed in the text, and evaluate the sentiment score.

[0047] A tone recognition model is used to recognize the tone features of the text information, including imperative, interrogative, and declarative. The tone recognition model is trained based on a large amount of text data with tone labels: the system uses a pre-trained tone recognition model to recognize the tone features of the text and determine the tone type of the text.

[0048] In the embodiment of the present application, it is assumed that the user inputs the following text message through the text input interface: "The weather is really nice today, let's go to the park together!"

[0049] The system receives and stores the text entered by the user: "It's a nice day today, let's go to the park together!"

[0050] The system uses natural language processing techniques to parse text and extract keywords, phrases, and sentence structure.

[0051] The system applies a sentiment analysis algorithm and determines that the sentiment type expressed in the text is "happy" with a sentiment score of 0.85.

[0052] The system uses the tone recognition model to determine that the tone feature of the text is "imperative" because the sentence "Let's go to the park together!" in the text is a suggestion or invitation and has an imperative tone.

[0053] Through the above optional scheme, the embodiment of the present invention can more accurately identify and analyze the emotional color and tone features in the text, so as to select more suitable basic speech segments in the subsequent speech synthesis process, and generate high-quality, highly natural, and emotionally rich speech output. Through the sentiment analysis algorithm, the system can more accurately identify the type and intensity of emotions in the text, and the generated speech can more realistically convey the emotions of the original text. Through the tone recognition model, the system can recognize a variety of tone features, and the generated speech is richer and more natural in tone expression. Through natural language processing technology, the system can deeply parse the keywords, phrases and sentence structures of the text to ensure that the generated speech is more reasonable in logic and coherence. These improvements significantly enhance the user's auditory experience and make the generated speech more practical and effective in a variety of application scenarios.

[0054] This application takes into account that in the prior art, sentiment analysis algorithms usually rely on a single model or simple rules, which results in low accuracy and robustness of sentiment recognition. In order to solve the above technical problems, the embodiment of the present invention proposes a more sophisticated sentiment analysis algorithm, which comprehensively captures the sentiment features in the text by combining convolutional neural networks, recursive neural networks and attention mechanisms, and improves the accuracy of sentiment recognition and the precision of sentiment intensity assessment. The optional solution specifically includes:

[0055] Optionally, the step 101 of using a sentiment analysis algorithm to evaluate the sentiment score includes: calculating the sentiment score using the following formula: : in, represents the sentiment weight of the word, represents the sentiment impact coefficient of the context, The sentiment regulation factor representing the text structure; Indicates taking and the maximum value between 0, ensuring the sentiment score A non-negative value indicates the parameter set of the model;

[0056] is the output of a convolutional neural network model, which is used to capture the local and global features between word sentiment, contextual sentiment, and text structure sentiment, and is defined as:

[0057] in, Indicates that , , Converted to embedding vector, represents the convolution operation, The parameter set of the model is shown; is the output of a recurrent neural network model, which is used to capture the temporal dependency and contextual information of sentiment features and is defined as: in, represents the long short-term memory network, Represents the parameter set of the model; The output of an attention mechanism model, which is used to capture the important parts of the sentiment feature, is defined as: in, is the number of words in the text, It is The attention weight of each word, It is The hidden state of a word, Represents the set of parameters for the model.

[0058] In this step, the sentiment score A numerical value used to quantify the sentiment intensity of a text, usually ranging from 0 to 1; the sentiment weight of a word Indicates the importance of each word in expressing emotion, obtained through pre-trained sentiment lexicon or model; sentiment influence coefficient of context Indicates the degree of influence of the context on the emotional expression of the current word, used to capture the contextual dependence of emotion; emotional regulation factor of text structure Indicates the impact of text structure on emotional expression and is used to adjust the expression of emotions in different sentence structures. Convolutional Neural Network It means capturing local features in text through convolution operation, which is particularly suitable for capturing local patterns of emotional features; recursive neural network Represents capturing the temporal dependency and contextual information in text through recursive operations, which is particularly suitable for processing sequence data; Long Short-Term Memory Network express A variant of , which can effectively solve the gradient disappearance and gradient explosion problems and is more suitable for processing long sequence data; attention mechanism Indicates that by assigning attention weights to each word, the important parts of the emotional features are highlighted to improve the accuracy of emotional recognition; the embedding vector Represents the conversion of words, context, and structural information in the text into a fixed-length vector representation; convolution operation Indicates that the convolution kernel slides on the input data to extract local features; hidden state express The internal state of the model when processing each word, including information about the current word and its context; attention weight Indicates the importance of each word in sentiment recognition, calculated through the attention mechanism.

[0059] The sentiment score is calculated using the following formula: : in, represents the sentiment weight of the word, represents the sentiment impact coefficient of the context, The sentiment regulation factor representing the text structure; Indicates taking and the maximum value between 0, ensuring the sentiment score A non-negative value indicates the parameter set of the model; is the output of a convolutional neural network model, which is used to capture the local and global features between word sentiment, contextual sentiment, and text structure sentiment, and is defined as: in, Indicates that , , Converted to embedding vector, represents the convolution operation, Represents the parameter set of the model; is the output of a recurrent neural network model, which is used to capture the temporal dependency and contextual information of sentiment features and is defined as: in, represents the long short-term memory network, Represents the parameter set of the model; is the output of an attention mechanism model, which is used to capture the important part of the sentiment feature and is defined as: in, is the number of words in the text, It is The attention weight of each word, It is The hidden state of a word, Represents the set of parameters for the model.

[0060] In the embodiment of the present application, it is assumed that the text message input by the user is: "Today's weather is really good, I feel very happy."

[0061] The sentiment score is calculated using the following formula: : The sentiment weight of the words in the text , the sentiment influence coefficient of the context and the emotional modulatory factors of text structure Convert to embedding vector.

[0062] The convolution operation extracts local features in the embedding vector, for example, the relationship between "weather" and "good". ; Input the embedding vector into The model captures temporal dependencies and contextual information, for example, the relationship between "Today's weather is great" and "I feel very happy". ;

[0063] Calculate the attention weight of each word, for example, "very" and "happy" are more important in expressing emotions. Assume ; Since sentiment scores usually range from 0 to 1, they can be converted to a suitable range through normalization, such as .

[0064] Through the above optional scheme, the embodiment of the present invention can more accurately identify and analyze the emotional features in the text, thereby generating higher quality, more natural, and more emotionally rich speech output in the subsequent speech synthesis process. , and The model enables the system to fully capture the emotional features in the text and reduce misjudgment. By considering the emotional weight of words, the emotional influence coefficient of the context, and the emotional adjustment factor of the text structure, the system can more accurately evaluate the emotional intensity. Through convolution and recursive operations, the system can capture the local and global correlation of emotional features and improve the robustness of emotion recognition.

[0065] These improvements significantly enhance the user's auditory experience and make the generated speech more practical and effective in a variety of application scenarios.

[0066] 102. Selecting a matching basic speech segment from a pre-built emotional speech library according to the emotional color and the tone feature;

[0067] In this step, selecting the basic speech segments that match the desired emotional color and tone characteristics is a key step when building an emotional speech synthesis system.

[0068] First, the target emotional color needs to be clearly defined. Emotional color can be basic emotions such as happiness, sadness, anger, surprise, etc., or more subtle emotional states such as calm, relaxed, nervous, etc. This step ensures that the subsequent selected voice clips can accurately convey the intended emotion.

[0069] Tone features involve the way someone speaks, including changes in speech speed, pitch, and intensity. For example, fast speech may be associated with excitement or anxiety, while slow speech may express sadness or contemplation. Analyzing these features helps to more accurately match the appropriate basic voice segment.

[0070] In order to support the selection of emotional speech, a library of speech segments containing a variety of emotional colors and tone features is required. These segments are usually recorded by professional readers in different emotional states and annotated to indicate the emotional color and tone features of each segment.

[0071] Based on the above definition and analysis of emotional color and tone characteristics, the most suitable voice clips are selected from the emotional speech library. This may involve automatic matching using algorithms or manual selection to ensure that the selected clips can effectively convey the target emotion and tone.

[0072] Once the appropriate voice segments are selected, the next step is to combine these segments through technical means to form complete sentences or paragraphs. In this process, some adjustments may also be required to the selected segments, such as adjusting pitch, speed, etc., to ensure that the entire output is more natural and smooth, and better fits the target emotion and tone.

[0073] This process is the core part of emotional speech synthesis technology, which aims to make the machine-generated voice not only sound natural, but also accurately express specific emotions and tones, thereby improving the quality of human-computer interaction and user experience.

[0074] The present application takes into account that although the existing emotional speech synthesis technology has been able to achieve a certain degree of emotional expression, it is still insufficient in accurately matching specific emotional colors and tone features. The main problem is that the existing speech synthesis system often relies on preset rules or simple template matching, and cannot fully take into account the diversity and complexity of language, especially when dealing with delicate emotional changes and specific tones. In addition, due to the lack of efficient emotional speech library management and search mechanism, it becomes difficult to find the best matching basic speech segment in large-scale data, which in turn affects the naturalness and emotional accuracy of the final synthesized speech. In order to solve the above technical problems, the embodiment of the present invention proposes a more sophisticated and intelligent emotional speech segment selection method, which aims to improve the accuracy and naturalness of emotional speech synthesis. By constructing a detailed annotated emotional speech library and adopting a multi-dimensional search algorithm based on similarity, the performance of the emotional speech synthesis system can be effectively improved, especially when dealing with complex emotions and tone requirements. The optional scheme specifically includes:

[0075] Optionally, the step 102 of selecting a matching basic speech segment from a pre-constructed emotional speech library according to the emotional color and the tone characteristics includes: constructing an emotional speech library, the emotional speech library containing basic speech segments with a variety of emotional colors and tone characteristics, each segment having a corresponding emotional label and tone label; searching for matching emotional labels and tone labels in the emotional speech library according to the emotional color and the tone characteristics; comparing the similarity between the emotional label and the tone label of each speech segment in the search results and the emotional color and the tone characteristics, and selecting the basic speech segment with the highest similarity as the candidate segment; if there are multiple candidate segments with the same similarity, selecting the basic speech segment that best fits the text context according to the specific content of the text information.

[0076] In this step, the emotional speech database is a database of basic speech segments containing a variety of emotional colors and tone features, each segment has a corresponding emotional label and tone label.

[0077] Emotion label: A label that describes the emotion type of the basic speech segment.

[0078] Tone label: A label that describes the tone type of the underlying speech segment.

[0079] Similarity: It is used to measure the matching degree between the emotion label and tone label of the basic speech segment and the recognition result.

[0080] Candidate segment: The basic speech segment found in the emotional speech database that best matches the emotional color and tone characteristics of the text.

[0081] Text context: The specific content and background information of the text, which is used to further select the most appropriate candidate fragments.

[0082] An emotional speech library is constructed that contains basic speech segments with various emotional colors and tone features, and each segment has corresponding emotional labels and tone labels.

[0083] The identified emotional color and tone features are compared with the labels in the emotional speech library, and the basic speech fragments that match the emotional labels and tone labels are searched.

[0084] The similarity between the emotion label and tone label of each speech segment in the search results and the recognition results is compared, and the segment with the highest similarity is selected as the candidate segment.

[0085] When there are multiple candidate segments with the same similarity, the specific content of the text is further analyzed to select the segment that best fits the text context.

[0086] In the embodiment of the present application, it is assumed that the text message input by the user is: "The weather is really nice today, let's go to the park together!"

[0087] The emotional speech library contains multiple basic speech segments, each of which has an emotional label and a tone label. For example:

[0088] Clip 1: Emotion label "happy", tone label "declarative"

[0089] Clip 2: Emotion label "happy", tone label "imperative"

[0090] Clip 3: Sentiment label "neutral", tone label "declarative"

[0091] The system recognizes that the emotion type of the text is "happy" and the tone type is "imperative".

[0092] Search the emotional speech library for segments with the emotion label "happy" and the tone label "imperative".

[0093] After comparison, the system found that the emotion label and tone label of segment 2 completely matched the recognition result and had the highest similarity, so segment 2 was selected as the candidate segment.

[0094] Assume that there is another segment 4 in the emotional speech database, with the emotional label “happy” and the tone label “imperative”, and the similarity is also the same.

[0095] The system further analyzed the specific content of the text and found that "Let's go to the park together!" is a suggestion or invitation, which is more in line with the imperative tone.

[0096] Therefore, the system finally selects segment 2 as the basic speech segment that best fits the text context.

[0097] Through the above optional scheme, the embodiment of the present invention can more accurately select basic speech segments that match the emotional color and tone characteristics of the text, thereby generating high-quality, highly natural, and emotionally rich speech output during the speech synthesis process. Specifically, through detailed matching of emotional tags and tone tags, it is ensured that the selected speech segment is highly consistent with the original text in terms of emotion and tone. When there are multiple candidate segments with the same similarity, the segment that best fits the text context is selected by analyzing the specific content of the text, thereby improving the rationality of the selection. The generated speech is natural and smooth in emotion and tone, avoiding abrupt switching, and improving the naturalness and coherence of the speech. The generated speech is closer to the user's real emotions and tone, improving the user's auditory experience, and making speech synthesis more practical and effective in a variety of application scenarios.

[0098] These improvements significantly enhance the quality of speech synthesis, making the generated speech richer and more natural in terms of emotional expression and tone characteristics. At the same time, along with the improvement in the quality of speech synthesis, the timbre adjustment processing in particular can ensure that the generated sound is more natural and realistic, similar to the speech of a real person; that is, this highly realistic timbre adjustment processing makes the synthesized speech highly consistent with the target speaker in timbre, intonation, rhythm and other dimensions, achieving an effect that is indistinguishable from the real thing.

[0099] Therefore, the system can accurately restore the speaker's voice changes in different emotional states, as well as the individual's unique tone habits and expressions, making the synthesized voice almost indistinguishable from real human voice.

[0100] 103. Using a dynamic time warping algorithm to adjust the time axis of the basic voice segment, so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color;

[0101] Dynamic Time Warping (DTW) is a technique used to measure the similarity between two sequences, and is particularly suitable for nonlinearly changing sequence data. In the field of speech processing, DTU is often used in speech recognition, speech synthesis and other scenarios to solve the problem of speech length mismatch caused by differences in speakers or speaking speeds.

[0102] The purpose of applying the dynamic time warping algorithm is to adjust the time axis of the basic speech segment so that it can match the speech speed of the specific text information, while ensuring that the adjusted speech segment maintains the original emotional color. Specifically, this process can be divided into the following steps:

[0103] First, analyze the basic speech segment and extract its feature parameters. Determine the target speech rate based on the information of the text to be synthesized. Determine the appropriate speech rate based on the emotional color of the text. Compare the feature parameters of the original speech segment with the ideal feature parameters at the target speech rate. Calculate the best matching path between the two sets of features through the dynamic time warping algorithm to find out how to adjust the time axis of the original speech segment so that the adjusted speech meets the target speech rate requirements and retains the original emotional color as much as possible. According to the best matching path obtained in the previous step, stretch or compress the original speech segment in time. In this process, it is important to note that although the time axis has changed, the key features of the speech signal should be kept as unaffected as possible to maintain the consistency of the emotional color. Finally, some post-processing operations, such as smoothing, are required for the adjusted speech to eliminate the unnatural feeling that may be caused by the time axis adjustment.

[0104] In summary, through the above steps, the dynamic time warping algorithm can help adjust the time axis of the speech segment to adapt it to the speech speed requirements of a specific text while maintaining the consistency and naturalness of emotional expression. This is of great significance for improving the performance of the speech synthesis system.

[0105] This application takes into account that in the prior art, when the text-to-speech system adjusts the timeline of the voice segment to adapt to the speed of the text information, it often ignores the consistency of the emotional color. In order to solve the above technical problems, the embodiment of the present invention proposes a more sophisticated optional solution, which ensures that when adjusting the timeline of the voice segment, it can adapt to the speed of the text information and maintain the consistency of the emotional color by introducing an emotion retention mechanism and a verification mechanism.

[0106] The options include:

[0107] Optionally, the step 103 uses a dynamic time warping algorithm to adjust the time axis of the basic voice segment so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color, including: applying a dynamic time warping algorithm to adjust the time axis of the basic voice segment to ensure that the adjusted voice segment can be played at the expected speech speed; in the process of adjusting the time axis, introducing an emotion retention mechanism to fine-tune the pitch and volume of the voice segment to ensure that the original emotional color is maintained unchanged when the playback speed is changed; verifying the degree of matching between the adjusted time axis and the speech speed of the text information, as well as the consistency of the emotional color.

[0108] In this step, the dynamic time warping algorithm is used to align time series, which is commonly used in speech signal processing to adjust the time axis of speech segments.

[0109] Emotion retention mechanism: When adjusting the timeline of a voice clip, the consistency of emotional color is ensured by fine-tuning features such as pitch and volume in the voice clip.

[0110] Pitch: The highness of a sound, usually expressed in Hertz (Hz).

[0111] Sound intensity: The loudness of a sound, usually expressed in decibels (dB).

[0112] Verification mechanism: After adjusting the timeline, a series of verification steps are performed to ensure that the adjusted voice clip matches the speed of the text information and the emotional color is consistent.

[0113] The dynamic time warping algorithm is used to adjust the time axis of the basic voice segment to ensure that the adjusted voice segment can be played at the expected speaking speed.

[0114] In the process of adjusting the timeline, an emotion retention mechanism is introduced to fine-tune the pitch and volume of the voice clips to ensure that the original emotional color remains unchanged when the playback speed is changed:

[0115] Introducing an emotion preservation mechanism to ensure the consistency of emotional color by fine-tuning the pitch and intensity of voice clips when adjusting the timeline. Specific methods include:

[0116] Pitch Adjustment: Preserve natural changes in pitch through interpolation or smoothing.

[0117] Sound intensity adjustment: Maintain the intensity of emotional expression by adjusting the dynamic range of sound intensity.

[0118] Verify whether the adjusted timeline matches the speech speed of the text information by calculating the difference between the average speech speed of the adjusted voice segment and the expected speech speed.

[0119] The consistency of emotional color can be verified by comparing the similarity of emotional features between the adjusted speech segment and the original speech segment.

[0120] In the embodiment of the present application, it is assumed that the text information input by the user is: "The weather is so nice today, let's go to the park together!" The expected speaking speed is 150 words per minute.

[0121] Select the basic speech clip with the emotion label of "happy" and the tone label of "command".

[0122] The dynamic time warping algorithm is used to adjust the timeline of the basic voice clip to ensure that the adjusted voice clip can be played at a speaking speed of 150 words per minute.

[0123] Assuming that the pitch range of the original speech clip is 100Hz to 300Hz, the adjusted pitch range should be kept in a similar range and the natural variation of pitch should be maintained through interpolation or smoothing.

[0124] Assuming that the sound intensity range of the original speech segment is 60dB to 80dB, the adjusted sound intensity range should be kept in a similar range. By adjusting the dynamic range of the sound intensity, the intensity of emotional expression can be maintained.

[0125] Calculate the average speaking speed of the adjusted speech segment. Assuming that the adjusted speech segment is 10 seconds long and contains 100 words, the average speaking speed is 600 words per minute. By adjusting the timeline, ensure that the average speaking speed reaches 150 words per minute.

[0126] Comparing the emotional features of the adjusted speech clips with the original speech clips, assuming that the average pitch of the original speech clips is 200Hz, and the average sound intensity is 70dB, the average pitch of the adjusted speech clips is 195Hz, and the average sound intensity is 68dB. The similarity is high, indicating that the emotional color is well maintained.

[0127] Through the above optional scheme, the embodiment of the present invention can more accurately adjust the time axis of the basic voice segment to ensure that the generated voice maintains the consistency of emotional color while adapting to the speed of the text information. Specifically, through the dynamic time warping algorithm, the system can flexibly adjust the time axis of the voice segment to ensure that the generated voice is adapted to the speed of the text information. Through the emotion preservation mechanism, the system can ensure the consistency of emotional color and avoid distortion of emotional expression by fine-tuning the pitch and volume when adjusting the speed. Through the verification mechanism, the system can ensure that the adjusted time axis matches the speed of the text information, as well as the consistency of the emotional color, thereby improving the quality and reliability of the generated voice.

[0128] These improvements significantly enhance the user's auditory experience, making the generated speech more practical and effective in a variety of application scenarios, and are suitable for scenarios such as intelligent customer service, audiobooks, voice assistants, etc.

[0129] 104. Using spectrum fusion technology, fuse the adjusted time axis with the phoneme sequence of the text information to generate speech output, wherein the spectrum fusion technology is used to ensure that basic speech segments with different emotional colors transition smoothly at the connection points.

[0130] Spectral fusion technology is a technology used in the field of speech synthesis to improve the naturalness of synthesized speech. It is mainly used to splice basic speech fragments from different sources or with different emotional colors to ensure that these fragments can transition smoothly at the connection, so that the final generated speech sounds more coherent and natural.

[0131] First, the ideal duration of each phoneme is calculated based on the desired speech content. This step is to ensure that the synthesized speech not only accurately expresses the text content, but also its rhythm and intonation conform to the characteristics of natural language.

[0132] Next, the text information is converted into a series of phonemes. This process involves text analysis, including part-of-speech tagging, grammatical structure analysis, etc., to determine the most appropriate phoneme sequence to express the text.

[0133] The phoneme sequence prepared in the above steps is combined with the pre-stored basic speech segments using spectrum fusion technology. The basic speech segments here are usually real human speech samples obtained through recordings, which may come from different speakers, or the pronunciation of the same speaker in different emotional states. The role of spectrum fusion technology is that it can reduce or eliminate the discontinuity at the splicing point by adjusting the spectral characteristics between adjacent speech segments, such as frequency response, harmonic structure, etc., so that the entire speech stream sounds smoother and more natural.

[0134] Finally, the processed data is converted into actual audio signals, which is called "voice output". This process may also include further adjustments to parameters such as volume and speed to better match the needs of specific application scenarios.

[0135] In short, spectrum fusion technology is one of the important means to achieve high-quality speech synthesis, especially when it is necessary to deal with changing emotional expressions and complex speech content.

[0136] The present application takes into account that in the prior art, when the text-to-speech system fuses the adjusted timeline with the phoneme sequence of the text information, it often faces the following problems: the spectral characteristics of the basic speech segments with different emotional colors may be greatly different at the connection, resulting in unnatural connection between the speech segments and abrupt switching. In the process of spectrum fusion, there is a lack of effective mechanism to ensure the emotional coherence of the basic speech segments with different emotional colors at the connection, resulting in incoherent emotional expression of the generated speech. In the process of spectrum fusion in the prior art, the adjustment of pitch and timbre is not fine enough, which affects the naturalness and realism of the generated speech. In order to solve the above technical problems, the embodiment of the present invention proposes a more sophisticated optional solution, which ensures that the basic speech segments with different emotional colors have a smooth transition at the connection while maintaining the coherence of the emotional color through spectrum analysis and spectrum fusion technology.

[0137] The options include:

[0138] Optionally, the spectrum fusion technology used in step 104 is used to fuse the adjusted time axis with the phoneme sequence of the text information to generate a speech output, including: aligning the phoneme sequence converted from the text information with the adjusted time axis to ensure that each phoneme in the phoneme sequence accurately matches the corresponding basic speech segment; performing spectrum analysis on each of the basic speech segments to extract spectrum features, wherein the spectrum features include fundamental frequency, harmonic structure and resonance peak position; applying spectrum fusion technology at the connection of adjacent basic speech segments to smooth the spectrum difference between two basic speech segments, and in the process of spectrum fusion, adjusting the pitch and timbre at the connection of adjacent basic speech segments to ensure the continuity of the emotional color; after spectrum fusion processing, all basic speech segments are combined in sequence for speech output.

[0139] In this step, spectral fusion technology: a technology used to smooth the spectral differences between different speech segments, ensures that the generated speech is natural and smooth at the connection.

[0140] Phoneme sequence: A series of phonemes converted into text information, each phoneme represents a pronunciation unit.

[0141] Spectral characteristics: frequency domain characteristics of speech signals, including fundamental frequency, harmonic structure, and resonance peak position.

[0142] Fundamental Frequency: The basic frequency of a sound, usually expressed in Hertz (Hz).

[0143] Harmonic structure: The harmonic components of sound, reflecting the periodic characteristics of sound.

[0144] Resonance peak position: The resonance peak position of a sound reflects the resonance characteristics of the sound.

[0145] Pitch: The highness of a sound, usually expressed in Hertz (Hz).

[0146] Timbre: The sound quality characteristics of sound, reflecting the sound quality and timbre of the sound.

[0147] Convert text information into a phoneme sequence. Align the phoneme sequence with the adjusted timeline to ensure that each phoneme accurately matches the corresponding basic speech segment. Perform spectral analysis on each basic speech segment to extract its spectral features such as fundamental frequency, harmonic structure, and formant position. Apply spectral fusion technology to smooth the spectral difference between two adjacent basic speech segments. Ensure the continuity of emotional color by adjusting the pitch and timbre of the connection. Combine the basic speech segments that have been processed by spectral fusion in sequence to generate the final speech output.

[0148] In this embodiment of the present application, it is assumed that the text information input by the user is: "The weather is so nice today, let's go to the park together!" After the previous steps of processing, the basic voice segment with the "happy" emotion label and the "imperative" tone label has been selected, and the timeline has been adjusted to adapt to the speaking speed of 150 words per minute.

[0149] Convert the text message "The weather is nice today, let's go to the park together!" into a phoneme sequence, for example: [t, j, t, a, n, q, i, z, h, e, n, h, ao, w, o, m, e, n, q, i, y, i, q, u, y, i, q, u, g, o, n, g, y, i, w, an, ba, ].

[0150] Align the phoneme sequence with the adjusted timeline to ensure that each phoneme accurately matches the corresponding underlying speech segment.

[0151] Perform spectral analysis on each basic speech segment to extract its fundamental frequency, harmonic structure and formant position. For example, the fundamental frequency of a segment is 200 Hz, the harmonic structure is [200, 400, 600, 800] Hz, and the formant position is [500, 1500, 2500] Hz.

[0152] At the junction of adjacent basic speech segments, spectrum fusion technology is applied to smooth the spectrum difference between the two segments. For example, assuming that the fundamental frequencies of the two segments are 200Hz and 220Hz respectively, the fundamental frequencies are gradually transitioned from 200Hz to 220Hz through interpolation or smoothing.

[0153] By adjusting the pitch and timbre of the connection, the continuity of emotional color can be ensured. For example, the pitch of the connection can be adjusted to smoothly transition from 200Hz to 220Hz, and the timbre can be adjusted at the same time to ensure the continuity of emotional expression.

[0154] Combine the basic speech segments processed by spectrum fusion in order to generate the final speech output. For example, combine all the segments in order to generate a natural, fluent and emotionally coherent speech: "It's a nice day today, let's go to the park together!"

[0155] Through the above optional scheme, the embodiment of the present invention can more accurately fuse the adjusted timeline with the phoneme sequence of the text information to generate high-quality, highly natural, and emotionally coherent speech output. Specifically, through spectrum fusion technology, the system can smoothly process the spectrum differences between basic speech segments with different emotional colors, ensuring that the generated speech is natural and smooth at the connection. By adjusting the pitch and timbre of the connection, the system can ensure the emotional coherence of the basic speech segments with different emotional colors at the connection, avoiding abrupt and unnatural emotional expression. Through sophisticated spectrum analysis and spectrum fusion technology, the system can generate more natural and realistic speech output, significantly improving the user's auditory experience.

[0156] These improvements enable the present invention to perform well in practical applications and is suitable for a variety of scenarios such as intelligent customer service, audio books, and voice assistants, significantly improving the quality of generated speech and user experience.

[0157] The present application takes into account that in the prior art, when the text-to-speech system fuses the adjusted timeline with the phoneme sequence of the text information, it often faces the following problems: the emotional features of the basic speech segments with different emotional colors may differ greatly at the connection points, resulting in the generated speech being incoherent in emotional expression and producing abrupt switches. In the process of spectrum fusion, there is a lack of effective mechanisms to ensure the smooth transition of the spectrum features of the basic speech segments with different emotional colors at the connection points, which affects the naturalness and realism of the generated speech. In the process of spectrum fusion, the prior art is not sophisticated enough in the adjustment of emotional features, especially in terms of time distance and harmony of emotional features, and lacks an effective control mechanism. In order to solve the above technical problems, the embodiment of the present invention proposes a more sophisticated optional solution, which ensures that the basic speech segments with different emotional colors have a smooth transition at the connection points while maintaining the consistency of the emotional features by introducing an emotion retention calculation formula.

[0158] The options include:

[0159] Optionally, the spectrum fusion technology in step 104, when generating speech output, includes:

[0160] The emotional retention is calculated using the following formula: , so as to adjust the basic speech segments with different emotional colors to smoothly transition at the connection by using the emotion preservation degree: in, represents the original basic speech segment, Indicates the basic voice segment on the adjusted timeline; Indicates A sentiment feature function, is the total number of sentiment feature functions; is a small positive number used to prevent division by zero errors; It is The importance weights of the emotional features reflect the relative importance of different emotional features in the emotional retention; is a decay factor used to control the effect of time distance; is another attenuation factor used to control the harmony effect of emotional features; Indicates Emotional features in the original speech segment ; Indicates Emotional features in the original speech segment ; Indicates Emotional features in the original speech segment The baseline value on is used to standardize the changes in emotional features; It is The distance between the emotional features on the time axis is defined as: in, and Respectively represent Emotional features in the original basic speech segment and the adjusted basic voice clip The time position in It is The harmony of the emotional characteristics is defined as: in, and Respectively represent Emotional features in the original speech segment and the adjusted voice clip The spectral phase in .

[0161] In this step, spectral fusion technology: a technology used to smooth the spectral differences between different speech segments, ensures that the generated speech is natural and smooth at the connection.

[0162] Emotional retention : An indicator for evaluating the emotional coherence of basic speech segments with different emotional colors at the junction.

[0163] Sentiment feature function : Function used to extract emotional features from speech fragments, such as fundamental frequency, harmonic structure, formant position, etc.

[0164] Benchmark value : No. Emotional features in the original speech segment The baseline value on is used to standardize the changes in emotional features.

[0165] Time distance : No. The distance between the emotional features on the time axis is used to measure the changes of emotional features over time.

[0166] Harmony : No. The harmony degree of emotional features is used to measure the similarity of emotional features in the spectral phase.

[0167] Attenuation Factor and : Used to control the impact of time distance and emotional characteristic harmony on emotional retention.

[0168] Importance Weight : No. The importance weights of the emotional features reflect the relative importance of different emotional features in the emotional retention.

[0169] The emotional retention is calculated using the following formula: , so as to adjust the basic speech segments with different emotional colors to smoothly transition at the connection by using the emotion preservation degree: in, represents the original basic speech segment, Indicates the basic voice segment on the adjusted timeline; Indicates A sentiment feature function, is the total number of sentiment feature functions; is a small positive number used to prevent division by zero errors; It is The importance weights of the emotional features reflect the relative importance of different emotional features in the emotional retention; is a decay factor used to control the effect of time distance; is another attenuation factor used to control the harmony effect of emotional features; Indicates Emotional features in the original speech segment ; Indicates Emotional features in the original speech segment ; Indicates Emotional features in the original speech segment The baseline value on is used to standardize the changes in emotional features; It is The distance between the emotional features on the time axis is defined as: in, and Respectively represent Emotional features in the original basic speech segment and the adjusted basic voice clip The time position in It is The harmony of the emotional characteristics is defined as: in, and Respectively represent Emotional features in the original speech segment and the adjusted voice clip According to the calculated emotion retention :

[0170] Adjust the pitch, timbre and other features of basic speech segments with different emotional colors at the connection points to ensure emotional continuity and smooth transition of the spectrum.

[0171] By fine-tuning the pitch and timbre, we ensure that the generated speech is natural and smooth at the connection and the emotional expression is coherent.

[0172] In this embodiment of the present application, it is assumed that the text information input by the user is: "The weather is so nice today, let's go to the park together!" After the previous steps of processing, the basic voice segment with the "happy" emotion label and the "imperative" tone label has been selected, and the timeline has been adjusted to adapt to the speaking speed of 150 words per minute.

[0173] Calculating emotional retention :

[0174] Assume that there are two emotional features: base frequency Strong harmony ; Original voice clip The fundamental frequency reference value , sound intensity reference value ; The basic voice clip on the adjusted timeline The fundamental frequency , sound intensity Importance weight =0.6, =0.4; attenuation factor , ; Small positive number . Calculate time distance :

[0175] Assume that the time position of the fundamental frequency Second, Seconds; time position of sound intensity Second, Second. Calculating harmony :

[0176] Assume that the spectral phase of the fundamental frequency ; Spectral phase of sound intensity .

[0177] Calculating emotional retention : According to the calculated emotional retention , adjust the pitch and timbre of basic speech segments with different emotional colors at the connection point to ensure emotional coherence and smooth transition of the spectrum.

[0178] For example, adjust the pitch at the connection to smoothly transition from 200Hz to 210Hz, and at the same time adjust the timbre to ensure the continuity of emotional expression.

[0179] Through the above optional scheme, the embodiment of the present invention can more accurately fuse the adjusted timeline with the phoneme sequence of the text information to generate high-quality, highly natural, and emotionally coherent speech output. Specifically, through the emotion retention calculation formula, the system can ensure the emotional coherence of basic speech segments with different emotional colors at the connection points, avoiding abrupt and unnatural emotional expressions. Through spectrum fusion technology and emotion retention calculation, the system can smoothly process the spectrum differences between basic speech segments with different emotional colors, ensuring that the generated speech is natural and smooth at the connection points. Through fine emotional feature adjustment and spectrum fusion technology, the system can generate more natural and realistic speech output, significantly improving the user's auditory experience.

[0180] These improvements enable the present invention to perform well in practical applications and is suitable for a variety of scenarios such as intelligent customer service, audio books, and voice assistants, significantly improving the quality of generated speech and user experience. Figure 2 A schematic diagram of the structure of an efficient text-to-speech system is provided for an embodiment of the present application. Figure 2 As shown, the system includes:

[0181] The receiving and identifying module 21 is used to receive text information input by the user, identify and analyze the emotional color and tone characteristics in the text information;

[0182] A selection module 22, configured to select a matching basic speech segment from a pre-built emotional speech library according to the emotional color and the tone feature;

[0183] An adjustment module 23, for adjusting the time axis of the basic voice segment by using a dynamic time warping algorithm, so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color;

[0184] The fusion module 24 is used to fuse the adjusted time axis with the phoneme sequence of the text information using spectrum fusion technology to generate speech output, wherein the spectrum fusion technology is used to ensure that basic speech segments with different emotional colors transition smoothly at the connection points.

[0185] Figure 2 The efficient text-to-speech system can perform Figure 1 The implementation principle and technical effect of the efficient text-to-speech method described in the illustrated embodiment will not be described in detail. The specific manner in which each module and unit performs operations in the efficient text-to-speech system in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.

[0186] In one possible design, Figure 2 The efficient text-to-speech system of the illustrated embodiment may be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0187] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0188] The processing component 32 is used to: receive text information input by a user, identify and analyze the emotional color and tone characteristics in the text information; select matching basic voice segments from a pre-built emotional speech library based on the emotional color and the tone characteristics; use a dynamic time warping algorithm to adjust the time axis of the basic voice segment so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color; use spectrum fusion technology to fuse the adjusted time axis with the phoneme sequence of the text information to generate speech output, wherein the spectrum fusion technology is used to ensure that basic voice segments with different emotional colors transition smoothly at the connection.

[0189] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0190] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0191] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0192] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0193] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0194] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0195] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The efficient text-to-speech method of the illustrated embodiment.

[0196] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0197] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0198] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An efficient text-to-speech method, characterized in that: include: Receiving text information input by a user, identifying and analyzing the emotional color and tone characteristics in the text information; According to the emotional color and the tone characteristics, selecting a matching basic speech segment from a pre-built emotional speech library; Using a dynamic time warping algorithm to adjust the time axis of the basic voice segment so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color; Using spectrum fusion technology, the adjusted time axis is merged with the phoneme sequence of the text information to generate speech output, wherein the spectrum fusion technology is used to ensure that basic speech segments with different emotional colors have a smooth transition at the connection; The receiving of text information input by the user, and identifying and analyzing the emotional color and tone characteristics in the text information, includes: Receive text information provided by the user through a text input interface; Analyze the text information using natural language processing technology to extract keywords, phrases and sentence structures of the text to understand the basic meaning of the text; Applying a sentiment analysis algorithm to perform sentiment analysis on the text information, determining the type of sentiment expressed in the text, and evaluating the sentiment score; Using a tone recognition model to recognize tone features of the text information, including imperative, interrogative, and declarative, the tone recognition model is trained based on a large amount of text data with tone labels; The method of using spectrum fusion technology to fuse the adjusted time axis with the phoneme sequence of the text information to generate speech output includes: Aligning the phoneme sequence converted from the text information with the adjusted time axis to ensure that each phoneme in the phoneme sequence accurately matches the corresponding basic speech segment; Performing spectrum analysis on each of the basic speech segments to extract spectrum features, wherein the spectrum features include fundamental frequency, harmonic structure and formant position; At the connection of adjacent basic speech segments, spectrum fusion technology is applied to smooth the spectrum difference between the two basic speech segments, and in the spectrum fusion process, the pitch and timbre at the connection of adjacent basic speech segments are adjusted to ensure the coherence of the emotional color; After spectrum fusion processing, all basic speech segments are combined in order to output the speech; The applying of the sentiment analysis algorithm to perform sentiment analysis on the text information, determining the type of sentiment expressed in the text, and evaluating the sentiment score includes: The sentiment score S is calculated using the following formula: S=max(CNN(W,C,T;θ1)+RNN(W,C,T;θ2)+Attention(W,C,T;θ3),0) Where W represents the sentiment weight of the word, C represents the sentiment influence coefficient of the context, and T represents the sentiment adjustment factor of the text structure; max(x, 0) means taking the maximum value between x and 0, ensuring that the sentiment score S is not a negative value, which represents the parameter set of the model; CNN(W, C, T; θ1) is the output of a convolutional neural network model to capture local and global features among word sentiment, contextual sentiment, and text structure sentiment, defined as: CNN(W,C,T;θ1)=Conv(Embed(W,C,T)) Among them, Embed(W, C, T) means converting W, C, T into an embedding vector, Conv means the convolution operation, and θ1 means the parameter set of the model; RNN(W, C, T; θ2) is the output of a recurrent neural network model to capture the temporal dependency and contextual information of sentiment features, defined as: RNN (W, C, T; θ2) = LSTM (Embed (W, C, T)) Among them, LSTM represents the long short-term memory network, and θ2 represents the parameter set of the model; Attention(W, C, T; θ3) is the output of an attention mechanism model, which is used to capture the important part of the sentiment feature and is defined as: Where N is the number of words in the text, α i is the attention weight of the i-th word, h i is the hidden state of the i-th word, and θ3 represents the parameter set of the model; The spectrum fusion technology, when generating speech output, includes: The emotion preservation degree E is calculated using the following formula, so as to adjust the basic speech segments with different emotional colors to smoothly transition at the connection point by using the emotion preservation degree: Wherein, V represents the original basic speech segment, V' represents the basic speech segment on the adjusted time axis; f i represents the i-th emotional feature function, n is the total number of emotional feature functions; ∈ is a small positive number used to prevent division by zero errors; w i is the importance weight of the i-th emotional feature, reflecting the relative importance of different emotional features in emotional retention; λ is a decay factor used to control the influence of time distance; μ is another decay factor used to control the influence of harmony of emotional features; f i (V') represents the i-th emotional feature in the original speech segment V'; f i (V) represents the i-th emotional feature in the original speech segment V; J i (V) represents the baseline value of the i-th emotional feature on the original speech segment V, which is used to standardize the change of emotional features; D i is the distance of the i-th sentiment feature on the time axis, defined as: D i =|t i (V')-t i (V)| Among them, t i (V) and t i (V') represents the time position of the i-th emotional feature in the original basic speech segment V and the adjusted basic speech segment V'; H i is the harmony degree of the i-th emotional feature, defined as: H i =1-cos(φ i (V),φ i (V')) Among them, φ i (V) and φ i (V') represents the spectral phase of the i-th emotional feature in the original speech segment V and the adjusted speech segment V' respectively.

2. The method according to claim 1, characterized in that The selecting a matching basic speech segment from a pre-built emotional speech library according to the emotional color and the tone feature comprises: Constructing an emotional speech library, wherein the emotional speech library contains basic speech segments with various emotional colors and tone characteristics, each segment has a corresponding emotional label and tone label; According to the emotional color and the tone characteristics, searching for matching emotional tags and tone tags in the emotional speech library; Comparing the similarity between the emotion label and the tone label of each voice segment in the search result and the emotion color and the tone feature, and selecting the basic voice segment with the highest similarity as the candidate segment; If there are multiple candidate segments with the same similarity, the basic voice segment that best fits the text context is selected according to the specific content of the text information.

3. The method according to claim 1, characterized in that The method of adjusting the time axis of the basic voice segment by using a dynamic time warping algorithm so that the time axis of the basic voice segment is adapted to the speech speed of the text information while maintaining the consistency of the emotional color includes: Applying a dynamic time warping algorithm to adjust the time axis of the basic voice segment to ensure that the adjusted voice segment can be played at an expected speaking speed; In the process of adjusting the timeline, an emotion retention mechanism is introduced to fine-tune the pitch and volume of the voice clips to ensure that the original emotional color remains unchanged when the playback speed is changed; Verify the matching degree between the adjusted timeline and the speech speed of the text information, as well as the consistency of the emotional color.

4. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement an efficient text-to-speech method as described in any one of claims 1-3.

5. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, an efficient text-to-speech method as described in any one of claims 1-3 is implemented.

Images