Artificial intelligence-based speech synthesis method, device, computer equipment and medium

Abstract

The present invention is applicable to the field of speech synthesis technology, and in particular relates to an artificial intelligence-based speech synthesis method, apparatus, computer equipment, and medium. The present invention inputs a target text into a first emotion classification model to obtain a first emotion prediction category and a first probability of the target text, inputs the context of the target text within a preset context range into a second emotion classification model to obtain a second emotion prediction category and a second probability of the target text, and if the probability difference between the first probability and the second probability does not meet a preset condition, adjusts the preset context range until the updated probability difference meets the preset condition to determine the emotion prediction category of the target text, and inputs the target text, the emotion prediction category of the target text, and a preset sound template into a speech generation model to obtain a target speech. By obtaining a more accurate preset context range for the target text, the accuracy of the emotion prediction category of the target text is improved, and the emotional effect of the synthesized speech is enhanced.

Description

Technical Field

[0001] The present invention is applicable to the field of speech synthesis technology, and in particular relates to a speech synthesis method, apparatus, computer equipment and medium based on artificial intelligence. Background Art

[0002] Speech synthesis is a technology that converts text into speech. With the rise of affective computing, sentiment analysis plays a crucial role in speech synthesis. Current mainstream speech synthesis technologies focus on the individual semantics of spoken text, ignoring the specific context of conversations. This reduces the accuracy of topic representation and the coherence of speech responses, resulting in poor emotional impact in speech synthesis.

[0003] Therefore, in the field of speech synthesis technology, how to improve the emotional effect of synthesized speech has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the embodiments of the present invention provide an artificial intelligence-based speech synthesis method, apparatus, computer equipment and medium to solve the problem in the prior art that the timbre of the synthesized speech is significantly different from the timbre of the target user when there are zero speech samples or lightweight speech samples.

[0005] In a first aspect, an embodiment of the present invention provides a speech synthesis method based on artificial intelligence, the speech synthesis method comprising:

[0006] Obtaining a target text, inputting the target text into a trained first sentiment classification model, obtaining a first sentiment prediction category of the target text, and a first probability of determining that the target text is in the first sentiment prediction category;

[0007] Obtaining a preceding and following context of the target text within a preset context range, inputting the preceding and following context into a trained second sentiment classification model to obtain a second sentiment prediction category of the target text and a second probability of determining the target text as belonging to the second sentiment prediction category;

[0008] Calculating a probability difference between the first probability and the second probability, determining whether the probability difference satisfies a preset condition, and if the probability difference does not satisfy the preset condition, adjusting the preset context range, re-obtaining an updated second emotion prediction category and the second probability, and recalculating the probability difference until the probability difference satisfies the preset condition;

[0009] Determining the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category;

[0010] A preset sound template is obtained, and the target text, the emotion prediction category of the target text, and the sound template are input into a trained speech generation model to obtain the target speech.

[0011] In a second aspect, an embodiment of the present invention provides a speech synthesis device based on artificial intelligence, the speech synthesis device comprising:

[0012] A first sentiment classification module is configured to obtain a target text, input the target text into a trained first sentiment classification model, obtain a first sentiment prediction category of the target text, and determine a first probability of the target text being classified into the first sentiment prediction category;

[0013] A second sentiment classification module is configured to obtain a preceding and following context of the target text within a preset context, input the preceding and following context into a trained second sentiment classification model, and obtain a second predicted sentiment category of the target text and a second probability of the target text being classified as the second predicted sentiment category;

[0014] a context range adjustment module, calculating a probability difference between the first probability and the second probability, determining whether the probability difference satisfies a preset condition, and if the probability difference does not satisfy the preset condition, adjusting the preset context range, re-obtaining the updated second emotion prediction category and the second probability, and recalculating the probability difference until the probability difference satisfies the preset condition;

[0015] a target emotion category judgment module, configured to determine the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category;

[0016] The target speech generation module is used to obtain a preset sound template, input the target text, the emotion prediction category of the target text and the sound template into the trained speech generation model to obtain the target speech.

[0017] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the speech synthesis method as described in the first aspect when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the speech synthesis method as described in the first aspect is implemented.

[0019] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows: by inputting the target text into a trained first emotion classification model, the first emotion prediction category of the target text and the first probability of being judged as the first emotion prediction category are obtained; the above and below contexts of the target text within a preset context range are input into a trained second emotion classification model to obtain the second emotion prediction category of the target text and the second probability of being judged as the second emotion prediction category; and whether the probability difference between the first probability and the second probability meets the preset conditions is judged; if the probability difference does not meet the preset conditions, the preset context range is adjusted, the updated second emotion prediction category and the second probability are obtained again, and the probability difference is recalculated until the probability difference meets the preset conditions, thereby determining the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category; finally, the target text, the emotion prediction category of the target text and the sound template are input into a trained speech generation model to obtain the target speech; by adjusting to obtain a more accurate preset context range for the target text, the accuracy of the emotion prediction category of the target text is improved by combining the target text and the context within the preset context range, and the emotional effect of the synthesized speech is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a schematic diagram of an application environment of a speech synthesis method provided in Example 1 of the present invention;

[0022] Figure 2 This is a flow chart of a speech synthesis method provided in Example 1 of the present invention;

[0023] Figure 3 This is a structural diagram of a speech synthesis device provided in Embodiment 2 of the present invention;

[0024] Figure 4 This is a structural diagram of a computer device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0025] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0027] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0028] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0029] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0031] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0032] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0033] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0034] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0035] A speech synthesis method provided in the first embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, a client communicates with a server. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0036] See also Figure 2 , is a flow chart of a speech synthesis method provided by the first embodiment of the present invention, the above-mentioned speech synthesis method can be applied to Figure 1 In the client, the speech synthesis method may include the following steps:

[0037] Step S201 : obtaining a target text, inputting the target text into a trained first sentiment classification model, obtaining a first sentiment prediction category of the target text, and a first probability of determining the target text as belonging to the first sentiment prediction category.

[0038] Among them, the target text is the text to be synthesized speech. The target text can be part of the text content in a paragraph, an article, or a book. There is corresponding contextual content, and it is in a specific situation together with the contextual content. Therefore, the emotion of the context can be combined to improve the accuracy of the emotional representation of the target text, so as to improve the emotional effect of the generated speech.

[0039] The first emotion prediction category output by the first emotion classification model is used to represent the emotion contained in the input text of the first emotion classification model. The specific first emotion prediction category can be set according to actual conditions and the first emotion prediction category training is performed accordingly. For example, the first emotion prediction category includes positive emotion, negative emotion, affirmative emotion, negative emotion, positive emotion, and negative emotion.

[0040] Specifically, the target text is first obtained, and the target text is input into the trained first sentiment classification model, and the sentiment features of the target text are extracted to output the first sentiment prediction category of the target text, and a first probability of being judged as the first sentiment prediction category is obtained, wherein the first sentiment prediction category is used to characterize the sentiment category contained in the target text, and the first probability is used to characterize the confidence of the target text in containing the sentiment category.

[0041] Optionally, the first sentiment classification model includes a first encoder and a first decoder, and the first sentiment classification model is trained using sample texts as training samples and actual sentiment categories of the samples as training labels.

[0042] The training process of the first sentiment classification model includes:

[0043] Input the sample text into the first encoder for feature extraction to obtain sample text features;

[0044] Performing Gaussian sampling on the sample text features to obtain a sample text sampling result, and inputting the sample text sampling result into a first decoder for reconstruction to obtain a first emotion prediction category of the sample text;

[0045] The first loss function is calculated according to the first emotion prediction category, the first probability and the actual emotion category of the sample text, and the parameters of the first encoder and the first decoder are reversely corrected according to the gradient descent method until the first loss function converges to obtain a trained first emotion classification model.

[0046] Among them, the training samples of the first sentiment classification model are a large number of sample texts, which can also be part of the text content in a paragraph, an article, or a book, and have corresponding contextual content. During the training process of the first sentiment classification model, the sample text is input into the first encoder for feature extraction to obtain sample text features, and Gaussian sampling is performed on the sample text features to obtain sample text sampling results, and then the sample text sampling results are input into the first decoder for reconstruction to obtain the first sentiment prediction category of the sample text, and the first probability of being judged as the first sentiment prediction category is obtained.

[0047] At the same time, in order to improve the sentiment classification accuracy of the first sentiment classification model, the first loss function is calculated based on the first sentiment prediction category, the first probability and the actual sentiment category of the sample text obtained, wherein the smaller the first loss function is, the higher the accuracy of the first sentiment prediction category and the first probability is, and conversely, the larger the first loss function is, the lower the accuracy of the first sentiment prediction category and the first probability is. It is necessary to reversely correct the parameters of the first encoder and the first decoder according to the gradient descent method until the first loss function converges to improve the classification accuracy of the first sentiment classification model and finally obtain a trained first sentiment classification model.

[0048] In one embodiment, during the training of the first sentiment classification model, in order to improve the sentiment classification accuracy of the first sentiment classification model, when calculating the first loss function based on the first sentiment prediction category, the first probability, and the actual sentiment category of the sample text obtained, it is first determined whether the first sentiment prediction category and the actual sentiment category are consistent. If the first sentiment prediction category and the actual sentiment category are inconsistent, the first loss function Loss1 is recorded as Loss1=1. If the first sentiment prediction category and the actual sentiment category are consistent, the first probability is recorded as Q1, and the first loss function Loss1 is:

[0049] Loss1=1-Q1

[0050] Where Q1 is the first probability of being judged as the first emotion prediction category.

[0051] Step S202, obtain the context of the target text within the preset context range, input the context into the trained second sentiment classification model, obtain the second sentiment prediction category of the target text, and the second probability of judging it as the second sentiment prediction category.

[0052] Among them, the above and below contexts belong to the same paragraph, article, or book as the target text, and are in the same specific context as the target text. The context range can be preset according to the source of the target text. For example, when the target text comes from part of the text content in an article, the preset context range is the upper and lower paragraphs of the target text. The above context is from the previous paragraph of the target text to the previous word of the target text, and the below context is from the next word of the target text to the next paragraph of the target text. The preset context range can be adjusted according to actual conditions to improve the accuracy of the representation of the emotions contained in the context to the emotions contained in the target text.

[0053] The second emotion prediction category output by the second emotion classification model is used to represent the emotion contained in the target text corresponding to the input text of the second emotion classification model. The specific second emotion prediction category is consistent with the first emotion prediction category corresponding to the first emotion classification model. For example, the second emotion prediction category and the first emotion prediction category both include positive emotion, negative emotion, affirmative emotion, negative emotion, positive emotion, and negative emotion.

[0054] Specifically, first determine the preset context range of the target text according to actual conditions, and obtain the context above and below the target text within the preset context range, input the context above and below into the trained second sentiment classification model, extract the sentiment features of the context above and below, and output the second sentiment prediction category of the target text corresponding to the context above and below, and obtain a second probability of being judged as the second sentiment prediction category, wherein the second sentiment classification model judges the emotion contained in the target text by the emotions contained in the context above and below, and uses the second sentiment prediction category to characterize the sentiment category contained in the target text, and uses the second probability to characterize the confidence of the target text containing the sentiment category.

[0055] Optionally, the second sentiment classification model includes a second encoder and a second decoder, and is trained using the sample context and the sample context as training samples and the actual sentiment category of the sample as a training label.

[0056] The training process of the second sentiment classification model includes:

[0057] Input the sample context and the sample context into the second encoder for feature extraction to obtain sample context features;

[0058] Performing Gaussian sampling on the sample context features to obtain a sample context sampling result, and inputting the sample context sampling result into a second decoder for reconstruction to obtain a second emotion prediction category of the sample text;

[0059] The second loss function is calculated according to the second emotion prediction category of the sample text and the actual emotion category of the sample, and the parameters of the second encoder and the second decoder are reversely corrected according to the gradient descent method until the second loss function converges to obtain a trained second emotion classification model.

[0060] Among them, the training samples of the second sentiment classification model are a large number of sample contexts and sample contexts. During the training process of the second sentiment classification model, the sample context and sample context are input into the second encoder for feature extraction to obtain sample context features, and the sample context features are Gaussian sampled to obtain sample context sampling results, and then the sample context sampling results are input into the second decoder for reconstruction to obtain the second sentiment prediction category of the corresponding sample text, and the second probability of being judged as the second sentiment prediction category is obtained.

[0061] At the same time, in order to improve the sentiment classification accuracy of the second sentiment classification model, the second loss function is calculated based on the second sentiment prediction category, the second probability and the actual sentiment category of the corresponding sample text. The smaller the second loss function is, the higher the accuracy of the second sentiment prediction category and the second probability is. Conversely, the larger the second loss function is, the lower the accuracy of the second sentiment prediction category and the second probability is. It is necessary to reversely correct the parameters of the second encoder and the second decoder according to the gradient descent method until the second loss function converges to improve the classification accuracy of the second sentiment classification model, and finally obtain a trained second sentiment classification model to more accurately represent the emotions contained in the corresponding target text according to the above and below contexts.

[0062] In one embodiment, during the training of the second sentiment classification model, in order to improve the sentiment classification accuracy of the second sentiment classification model, when calculating the second loss function based on the obtained second sentiment prediction category, the second probability and the actual sentiment category of the corresponding sample text, it is first determined whether the second sentiment prediction category and the actual sentiment category are consistent. If the second sentiment prediction category and the actual sentiment category are inconsistent, the second loss function Loss2 is recorded as Loss2=1. If the second sentiment prediction category and the actual sentiment category are consistent, the second probability is recorded as Q2, and the second loss function Loss2 is:

[0063] Loss2=1-Q2

[0064] Where Q2 is the second probability of being judged as the second emotion prediction category.

[0065] Step S203, calculate the probability difference between the first probability and the second probability, and determine whether the probability difference meets the preset conditions. If the probability difference does not meet the preset conditions, adjust the preset context range, obtain the updated second emotion prediction category and second probability, and recalculate the probability difference until the probability difference meets the preset conditions.

[0066] Among them, the first probability is the probability that the target text is judged as the first emotion prediction category, and the first emotion prediction category is obtained based on feature extraction and emotion analysis of the target text. The second probability is the probability that the target text is judged as the second emotion prediction category, and the second emotion prediction category is obtained based on feature extraction and emotion analysis of the previous and next contexts of the target text within a preset context. Therefore, when the first emotion prediction category and the second emotion prediction category are consistent, the probability difference between the first probability and the second probability is calculated, and it is determined whether the probability difference meets the preset conditions. The preset conditions are, for example, a preset difference threshold based on actual conditions.

[0067] If the probability difference meets the preset conditions, it means that the difference between the second emotion prediction category and the first emotion prediction category is small, and the accuracy of the representation of the emotion contained in the target text by the previous and next contexts is high. Conversely, if the probability difference does not meet the preset conditions, it means that the difference between the second emotion prediction category and the first emotion prediction category is large. In order to improve the accuracy of the representation of the emotion contained in the target text by the previous and next contexts, the preset context range is adjusted, and the previous and next contexts with a larger range are selected. The degree of representation of the emotion contained in the target text is improved by sacrificing the classification efficiency of the second emotion classification model, and the updated second emotion prediction category and second probability are obtained again, and the corresponding probability difference is recalculated until the updated probability difference meets the preset conditions.

[0068] Optionally, determining whether the probability difference satisfies a preset condition includes:

[0069] When the first emotion prediction category and the second emotion prediction category are consistent, determining whether the probability difference meets a preset condition based on the probability difference and a preset threshold value;

[0070] If the probability difference is less than the preset threshold value, it is determined that the probability difference meets the preset condition;

[0071] If the probability difference is not less than the preset threshold value, it is determined that the probability difference does not meet the preset condition.

[0072] Among them, when the first emotion prediction category and the second emotion prediction category are consistent, a threshold value is first preset according to actual conditions. For example, in this embodiment, the threshold value is preset to 0.3 according to actual conditions. The probability difference obtained by calculation and the preset threshold value are used to determine whether the probability difference meets the preset conditions. If the probability difference is less than the preset threshold value, it is determined that the probability difference meets the preset conditions. If the probability difference is not less than the preset threshold value, it is determined that the probability difference does not meet the preset conditions, and it is necessary to increase the preset context range and obtain the updated second emotion prediction category again to improve the degree of representation of the emotions contained in the target text by the above and below contexts.

[0073] Optionally, determining whether the probability difference satisfies a preset condition further includes:

[0074] When the first emotion prediction category and the second emotion prediction category are inconsistent, determining a parameter adjustment object according to the first probability and the second probability;

[0075] If the first probability is not less than the second probability, determining that the parameter adjustment object is the second emotion classification model, adjusting the parameters of the second emotion classification model, and re-obtaining the adjusted second emotion prediction category until the first emotion prediction category and the second emotion prediction category are consistent;

[0076] If the first probability is less than the second probability, determine that the parameter adjustment object is the first emotion classification model, adjust the parameters of the first emotion classification model, and obtain the adjusted first emotion prediction category again until the first emotion prediction category is consistent with the second emotion prediction category.

[0077] Among them, when the first emotion prediction category and the second emotion prediction category are inconsistent, the calculated probability difference between the first probability and the second probability cannot represent the difference in the characterization results of characterizing the emotions contained in the target text according to the target text and the corresponding context, that is, it is impossible to judge whether the preset context range needs to be adjusted by judging whether the probability difference meets the preset conditions. Therefore, it is first necessary to optimize the first emotion prediction category or the second emotion prediction category by adjusting the parameters of the model so that the first emotion prediction category and the second emotion prediction category are consistent as the basis for adjusting the preset context range.

[0078] Specifically, when the first emotion prediction category and the second emotion prediction category are inconsistent, the parameter adjustment object is determined according to the first probability and the second probability:

[0079] If the first probability is not less than the second probability, it indicates that the confidence level of the first emotion prediction category contained in the target text is not less than the confidence level of the second emotion prediction category contained in the target text. Considering the parameter adjustment efficiency and parameter adjustment effect, the parameter adjustment object is determined to be the second emotion classification model. By adjusting the parameters of the second emotion classification model, for example, adjusting the neuron weights, biases or convolutional kernel weights according to the actual gradient inside the second emotion classification model, the adjusted second emotion prediction category is obtained again until the first emotion prediction category and the second emotion prediction category are the same.

[0080] If the first probability is less than the second probability, it indicates that the confidence level of the first emotion prediction category contained in the target text is less than the confidence level of the second emotion prediction category contained in the target text. Considering the parameter adjustment efficiency and parameter adjustment effect, the parameter adjustment object is determined to be the first emotion classification model. By adjusting the parameters of the first emotion classification model, for example, adjusting the neuron weights, biases or convolutional kernel weights according to the actual gradient inside the first emotion classification model, the adjusted first emotion prediction category is obtained again until the first emotion prediction category and the second emotion prediction category are the same.

[0081] In one implementation, when the first emotion prediction category and the second emotion prediction category are the same, the first probability is Q1 and the second probability is Q2. According to the actual situation, the preset threshold value is Q0, and the difference between the first probability Q1 and the second probability Q2 is denoted as ΔQ. Then the probability difference ΔQ is:

[0082] ΔQ = |Q1 - Q2|

[0083] In the formula, Q1 is the first probability and Q2 is the second probability.

[0084] If ΔQ < Q0, it is determined that the probability difference meets the preset condition. If ΔQ ≥ Q0, it is determined that the probability difference does not meet the preset condition, and it is necessary to increase the preset context range to obtain the updated second emotion prediction category again, where the preset threshold value Q0 = 0.3 according to the actual situation.

[0085] In one implementation, when the first emotion prediction category and the second emotion prediction category are not the same, the first probability is Q1 and the second probability is Q2. If Q1 ≥ Q2, the parameter adjustment object is determined to be the second emotion classification model, and the parameters of the second emotion classification model are adjusted to obtain the adjusted second emotion prediction category again until the first emotion prediction category and the second emotion prediction category are the same; if Q1 < Q2, the parameter adjustment object is determined to be the first emotion classification model, and the parameters of the first emotion classification model are adjusted to obtain the adjusted first emotion prediction category again until the first emotion prediction category and the second emotion prediction category are the same.

[0086] Step S204: determining the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category.

[0087] Among them, after adjusting the preset context range and the model parameters in step S203, the first emotion prediction category obtained by feature extraction and sentiment analysis of the target text is consistent with the second emotion prediction category obtained by feature extraction and sentiment analysis of the context before and after the target text within the preset context range. Therefore, the first emotion prediction category and the second emotion prediction category are determined to be the emotion prediction category of the target text. Combined with the target text and the context before and after the target text within the preset context range, the accuracy of the emotion prediction category of the target text is improved.

[0088] Step S205: Obtain a preset sound template, input the target text, the emotion prediction category of the target text, and the sound template into the trained speech generation model to obtain the target speech.

[0089] Among them, the speech generation model is a pre-trained model, including a trained third encoder, a trained generator, a trained third decoder and a trained vocoder. The target text is input into the trained third encoder for feature extraction to obtain text encoding features, and the text encoding features, the obtained emotion prediction category of the target text and the preset sound template are input into the trained generator to obtain speech features, and the speech features are input into the trained third decoder for reconstruction to obtain the mel spectrogram of the generated speech, and finally the mel spectrogram is input into the trained vocoder to obtain the target speech that conforms to the sound template, matches the target text and contains the emotion prediction category of the target text.

[0090] The embodiment of the present invention inputs the target text into a trained first emotion classification model to obtain the first emotion prediction category of the target text and the first probability of being judged as the first emotion prediction category, inputs the above and below context of the target text within a preset context range into a trained second emotion classification model to obtain the second emotion prediction category of the target text and the second probability of being judged as the second emotion prediction category, and judges whether the probability difference between the first probability and the second probability meets the preset conditions. If the probability difference does not meet the preset conditions, the preset context range is adjusted to obtain the updated second emotion prediction category and the second probability, and the probability difference is recalculated until the probability difference meets the preset conditions, thereby determining the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category, and finally inputs the target text, the emotion prediction category of the target text and the sound template into the trained speech generation model to obtain the target speech. By adjusting the preset context range to obtain a more accurate preset context range for the target text, the accuracy of the emotion prediction category of the target text is improved by combining the target text and the context within the preset context range, and the emotional effect of the synthesized speech is improved.

[0091] Corresponding to the speech synthesis method of the above embodiment, Figure 3 A structural block diagram of a speech synthesis device provided in the second embodiment of the present invention is given. For ease of explanation, only the parts related to the embodiment of the present invention are shown.

[0092] See also Figure 3 , the speech synthesis device comprises:

[0093] A first sentiment classification module 31 is configured to obtain a target text, input the target text into a trained first sentiment classification model, and obtain a first predicted sentiment category of the target text and a first probability of the target text being classified as the first predicted sentiment category;

[0094] A second sentiment classification module 32 is configured to obtain the context of the target text within a preset context, input the context into a trained second sentiment classification model, and obtain a second sentiment prediction category of the target text and a second probability of the target text being classified as the second sentiment prediction category.

[0095] A context range adjustment module 33 is configured to calculate a probability difference between the first probability and the second probability, determine whether the probability difference satisfies a preset condition, and if not, adjust the preset context range, obtain an updated second emotion prediction category and second probability, and recalculate the probability difference until the probability difference satisfies the preset condition.

[0096] A target emotion category determination module 34 is configured to determine the emotion prediction category of the target text based on the first emotion prediction category and the second emotion prediction category;

[0097] The target speech generation module 35 is used to obtain a preset sound template, input the target text, the emotion prediction category of the target text and the sound template into the trained speech generation model to obtain the target speech.

[0098] Optionally, the first emotion classification module 31 includes:

[0099] The first training submodule is used to determine that the first sentiment classification model includes a first encoder and a first decoder, and train the first sentiment classification model using sample text as a training sample and the actual sentiment category of the sample as a training label.

[0100] Optionally, the first training submodule includes:

[0101] A first feature extraction unit is used to input the sample text into the first encoder for feature extraction to obtain sample text features;

[0102] A first category prediction unit is configured to perform Gaussian sampling on the sample text features to obtain a sample text sampling result, and input the sample text sampling result into a first decoder for reconstruction to obtain a first emotion prediction category of the sample text;

[0103] The first parameter correction unit is used to calculate the first loss function based on the first emotion prediction category, the first probability and the actual emotion category of the sample text, and reversely correct the parameters of the first encoder and the first decoder according to the gradient descent method until the first loss function converges to obtain a trained first emotion classification model.

[0104] Optionally, the second emotion classification module 32 includes:

[0105] The second training submodule is used to determine that the second emotion classification model includes a second encoder and a second decoder, uses the sample context and the sample context as training samples, and uses the actual emotion category of the sample as a training label to train the second emotion classification model.

[0106] Optionally, the second training submodule includes:

[0107] A second feature extraction unit is used to input the sample context and the sample context into a second encoder for feature extraction to obtain sample context features;

[0108] The second category prediction unit is used to perform Gaussian sampling on the sample context feature to obtain a sample context sampling result, and input the sample context sampling result into the second decoder for reconstruction to obtain a second emotion prediction category of the sample text;

[0109] The second parameter correction unit is used to calculate the second loss function based on the second emotion prediction category, the second probability and the actual emotion category of the sample text, and reversely correct the parameters of the second encoder and the second decoder according to the gradient descent method until the second loss function converges to obtain a trained second emotion classification model.

[0110] Optionally, the context range adjustment module 33 includes:

[0111] A condition determination submodule, configured to determine whether the probability difference satisfies a preset condition based on the probability difference and a preset threshold value when the first emotion prediction category and the second emotion prediction category are consistent;

[0112] The parameter adjustment submodule is used to determine a parameter adjustment object according to the first probability and the second probability when the first emotion prediction category and the second emotion prediction category are inconsistent.

[0113] Optionally, the above-mentioned condition determination submodule includes:

[0114] A first condition judgment unit, configured to determine that the probability difference satisfies a preset condition if the probability difference is less than a preset threshold value;

[0115] The second condition judgment unit is used to determine that the probability difference does not meet the preset condition if the probability difference is not less than the preset threshold value.

[0116] Optionally, the parameter adjustment submodule includes:

[0117] a first parameter adjustment unit, configured to, if the first probability is not less than the second probability, determine that the parameter adjustment object is the second emotion classification model, adjust the parameters of the second emotion classification model, and obtain the adjusted second emotion prediction category again, until the first emotion prediction category and the second emotion prediction category are consistent;

[0118] The second parameter adjustment unit is used to determine that the parameter adjustment object is the first emotion classification model if the first probability is less than the second probability, adjust the parameters of the first emotion classification model, and re-obtain the adjusted first emotion prediction category until the first emotion prediction category is consistent with the second emotion prediction category.

[0119] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0120] Figure 4 This is a schematic diagram of the structure of a computer device provided in the third embodiment of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned embodiments of the speech synthesis method are implemented.

[0121] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 4 The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0122] The processor may be a CPU, or other general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor, or any conventional processor.

[0123] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be the hard disk of the computer device, and in other embodiments, it can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory can also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, boot loaders (BootLoader), data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or is about to be output.

[0124] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiment. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0125] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0126] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0127] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0128] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A speech synthesis method based on artificial intelligence, characterized in that: The speech synthesis method comprises: Obtaining a target text, inputting the target text into a trained first sentiment classification model, obtaining a first sentiment prediction category of the target text, and a first probability of determining that the target text is in the first sentiment prediction category; Obtaining a preceding and following context of the target text within a preset context range, inputting the preceding and following context into a trained second sentiment classification model to obtain a second sentiment prediction category of the target text and a second probability of determining the target text as belonging to the second sentiment prediction category; Calculating a probability difference between the first probability and the second probability, determining whether the probability difference satisfies a preset condition, and if the probability difference does not satisfy the preset condition, adjusting the preset context range, re-obtaining an updated second emotion prediction category and the second probability, and recalculating the probability difference until the probability difference satisfies the preset condition; Determining the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category; A preset sound template is obtained, and the target text, the emotion prediction category of the target text, and the sound template are input into a trained speech generation model to obtain the target speech.

2. The speech synthesis method according to claim 1, wherein: The first sentiment classification model includes a first encoder and a first decoder. The first sentiment classification model is trained using sample texts as training samples and the actual sentiment categories of the samples as training labels.

3. The speech synthesis method according to claim 2, wherein: The training process of the first sentiment classification model includes: Inputting the sample text into the first encoder for feature extraction to obtain sample text features; Performing Gaussian sampling on the sample text features to obtain a sample text sampling result, and inputting the sample text sampling result into the first decoder for reconstruction to obtain a first emotion prediction category of the sample text; A first loss function is calculated based on the first emotion prediction category of the sample text, the first probability and the actual emotion category of the sample, and the parameters of the first encoder and the first decoder are reversely corrected according to the gradient descent method until the first loss function converges to obtain a trained first emotion classification model.

4. The speech synthesis method according to claim 1, wherein: The second emotion classification model includes a second encoder and a second decoder. The second emotion classification model is trained using the sample context and the sample context as training samples and the actual emotion category of the sample as a training label.

5. The speech synthesis method according to claim 4, characterized in that The training process of the second sentiment classification model includes: Inputting the sample context and the sample context into the second encoder for feature extraction to obtain sample context features; Performing Gaussian sampling on the sample context feature to obtain a sample context sampling result, and inputting the sample context sampling result into the second decoder for reconstruction to obtain a second emotion prediction category of the training sample; A second loss function is calculated based on the second emotion prediction category of the training sample, the second probability and the actual emotion category of the sample, and the parameters of the second encoder and the second decoder are reversely corrected according to the gradient descent method until the second loss function converges to obtain a trained second emotion classification model.

6. The speech synthesis method according to claim 1, wherein: Determining whether the probability difference satisfies a preset condition includes: When the first emotion prediction category and the second emotion prediction category are consistent, determining whether the probability difference meets a preset condition according to the probability difference and a preset threshold value; If the probability difference is less than the preset threshold value, it is determined that the probability difference meets the preset condition; If the probability difference is not less than the preset threshold value, it is determined that the probability difference does not meet the preset condition.

7. The speech synthesis method according to claim 6, characterized in that: Determining whether the probability difference satisfies a preset condition further includes: When the first emotion prediction category and the second emotion prediction category are inconsistent, determining a parameter adjustment object according to the first probability and the second probability; If the first probability is not less than the second probability, determining that the parameter adjustment object is the second emotion classification model, adjusting the parameters of the second emotion classification model, and re-obtaining an adjusted second emotion prediction category until the first emotion prediction category and the adjusted second emotion prediction category are consistent; If the first probability is less than the second probability, determine that the parameter adjustment object is the first emotion classification model, adjust the parameters of the first emotion classification model, and obtain the adjusted first emotion prediction category again until the first emotion prediction category is consistent with the adjusted second emotion prediction category.

8. A speech synthesis device based on artificial intelligence, characterized in that: The speech synthesis device comprises: A first sentiment classification module is configured to obtain a target text, input the target text into a trained first sentiment classification model, obtain a first sentiment prediction category of the target text, and determine a first probability of the target text being classified into the first sentiment prediction category; A second sentiment classification module is configured to obtain a preceding and following context of the target text within a preset context, input the preceding and following context into a trained second sentiment classification model, and obtain a second predicted sentiment category of the target text and a second probability of the target text being classified as the second predicted sentiment category; a context range adjustment module, configured to calculate a probability difference between the first probability and the second probability, determine whether the probability difference satisfies a preset condition, and if the probability difference does not satisfy the preset condition, adjust the preset context range, re-obtain an updated second emotion prediction category and the second probability, and recalculate the probability difference until the probability difference satisfies the preset condition; a target emotion category judgment module, configured to determine the emotion prediction category of the target text according to the first emotion prediction category and the second emotion prediction category; The target speech generation module is used to obtain a preset sound template, input the target text, the emotion prediction category of the target text and the sound template into the trained speech generation model to obtain the target speech.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the speech synthesis method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech synthesis method according to any one of claims 1 to 7 is implemented.

Images