Model training method, voice processing method and related equipment

Abstract

The embodiment of the invention provides a model training method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the text-to-voice conversion of a first text of a first text paragraph, and obtaining the first voice data of the first text; performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; determining a first value of the first voice data according to the first semantic feature and the second semantic feature; adjusting at least one of speed, volume and intonation of the first voice data according to the first value to obtain second voice data; and training a model according to the second voice data and the third voice data of the first text. Therefore, the voice generation effect of the model can be optimized.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, a speech processing method, a device, an electronic device, a computer storage medium, and a computer program product. Background Art

[0002] Text-to-speech (TTS) is a technology that converts text information into speech output and is widely used in daily life, work, and study.

[0003] In practical applications, text is often converted into standardized voice data through models and then broadcast. However, this may result in the broadcasted voice data being always bland, regardless of whether there are important turns in the text content, making it difficult for users listening to the broadcast to capture key information. Summary of the Invention

[0004] The embodiments of the present application provide a model training method, a speech processing method, an apparatus, an electronic device, a computer storage medium, and a computer program product.

[0005] In a first aspect, an embodiment of the present application provides a model training method, comprising: Performing text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; Performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; determining a first value of the first speech data according to the first semantic feature and the second semantic feature; adjusting at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; A model is trained based on the second voice data and the third voice data of the first text.

[0006] In a second aspect, an embodiment of the present application provides a speech generation method, comprising: Performing text-to-speech conversion on the second text of the second text paragraph to obtain fourth voice data of the second text; performing feature extraction on the second text to obtain a third semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a fourth semantic feature of the second text paragraph; determining a second value of the fourth speech data according to the third semantic feature and the fourth semantic feature; At least one of the speech rate, volume, and tone of the fourth voice data is adjusted according to the second value to obtain fifth voice data.

[0007] In a third aspect, an embodiment of the present application provides a model training device, comprising: a first conversion unit, configured to perform text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; a first extraction unit configured to perform feature extraction on the first text to obtain a first semantic feature of the first text, and to perform context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; a first determining unit, configured to determine a first value of the first speech data according to the first semantic feature and the second semantic feature; a first adjusting unit, configured to adjust at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; A training unit is used to train a model based on the second voice data and the third voice data of the first text.

[0008] In a fourth aspect, an embodiment of the present application provides a speech generation device, comprising: a second conversion unit, configured to perform text-to-speech conversion on a second text of the second text paragraph to obtain fourth voice data of the second text; a second extraction unit configured to perform feature extraction on the second text to obtain a third semantic feature of the second text, and to perform context feature extraction on the second text paragraph to obtain a fourth semantic feature of the second text paragraph; a second determining unit, configured to determine a second value of the fourth speech data according to the third semantic feature and the fourth semantic feature; The second adjustment unit is configured to adjust at least one of the speech rate, volume, and intonation of the fourth voice data according to the second value to obtain fifth voice data.

[0009] In a fifth aspect, an embodiment of the present application provides an electronic device comprising: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to execute the model training method as described in the first aspect, or the speech generation method as described in the second aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium for storing computer-executable instructions, which, when executed by a processor, implement the model training method as described in the first aspect, or the speech generation method as described in the second aspect.

[0011] In the seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the model training method as described in the first aspect, or the speech generation method as described in the second aspect.

[0012] It can be seen that in an embodiment of the present application, first, text-to-speech is performed on the first text of the first text paragraph to obtain first voice data of the first text; secondly, feature extraction is performed on the first text to obtain a first semantic feature of the first text, and context feature extraction is performed on the first text paragraph to obtain a second semantic feature of the first text paragraph; then, based on the first semantic feature and the second semantic feature, a first value of the first voice data is determined; then, based on the first semantic feature, at least one of the speaking speed, volume and intonation of the first voice data is adjusted according to the first value to obtain second voice data; finally, a model is trained based on the second voice data and the third voice data of the first text. In this way, by determining the first value of the first voice data according to the first semantic feature and the second semantic feature and adjusting the first voice data of the first text according to the first value to obtain the second voice data, the semantic features of the first text itself and the context features of the first text paragraph including the first text can be referred to at the same time during the generation of the second voice data. This is conducive to perceiving the changing trend of the text content of the first text paragraph. Therefore, at least one of the speech speed, volume and intonation of the second voice data obtained by adjusting the first voice data according to the first value is more in line with the changing trend. Furthermore, through model training, the model can fully learn how to make at least one of the speech speed, volume and intonation of the generated second voice data match the changing trend of the text content of the first text paragraph as much as possible, thereby optimizing the voice generation effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments described in this specification. Those skilled in the art can also derive other drawings based on these drawings without inventive work. Figure 1 A schematic diagram of an implementation environment for a model training method provided in an embodiment of the present application; Figure 2 A processing flow chart of a model training method provided in an embodiment of the present application; Figure 3 A processing flow chart of another model training method provided in an embodiment of the present application; Figure 4A data flow diagram of another model training method provided in an embodiment of the present application; Figure 5 A schematic diagram of the data flow of a dimension conversion module in a speech generation model provided in an embodiment of the present application; Figure 6A A schematic diagram of the structure of a speech generation model provided in an embodiment of the present application; Figure 6B A schematic diagram of the data flow of a control information determination module in a speech generation model provided in an embodiment of the present application; Figure 7 A processing flow chart of a speech generation method provided in an embodiment of the present application; Figure 8 A schematic diagram of a model training device provided in an embodiment of the present application; Figure 9 A schematic diagram of a speech generation device provided in an embodiment of the present application; Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0015] In intelligent question-and-answer scenarios, robots and users can interact through voice. For example, a user asks a question, and the robot generates a text response based on the question, converts the response into voice data, and then reads it.

[0016] In actual applications, robots often use speech generation models to convert response text into standardized voice data and broadcast it. The standardized voice data may have the following characteristics: uniform speaking speed, constant volume, no fluctuation in tone, etc.

[0017] When the voice data read by the robot is long, regardless of whether the text content has important turning points, the voice data does not have obvious ups and downs. This may cause the user to be distracted halfway through listening and miss important information, which is not conducive to the user listening to the broadcast to capture the key information.

[0018] Therefore, in order to solve the above problems, an embodiment of the present application provides a model training method.

[0019] The model training method provided in one or more embodiments of this specification may be applicable to an implementation environment of the model training method, which implementation environment includes at least a server 101 for model training.

[0020] Among them, server 101 can be a server, or a server cluster composed of several servers, or one or more cloud servers in a cloud computing platform, used for model training.

[0021] In this implementation environment, during the model training process, the server 101 first performs text-to-speech conversion on the first text of the first text paragraph to obtain first voice data of the first text; secondly, performs feature extraction on the first text to obtain a first semantic feature of the first text, and performs context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; then, determines a first value of the first voice data based on the first semantic feature and the second semantic feature; then, adjusts at least one of the speaking speed, volume, and intonation of the first voice data based on the first value to obtain second voice data; finally, trains the model based on the second voice data and the third voice data of the first text. In this way, by determining the first value of the first voice data according to the first semantic feature and the second semantic feature and adjusting the first voice data of the first text according to the first value to obtain the second voice data, the semantic features of the first text itself and the context features of the first text paragraph including the first text can be referred to at the same time during the generation of the second voice data. This is conducive to perceiving the changing trend of the text content of the first text paragraph. Therefore, at least one of the speech speed, volume and intonation of the second voice data obtained by adjusting the first voice data according to the first value is more in line with the changing trend. Furthermore, through model training, the model can fully learn how to make at least one of the speech speed, volume and intonation of the generated second voice data match the changing trend of the text content of the first text paragraph as much as possible, thereby optimizing the voice generation effect of the model.

[0022] Figure 2 This is a processing flow chart of a model training method provided in an embodiment of the present application. Figure 2 The model training method provided in this embodiment specifically includes steps S202 to S210.

[0023] Step S202 : performing text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text.

[0024] A text paragraph is a basic structural unit in written language. It consists of a group of related sentences and is usually used to express a complete idea or theme.

[0025] Specifically in one or more embodiments of this specification, the first text paragraph may include multiple sentences, and the sentences are logically related to each other to form a meaningful and organized whole.

[0026] For example, in a call scenario, the content of the first text paragraph may be a conversation text. In a broadcast scenario, the content of the first text paragraph may be a report text. In a novel reading scenario, the content of the first text paragraph may be a novel text, and so on.

[0027] The first text may be a sentence in the first text paragraph, or the first text may be a plurality of consecutive sentences in the first text paragraph.

[0028] For example, the first text paragraph includes sentence S1, sentence S2, sentence S3 and sentence S4, wherein the first text is sentence S4.

[0029] For another example, the first text paragraph includes sentence S1, sentence S2, sentence S3, sentence S4, and sentence S5. The first text includes sentence S3 and sentence S4.

[0030] In the embodiments of this specification, ordinal numbers such as "first" and "second" are used to distinguish different technical features and have no actual meaning, and will not be further described below.

[0031] Text-to-speech (TTS) is a technology that converts written text into spoken audio. It uses computer algorithms and speech synthesis technology to convert text into natural, fluent speech that simulates the sound of human speech.

[0032] Performing text-to-speech conversion on the first text of the first text paragraph to obtain first voice data of the first text can be regarded as using text-to-speech technology to convert text data into audio data, and the data content remains unchanged before and after the conversion.

[0033] Step S204 : performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph.

[0034] Feature extraction is performed on the first text to obtain a first semantic feature of the first text. The first text may be encoded to obtain a first text vector representing the semantics of the first text, and the first text vector is used as the first semantic feature.

[0035] Feature extraction is performed on the first text to obtain a first semantic feature of the first text. Alternatively, word segmentation processing is performed on the first text to obtain a first word in the first text, and the first word is encoded to obtain a second text vector representing the semantics of the first word, and the second text vector is used as the first semantic feature.

[0036] The first semantic feature of the first text may include only the first text vector, only the second text vector, or both the first text vector and the second text vector.

[0037] Context feature extraction is performed on the first text paragraph to obtain the second semantic feature of the first text paragraph. The first text paragraph can be encoded to obtain a third text vector representing the semantics of the first text paragraph, and the third text vector is used as the second semantic feature of the first text paragraph.

[0038] Contextual features are features used in text analysis to describe the surrounding information of a language unit. These features help us understand the meaning and function of the language unit in a specific context. A language unit can be a word, phrase, sentence, etc.

[0039] Semantic features are meaning-related features extracted from text. They capture the deeper meaning of text and, combined with contextual information, improve the accuracy of natural language processing tasks. Therefore, semantic features can be considered a type of contextual feature.

[0040] Specifically in the embodiment of the present application, since the first text paragraph includes the first text, the semantic feature of the first text paragraph can be regarded as a context feature of the first text.

[0041] Step S206: Determine a first value of the first speech data according to the first semantic feature and the second semantic feature.

[0042] Determining the first value of the first speech data based on the first semantic feature and the second semantic feature can be done by taking the first semantic feature and the second semantic feature as input data of a first encoder, encoding the input data through the first encoder to obtain a corresponding encoding value, and using the encoding value as the first value.

[0043] Alternatively, determining the first value of the first speech data based on the first semantic feature and the second semantic feature may be determining a comprehensive semantic feature based on the first semantic feature and the second semantic feature, and performing mapping processing on the comprehensive semantic feature to obtain the first value.

[0044] Taking the example where the first semantic feature includes the first text vector and the second semantic feature includes the third text vector, the process of determining the comprehensive semantic feature based on the first semantic feature and the second semantic feature is exemplified: A weighted sum is performed based on the first text vector, the third text vector and a pre-configured weight parameter to obtain a comprehensive text vector, which is used as a comprehensive semantic feature.

[0045] Alternatively, the first text vector and the third text vector are concatenated to obtain a comprehensive text vector, and the comprehensive text vector is used as the comprehensive semantic feature.

[0046] Alternatively, the similarity between the first text vector and the third text vector is calculated, and the similarity is used as the comprehensive semantic feature, and so on.

[0047] The above-mentioned mapping process may be linear mapping or nonlinear mapping.

[0048] The first value can be used to adjust the numerical value of the speaking speed of the first voice data. The numerical value of the speaking speed is used to indicate the speed of the speaking. The larger the numerical value of the speaking speed, the faster the speaking speed of the first voice data. Conversely, the smaller the numerical value of the speaking speed, the slower the speaking speed of the first voice data.

[0049] The first value can be used to adjust the volume value of the first voice data. The volume value is used to indicate the volume size. The larger the volume value, the louder the volume of the first voice data. Conversely, the smaller the volume value, the smaller the volume of the first voice data.

[0050] The first value can be used to adjust the intonation type of the first voice data, where the intonation type is used to indicate the rise and fall of the voice. For example, intonation type 1 indicates that the voice is rising, intonation type 2 indicates that the voice is falling, and so on.

[0051] The first value may be used to adjust one or more of the speaking speed, volume, and tone of the first voice data.

[0052] Before step S206 is executed, the numerical value of the speaking rate of the first voice data may be a preconfigured first speaking rate value, the numerical value of the volume of the first voice data may be a preconfigured first volume value, and the intonation type of the first voice data may be a preconfigured intonation type x, which indicates that the sound of the first voice data is smooth and there is no obvious ups and downs.

[0053] Any of the aforementioned speech speed, volume, and intonation type may be used to change the auditory effect of the first voice data.

[0054] In addition, the first value may also be used to adjust the pitch, timbre, rhythm, pause, emphasis, or any other feature of the first voice data for changing the auditory effect of the first voice data.

[0055] Pitch refers to the high and low frequencies of a sound. For example, a high note sounds brisk or tense, while a low note sounds calm or serious. Timbre refers to the texture or color of a sound, determined by overtones and resonance. Rhythm refers to the intensity and duration of speech. For example, a fast tempo sounds lively, while a slow tempo sounds solemn. Pauses indicate brief silences in speech. Stress indicates the emphasis on certain syllables or words, and so on.

[0056] In addition, the intention information of the first text paragraph can also be obtained. If it is determined that the first text paragraph includes a preset intention based on the intention information of the first text paragraph, the first value of the first voice data is determined based on the preset intention, the first semantic feature and the second semantic feature.

[0057] Preset intentions include but are not limited to: repeat conversation intention, volume adjustment intention, speech speed adjustment intention, etc.

[0058] For example, the first paragraph text is a dialogue text, and the first paragraph text includes "You speak too fast, I can't hear clearly, please speak slower", and the intention information S1 of the first paragraph text is obtained. According to S1, it can be determined that the first paragraph text includes the intention to adjust the speaking speed, and then, according to the intention to adjust the speaking speed, the first semantic feature and the second semantic feature, the first value of the first voice data is determined.

[0059] In a specific implementation, the first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of the first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph; based on the first semantic feature and the second semantic feature, determining the first value of the first speech data includes: calculating the similarity between the first text vector and the third text vector to obtain a first similarity; calculating the similarity between the second text vector and the third text vector to obtain a second similarity; calculating the similarity between the first text vector and the second text vector to obtain a third similarity; determining the first value of the first speech data based on the first similarity, the second similarity and the third similarity.

[0060] The first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of the first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph. The acquisition process of the first text vector, the second text vector and the third text vector can refer to the corresponding description part of step S204.

[0061] In the process of calculating the similarity between the first text vector and the third text vector to obtain the first similarity, a preset similarity algorithm may be used.

[0062] For example, the preset similarity algorithm may be one of the following: cosine similarity, Euclidean distance, WMD (Word Mover's Distance), WRD (Word Rotator's Distance), and the like.

[0063] Cosine similarity is a commonly used similarity calculation method, particularly suitable for calculating similarity on text data. It measures the similarity between two vectors by calculating the cosine of the angle between them. The closer the cosine of the angle is to 1, the more similar the two vectors are; the closer the cosine of the angle is to -1, the less similar the two vectors are.

[0064] The calculation formula of cosine similarity can be expressed as: (1) in, (Similarity) representation and The cosine similarity between and are two vectors, " represents the dot product operation of the vector, To represent vectors The model, Used to represent vectors Model.

[0065] Euclidean distance, also known as Euclidean distance, is a method used to measure the distance between two vectors. It calculates the straight-line distance between the two vectors. The smaller the Euclidean distance, the greater the similarity between them.

[0066] The calculation formula of Euclidean distance can be expressed as: (2) in, express and The Euclidean distance between and is an n-dimensional vector, where n is an integer greater than or equal to 1. Represents a vector In the The value of the dimension, Represents a vector In the The value of a dimension.

[0067] The WMD algorithm is an algorithm used to calculate text similarity, particularly in the field of Natural Language Processing (NLP). Based on word embeddings, the WMD algorithm measures the minimum distance required to "move" all the words in one document to all the words in another document. This distance reflects the semantic similarity between the two documents.

[0068] The WRD algorithm is another algorithm for calculating text similarity. It was developed based on the WMD algorithm. Compared with WMD, the WRD algorithm more accurately measures the semantic similarity between texts by considering the perspective of word vectors rather than just the Euclidean distance between word vectors.

[0069] The algorithms listed above are some optional and feasible algorithms used when calculating the similarity between the first text vector and the third text vector. In actual application scenarios, there can also be a variety of different similarity algorithms, which may vary depending on the actual application scenarios. The embodiments of this specification do not make specific limitations on this.

[0070] The following uses the cosine similarity algorithm as an example to illustrate a specific implementation process of calculating the similarity between the first text vector and the third text vector to obtain the first similarity: determining the modulus of the first text vector and the modulus of the third text vector, performing a dot product calculation on the modulus of the first text vector and the modulus of the third text vector to obtain a first calculation result, performing a dot product calculation on the first text vector and the third text vector to obtain a second calculation result, calculating the quotient of the second calculation result and the first calculation result to obtain the cosine similarity between the first text vector and the third text vector, and using the cosine similarity as the first similarity.

[0071] Since the concepts are the same, the similarity between the second text vector and the third text vector is calculated to obtain the second similarity, and the calculation process of the first similarity can be referred to; the similarity between the first text vector and the second text vector is calculated to obtain the third similarity, and the calculation process of the first similarity can be referred to.

[0072] Determining the first value of the first voice data based on the first similarity, the second similarity, and the third similarity can be performed by using the first similarity, the second similarity, and the third similarity as input data of a second encoder, encoding the input data through the second encoder to obtain a corresponding encoding value, and using the encoding value as the first value of the first voice data.

[0073] Alternatively, the first value of the first voice data is determined based on the first similarity, the second similarity, and the third similarity. Alternatively, a comprehensive similarity is determined based on the first similarity, the second similarity, and the third similarity, and the comprehensive similarity is mapped to obtain the first value.

[0074] The above-mentioned mapping process may be linear mapping or nonlinear mapping.

[0075] In addition, the first value of the first voice data may also be determined according to at least one of the first similarity, the second similarity, and the third similarity and a preset similarity threshold.

[0076] The preset similarity threshold may be a custom-configured similarity threshold.

[0077] Next, an example is given of determining the first value of the first voice data according to the first similarity and the preset similarity threshold: The preset similarity threshold includes a first similarity threshold and a second similarity threshold, and the first similarity threshold is smaller than the second similarity threshold.

[0078] If the first similarity is less than or equal to the first similarity threshold, the first preset value is determined as the first value of the first voice data; if the first similarity is greater than the first similarity threshold and less than or equal to the second similarity threshold, the second preset value is determined as the first value of the first voice data; if the first similarity is greater than the second similarity threshold, the third preset value is determined as the first value of the first voice data.

[0079] The first preset value can be used to adjust at least one of the speaking speed, volume and tone of the first voice data, the second preset value can be used to adjust at least one of the speaking speed, volume and tone of the first voice data, and the third preset value can be used to adjust at least one of the speaking speed, volume and tone of the first voice data.

[0080] The comparison result between the first similarity and the preset similarity threshold can be used to determine the first value of the first voice data, as shown in the following example: For example, the robot speaks a long paragraph and the last sentence it says is a summary of the above content. In this case, the semantics of the first text and the first text paragraph are likely to be similar. The first text needs to be emphasized to attract the attention of the person listening to the robot (i.e., the listener) to prevent the listener from missing the important summary information.

[0081] For example, the robot spoke a long paragraph at the beginning, but this paragraph was all about point A. At the end, it said "but" and stated point B, which was opposite to point A. In this case of a transition, the semantics of the first text and the first text paragraph are likely to be very different. The first text needs to be emphasized to attract the listener's attention so that the listener does not miss the turning point.

[0082] In this implementation, by calculating the first similarity, the second similarity and the third similarity, the information describing the relationship between the first text, the first word and the first text paragraph can be fully mined, as well as the changing trend of the text content of the first text paragraph can be perceived. Then, the first value of the first voice data can be determined based on the first similarity, the second similarity and the third similarity. The mined information and the perceived changing trend can be used to determine how to adjust the first voice data, thereby optimizing the auditory effect of the second voice data obtained after adjustment.

[0083] In a specific implementation, the first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of the first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph; based on the first semantic feature and the second semantic feature, determining the first value of the first speech data includes: calculating the similarity between the second text vector and the third text vector to obtain the second similarity; calculating the similarity between the first text vector and the second text vector to obtain the third similarity; determining the first information density of the first word based on the second similarity and / or the third similarity; and determining the first value of the first speech data based on the first information density of the first word.

[0084] The first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of the first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph. The acquisition process of the first text vector, the second text vector and the third text vector can refer to the corresponding description part of step S204.

[0085] When calculating the similarity between the second text vector and the third text vector to obtain the second similarity, a preset similarity algorithm can be used. For details, please refer to the corresponding description above. When calculating the similarity between the first text vector and the second text vector to obtain the third similarity, a preset similarity algorithm can be used. For details, please refer to the corresponding description above.

[0086] A first information density of the first word is determined according to the second similarity.

[0087] Alternatively, the first information density of the first word is determined based on the third similarity.

[0088] Alternatively, the first information density of the first word is determined according to the second similarity and the third similarity.

[0089] Information density refers to the amount of information contained within a given amount of space, time, or data. This concept has different applications and interpretations in different fields, but fundamentally refers to the richness or compactness of information content within a given carrier or medium.

[0090] In text analysis and natural language processing, information density refers to the amount of effective information in a text—that is, the amount of meaningful content conveyed after removing redundant and useless information. Texts with high information density typically contain more substantive content, while texts with low information density may contain a large amount of repetition, filler words, or meaningless sentences.

[0091] Specifically in the embodiment of the present application, the first information density of the first word refers to the content of effective information in the first word.

[0092] Considering that the first text paragraph includes the first text, and the first text includes multiple first words, for any first word in the first text, the higher the similarity between the semantics of the first text paragraph and the semantics of the first word, the more the first word can reflect the central idea of ​​the first text paragraph, and therefore the first information density of the first word is higher; conversely, the lower the similarity between the semantics of the first text paragraph and the semantics of the first word, the more the first word deviates from the central idea of ​​the first text paragraph, and therefore the first information density of the first word is lower.

[0093] Therefore, the higher the value of the second similarity between the third text vector representing the semantics of the first text paragraph and the second text vector representing the semantics of the first word in the first text, the higher the first information density of the first word can be determined; conversely, the lower the value of the second similarity, the lower the first information density of the first word can be determined.

[0094] Based on similar ideas, for any first word in the first text, the higher the similarity between the semantics of the first text and the semantics of the first word, the more the first word can reflect the central idea of ​​the first text, and therefore the first information density of the first word is higher; conversely, the lower the similarity between the semantics of the first text and the semantics of the first word, the more the first word deviates from the central idea of ​​the first text, and therefore the first information density of the first word is lower.

[0095] Therefore, the higher the value of the third similarity between the first text vector representing the semantics of the first text and the second text vector representing the semantics of the first word in the first text, the higher the first information density of the first word; conversely, the lower the value of the third similarity, the lower the first information density of the first word.

[0096] Determining the first information density of the first word based on the second similarity and the third similarity may be determining a third value representing the information density of the first word based on the second similarity, determining a fourth value representing the information density of the first word based on the third similarity, and determining the first information density of the first word based on the third value and the fourth value.

[0097] Determining the first value of the first voice data based on the first information density of the first word can involve encoding the first information density to obtain an encoded value of the first information density, and using the encoded value as the first value of the first voice data.

[0098] The first value of the first speech data is determined based on the first information density of the first word, or the first value of the first speech data is obtained by performing a mapping process on the first information density. The mapping process may be a linear mapping or a nonlinear mapping.

[0099] In this implementation, by determining the first information density of the first word based on the second similarity and / or the third similarity, the importance of different first words can be distinguished when the first text includes multiple first words. Furthermore, by determining the first value of the first word based on the first information density, it can be used to adjust the voice segment corresponding to the first word, so that the second voice data obtained after adjustment sounds like it emphasizes the important words, thereby optimizing its auditory effect.

[0100] In a specific implementation, there are multiple first words; there are multiple second similarities; the first words correspond to the second similarities one-to-one; based on the second similarities, the first information density of the first word is determined, including: determining the natural exponential function value of each second similarity among the multiple second similarities; summing the natural exponential function value of each second similarity to obtain the first function value; determining the first information density of the second word based on the natural exponential function value of the second similarity corresponding to the second word and the first function value; the second word is any one of the multiple first words.

[0101] In the process of determining the first information density of the first word according to the second similarity, a normalization function, a probability distribution function, or any method capable of calculating the similarity ratio in a numerical manner may be used.

[0102] Among the plurality of second similarities, a natural exponential function value of each second similarity is determined.

[0103] For example, the number of first words is n; the number of second similarities between the second text vector and the third text vector is n; n first words correspond to n second similarities. n is an integer greater than 1. , where n represents the number of first words in the first text, Used to indicate the second similarity corresponding to the first word in the first text, It is used to indicate the second similarity corresponding to the second first word in the first text, ..., Used to represent the second similarity corresponding to the nth first word in the first text.

[0104] Among n second similarities, determine the i-th second similarity The natural exponential function value of i is an integer greater than or equal to 1 and less than or equal to n. e is a natural constant. The natural exponential function is an exponential function with the natural constant e as its base.

[0105] The natural exponential function values ​​of each second similarity are summed to obtain the first function value.

[0106] For example, the first function value X1= . Where j is the summation variable of the accumulation function.

[0107] The first information density of the second word is determined according to the natural exponential function value of the second similarity corresponding to the second word and the first function value, where the second word is any one of the multiple first words.

[0108] by Taking the probability distribution function as an example, the calculation process of determining the first information density of the first word according to the second similarity is as follows: (3) Among them, the second word can be the i-th first word, for representing the first information density of the second word, Indicates the second similarity of the i-th The natural exponential function value of That is the first function value X1 mentioned above.

[0109] Alternatively, determining the first information density of the first word based on the second similarity may also include: summing each second similarity to obtain a second function value; determining the first information density of the third word based on the second similarity and the second function value corresponding to the third word; the third word is any one of the multiple first words.

[0110] by For example, the calculation process of determining the first information density of the first word according to the second similarity is as follows: (4) Among them, the third word can be the i-th first word, used to represent the first information density of the third word, Indicates the second similarity of the i-th , can represent the second function value.

[0111] Due to the same concept, the above-mentioned "determining the first information density of the first word based on the third similarity", "determining the third value representing the information density of the first word based on the second similarity", and "determining the fourth value representing the information density of the first word based on the third similarity" can adopt a normalization function, a probability distribution function, or any method that can calculate the similarity ratio in a numerical form. For details, please refer to the corresponding instructions in this implementation method.

[0112] In addition, the first text may include multiple first words, and some of the multiple first words may be stop words. In order to improve data processing efficiency, before determining the first value of the first speech data based on the first semantic feature and the second semantic feature, the second text vector representing the semantics of the stop words can be deleted from the multiple second text vectors corresponding to the first text.

[0113] Stop words are a type of vocabulary that is often filtered out in text processing and NLP. These words are often used frequently in a language, but in most cases do not carry significant meaning or information, and are therefore often considered unimportant when analyzing text content. In text analysis, search engine optimization, information retrieval, and other NLP applications, removing stop words can help reduce data noise and focus analysis on words that contribute substantially to the text content. This improves analysis efficiency and accuracy, especially when processing large amounts of text data.

[0114] Deleting the second text vector representing the semantic meaning of stop words helps improve the computational efficiency of calculating the first information density and the efficiency of generating the first value corresponding to the first information density. This allows, when determining the first value used to adjust the first speech data, to focus on first words that substantially contribute to the text content, thereby improving data processing efficiency.

[0115] Step S208: Adjust at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data.

[0116] The second voice data is voice data obtained after adjusting the first voice data according to the first value.

[0117] At least one of the speech rate, volume, and tone of the first voice data is adjusted according to the first value to obtain second voice data.

[0118] In addition, when the first value can also be used to adjust the pitch, timbre, rhythm, pauses, stress and any other characteristics of the first voice data used to change the auditory effect of the first voice data, the speaking speed, volume, voice pitch, timbre, rhythm, pauses, stress and at least one of any other characteristics of the first voice data used to change the auditory effect of the first voice data can be adjusted according to the first value to obtain the second voice data.

[0119] In a specific implementation, the first voice data is adjusted according to the first value to obtain the second voice data, including: if the first value is used to adjust the speaking speed of the first voice data, the playback speed of the first voice data is increased or decreased from the preset speaking speed value to the target speaking speed value corresponding to the first value; if the first value is used to adjust the volume of the first voice data, the volume of the first voice data is increased or decreased from the preset volume value to the target volume value corresponding to the first value; if the first value is used to adjust the tone of the first voice data, the tone type of the first voice data is updated from the preset tone type to the target tone type corresponding to the first value.

[0120] In the case where the first value is used to adjust the speech rate of the first voice data, the first value may have a corresponding target speech rate value, and the target speech rate value may be regarded as an adjustment target for adjusting the speech rate of the first voice data.

[0121] If the first value is used to adjust the speech speed of the first voice data, the playback speed of the first voice data is increased or decreased from the preset speech speed value to the target speech speed value corresponding to the first value.

[0122] The preset speech speed value may indicate the speed of playing the first voice data before adjustment.

[0123] For example, the playback speed of the first voice data is the preset speech speed value v1 before adjustment, v1 is less than the target speech speed value v2, the playback speed of the first voice data is increased from v1 to v2, and the second voice data is obtained, and the playback speed of the second voice data is v2.

[0124] Alternatively, the playback speed of the first voice data before adjustment is a preset speech speed value v3, v3 is greater than the target speech speed value v2, the playback speed of the first voice data is reduced from v3 to v2, and the second voice data is obtained, and the playback speed of the second voice data is v2.

[0125] In the case where the first value is used to adjust the volume of the first voice data, the first value may have a corresponding target volume value, and the target volume value may be regarded as an adjustment target for adjusting the volume of the first voice data.

[0126] If the first value is used to adjust the volume of the first voice data, the volume of the first voice data is increased or decreased from the preset volume value to the target volume value corresponding to the first value.

[0127] The preset volume value may represent the volume of the first voice data before adjustment.

[0128] For example, the volume of the first voice data is a preset volume value l1 before adjustment, and l1 is less than the target volume value l2. The volume of the first voice data is increased from l1 to l2 to obtain the second voice data, and the volume of the second voice data is l2.

[0129] Alternatively, the volume of the first voice data is a preset volume value l3 before adjustment, l3 is greater than the target volume value l2, the volume of the first voice data is reduced from l3 to l2, and the second voice data is obtained, and the volume of the second voice data is l2.

[0130] In the case where the first value is used to adjust the intonation of the first voice data, the first value may have a corresponding target intonation type, and the target intonation type may be regarded as an adjustment target for adjusting the intonation of the first voice data.

[0131] If the first value is used to adjust the intonation of the first voice data, the intonation type of the first voice data is updated from the preset intonation type to the target intonation type corresponding to the first value.

[0132] The preset intonation type may represent the intonation type of the first voice data before adjustment.

[0133] This implementation method exemplifies a feasible voice adjustment method. In actual applications, voice adjustment methods corresponding to various features can be flexibly configured in combination with specific scenarios.

[0134] In a specific implementation method, the first voice data is adjusted according to the first value to obtain the second voice data, including: in the first voice data, determining the first voice segment and the second voice segment of the first word corresponding to the first value, the second voice segment being a voice segment in the first voice data that does not include the first voice segment; adjusting the first voice segment according to the first value; and generating the second voice data based on the second voice segment and the adjusted first voice segment.

[0135] The first value of the first voice data can be used to adjust the first voice data, and the first value of the first voice data can also be used to adjust a voice segment in the first voice data.

[0136] In the first speech data, a first speech segment and a second speech segment of a first word corresponding to a first value are determined.

[0137] The first voice data is voice data obtained by performing voice-to-text conversion on a first text. The first text includes a plurality of first words, wherein each first word has a corresponding first voice segment in the first voice data.

[0138] For any first word, a speech segment other than the first speech segment of the first word in the first speech data can be used as the second speech segment of the first word.

[0139] For example, if the first text is "ABCDEFG" and the first word is "CD", in the first voice data, the voice segment corresponding to "CD" can be regarded as the first voice segment of the first word "CD", the voice segment corresponding to "AB" can be regarded as a second voice segment of the first word "CD", and the voice segment corresponding to "EFG" can be regarded as another second voice segment of the first word "CD".

[0140] Adjusting the first voice segment according to the first value may be adjusting at least one of the speaking speed, volume, and intonation of the first voice segment according to the first value. Specifically, reference may be made to the corresponding description section above of adjusting the first voice data according to the first value to obtain the second voice data.

[0141] In this implementation, by adjusting the voice segments in the first voice data, it is possible to emphasize individual words, so that the listener's attention is more easily attracted to the important first words, thereby optimizing the auditory effect of the first voice data.

[0142] Step S210: training a model based on the second voice data and the third voice data of the first text.

[0143] The model to be trained can be used to convert text data input into corresponding speech data.

[0144] The third voice data of the first text can be pre-configured voice data with changes in speaking speed, volume, and intonation. The closer the second voice data generated by the model is to the third voice data, the better the voice generation effect of the model is, and the better it can convert the text input into the model into voice data with changes in speaking speed, volume, and intonation.

[0145] During the process of training the model based on the second speech data and the third speech data of the first text, a model training loss may be generated based on the second speech data and the third speech data of the first text, and the model training loss is used to adjust model parameters. The model parameters may include: trainable parameters of the first encoder, trainable parameters of the second encoder, etc.

[0146] For example, the trainable parameters of the first encoder may include a weight matrix of the first encoder, a bias term of the first encoder, and the like. The weight matrix refers to the connection weights of each layer in the first encoder, used to map input data to output. The bias term refers to an additional parameter of each layer of the first encoder, used to adjust the offset of the output.

[0147] For example, by continuously adjusting the model parameters during the model training process, the first encoder can fully learn how to encode two input data, the first semantic feature and the second semantic feature, into a first value.

[0148] For another example, by continuously adjusting the model parameters during the model training process, the second encoder can fully learn how to encode the three input data of the first similarity, the second similarity, and the third similarity into the first value.

[0149] In the process of generating the model training loss, the preset reference speech data acts similarly to a sample label. Specifically, generating the model training loss based on the second speech data and the third speech data of the first text can be performed by calculating the model training loss based on the second speech data, the third speech data of the first text, and a preset loss function.

[0150] The loss function is a key concept in machine learning and statistics. It measures the difference between a model's predicted values ​​and the true values. The goal of the loss function is to optimize model parameters by minimizing this difference. Different tasks and models may use different loss functions.

[0151] Exemplarily, the preset loss function may be a cross entropy function, a mean square error, or the like.

[0152] The above-mentioned model can perform the following steps in each training process: perform text-to-speech conversion on the first text of the first text paragraph to obtain the first voice data of the first text; perform feature extraction on the first text to obtain the first semantic feature of the first text, and perform context feature extraction on the first text paragraph to obtain the second semantic feature of the first text paragraph; determine the first value of the first voice data based on the first semantic feature and the second semantic feature, and the first value is used to adjust at least one of the speaking speed, volume and tone of the first voice data; adjust the first voice data according to the first value to obtain the second voice data; generate a model training loss based on the second voice data and the third voice data of the first text, and adjust the model parameters according to the model training loss.

[0153] When the number of training times reaches a preset threshold, or when the training stop condition is determined to be met based on the model training loss, the model training can be terminated.

[0154] The training stopping condition can be that the model training loss is less than the preset loss threshold, or that the model training loss tends to be stable, etc.

[0155] In addition, it should be emphasized that, during the model training process, each training sample includes a first text and a first text paragraph including the first text, rather than just a first text.

[0156] In a specific implementation method, a model is trained based on the second voice data and the third voice data of the first text, including: comparing the first phoneme in the second voice data with the second phoneme in the third voice data to obtain a comparison result of the first phoneme and the second phoneme; determining the speech generation accuracy of the first text based on the comparison result of the first phoneme and the second phoneme; and training the model based on the speech generation accuracy.

[0157] Phoneme is a basic concept in linguistics, which refers to the smallest phonetic unit in human language that can distinguish meaning.

[0158] Specifically in the embodiment of this specification, the second voice data may include one or more first phonemes, and the third voice data may include one or more second phonemes.

[0159] In the process of comparing the first phoneme in the second speech data with the second phoneme in the third speech data to obtain a comparison result of the first phoneme and the second phoneme: comparing the first first phoneme in the second speech data with the first second phoneme in the third speech data to obtain a comparison result of the first first phoneme and the first second phoneme; comparing the second first phoneme in the second speech data with the second second phoneme in the third speech data to obtain a comparison result of the second first phoneme and the second second phoneme; ...; comparing the i-th first phoneme in the second speech data with the i-th second phoneme in the third speech data to obtain a comparison result of the i-th first phoneme and the i-th second phoneme; ... i is an integer greater than 1.

[0160] The comparison result may indicate that the first phoneme and the second phoneme are the same, or may indicate that the first phoneme and the second phoneme are different.

[0161] The speech generation accuracy of the first text can be used to represent the accuracy of the second speech data based on the third speech data.

[0162] For example, the second voice data includes 100 first phonemes, and the third voice data includes 100 second phonemes, among which 90 first phonemes are the same as 90 second phonemes, and the remaining 10 first phonemes are different from the remaining 10 second phonemes. It can be determined that the voice generation accuracy of the first text is 90%.

[0163] The speech generation accuracy of the first text is determined based on the comparison results of the first phoneme and the second phoneme. This can be done by counting the number of comparison results to obtain a first number, counting the number of first comparison results indicating that the first phoneme is identical to the second phoneme among the first number of comparison results to obtain a second number, and finding the quotient of the second number and the first number to obtain the speech generation accuracy of the first text.

[0164] The model is trained based on the speech generation accuracy.

[0165] The speech generation accuracy is used to generate the model training loss. A higher speech generation accuracy indicates better model training results. When the accuracy exceeds the preset accuracy threshold, model training is terminated. A lower speech generation accuracy indicates that the model's current speech generation performance is not meeting expectations and requires further adjustment of model parameters.

[0166] In this implementation, by comparing phonemes, the third voice data can be used to guide the model to better learn how to convert the input text data into voice data that is more in line with the rules of natural language communication, thereby optimizing the voice generation effect of the model.

[0167] In such Figure 2 In the embodiment shown, first, text-to-speech is performed on the first text of the first text paragraph to obtain first voice data of the first text; second, feature extraction is performed on the first text to obtain a first semantic feature of the first text, and context feature extraction is performed on the first text paragraph to obtain a second semantic feature of the first text paragraph; then, a first value of the first voice data is determined based on the first semantic feature and the second semantic feature; then, at least one of the speaking speed, volume and intonation of the first voice data is adjusted based on the first value to obtain second voice data; finally, a model is trained based on the second voice data and the third voice data of the first text. In this way, by determining the first value of the first voice data according to the first semantic feature and the second semantic feature and adjusting the first voice data of the first text according to the first value to obtain the second voice data, the semantic features of the first text itself and the context features of the first text paragraph including the first text can be referred to at the same time during the generation of the second voice data. This is conducive to perceiving the changing trend of the text content of the first text paragraph. Therefore, at least one of the speech speed, volume and intonation of the second voice data obtained by adjusting the first voice data according to the first value is more in line with the changing trend. Furthermore, through model training, the model can fully learn how to make at least one of the speech speed, volume and intonation of the generated second voice data match the changing trend of the text content of the first text paragraph as much as possible, thereby optimizing the voice generation effect of the model.

[0168] Figure 3 A processing flow chart of another model training method provided in an embodiment of the present application.

[0169] like Figure 3 As shown, in step S302, target sample data is obtained; the target sample data includes: a first text, a first text paragraph, and third voice data of the first text.

[0170] The first text paragraph includes the first text. The first text, the first text paragraph, and the third voice data of the first text can refer to the corresponding description in the aforementioned method embodiment.

[0171] Step S304: input the target sample data into the speech generation model to be trained to perform speech generation, and obtain second speech data of the target sample data.

[0172] Taking a target sample data as an example, in the process of speech generation, the following steps are performed: performing text-to-speech conversion on the first text to obtain the first speech data of the first text; performing feature extraction on the first text to obtain the first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain the second semantic feature of the first text paragraph; determining the first value of the first speech data based on the first semantic feature and the second semantic feature, the first value is used to adjust at least one of the speaking speed, volume and intonation of the first speech data; adjusting the first speech data according to the first value to obtain the second speech data.

[0173] For details, please refer to the corresponding description of steps S202 to S208 of the aforementioned method embodiment.

[0174] Step S306: Perform model training based on the second voice data and the third voice data to obtain a trained speech generation model.

[0175] This step may refer to the corresponding description of step S210 in the aforementioned method embodiment.

[0176] Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above.

[0177] Figure 4 A data flow diagram of another model training method provided in an embodiment of the present application.

[0178] like Figure 4 As shown, the speech generation model 402 to be trained includes: a feature extraction module 404, a phoneme embedding module 408, an audio encoder 410, a volume adjuster 412 and an audio decoder 414 connected in series.

[0179] The speech generation model 402 to be trained further includes a speech rate / volume / intonation encoder 406. The feature extraction module 404 is connected to the speech rate / volume / intonation encoder 406, and the data output by the feature extraction module 404 can be transmitted to the speech rate / volume / intonation encoder 406; the speech rate / volume / intonation encoder 406 is connected to the audio encoder 410, and the data output by the speech rate / volume / intonation encoder 406 can be transmitted to the audio encoder 410; and the speech rate / volume / intonation encoder 406 is connected to the volume adjuster, and the data output by the speech rate / volume / intonation encoder 406 can be transmitted to the volume adjuster 412.

[0180] The target sample data is input into the speech generation model 402 to be trained for speech generation to obtain the second speech data 416 of the target sample data. The corresponding description of step S304 of the aforementioned method embodiment can be referred to.

[0181] The feature extraction module 404 may be configured to perform feature extraction on the first text to obtain a first semantic feature of the first text, and to perform context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph.

[0182] Additionally, feature extraction module 404 may include a dimensionality conversion submodule for performing dimensionality conversion on the first semantic feature and the second speech feature. This dimensionality conversion allows the dimension of the output data of feature extraction module 404 to be aligned with the dimension of the input data of phoneme embedding module 408. Furthermore, the dimension of the output data of feature extraction module 404 can be aligned with the dimension of the input data of speech rate / volume / intonation encoder 406. This avoids data recognition anomalies caused by dimensionality inconsistencies.

[0183] The speech rate / volume / intonation encoder 406 may be configured to determine a first value of the first speech data according to the first semantic feature and the second semantic feature output by the feature extraction module 404 .

[0184] The phoneme embedding module 408 can be used to embed pronunciation phonemes according to the information of the first text to obtain an original phoneme file. In other words, the phoneme embedding module 408 can be used to perform text-to-speech conversion on the first text to obtain first speech data of the first text.

[0185] The audio encoder 410 can be used to adjust the speaking speed of the first voice data according to the first value of the first voice data output by the speaking speed / volume / intonation encoder 406; it can also adjust the volume of the first voice data according to the first value of the first voice data output by the speaking speed / volume / intonation encoder 406; it can also control and adjust the tone of the first voice data according to the first value of the first voice data output by the speaking speed / volume / intonation encoder 406.

[0186] The volume adjuster 412 may be configured to adjust the volume of the first voice data according to a first value.

[0187] It should be noted that the reason why both the audio encoder 410 and the volume adjuster 412 can be used to adjust the volume of the first voice data according to the first value is that although the ultimate goal of adjusting the volume through the audio encoder 410 and adjusting the volume through the volume adjuster 412 is to change the volume of the first voice data, there are obvious differences between the two.

[0188] The audio encoder 410's primary task is to compress audio signals into a specific format, but it can also adjust the volume during the encoding process. For example, it can scale the amplitude of the audio signal during encoding. The volume leveler 412 is a real-time processing tool that dynamically adjusts the volume during playback or transmission, offering greater flexibility.

[0189] The audio decoder 414 may be configured to decode the encoded information output by the volume adjuster 412 to obtain second voice data.

[0190] After obtaining the second speech data 416, if the training goal is achieved, the model training is terminated and the trained speech generation model 418 is obtained; if the training goal is not achieved, the next round of training is continued, and the corresponding description part of step S210 of the aforementioned method embodiment can be referred to.

[0191] Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above.

[0192] Figure 5 A schematic diagram of the data flow of a dimensionality conversion module in a speech generation model provided in an embodiment of the present application. Figure 5 The dimension conversion module 502 in the embodiment may have the same structure as the dimension conversion submodule in the aforementioned feature extraction module 404 .

[0193] like Figure 5 As shown, the dimension conversion module 502 performs dimension conversion on the first semantic feature and the second semantic feature according to the first text information dimension 504, the control instruction encoding dimension 506, the second text information dimension 508 and the control instruction encoding dimension 510, and the first semantic feature after dimension conversion and the second semantic feature after dimension conversion conform to the speech generation model feature dimension 512.

[0194] The first text information dimension 504 is used to represent the dimension information of the first semantic feature before dimension conversion, the control instruction encoding dimension 506 is used to represent a dimension information pre-configured for the first value, the second text information dimension 508 is used to represent the dimension information of the second semantic feature before dimension conversion, and the control instruction encoding dimension 510 is used to represent another dimension information pre-configured for the first value.

[0195] The speech generation model feature dimension 512 may represent feature dimension information after dimension conversion.

[0196] Through dimensionality conversion, the dimension of the output data of the dimensionality conversion module 502 can be made the same as the dimension of the input data of other modules connected to the dimensionality conversion module 502 in the speech generation model, thereby avoiding data recognition anomalies caused by dimensional inconsistency.

[0197] Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above.

[0198] Figure 6A A schematic diagram of the structure of a speech generation model provided in an embodiment of the present application.

[0199] like Figure 6A As shown, the speech generation model 602 to be trained includes: a first text feature extraction module 604 , a second text feature extraction module 606 , a control information determination module 608 , a speech rate / volume / intonation encoder 610 and an audio generation module 602 .

[0200] The first text feature extraction module 604 is used to extract features from the first text to obtain a first semantic feature of the first text, and the second text feature extraction module 606 is used to extract context features from the first text paragraph to obtain a second semantic feature of the first text paragraph. For details, please refer to the corresponding description of step S204 of the aforementioned method embodiment.

[0201] The first text feature extraction module 604 may be an encoder, and the second text feature extraction module 606 may be another encoder with the same structure as the first text feature extraction module 604 , or another encoder with a different structure from the first text feature extraction module 604 .

[0202] For example, the data processing flow of the control information determination module 608 can refer to the following Figure 6B .

[0203] Figure 6B A schematic diagram of the data flow of a control information determination module in a speech generation model provided in an embodiment of the present application.

[0204] like Figure 6B As shown, the first text vector of the first text and the second text vector 614 of the first word in the first text may include one first text vector and multiple second text vectors.

[0205] The similarity between the first text vector and the third text vector is calculated to obtain a first similarity 618 ; the similarity between the second text vector and the third text vector is calculated to obtain a second similarity 620 ; and the similarity between the first text vector and the second text vector is calculated to obtain a third similarity 622 .

[0206] After obtaining the first similarity 618, the second similarity 620 and the third similarity 622, speech rate control information 624 can be generated based on the first similarity 618, the second similarity 620 and the third similarity 622. Volume control information 626 can also be generated based on the first similarity 618, the second similarity 620 and the third similarity 622; and intonation control information 628 can also be generated based on the first similarity 618, the second similarity 620 and the third similarity 622.

[0207] The above-mentioned speech rate control information 624, volume control information 626 and intonation control information 628 can refer to the corresponding description of the first value of the first voice data in the above-mentioned method embodiment.

[0208] The speech rate / volume / intonation encoder 610 may be configured to generate corresponding speech rate control instructions based on the speech rate control information 624 , generate corresponding volume control instructions based on the volume control information 626 , and generate corresponding intonation control instructions based on the intonation control information 628 .

[0209] It should be noted that in the above Figure 4 In the illustrated embodiment, the speech rate / volume / intonation encoder 406 is configured to determine a first value of the first speech data based on the first semantic feature and the second semantic feature output by the feature extraction module 404 .

[0210] In this embodiment, the control information determination module 608 is used to determine the first value of the first voice data based on the first semantic feature and the second semantic feature, and the speech rate / volume / intonation encoder 610 is used to encode the first value of the first voice data into a control instruction recognizable by the audio generation module 612, such as a speech rate control instruction, a volume control instruction, and an intonation control instruction. That is, the speech rate / volume / intonation encoder 610 and the speech rate / volume / intonation encoder 610 in this embodiment are Figure 4 The functions of the speech rate / volume / intonation encoder 406 in the embodiments are different.

[0211] The audio generation module 612 is configured to adjust the first voice data to obtain the second voice data. In the process of generating the second voice data, the audio generation module 612 may adjust the first voice data according to at least one of a speech rate control instruction, a volume control instruction, and a tone control instruction.

[0212] Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above.

[0213] Based on the same technical concept, an embodiment of the present application also provides a speech generation method. Figure 7 A processing flow chart of a speech generation method provided in an embodiment of the present application.

[0214] like Figure 7 As shown, in step S702, text-to-speech is performed on the second text of the second text paragraph to obtain fourth voice data of the second text.

[0215] The second text paragraph can refer to the corresponding description part of the first text paragraph. The second text can refer to the corresponding description part of the first text. The fourth voice data can refer to the corresponding description part of the first voice data.

[0216] For this step, reference may be made to the corresponding description of the aforementioned step S202 .

[0217] Step S704 : performing feature extraction on the second text to obtain a third semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a fourth semantic feature of the second text paragraph.

[0218] The third semantic feature of the second text can refer to the corresponding description of the first semantic feature of the first text. The fourth semantic feature of the second text paragraph can refer to the corresponding description of the second semantic feature of the second text.

[0219] For this step, reference may be made to the corresponding description of the aforementioned step S204.

[0220] Step S706: Determine a second value of the fourth speech data according to the third semantic feature and the fourth semantic feature.

[0221] For the second value, please refer to the corresponding description of the first value.

[0222] For this step, reference may be made to the corresponding description of the aforementioned step S206 .

[0223] Step S708: Adjust at least one of the speech rate, volume, and intonation of the fourth voice data according to the second value to obtain fifth voice data.

[0224] The fifth voice data may refer to the corresponding description of the second voice data.

[0225] For this step, reference may be made to the corresponding description of the aforementioned step S208.

[0226] In addition, the second text and the second text paragraph can also be input into the target model for speech generation processing to obtain fifth speech data. The target model can be trained by the model training method provided by the aforementioned method embodiment.

[0227] Alternatively, you can first obtain the second text paragraph, take the last sentence in the second text paragraph as the second text, input the second text paragraph into the target model for speech generation processing, and obtain the fifth speech data. The target model can be trained by the model training method provided in the aforementioned method embodiment.

[0228] Since the technical concept is the same, the description in this embodiment is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above.

[0229] An embodiment of a model training device provided in this specification is as follows: In the above-mentioned embodiment, a model training method is provided. Based on the same technical concept, the embodiment of the present application also provides a model training device, which is described below with reference to the accompanying drawings.

[0230] Figure 8 Schematic diagram of a model training device provided in an embodiment of the present application. This embodiment provides a model training device 800, including: A first conversion unit 802 is configured to perform text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; A first extraction unit 804 is configured to perform feature extraction on the first text to obtain a first semantic feature of the first text, and to perform context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; A first determining unit 806 is configured to determine a first value of the first speech data according to the first semantic feature and the second semantic feature; a first adjusting unit 808, configured to adjust at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; The training unit 810 is configured to train a model based on the second voice data and the third voice data of the first text.

[0231] Optionally, the first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of a first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph; and when the first determining unit 806 determines the first value of the first speech data based on the first semantic feature and the second semantic feature, it performs the following steps: Calculating the similarity between the first text vector and the third text vector to obtain a first similarity; Calculating the similarity between the second text vector and the third text vector to obtain a second similarity; Calculating the similarity between the first text vector and the second text vector to obtain a third similarity; A first value of the first voice data is determined according to the first similarity, the second similarity, and the third similarity.

[0232] Optionally, the first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of a first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph; and when the first determining unit 806 determines the first value of the first speech data based on the first semantic feature and the second semantic feature, it performs the following steps: Calculating the similarity between the second text vector and the third text vector to obtain a second similarity; Calculating the similarity between the first text vector and the second text vector to obtain a third similarity; determining a first information density of the first word according to the second similarity and / or the third similarity; A first value of the first speech data is determined according to a first information density of the first word.

[0233] Optionally, when the first adjusting unit 808 adjusts the first voice data according to the first value to obtain the second voice data, the first adjusting unit 808 performs the following steps: If the first value is used to adjust the speech speed of the first voice data, the playback speed of the first voice data is increased or decreased from the preset speech speed value to the target speech speed value corresponding to the first value; If the first value is used to adjust the volume of the first voice data, increasing or decreasing the volume of the first voice data from a preset volume value to a target volume value corresponding to the first value; If the first value is used to adjust the intonation of the first voice data, the intonation type of the first voice data is updated from a preset intonation type to a target intonation type corresponding to the first value. Optionally, when the first adjusting unit 808 adjusts the first voice data according to the first value to obtain the second voice data, the first adjusting unit 808 performs the following steps: Determining, in the first voice data, a first voice segment and a second voice segment of the first word corresponding to the first value, where the second voice segment is a voice segment in the first voice data that does not include the first voice segment; adjusting the first speech segment according to the first value; The second voice data is generated according to the second voice segment and the adjusted first voice segment.

[0234] Optionally, there are multiple first words; there are multiple second similarities; the first words correspond to the second similarities one-to-one; and when the first determining unit 806 determines the first information density of the first word based on the second similarity, it performs the following steps: determining a natural exponential function value of each second similarity among the plurality of second similarities; Sum the natural exponential function values ​​of each second similarity to obtain the first function value; The first information density of the second word is determined according to the natural exponential function value of the second similarity corresponding to the second word and the first function value; the second word is any one of the multiple first words.

[0235] Optionally, when training a model based on the second voice data and the third voice data of the first text, the training unit 810 performs the following steps: comparing a first phoneme in the second voice data with a second phoneme in the third voice data to obtain a comparison result between the first phoneme and the second phoneme; Determining a speech generation accuracy rate of the first text based on a comparison result of the first phoneme and the second phoneme; The model is trained according to the speech generation accuracy.

[0236] In an embodiment of the present application, the model training device includes: a first conversion unit, used to perform text-to-speech conversion on the first text of the first text paragraph to obtain first voice data of the first text; a first extraction unit, used to perform feature extraction on the first text to obtain a first semantic feature of the first text, and to perform context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; a first determination unit, used to determine a first value of the first voice data based on the first semantic feature and the second semantic feature; a first adjustment unit, used to adjust at least one of the speaking speed, volume and intonation of the first voice data according to the first value to obtain second voice data; and a training unit, used to train a model based on the second voice data and the third voice data of the first text. In this way, by determining the first value of the first voice data according to the first semantic feature and the second semantic feature and adjusting the first voice data of the first text according to the first value to obtain the second voice data, the semantic features of the first text itself and the context features of the first text paragraph including the first text can be referred to at the same time during the generation of the second voice data. This is conducive to perceiving the changing trend of the text content of the first text paragraph. Therefore, at least one of the speech speed, volume and intonation of the second voice data obtained by adjusting the first voice data according to the first value is more in line with the changing trend. Furthermore, through model training, the model can fully learn how to make at least one of the speech speed, volume and intonation of the generated second voice data match the changing trend of the text content of the first text paragraph as much as possible, thereby optimizing the voice generation effect of the model.

[0237] An embodiment of a speech generation device provided in this specification is as follows: In the above embodiment, a speech generation method is provided. Based on the same technical concept, the embodiment of the present application also provides a speech generation device, which is described below with reference to the accompanying drawings.

[0238] Figure 9 Schematic diagram of a speech generation device provided in an embodiment of the present application. This embodiment provides a speech generation device 900, including: A second conversion unit 902 is configured to perform text-to-speech conversion on a second text of a second text paragraph to obtain fourth speech data of the second text; A second extraction unit 904 is configured to perform feature extraction on the second text to obtain a third semantic feature of the second text, and to perform context feature extraction on the second text paragraph to obtain a fourth semantic feature of the second text paragraph; A second determining unit 906 is configured to determine a second value of the fourth speech data according to the third semantic feature and the fourth semantic feature; The second adjusting unit 908 is configured to adjust at least one of the speech rate, volume, and intonation of the fourth voice data according to the second value to obtain fifth voice data.

[0239] The speech generation device provided in this embodiment includes: a second conversion unit for performing text-to-speech conversion on a second text of a second text paragraph to obtain fourth speech data of the second text; a second extraction unit for performing feature extraction on the second text to obtain a third semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a fourth semantic feature of the second text paragraph; a second determination unit for determining a second value of the fourth speech data based on the third semantic feature and the fourth semantic feature; and a second adjustment unit for adjusting at least one of the speech rate, volume, and intonation of the fourth speech data based on the second value to obtain fifth speech data. In this way, by determining the second value of the fourth speech data based on the third semantic feature and the fourth semantic feature and adjusting the fourth speech data of the second text based on the second value to obtain the fifth speech data, the semantic features of the second text itself and the context features of the second text paragraph including the second text can be simultaneously referenced during the generation of the fifth speech data. This facilitates the perception of the changing trend of the text content of the second text paragraph. Therefore, at least one of the speech rate, volume, and intonation of the fifth speech data obtained by adjusting the fourth speech data based on the second value better matches the changing trend, thereby optimizing the speech generation effect.

[0240] Corresponding to the model training method or speech generation method described above, based on the same technical concept, an embodiment of the present application also provides an electronic device, which is used to execute the model training method or speech generation method provided above. Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0241] like Figure 10 As shown, electronic devices can vary significantly due to different configurations or performance. They may include one or more processors 1001 and memory 1002. Memory 1002 may store one or more applications or data. Memory 1002 may be either ephemeral or persistent. Applications stored in memory 1002 may include one or more modules (not shown), each of which may include a series of computer-executable instructions within the electronic device. Furthermore, processor 1001 may be configured to communicate with memory 1002 to execute the series of computer-executable instructions within memory 1002 on the electronic device. The electronic device may also include one or more power supplies 1003, one or more wired or wireless network interfaces 1004, one or more input / output interfaces 1005, one or more keyboards 1006, and the like.

[0242] In a specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following: Performing text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; Performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; determining a first value of the first speech data according to the first semantic feature and the second semantic feature; adjusting at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; A model is trained based on the second voice data and the third voice data of the first text.

[0243] In another specific embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, including computer-executable instructions for performing the following: Performing text-to-speech conversion on the second text of the second text paragraph to obtain fourth voice data of the second text; performing feature extraction on the second text to obtain a second semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a third semantic feature of the second text paragraph; determining a second value of the fourth speech data according to the second semantic feature and the third semantic feature; At least one of the speech rate, volume, and tone of the fourth voice data is adjusted according to the second value to obtain fifth voice data.

[0244] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process can be implemented: Performing text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; Performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; determining a first value of the first speech data according to the first semantic feature and the second semantic feature; adjusting at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; A model is trained based on the second voice data and the third voice data of the first text.

[0245] It should be noted that the above-mentioned embodiment of the computer-readable storage medium in this specification and the embodiment of the model training method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method mentioned above, and the repeated parts will not be repeated.

[0246] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions. When the computer-executable instructions are executed by a processor, the following process can be implemented: Performing text-to-speech conversion on the second text of the second text paragraph to obtain fourth voice data of the second text; performing feature extraction on the second text to obtain a second semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a third semantic feature of the second text paragraph; determining a second value of the fourth speech data according to the second semantic feature and the third semantic feature; At least one of the speech rate, volume, and tone of the fourth voice data is adjusted according to the second value to obtain fifth voice data.

[0247] It should be noted that the above-mentioned embodiment of the computer-readable storage medium in this specification and the embodiment of the speech generation method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method mentioned above, and the repeated parts will not be repeated.

[0248] Another embodiment of the present disclosure further provides a computer program product, the computer program product including a computer program, which can implement the following process when executed by a processor: Performing text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; Performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; determining a first value of the first speech data according to the first semantic feature and the second semantic feature; adjusting at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; A model is trained based on the second voice data and the third voice data of the first text.

[0249] The computer program product in the embodiment of the present disclosure can implement each process of the above-mentioned model training method embodiment and achieve the same effects and functions, which will not be repeated here.

[0250] The computer program product includes a computer program, and when the computer program is executed by a processor, the following process can be implemented: Performing text-to-speech conversion on the second text of the second text paragraph to obtain fourth voice data of the second text; performing feature extraction on the second text to obtain a second semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a third semantic feature of the second text paragraph; determining a second value of the fourth speech data according to the second semantic feature and the third semantic feature; At least one of the speech rate, volume, and tone of the fourth voice data is adjusted according to the second value to obtain fifth voice data.

[0251] The computer program product in the embodiment of the present disclosure can implement each process of the above-mentioned speech generation method embodiment and achieve the same effects and functions, which will not be repeated here.

[0252] In an embodiment of the present application, first, text-to-speech is performed on the first text of the first text paragraph to obtain first voice data of the first text; second, feature extraction is performed on the first text to obtain a first semantic feature of the first text, and context feature extraction is performed on the first text paragraph to obtain a second semantic feature of the first text paragraph; then, a first value of the first voice data is determined based on the first semantic feature and the second semantic feature; then, at least one of the speaking speed, volume and intonation of the first voice data is adjusted based on the first value to obtain second voice data; finally, a model is trained based on the second voice data and the third voice data of the first text. In this way, by determining the first value of the first voice data according to the first semantic feature and the second semantic feature and adjusting the first voice data of the first text according to the first value to obtain the second voice data, the semantic features of the first text itself and the context features of the first text paragraph including the first text can be referred to at the same time during the generation of the second voice data. This is conducive to perceiving the changing trend of the text content of the first text paragraph. Therefore, at least one of the speech speed, volume and intonation of the second voice data obtained by adjusting the first voice data according to the first value is more in line with the changing trend. Furthermore, through model training, the model can fully learn how to make at least one of the speech speed, volume and intonation of the generated second voice data match the changing trend of the text content of the first text paragraph as much as possible, thereby optimizing the voice generation effect of the model.

[0253] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0254] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0255] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable electronic device to produce a machine, so that the instructions executed by the processor of the computer or other programmable electronic device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0256] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable electronic device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0257] These computer program instructions can also be loaded onto a computer or other programmable electronic device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0258] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0259] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0260] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0261] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0262] The embodiments of the present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0263] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0264] The foregoing description is merely an example of the present invention and is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims herein.

Claims

1. A model training method, characterized in that: include: Performing text-to-speech conversion on a first text of a first text paragraph to obtain first speech data of the first text; Performing feature extraction on the first text to obtain a first semantic feature of the first text, and performing context feature extraction on the first text paragraph to obtain a second semantic feature of the first text paragraph; determining a first value of the first speech data according to the first semantic feature and the second semantic feature; adjusting at least one of the speech rate, volume, and intonation of the first voice data according to the first value to obtain second voice data; A model is trained based on the second voice data and the third voice data of the first text.

2. The method according to claim 1, characterized in that The first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of a first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph; and determining the first value of the first speech data based on the first semantic feature and the second semantic feature includes: Calculating the similarity between the first text vector and the third text vector to obtain a first similarity; Calculating the similarity between the second text vector and the third text vector to obtain a second similarity; Calculating the similarity between the first text vector and the second text vector to obtain a third similarity; A first value of the first voice data is determined according to the first similarity, the second similarity, and the third similarity.

3. The method according to claim 1, characterized in that The first semantic feature includes a first text vector representing the semantics of the first text and a second text vector representing the semantics of a first word in the first text; the second semantic feature includes a third text vector representing the semantics of the first text paragraph; and determining the first value of the first speech data based on the first semantic feature and the second semantic feature includes: Calculating the similarity between the second text vector and the third text vector to obtain a second similarity; Calculating the similarity between the first text vector and the second text vector to obtain a third similarity; determining a first information density of the first word according to the second similarity and / or the third similarity; A first value of the first speech data is determined according to a first information density of the first word.

4. The method according to claim 3, characterized in that The adjusting the first voice data according to the first value to obtain second voice data includes: Determining, in the first voice data, a first voice segment and a second voice segment of the first word corresponding to the first value, where the second voice segment is a voice segment in the first voice data that does not include the first voice segment; adjusting the first speech segment according to the first value; The second voice data is generated according to the second voice segment and the adjusted first voice segment.

5. The method according to claim 3, characterized in that There are multiple first words; there are multiple second similarities; and there is a one-to-one correspondence between the first words and the second similarities; Determining a first information density of the first word according to the second similarity includes: determining a natural exponential function value of each second similarity among the plurality of second similarities; Sum the natural exponential function values ​​of each second similarity to obtain the first function value; The first information density of the second word is determined according to the natural exponential function value of the second similarity corresponding to the second word and the first function value; the second word is any one of the multiple first words.

6. The method according to claim 1, characterized in that The training model based on the second voice data and the third voice data of the first text includes: comparing a first phoneme in the second voice data with a second phoneme in the third voice data to obtain a comparison result between the first phoneme and the second phoneme; Determining a speech generation accuracy rate of the first text based on a comparison result of the first phoneme and the second phoneme; The model is trained according to the speech generation accuracy.

7. A speech generation method, characterized in that: include: Performing text-to-speech conversion on the second text of the second text paragraph to obtain fourth voice data of the second text; performing feature extraction on the second text to obtain a third semantic feature of the second text, and performing context feature extraction on the second text paragraph to obtain a fourth semantic feature of the second text paragraph; determining a second value of the fourth speech data according to the third semantic feature and the fourth semantic feature; At least one of the speech rate, volume, and tone of the fourth voice data is adjusted according to the second value to obtain fifth voice data.

8. An electronic device, characterized in that: include: processor; And, a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the model training method as described in any one of claims 1 to 6, or the speech generation method as described in claim 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store computer-executable instructions, which, when executed by a processor, implement the model training method according to any one of claims 1 to 6, or the speech generation method according to claim 7.

10. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements the model training method as described in any one of claims 1 to 6, or the speech generation method as described in claim 7.

Images