Emotional audio generation method, device, equipment and storage medium

Abstract

The present invention relates to the field of audio generation, and discloses an audio generation method, device, equipment and storage medium with emotion. The method comprises: receiving a plurality of speech data with emotion tags, generating an emotion speech set; receiving a text string to be converted, performing sentence processing on the text string, obtaining N short sentence characters and short sentence sorting; performing emotion analysis processing on the short sentence characters according to a preset recognition algorithm, obtaining the emotion tags corresponding to the short sentence characters; performing label matching processing on the emotion speech set based on the emotion tags corresponding to the short sentence characters, obtaining the emotion speech matched with the emotion tags; performing audio generation processing on the short sentence characters based on the emotion speech matched with the emotion tags, obtaining the short sentence audio; performing splicing processing on the N short sentence audios according to the short sentence sorting, obtaining the text emotion audio corresponding to the text string.

Description

Technical Field

[0001] The present invention relates to the field of audio generation, and in particular to an audio generation method, device, equipment and storage medium with emotion. Background Art

[0002] In developed cities in my country, the number of minors suffering from myopia remains high. Excessive use of eyes is one of the causes of myopia that cannot be ignored. The emergence of audio books has alleviated the probability of minors suffering from myopia to a certain extent.

[0003] However, when reading text, the sound of audio books is too flat. The sound can only pronounce the text without any emotional changes to the text meaning, which makes people feel tired when listening to the relevant audio text and cannot convey the emotions contained in the text. Therefore, a technology is needed to solve the problem of lack of emotional changes in the current audio book reading process. Summary of the invention

[0004] The main purpose of the invention is to solve the technical problem of lack of emotional changes in the current audio book reading process.

[0005] The first aspect of the present invention provides a method for generating audio with emotion, comprising the steps of:

[0006] Receive multiple speech data with emotion labels and generate an emotion speech set;

[0007] Receiving a text string to be converted, and performing sentence processing on the text string to obtain N short sentence characters and short sentence order, wherein N is a positive integer;

[0008] According to a preset recognition algorithm, sentiment analysis is performed on the short sentence characters to obtain sentiment labels corresponding to the short sentence characters;

[0009] Based on the emotion labels corresponding to the characters of the short sentences, the emotion speech set is subjected to label matching processing to obtain emotion speech with matching emotion labels;

[0010] Based on the emotional speech matched by the emotional tag, performing audio generation processing on the short sentence characters to obtain short sentence audio;

[0011] According to the short sentence sorting, the N short sentence audios are spliced ​​to obtain the text emotion audio corresponding to the text string; wherein, according to the preset recognition algorithm, the short sentence characters are subjected to emotion analysis processing to obtain the emotion label corresponding to the short sentence characters, including:

[0012] Sequentially splitting the short sentence characters to obtain M language characters and character sequences, where M is a positive integer;

[0013] According to a preset character vector table, performing vector conversion processing on the language character to obtain a character vector;

[0014] Based on the character sorting, the M character vectors are concatenated to obtain a phrase concatenation matrix;

[0015] According to a preset overall assignment algorithm, dimension assignment processing is performed on the phrase concatenation matrix to obtain a phrase assignment matrix;

[0016] According to a preset classification algorithm, the phrase value matrix is ​​classified to obtain the sentiment labels corresponding to the characters of the short sentences.

[0017] Optionally, in a first implementation of the first aspect of the present invention, performing dimension assignment processing on the phrase concatenation matrix according to a preset overall assignment algorithm to obtain a phrase assignment matrix includes:

[0018] Reading a phrase concatenation element of the phrase concatenation matrix, calculating the Euclidean distance between the phrase concatenation element and an adjacent element, and obtaining a preceding element distance and a succeeding element distance;

[0019] Analyzing the magnitude relationship between the preceding element distance and a preset distance threshold, and analyzing the magnitude relationship between the succeeding element distance and a preset distance threshold;

[0020] Classify the distance corresponding elements exceeding the distance threshold as non-associated elements, and classify the distance corresponding elements not exceeding the distance threshold as associated elements;

[0021] Based on a preset association distinction algorithm, a distinction assignment process is performed on the association elements in the phrase concatenation matrix to obtain a phrase assignment matrix.

[0022] Optionally, in a second implementation of the first aspect of the present invention, the step of performing splicing processing on N short sentence audios according to the short sentence sorting to obtain the text emotion audio corresponding to the text string includes:

[0023] Create an audio frame;

[0024] Based on the short sentence sorting, read each frame data of the N short sentence audios in sequence to obtain a frame sequence data set;

[0025] The frame sequence data set is written into the audio framework to generate text emotion audio corresponding to the text string.

[0026] Optionally, in a third implementation of the first aspect of the present invention, the emotional speech based on the emotional tag matching, performing audio generation processing on the short sentence characters to obtain the short sentence audio includes:

[0027] According to a preset sampling frame, the emotional speech matched by the emotional label is segmented and sampled to obtain an audio feature set;

[0028] Based on the audio feature set, the short sentence characters are subjected to voiceprint simulation processing to obtain short sentence audio.

[0029] Optionally, in a fourth implementation of the first aspect of the present invention, the emotion tags include: a happy tag, a sad tag, an angry tag, a surprised tag, and a peaceful tag.

[0030] Optionally, in a fifth implementation of the first aspect of the present invention, after the N short sentence audios are spliced ​​according to the short sentence sorting to obtain the text emotion audio corresponding to the text string, the method further includes:

[0031] Based on the emotion tag, the text emotion audio is subjected to segment tagging processing to obtain the text emotion audio with multiple emotion tags.

[0032] A second aspect of the present invention provides an emotional audio generation device, the emotional audio generation device comprising:

[0033] A receiving module, used for receiving a plurality of speech data with emotion labels and generating an emotion speech set;

[0034] A sentence segmentation module is used to receive a text string to be converted, perform sentence segmentation processing on the text string, and obtain N short sentence characters and short sentence sequence, wherein N is a positive integer;

[0035] The sentiment analysis module is used to perform sentiment analysis on the short sentence characters according to a preset recognition algorithm to obtain sentiment labels corresponding to the short sentence characters;

[0036] A label matching module, used for performing label matching processing on the emotional speech set based on the emotional labels corresponding to the short sentence characters, to obtain emotional speech with matching emotional labels;

[0037] An audio generation module, used to perform audio generation processing on the short sentence characters based on the emotional speech matched by the emotional tag to obtain short sentence audio;

[0038] The splicing module is used to splice the N short sentence audios according to the short sentence sorting to obtain the text emotion audio corresponding to the text string.

[0039] The third aspect of the present invention provides an emotional audio generating device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the emotional audio generating device executes the above-mentioned emotional audio generating method.

[0040] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned method for generating audio with emotion.

[0041] In the embodiment of the present invention, a solution is provided for recording and simulating emotional voices to read any article with emotion. The user only needs to record a few audios with different emotions to generate a complete audio book, which greatly reduces the time and energy consumed by the user and solves the technical problem of lack of emotional changes in the current audio book reading process. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of an embodiment of a method for generating audio with emotion in an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of an embodiment of a device for generating audio with emotion in an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of another embodiment of the device for generating audio with emotion in an embodiment of the present invention;

[0045] Figure 4 It is a schematic diagram of an embodiment of an audio generating device with emotion in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] Embodiments of the present invention provide a method, apparatus, device and storage medium for generating emotional audio.

[0047] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0048] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of the method for generating audio with emotion in the embodiment of the present invention includes:

[0049] 101. Receive multiple speech data with emotion labels and generate an emotion speech set;

[0050] In this embodiment, the voice data with emotion labels can be selected by the user himself, such as "happy label", "sad label", "angry label", "surprised label" and "peaceful label". After selecting the label, the voice recorded with the emotion label is used as the data in the emotion voice set. Here, all five types of labels need to be entered into the voice data.

[0051] 102. Receive a text string to be converted, and perform sentence processing on the text string to obtain N short sentence characters and short sentence sequence, where N is a positive integer;

[0052] In this embodiment, the received sentence "Stand in front of a flower branch, close your eyes, hold your breath and listen to the sound of petals blooming. Flower blooming is a language that interprets the melody of nature." is obtained, and the result after sentence segmentation is {Stand in front of a flower branch, close your eyes, hold your breath and listen to the sound of petals blooming. Flower blooming is a language that interprets the melody of nature}, and the relevant sentences are accompanied by the order of short sentences.

[0053] 103. Perform sentiment analysis on the short sentence characters according to a preset recognition algorithm to obtain sentiment labels corresponding to the short sentence characters;

[0054] In this embodiment, different Chinese characters contained in the recognized phrase characters express different meanings. For example, "interpreting the melody of nature" and "holding your breath and listening to the sound of petals blooming" have different meanings. After recognizing the emotions contained in the short sentence characters, an emotional label of "happy label", "sad label", "angry label", "surprised label" and "peaceful label" is set as the label of the short sentence characters.

[0055] Further, the following steps can be executed in 103:

[0056] 1031. Perform sequential splitting processing on the short sentence characters to obtain M language characters and character sorting, where M is a positive integer;

[0057] 1032. According to the preset character vector table, perform vector conversion processing on the language characters to obtain character vectors;

[0058] 1033. Based on the character sorting, splice the M character vectors to obtain a phrase splicing matrix;

[0059] 1034. According to the preset overall assignment algorithm, perform dimension assignment processing on the phrase splicing matrix to obtain a phrase assignment matrix;

[0060] 1035. According to the preset classification algorithm, perform classification processing on the phrase assignment matrix to obtain the sentiment label corresponding to the short sentence characters.

[0061] In steps 1031 - 1035, first disassemble "interpreting the melody of nature" to obtain 9 language characters "诠, 释, 着, 大, 自, 然, 的, 旋, 律" and the corresponding character sorting.

[0062] Through the trained character mapping vector table, query the vector values corresponding to the 9 language characters "诠, 释, 着, 大, 自, 然, 的, 旋, 律" respectively to obtain 9 character vectors;

[0063] Based on the character sorting, the character vectors are concatenated to become a phrase concatenation matrix such as (3.7526, 4.56145669955, 3.662562356, 7.523645654, 9.5622222, 9.545544545, 22110.054500, 1.562222, 1.555369745), which is actually an array. However, this processing can only simply concatenate the mapping characters themselves, and does not reflect the relationship between the characters. Therefore, it is also necessary to divide the relevant character levels. For example, phrases such as "interpreter", "nature" and "melody" are of the same category. In this way, the phrase concatenation matrix can be modified as a whole to (3.7526+50, 4.56145669955+50, 3.662562356+50, 7.523645654+30, 9.5622222+30, 9.545544545+30, 22110.054500+20, 1.562222+10, 1.555369745+10). After this adjustment, the phrase assignment matrix is ​​obtained. Finally, the modified phrase assignment matrix (53.7526, 54.56145669955, 53.662562356, 37.523645654, 39.5622222, 39.545544545, 22130.054500, 11.562222, 11.555369745) is used to identify the emotional labels and obtain the emotional labels of the short sentence characters. This processing not only reflects the overall interrelationship, but also shows the details of each character in the entire phrase, verifying that the short sentence labels are more accurate and effective.

[0064] Furthermore, the following steps may be performed in 1034:

[0065] 10341. Read the phrase concatenation element of the phrase concatenation matrix, calculate the Euclidean distance between the phrase concatenation element and the adjacent element, and obtain the front element distance and the back element distance;

[0066] 10342. Analyze the relationship between the element distance before and the preset distance threshold, and analyze the relationship between the element distance after and the preset distance threshold;

[0067] 10343. Classify the distance corresponding elements exceeding the distance threshold as non-associated elements, and classify the distance corresponding elements not exceeding the distance threshold as associated elements;

[0068] 10344. Based on a preset association distinction algorithm, the associated elements in the phrase concatenation matrix are distinguished and assigned to obtain a phrase assignment matrix.

[0069] In steps 10341-10344, (3.7526, 4.56145669955, 3.662562356, 7.523645654, 9.5622222, 9.545544545, 22110.054500, 1.562222, 1.555369745) read 4.56145669955, calculated 4.56145669 The Euclidean distance between 955 and 3.7526 is the front element distance, and the Euclidean distance between 4.56145669955 and 3.662562356 is calculated as the back element distance. It is determined whether the back element distance and the front element distance of the two exceed the distance threshold. If they do not exceed the distance threshold, they are considered to be related to each other, and 4.56145669955, 3.7526, and 3.662562356 are classified as related elements. However, the Euclidean distance between 3.662562356 and 7.523645654 exceeds the threshold, so 3.662562356 and 7.523645654 are classified as non-related elements, and then they are assigned values ​​based on the difference between related and non-related elements, and different related elements are assigned different values ​​to generate a phrase assignment matrix. For example, there are 5 categories, 3.7526, 4.56145669955, 3.662562356 all increase by 50, 7.523645654, 9.5622222, 9.545544545 all increase by 30, 22110.054500 increase by 20, 1.562222, 1.555369745 all increase by 10, to achieve the distinction between different categories.

[0070] 104. Based on the emotion labels corresponding to the short sentence characters, perform label matching processing on the emotion speech set to obtain the emotion speech with matching emotion labels;

[0071] In this embodiment, the emotional label of "interpreting the melody of nature" is "happy label", then the template voice of "happy label" is found in the emotional voice set, and then the voiceprint of the template voice is imitated, or the vioceclone technology can be used to replicate the voice.

[0072] 105. Based on the emotional speech matched with the emotional label, audio generation processing is performed on the short sentence characters to obtain short sentence audio;

[0073] In this embodiment, based on the emotional speech of the "happy label", "interpreting the melody of nature" is converted into short sentence audio, and based on the vioce clone technology, the short sentence audio of "interpreting the melody of nature" is gradually generated.

[0074] Furthermore, at 105, the following steps may be performed:

[0075] 1051. According to a preset sampling frame, segment and sample the emotional speech matched with the emotional label to obtain an audio feature set;

[0076] 1052. Based on the audio feature set, perform voiceprint simulation processing on the short sentence characters to obtain the short sentence audio.

[0077] In this embodiment, the emotional speech can be segmented based on the sampling rate and the size of the sampling frame, and then each segment obtains audio features. The short sentence characters are simulated according to the voiceprint of the segment. The voiceprint information carries emotional state information. The emotional information of the short sentence audio and the emotional speech obtained is consistent, which is mainly obtained by copying and simulating the voiceprint information.

[0078] 106. According to the sorting of the short sentences, the N short sentence audios are concatenated to obtain the text emotion audio corresponding to the text string.

[0079] In this embodiment, {Stand in front of the flower branch, close your eyes, hold your breath and listen to the sound of the petals blooming. The blooming of flowers is a language that interprets the melody of nature} has a total of 6 short sentence audios. Based on the order of the short sentences, a complete "Stand in front of the flower branch, close your eyes. Hold your breath and listen to the sound of the petals blooming. The blooming of flowers is a language that interprets the melody of nature." is generated with the tone of "happy label" text emotion audio.

[0080] Furthermore, at 106, the following steps may be performed:

[0081] 1061. Create an audio framework;

[0082] 1062. Based on the short sentence sorting, read each frame data of the N short sentence audios in sequence to obtain a frame sequence data set;

[0083] 1063. Write the frame sequence data set into the audio framework to generate text emotion audio corresponding to the text string.

[0084] In steps 1061-1063, first create an empty folder, read each frame of audio in sequence, and then copy it into the empty folder, that is, copy it into the audio frame, and combine the audio clips in sequence in the folder to obtain text emotional audio.

[0085] Furthermore, the emotion tags include: a happy tag, a sad tag, an angry tag, a surprised tag, and a peaceful tag. After step 106, the following steps may be performed:

[0086] 1064. Based on the emotion tag, perform segment tagging processing on the text emotion audio to obtain text emotion audio with multiple emotion tags.

[0087] In this embodiment, because different text segments have different emotions when read, in order to facilitate user search, the emotion of each text audio segment is marked, so that the user can retrieve the relevant segment and play it directly.

[0088] In the embodiment of the present invention, a solution is provided for recording and simulating emotional voices to read any article with emotion. The user only needs to record a few audios with different emotions to generate a complete audio book, which greatly reduces the time and energy consumed by the user and solves the technical problem of lack of emotional changes in the current audio book reading process.

[0089] The above describes the method for generating audio with emotion in the embodiment of the present invention. The following describes the device for generating audio with emotion in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, an audio generating device with emotion includes:

[0090] The receiving module 201 is used to receive a plurality of speech data with emotion tags and generate an emotion speech set;

[0091] The sentence segmentation module 202 is used to receive a text string to be converted, perform sentence segmentation processing on the text string, and obtain N short sentence characters and short sentence ranking, where N is a positive integer;

[0092] The sentiment analysis module 203 is used to perform sentiment analysis on the short sentence characters according to a preset recognition algorithm to obtain sentiment labels corresponding to the short sentence characters;

[0093] The label matching module 204 is used to perform label matching processing on the emotional speech set based on the emotional labels corresponding to the short sentence characters to obtain emotional speech with matching emotional labels;

[0094] The audio generation module 205 is used to perform audio generation processing on the short sentence characters based on the emotional speech matched by the emotional tag to obtain short sentence audio;

[0095] The splicing module 206 is used to splice the N short sentence audios according to the short sentence sorting to obtain the text emotion audio corresponding to the text string. In the embodiment of the present invention, the beneficial effect;

[0096] The sentiment analysis module 203 is specifically used for:

[0097] Sequentially splitting the short sentence characters to obtain M language characters and character sequences, where M is a positive integer;

[0098] According to a preset character vector table, performing vector conversion processing on the language character to obtain a character vector;

[0099] Based on the character sorting, the M character vectors are concatenated to obtain a phrase concatenation matrix;

[0100] According to a preset overall assignment algorithm, dimension assignment processing is performed on the phrase concatenation matrix to obtain a phrase assignment matrix;

[0101] According to a preset classification algorithm, the phrase value matrix is ​​classified to obtain the sentiment labels corresponding to the characters of the short sentences.

[0102] In the embodiment of the present invention, a solution is provided for recording and simulating emotional voices to read any article with emotion. The user only needs to record a few audios with different emotions to generate a complete audio book, which greatly reduces the time and energy consumed by the user and solves the technical problem of lack of emotional changes in the current audio book reading process.

[0103] See also Figure 3 Another embodiment of the apparatus for generating audio with emotion in the embodiment of the present invention includes:

[0104] The receiving module 201 is used to receive a plurality of speech data with emotion tags and generate an emotion speech set;

[0105] The sentence segmentation module 202 is used to receive a text string to be converted, perform sentence segmentation processing on the text string, and obtain N short sentence characters and short sentence ranking, where N is a positive integer;

[0106] The sentiment analysis module 203 is used to perform sentiment analysis on the short sentence characters according to a preset recognition algorithm to obtain sentiment labels corresponding to the short sentence characters;

[0107] The label matching module 204 is used to perform label matching processing on the emotional speech set based on the emotional labels corresponding to the short sentence characters to obtain emotional speech with matching emotional labels;

[0108] The audio generation module 205 is used to perform audio generation processing on the short sentence characters based on the emotional speech matched by the emotional tag to obtain short sentence audio;

[0109] The splicing module 206 is used to splice the N short sentence audios according to the short sentence sorting to obtain the text emotion audio corresponding to the text string. In the embodiment of the present invention, there are beneficial effects.

[0110] The sentiment analysis module 203 is specifically used for:

[0111] Sequentially splitting the short sentence characters to obtain M language characters and character sequences, where M is a positive integer;

[0112] According to a preset character vector table, performing vector conversion processing on the language character to obtain a character vector;

[0113] Based on the character sorting, the M character vectors are concatenated to obtain a phrase concatenation matrix;

[0114] According to a preset overall assignment algorithm, dimension assignment processing is performed on the phrase concatenation matrix to obtain a phrase assignment matrix;

[0115] According to a preset classification algorithm, the phrase value matrix is ​​classified to obtain the sentiment labels corresponding to the characters of the short sentences.

[0116] The sentiment analysis module 203 may also be specifically used for:

[0117] Reading a phrase concatenation element of the phrase concatenation matrix, calculating the Euclidean distance between the phrase concatenation element and an adjacent element, and obtaining a preceding element distance and a succeeding element distance;

[0118] Analyzing the magnitude relationship between the preceding element distance and a preset distance threshold, and analyzing the magnitude relationship between the succeeding element distance and a preset distance threshold;

[0119] Classify the distance corresponding elements exceeding the distance threshold as non-associated elements, and classify the distance corresponding elements not exceeding the distance threshold as associated elements;

[0120] Based on a preset association distinction algorithm, a distinction assignment process is performed on the association elements in the phrase concatenation matrix to obtain a phrase assignment matrix.

[0121] The splicing module 206 is specifically used for:

[0122] Create an audio frame;

[0123] Based on the short sentence sorting, read each frame data of the N short sentence audios in sequence to obtain a frame sequence data set;

[0124] The frame sequence data set is written into the audio framework to generate text emotion audio corresponding to the text string.

[0125] The audio generation module 205 is specifically used for:

[0126] According to a preset sampling frame, the emotional speech matched by the emotional label is segmented and sampled to obtain an audio feature set;

[0127] Based on the audio feature set, the short sentence characters are subjected to voiceprint simulation processing to obtain short sentence audio.

[0128] The emotion tags include: a happy tag, a sad tag, an angry tag, a surprised tag, and a peaceful tag. The emotional audio generation device also includes a marking module 207. The marking module 207 is specifically used for:

[0129] Based on the emotion tag, the text emotion audio is subjected to segment tagging processing to obtain the text emotion audio with multiple emotion tags.

[0130] In the embodiment of the present invention, a solution is provided for recording and simulating emotional voices to read any article with emotion. The user only needs to record a few audios with different emotions to generate a complete audio book, which greatly reduces the time and energy consumed by the user and solves the technical problem of lack of emotional changes in the current audio book reading process.

[0131] above Figure 2 and Figure 3 The emotional audio generating device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the emotional audio generating device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0132] Figure 4 4 is a schematic diagram of the structure of an audio generating device with emotion provided by an embodiment of the present invention. The audio generating device with emotion 400 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 410 (for example, one or more processors) and a memory 420, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 433 or data 432. Among them, the memory 420 and the storage medium 430 can be short-term storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the audio generating device with emotion 400. Furthermore, the processor 410 may be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the audio generating device with emotion 400.

[0133] The emotion-based audio generation device 400 may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input and output interfaces 460, and / or one or more operating systems 431, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 4 The structure of the emotional audio generating device shown does not constitute a limitation on the emotional audio generating device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0134] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the method for generating emotional audio.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0136] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0137] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating emotional audio, characterized in that: Includes steps: Receive multiple speech data with emotion labels and generate an emotion speech set; Receiving a text string to be converted, and performing sentence processing on the text string to obtain N short sentence characters and short sentence order, wherein N is a positive integer; According to a preset recognition algorithm, sentiment analysis is performed on the short sentence characters to obtain sentiment labels corresponding to the short sentence characters; Based on the emotion labels corresponding to the characters of the short sentences, the emotion speech set is subjected to label matching processing to obtain emotion speech with matching emotion labels; Based on the emotional speech matched by the emotional tag, performing audio generation processing on the short sentence characters to obtain short sentence audio; According to the sorting of the short sentences, the N short sentence audios are spliced ​​to obtain the text emotion audio corresponding to the text string; The step of performing sentiment analysis on the short sentence characters according to a preset recognition algorithm to obtain sentiment labels corresponding to the short sentence characters includes: Sequentially splitting the short sentence characters to obtain M language characters and character sequences, where M is a positive integer; According to a preset character vector table, performing vector conversion processing on the language character to obtain a character vector; Based on the character sorting, the M character vectors are concatenated to obtain a phrase concatenation matrix; According to a preset overall assignment algorithm, dimension assignment processing is performed on the phrase concatenation matrix to obtain a phrase assignment matrix; According to a preset classification algorithm, the phrase assignment matrix is ​​classified to obtain the sentiment labels corresponding to the characters of the short sentences; The phrase concatenation matrix is ​​dimensionally assigned according to a preset overall assignment algorithm to obtain a phrase assignment matrix including: Reading a phrase concatenation element of the phrase concatenation matrix, calculating the Euclidean distance between the phrase concatenation element and an adjacent element, and obtaining a preceding element distance and a succeeding element distance; Analyzing the magnitude relationship between the preceding element distance and a preset distance threshold, and analyzing the magnitude relationship between the succeeding element distance and a preset distance threshold; Classify the distance corresponding elements exceeding the distance threshold as non-associated elements, and classify the distance corresponding elements not exceeding the distance threshold as associated elements; Based on a preset association distinction algorithm, the association elements in the phrase concatenation matrix are distinguished and assigned values ​​to obtain a phrase assignment matrix.

2. The method for generating emotional audio according to claim 1, characterized in that: The step of performing splicing processing on the N short sentence audios according to the short sentence sorting to obtain the text emotion audio corresponding to the text string includes: Create an audio frame; Based on the short sentence sorting, read each frame data of the N short sentence audios in sequence to obtain a frame sequence data set; The frame sequence data set is written into the audio framework to generate text emotion audio corresponding to the text string.

3. The method for generating emotional audio according to claim 1, characterized in that: The emotional speech based on the emotional tag matching, performing audio generation processing on the short sentence characters to obtain the short sentence audio includes: According to a preset sampling frame, the emotional speech matched by the emotional label is segmented and sampled to obtain an audio feature set; Based on the audio feature set, the short sentence characters are subjected to voiceprint simulation processing to obtain short sentence audio.

4. The method for generating emotional audio according to any one of claims 1 to 3, characterized in that: The emotion labels include: a happy label, a sad label, an angry label, a surprised label, and a peaceful label.

5. The method for generating emotional audio according to claim 4, characterized in that: After the N short sentence audios are spliced ​​according to the short sentence sorting to obtain the text emotion audio corresponding to the text string, the method further includes: Based on the emotion tag, the text emotion audio is subjected to segment tagging processing to obtain the text emotion audio with multiple emotion tags.

6. An emotional audio generation device, characterized in that: The emotional audio generation device comprises: A receiving module, used for receiving a plurality of speech data with emotion labels and generating an emotion speech set; A sentence segmentation module is used to receive a text string to be converted, perform sentence segmentation processing on the text string, and obtain N short sentence characters and short sentence sequence, wherein N is a positive integer; The sentiment analysis module is used to perform sentiment analysis on the short sentence characters according to a preset recognition algorithm to obtain sentiment labels corresponding to the short sentence characters; A label matching module, used for performing label matching processing on the emotional speech set based on the emotional labels corresponding to the characters of the short sentences, to obtain emotional speech with matching emotional labels; An audio generation module, used to perform audio generation processing on the short sentence characters based on the emotional speech matched by the emotional tag to obtain short sentence audio; A splicing module, used for splicing N short sentence audios according to the short sentence sorting, to obtain the text emotion audio corresponding to the text string; Wherein, the sentiment analysis module is specifically used for: Sequentially splitting the short sentence characters to obtain M language characters and character sequences, where M is a positive integer; According to a preset character vector table, performing vector conversion processing on the language character to obtain a character vector; Based on the character sorting, the M character vectors are concatenated to obtain a phrase concatenation matrix; According to a preset overall assignment algorithm, dimension assignment processing is performed on the phrase concatenation matrix to obtain a phrase assignment matrix; According to a preset classification algorithm, the phrase assignment matrix is ​​classified to obtain the sentiment labels corresponding to the characters of the short sentences; The sentiment analysis module is also specifically used for: Reading a phrase concatenation element of the phrase concatenation matrix, calculating the Euclidean distance between the phrase concatenation element and an adjacent element, and obtaining a preceding element distance and a succeeding element distance; Analyzing the magnitude relationship between the preceding element distance and a preset distance threshold, and analyzing the magnitude relationship between the succeeding element distance and a preset distance threshold; Classify the distance corresponding elements exceeding the distance threshold as non-associated elements, and classify the distance corresponding elements not exceeding the distance threshold as associated elements; Based on a preset association distinction algorithm, a distinction assignment process is performed on the association elements in the phrase concatenation matrix to obtain a phrase assignment matrix.

7. An audio generating device with emotion, characterized in that: The emotional audio generating device comprises: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the emotional audio generation device to execute the emotional audio generation method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating emotional audio according to any one of claims 1 to 5 is implemented.

Images