Large model prompt information screening method and device for speech synthesis scene, equipment and medium

Abstract

The invention is suitable for the technical field of artificial intelligence, and particularly relates to a large model prompt information screening method and device for a speech synthesis scene, equipment and a medium. According to the method, static quality detection is carried out on initial prompt speech in a speech library, evaluation on the quality of the initial prompt speech is completed in advance; and the initial prompt voice with relatively high quality is selected for prompting the large model to carry out a voice synthesis task. Meanwhile, the semantic similarity between the initial prompt voice and the voice synthesis text is also considered, so that the target prompt voice most conforming to the semantics of the voice synthesis text is dynamically selected from the candidate prompt voices. Therefore, the target prompt voice obtained through screening has relatively high quality and accords with the semantics of the voice synthesis text, and the accuracy of large model voice synthesis is improved, so that a large language model in a financial scene and a medical scene outputs a voice synthesis result with a better effect.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for screening large model prompt information in a speech synthesis scenario. Background Art

[0002] With the development of artificial intelligence technology, large language models have achieved remarkable success in natural language processing, speech synthesis, and human-computer interaction. At present, most speech synthesis technologies based on large language models use autoregressive generation, that is, the method of using input prompts and text generation tags. Only a few seconds of prompt speech is needed to achieve zero-sample speech cloning. The quality of the input prompt is the key factor in outputting synthesized speech with good timbre and high quality.

[0003] However, input prompts are generally set specifically for different scenarios, and randomly selected input prompts usually cannot provide rich emotional information and expressiveness, which limits the emotional expression of synthesized speech and leads to poor speech synthesis effects. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, equipment and medium for screening large model prompt information in a speech synthesis scenario, so as to solve the problem in the prior art that randomly selected input prompts lead to poor speech synthesis effect.

[0005] A method for screening large model prompt information in a speech synthesis scenario, comprising: Acquire an initial prompt voice library; the initial prompt voice library includes multiple initial prompt voices; Performing static quality detection on the initial prompt voice to determine a static quality value corresponding to each initial prompt voice; Selecting a first preset number of initial prompt voices with the highest static quality values ​​from all the initial prompt voices as candidate prompt voices; Receiving a speech synthesis instruction including a speech synthesis text, and determining a semantic similarity between the speech synthesis text and each of the candidate prompt speech; The candidate prompt speech with the highest semantic similarity is determined as the target prompt speech of the speech synthesis text.

[0006] A large model prompt information screening device for a speech synthesis scene, comprising: A voice library acquisition module, used to acquire an initial prompt voice library; the initial prompt voice library includes multiple initial prompt voices; A static quality detection module, used to perform static quality detection on the initial prompt voices to determine a static quality value corresponding to each of the initial prompt voices; A first prompt voice screening module, used for selecting a first preset number of initial prompt voices with the highest static quality values ​​from all the initial prompt voices as candidate prompt voices; A similarity detection module, configured to receive a speech synthesis instruction including a speech synthesis text, and determine a semantic similarity between the speech synthesis text and each of the candidate prompt speech; The second prompt voice screening module is used to determine the candidate prompt voice with the highest semantic similarity as the target prompt voice of the speech synthesis text.

[0007] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for screening large model prompt information for the above-mentioned speech synthesis scenario is implemented.

[0008] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the large model prompt information screening method for the above-mentioned speech synthesis scenario.

[0009] The method, device, equipment and medium for screening prompt information of the large model in the above-mentioned speech synthesis scenario, the method performs static quality detection on the initial prompt voice in the voice library, completes the evaluation of the quality of the initial prompt voice in advance, and selects the initial prompt voice with higher quality to prompt the large model to perform the speech synthesis task. At the same time, the semantic similarity between the initial prompt voice and the speech synthesis text is also taken into account, so as to dynamically select the target prompt voice that best conforms to the semantics of the speech synthesis text from the candidate prompt voices. In this way, the screened target prompt voice has higher quality and conforms to the semantics of the speech synthesis text, which improves the accuracy of speech synthesis of the speech synthesis text, and the speech synthesis result output by the large model has better effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0011] Figure 1 2 is a schematic diagram of an application environment of a large model prompt information screening method for a speech synthesis scenario in an embodiment of the present invention; Figure 2 is a flow chart of a method for screening large model prompt information for a speech synthesis scenario in one embodiment of the present invention; Figure 3is a flow chart of step S20 in a method for screening large model prompt information for a speech synthesis scenario in an embodiment of the present invention; Figure 4 It is a principle block diagram of a large model prompt information screening device for a speech synthesis scenario in one embodiment of the present invention; Figure 5 It is a principle block diagram of a static quality detection module in a large model prompt information screening device for a speech synthesis scenario in one embodiment of the present invention; Figure 6 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The large model prompt information screening method for the speech synthesis scene provided by the embodiment of the present invention can be applied as follows: Figure 1 Specifically, the large model prompt information screening method of the speech synthesis scene is applied in the large model prompt information screening system of the speech synthesis scene, and the large model prompt information screening system of the speech synthesis scene includes the following Figure 1 The client, server and smart cabinet shown in the figure, the client communicates with the server through the network, and the server and the smart cabinet are connected by service, which is used to solve the problems in the prior art. Among them, the client is also called the user end, which refers to the program corresponding to the server and provides local services to customers. The client can be installed on but not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. Among them, the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0014] In one embodiment, if Figure 2 As shown, a large model prompt information screening method for speech synthesis scenarios is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps: S10: Acquire an initial prompt voice library; the initial prompt voice library includes a plurality of initial prompt voices.

[0015] It is understandable that the initial prompt voice can be voice data in different application scenarios, and the initial prompt voice can be crawled from the Internet, voice database or intelligent question-and-answer interactive system by crawling. The initial prompt voice can be used to guide the large model, so that the large model can convert text data into corresponding voice data based on the initial prompt voice when performing speech synthesis tasks, and realize sound reproduction, reproduction of different speech contents of the same speaker, etc. However, since the quality detection and analysis of the initial prompt voice is not performed when crawling the initial prompt voice, these initial prompt voices may have problems such as low volume, unclearness, excessive noise, missing audio or scene incompatibility. Therefore, this application aims to first perform quality detection and screening on the initial prompt voice to improve the accuracy of the large model in performing speech synthesis tasks.

[0016] S20: Perform static quality detection on the initial prompt voice to determine a static quality value corresponding to each initial prompt voice.

[0017] It can be understood that the static quality detection of the present application detects and evaluates the initial prompt speech from four dimensions. The four dimensions are: the strength of emotional expression, the clarity, naturalness and overall quality of the speech, the consistency of text and emotion, and the performance of model generation. Among them, the strength of emotional expression is evaluated by the pitch feature information of the initial prompt speech. The clarity, naturalness and overall quality of the speech are evaluated by a preset audio quality evaluation model. The consistency of text and emotion is evaluated by a preset text quality evaluation model. The performance of model generation is evaluated by the character error rate, feature similarity, and emotional similarity of the synthetic speech generated by the large model based on the initial prompt speech. The static quality value comprehensively reflects the quality level of the initial prompt speech in multiple dimensions such as the audio itself, emotional features, and consistency between text and speech.

[0018] S30: Selecting a first preset number of initial prompt voices with the highest static quality values ​​from all the initial prompt voices as candidate prompt voices.

[0019] The first preset number can be set according to the demand or according to the number of initial prompt voices. For example, when the number of initial prompt voices exceeds 50, the first preset number can be set to 10% of the total number of initial prompt voices (e.g., when the number of initial prompt voices is 100, the first preset number can be set to 10).

[0020] Specifically, after performing static quality detection on the initial prompt voices and determining the static quality value corresponding to each of the initial prompt voices, the initial prompt voices are sorted based on the static quality values ​​to form a prompt voice sequence, and then starting from the first initial prompt voice in the prompt voice sequence, a first preset number of initial prompt voices are selected, and the selected initial prompt voices are used as candidate prompt voices.

[0021] S40: receiving a speech synthesis instruction including a speech synthesis text, and determining a semantic similarity between the speech synthesis text and each of the candidate prompt voices.

[0022] It can be understood that the speech synthesis instruction can be sent through the user's client, or it can be automatically generated after the user uploads the speech synthesis text. The speech synthesis text is the text data that the user needs to convert into speech data. For example, the speech synthesis text can be: I am very happy that you can share this with me. After listening to your sharing, I feel very excited and can't wait to participate in this event! At this time, the user wants to convert the above speech synthesis text into speech data.

[0023] Furthermore, since the speech synthesis text is text-type data and the candidate prompt speech is speech-type data, in order to facilitate the determination of the semantic similarity between the speech synthesis text and the candidate prompt speech, the candidate prompt speech can be first converted into a text representation by speech recognition, or when constructing the initial prompt speech library, each initial prompt speech and its corresponding text representation will be bound, and the text representation can be directly obtained at this time. In this way, after obtaining the candidate speech text corresponding to the candidate prompt speech, the semantic similarity between the speech synthesis text and the candidate speech text can be detected. The semantic similarity is the correlation relationship between the semantic expression of the candidate speech text and the speech synthesis text. The higher the semantic similarity, the closer the semantics between the candidate prompt speech and the speech synthesis text. On the contrary, the lower the semantic similarity, the more different the semantics between the candidate prompt speech and the speech synthesis text.

[0024] S50: Determine the candidate prompt speech with the highest semantic similarity as the target prompt speech of the speech synthesis text.

[0025] Specifically, after determining the semantic similarity between the speech synthesis text and each of the candidate prompt speech, the semantic similarity corresponding to each candidate prompt speech can be compared, so as to determine the candidate prompt speech with the highest semantic similarity as the target prompt speech of the speech synthesis text.

[0026] Furthermore, after determining the target prompt voice, the speech synthesis text and the target prompt voice can be simultaneously input into the large model, and the target prompt voice is used as a prompt for the large model, so that the large model can learn the voice features of the target prompt voice, and then perform speech synthesis processing on the speech synthesis text based on the voice features, and output the speech synthesis result. In addition, users can also put forward speech synthesis requirements, such as synthesizing a speech synthesis result with the same speaker characteristics as the target prompt voice, or synthesizing a speech synthesis result with the same emotional expression as the target prompt voice, etc.

[0027] In this embodiment, by performing static quality detection on the initial prompt voice in the voice library, the quality of the initial prompt voice is evaluated in advance, and the initial prompt voice with higher quality is selected to prompt the large model to perform the speech synthesis task. At the same time, the semantic similarity between the initial prompt voice and the speech synthesis text is also taken into account, so as to dynamically select the target prompt voice that best conforms to the semantics of the speech synthesis text from the candidate prompt voices. In this way, the screened target prompt voice has higher quality and conforms to the semantics of the speech synthesis text, which improves the accuracy of speech synthesis of the speech synthesis text, and the speech synthesis result output by the large model has better effect.

[0028] For example, in financial scenarios, it is often necessary to set up artificial intelligence customer service or develop a guide for digital intelligence to answer some basic questions of users, and it is hoped that artificial intelligence customer service or digital intelligence can output voice data that is more suitable for humans. Therefore, it is necessary to use large-scale voice data to train artificial intelligence customer service or digital intelligence, and the workload is large and time-consuming to manually collect voice data. At this time, the method of the present application can be used to screen out suitable large model prompt information, so that based on the large model prompt information, the large model can quickly and massively output the required voice data, and the training of artificial intelligence customer service or digital intelligence can be realized through the voice data output by the large model.

[0029] Another example, in a medical scenario, some patients speak in dialect rather than Mandarin, which is difficult for some doctors to understand. In order to take care of these patients who express themselves in dialect, a dialect recognition model is often trained to identify the patient's voice data in a specific dialect and convert it into Mandarin. In order to train the dialect recognition model, a large amount of dialect voice data needs to be collected, and the method of the present application is used to screen out appropriate large model prompt information, so that the large model can quickly and massively output the corresponding dialect voice data based on the large model prompt information, and the dialect recognition model is trained through the dialect voice data output by the large model.

[0030] Thus, it can be seen from the above two examples that by screening out appropriate big model prompt information through the method of the present application, the big model can be better guided, whether in the model training task or in the speech cloning task, so that the big model can refer to the most matching big model prompt information to generate better speech synthesis results.

[0031] In one embodiment, if Figure 3 As shown, in step S20, that is, performing static quality detection on the initial prompt voice to determine the static quality value corresponding to each initial prompt voice, includes: S201, performing emotion classification processing on the initial prompt voice to determine multiple prompt information emotion groups; each prompt information emotion group contains multiple emotion prompt voices; the emotion prompt voice is an initial prompt voice that meets the preset emotion category.

[0032] It can be understood that the emotion classification process is to classify the initial prompt voice into different emotional states, that is, after screening out the initial prompt voice that meets the preset emotion category, it is further subdivided based on the emotional state of each initial prompt voice to form multiple prompt information emotion groups. Among them, the preset emotion category may include but is not limited to: sadness, happiness, surprise, panic, etc. Further, the initial prompt voice without emotion expression will not be classified into any prompt information emotion group.

[0033] Furthermore, there are many methods for emotional classification of the initial prompt speech, such as the characteristics of the pitch shape of the speech itself, such as slope, curvature and turning point, but these data are relatively simple in describing emotions and cannot capture the subtle differences between different emotional states. As a suprasegmental feature, the information conveyed by pitch covers a longer time than segmental features (such as spectral envelope). In this embodiment, classification is performed based on the average and variance of the pitch features of the initial prompt speech, and these two data are more likely to cause emotional resonance.

[0034] S202: Perform voice quality detection on the emotion prompt voice in each of the prompt information emotion groups to determine the voice quality score of each of the emotion prompt voices.

[0035] It can be understood that the voice quality detection is used to detect the clarity, naturalness and overall quality of the emotional prompt voice and the consistency between the text and the emotion. Among them, this embodiment uses a preset audio quality assessment model to detect the clarity, naturalness and overall quality of the emotional prompt voice, and the preset audio quality assessment model can use the DNSMOS tool. A preset text quality assessment model is used to detect the consistency between the text and the emotion, and the preset text quality assessment model can use the ChatGPT API, or a self-trained model.

[0036] Furthermore, after the initial prompt voice is subjected to emotion classification processing and multiple prompt information emotion groups are determined, the emotional prompt voice is input into a preset audio quality assessment model, thereby determining the clarity, naturalness and overall quality of each emotional prompt voice through the preset audio quality assessment model, and then determining the first quality score of the emotional prompt voice based on the clarity, naturalness and overall quality.

[0037] Furthermore, after performing emotion classification processing on the initial prompt voice and determining multiple prompt information emotion groups, the voice prompt text corresponding to the emotion prompt voice is obtained, and the first emotion category information corresponding to the emotion prompt voice is determined based on the prompt information emotion group corresponding to the emotion prompt voice. The voice prompt text and the first emotion category information are input into a preset text quality assessment model, and the preset text quality assessment model is used to determine whether the emotion information reflected by the voice prompt text is the same as the first emotion category information, and a second quality score is output. Among them, the voice prompt text is a text representation of the emotion prompt voice. The first emotion category information expresses the emotional state of the emotion prompt voice. The second quality score measures whether the emotion expressed in the text of the emotion prompt voice is coherent and consistent with the pre-classified first emotion category information.

[0038] S203: Select a second preset number of emotion prompt voices with the highest voice quality scores from all the emotion prompt voices as screening prompt voices.

[0039] The second preset number can be set by user, or can be determined according to the number of emotion prompt voices obtained by screening. For example, if the number of emotion prompt voices is 20, the second preset number can be 30% of the number of emotion prompt voices, that is, the second preset number is 6.

[0040] Furthermore, after performing voice quality detection on the emotional prompt voice in each of the prompt information emotion groups and determining the voice quality score of each of the emotional prompt voices, the emotional prompt voices are sorted based on the voice quality scores to form an emotional voice sequence, and then starting from the first emotional prompt voice in the emotional voice sequence, a second preset number of emotional prompt voices are selected, and the selected emotional prompt voices are used as screening prompt voices.

[0041] S204: Perform large model prompt detection on the screening prompt voice to determine the static quality value of each screening prompt voice.

[0042] Understandably, in speech synthesis systems based on large language models, we not only focus on evaluating the quality of the prompt speech, but also realize that even with the same prompt speech input, different models may produce completely different outputs. This difference mainly stems from the selection strategy of the prompt speech markers. To solve this problem, we propose large model prompt detection, which focuses on the performance of different models when processing the same prompt speech, mainly including three parts: character error rate, feature similarity, and sentiment similarity.

[0043] In this embodiment, the initial prompt speech is evaluated from four dimensions: the strength of emotional expression, the clarity of speech, the consistency between text and emotion, and the performance of model generation, so that the screened candidate prompt speech can improve the accuracy of large models in performing speech synthesis tasks.

[0044] In one embodiment, in step S201, that is, performing emotion classification processing on the initial prompt voice to determine multiple prompt information emotion groups, the steps include: The pitch feature information of each of the initial prompt voices is obtained, and the pitch average value and the pitch variance value of each of the initial prompt voices are determined based on the pitch feature information.

[0045] It can be understood that pitch feature information is a feature before language is produced, which gives tone and rhythm to speech, that is, pitch feature information is the pitch data of the initial prompt speech at different times. Among them, the pitch average refers to the average value of the pitch at different time points in the initial prompt speech. It can reflect the general tone or emotion of the speaker. The pitch variance value measures the change of pitch over time. A larger pitch variance value may indicate that the speech is more vivid or emotional, while a smaller pitch variance value may indicate monotonous communication. Different emotional states are associated with different pitch patterns. For example, both the sad category and the comfortable category show relatively low pitch averages and variances, indicating calmer and lower pitch characteristics, and sadness is slightly more suppressed. On the other hand, emotions such as the happy category and the surprised category show higher averages and variances, reflecting more obvious emotional intensity.

[0046] All initial prompt voices are clustered based on the pitch average value and the pitch variance value to determine a plurality of initial emotion groups.

[0047] Specifically, some speech data containing emotional states are manually classified in advance, and then based on the pitch average and pitch variance values ​​of the speech data under each emotional state after classification, the data range of the pitch average and the data range of the pitch variance values ​​under different emotional states are determined. In this way, after the pitch average and pitch variance values ​​of each of the initial prompt speech are determined based on the pitch feature information, the pitch average of the initial prompt speech and the data range of the pitch average of each emotional state can be compared, and the pitch variance value of the initial prompt speech and the data range of the pitch variance value of each emotional state can also be compared, so that the initial prompt speech that belongs to both the data range of the pitch average and the data range of the pitch variance value can be clustered into the emotional state and classified into multiple initial emotional groups.

[0048] The emotion average value of the initial emotion group is determined based on the pitch average value of all the initial prompt voices in the initial emotion group, and the variance average value of the initial emotion group is determined based on the pitch variance value of all the initial prompt voices in the initial emotion group.

[0049] Specifically, after clustering all the initial prompt voices based on the pitch mean value and pitch variance value to determine multiple initial emotion groups, the emotional mean value of the initial emotion group can be determined based on the average value of the pitch mean values ​​of all the initial prompt voices in each initial emotion group, and the variance mean value of the initial emotion group can be determined based on the average value of the pitch variance values ​​of all the initial prompt voices in the initial emotion group.

[0050] A strong emotion threshold group and a weak emotion threshold group are obtained; the strong emotion threshold group includes a first average threshold and a first variance threshold, and the weak emotion threshold group includes a second average threshold and a second variance threshold.

[0051] Understandably, when all initial prompt information is clustered into multiple initial emotion groups, the emotional average and / or variance average of some initial emotion groups are relatively smooth, that is, the emotional average and / or variance average cannot experience the emotional states at two extremes (such as sadness and excitement, surprise, etc.). In the speech synthesis task, relatively peaceful emotional states do not require special training. Therefore, a strong emotion threshold group and a weak emotion threshold group are introduced to screen the initial emotion groups to screen out speech data that can more directly represent human emotional characteristics. Among them, the first average threshold is greater than the second average threshold, and the first variance threshold is greater than the second variance threshold.

[0052] The initial emotion group whose average emotion value is greater than the first average threshold and whose average variance value is greater than the first variance threshold is determined as the prompt information emotion group, and the initial emotion group whose average emotion value is less than the second average threshold and whose average variance value is less than the second variance threshold is determined as the prompt information emotion group.

[0053] Specifically, after obtaining the strong emotion threshold group and the weak emotion threshold group, the emotion average value is compared with the first average threshold value and the second average threshold value, and the emotion variance value is compared with the first variance threshold value and the second variance threshold value. The initial emotion group whose emotion average value is greater than the first average threshold value and whose variance average value is greater than the first variance threshold value is determined as the prompt information emotion group (the emotion prompt voice in the prompt information emotion group is a strong emotion category, such as excitement, happiness, and surprise), and the initial emotion group whose emotion average value is less than the second average threshold value and whose variance average value is less than the second variance threshold value is determined as the prompt information emotion group (the emotion prompt voice in the prompt information emotion group is a weaker or calm emotion category, such as sadness and anxiety). In this embodiment, the initial emotion group whose emotion average value is less than or equal to the first average threshold value, greater than or equal to the second average threshold value, and whose emotion variance value is less than or equal to the first variance threshold value, and greater than or equal to the second variance threshold value is eliminated.

[0054] In one embodiment, in step S204, that is, performing large model prompt detection on the screening prompt voice to determine the static quality value of each screening prompt voice, includes: Get the preset large language model and training speech text.

[0055] The preset large language model may be a model with speech synthesis capability, and the training speech text may be random text or text data in the same scenario.

[0056] The training speech text and the screening prompt speech are input into the preset large language model, so as to generate a training synthetic speech corresponding to the training speech text based on the screening prompt speech through the preset large language model.

[0057] Specifically, after obtaining the preset large language model and the training speech text, the training speech text and the screening prompt speech can be input into the preset large language model, so that the large language model generates the training synthesized speech corresponding to the training speech text based on the speech features of the screening prompt speech.

[0058] Based on the screening prompt voice and the corresponding training synthesized voice, a static quality value of each screening prompt voice is determined.

[0059] Specifically, after generating the training synthesized speech corresponding to the training speech text based on the screening prompt speech through the preset large language model, the static quality value of each screening prompt speech is determined based on the screening prompt speech and the corresponding training synthesized speech.

[0060] In a specific real-time manner, determining the static quality value of each screening prompt voice based on the screening prompt voice and the corresponding training synthesized voice includes: Based on the training speech text, a character error rate of the training synthesized speech is determined.

[0061] It can be understood that the character error rate reflects the accuracy of the training synthesized speech to the training speech text. In this way, the text content corresponding to the training synthesized speech can be determined, and the text content can be compared with the training speech text to detect whether there are erroneous characters in the training synthesized speech. The character error rate is determined based on the proportion of erroneous characters in all characters in the text content corresponding to the training synthesized speech.

[0062] Speaker features are extracted from the screening prompt speech and the corresponding training synthesized speech, and feature similarity between the speaker features of the screening prompt speech and the speaker features of the training synthesized speech is determined.

[0063] It can be understood that the feature similarity reflects the similarity between the voice of the speaker of the screening prompt speech and the training synthesized speech. The higher the feature similarity, the more similar the speaker voice of the screening prompt speech and the training synthesized speech is, and the lower the feature similarity, the more different the speaker voice of the screening prompt speech and the training synthesized speech is. Among them, the Resemblyzer and WavLM tools can be used to evaluate the feature similarity between the speaker features of the screening prompt speech and the speaker features of the training synthesized speech.

[0064] The first emotion category information corresponding to the screening prompt voice is determined based on the prompt information emotion group corresponding to the screening prompt voice, and the training synthetic voice is subjected to emotion classification processing to determine the second emotion category information of the training synthetic voice.

[0065] It can be understood that the first emotion category information represents the emotion category of the screening prompt speech, and the second emotion category information represents the emotion category of the training synthesis speech.

[0066] Based on the first emotion category information and the second emotion category information, the emotion similarity between the screening prompt speech and the training synthesized speech is determined.

[0067] The emotion2vec model may be used to determine the emotion similarity between the screening prompt speech and the training synthesized speech based on the first emotion category information and the second emotion category information.

[0068] Based on the character error rate, feature similarity and emotion similarity corresponding to the same screening prompt voice, the static quality value of each screening prompt voice is determined.

[0069] Specifically, after determining the character error rate, feature similarity, and emotion similarity of the training synthesized speech corresponding to the screening prompt speech, the static quality value of each screening prompt speech can be determined.

[0070] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0071] In one embodiment, a large model prompt information screening device for a speech synthesis scenario is provided, and the large model prompt information screening device for a speech synthesis scenario corresponds one-to-one to the large model prompt information screening method for a speech synthesis scenario in the above embodiment. Figure 4 As shown, the large model prompt information screening device for the speech synthesis scene includes a speech library acquisition module 10, a static quality detection module 20, a first prompt speech screening module 30, a similarity detection module 40, and a second prompt speech screening module 50. The functional modules are described in detail as follows: The voice library acquisition module 10 is used to acquire an initial prompt voice library; the initial prompt voice library includes a plurality of initial prompt voices; A static quality detection module 20, configured to perform static quality detection on the initial prompt voices to determine a static quality value corresponding to each of the initial prompt voices; A first prompt voice screening module 30, configured to select a first preset number of initial prompt voices with the highest static quality values ​​from all the initial prompt voices as candidate prompt voices; A similarity detection module 40 is used to receive a speech synthesis instruction including a speech synthesis text, and determine the semantic similarity between the speech synthesis text and each of the candidate prompt speech; The second prompt voice screening module 50 is used to determine the candidate prompt voice with the highest semantic similarity as the target prompt voice of the speech synthesis text.

[0072] In one embodiment, if Figure 5 As shown, the static quality detection module 20 includes: The emotion classification submodule 201 is used to perform emotion classification processing on the initial prompt voice to determine a plurality of prompt information emotion groups; each of the prompt information emotion groups includes a plurality of emotion prompt voices; the emotion prompt voice is an initial prompt voice that meets a preset emotion category; The quality detection submodule 202 is used to perform voice quality detection on the emotional prompt voice in each of the prompt information emotional groups, and determine the voice quality score of each of the emotional prompt voices; A first screening submodule 203 is used to select a second preset number of emotional prompt voices with the highest voice quality scores from all the emotional prompt voices as screening prompt voices; The large model prompt detection submodule 204 is used to perform large model prompt detection on the screening prompt voice to determine the static quality value of each screening prompt voice.

[0073] In one embodiment, the sentiment classification submodule 201 includes: A pitch acquisition unit, used to acquire pitch feature information of each of the initial prompt voices, and determine a pitch average value and a pitch variance value of each of the initial prompt voices based on the pitch feature information; A clustering unit, used for clustering all initial prompt voices based on the pitch average value and the pitch variance value to determine a plurality of initial emotion groups; An average value calculation unit, used to determine the emotion average value of the initial emotion group based on the pitch average value of all initial prompt voices in the initial emotion group, and determine the variance average value of the initial emotion group based on the pitch variance value of all initial prompt voices in the initial emotion group; A threshold acquisition unit, used to acquire a strong emotion threshold group and a weak emotion threshold group; the strong emotion threshold group includes a first average threshold and a first variance threshold, and the weak emotion threshold group includes a second average threshold and a second variance threshold; A threshold comparison unit is used to determine the initial emotion group whose emotion average value is greater than a first average threshold and whose variance average value is greater than a first variance threshold as the prompt information emotion group, and to determine the initial emotion group whose emotion average value is less than a second average threshold and whose variance average value is less than a second variance threshold as the prompt information emotion group.

[0074] In one embodiment, the quality detection submodule 202 includes: An audio quality evaluation unit, used for inputting the emotion prompt speech into a preset audio quality evaluation model to determine a first quality score of each emotion prompt speech through the preset audio quality evaluation model; A text quality assessment unit, used for obtaining a preset text quality assessment model, and determining a second quality score of each of the emotion prompting speech through the preset text quality assessment model; A quality detection unit is used to determine the sum of the first quality score and the second quality score as the voice quality score of the emotion prompt voice.

[0075] In one embodiment, the text quality assessment unit includes: An emotion category determination subunit, used for acquiring a voice prompt text corresponding to the emotion prompt voice, and determining first emotion category information corresponding to the emotion prompt voice based on a prompt information emotion group corresponding to the emotion prompt voice; The text quality assessment subunit is used to input the speech prompt text and the first emotion category information into the preset text quality assessment model to determine the second quality score corresponding to the emotion prompt speech through the preset text quality assessment model.

[0076] In one embodiment, the large model hint detection submodule 204 includes: A data acquisition unit, used to acquire a preset large language model and training speech text; A speech synthesis unit, used for inputting the training speech text and the screening prompt speech into the preset large language model, so as to generate a training synthesized speech corresponding to the training speech text based on the screening prompt speech through the preset large language model; The speech comparison unit is used to determine the static quality value of each screening prompt speech based on the screening prompt speech and the corresponding training synthetic speech.

[0077] In one embodiment, the speech comparison unit comprises: A character error rate determination subunit, used to determine the character error rate of the training synthesized speech based on the training speech text; A feature similarity determination subunit, used for extracting speaker features from the screening prompt speech and the corresponding training synthesized speech, and determining the feature similarity between the speaker features of the screening prompt speech and the speaker features of the training synthesized speech; The emotion classification subunit is used to determine the first emotion category information corresponding to the screening prompt voice based on the prompt information emotion group corresponding to the screening prompt voice, and at the same time perform emotion classification processing on the training synthesized voice to determine the second emotion category information of the training synthesized voice; An emotion similarity determination subunit, used to determine the emotion similarity between the screening prompt voice and the training synthesized voice based on the first emotion category information and the second emotion category information; The static quality value determination subunit is used to determine the static quality value of each screening prompt voice based on the character error rate, feature similarity and emotion similarity corresponding to the same screening prompt voice.

[0078] For the specific definition of the large model prompt information screening device for the speech synthesis scenario, please refer to the definition of the large model prompt information screening method for the speech synthesis scenario above, which will not be repeated here. The various modules in the large model prompt information screening device for the above-mentioned speech synthesis scenario can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0079] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used by the large model prompt information screening method for the speech synthesis scene in the above-mentioned embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a large model prompt information screening method for a speech synthesis scene is implemented.

[0080] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the large model prompt information screening method for the speech synthesis scenario in the above embodiment is implemented.

[0081] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the large model prompt information screening method for the speech synthesis scenario in the above embodiment is implemented.

[0082] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0083] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0084] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for screening large model prompt information in a speech synthesis scenario, characterized in that: include: Get the initial prompt voice library; The initial prompt voice library includes multiple initial prompt voices; Performing static quality detection on the initial prompt voice to determine a static quality value corresponding to each initial prompt voice; Selecting a first preset number of initial prompt voices with the highest static quality values ​​from all the initial prompt voices as candidate prompt voices; Receiving a speech synthesis instruction including a speech synthesis text, and determining a semantic similarity between the speech synthesis text and each of the candidate prompt speech; The candidate prompt speech with the highest semantic similarity is determined as the target prompt speech of the speech synthesis text.

2. The method for selecting large model prompt information for speech synthesis scenarios according to claim 1, characterized in that: The performing static quality detection on the initial prompt voice to determine the static quality value corresponding to each initial prompt voice includes: Performing emotion classification processing on the initial prompt voice to determine a plurality of prompt information emotion groups; each of the prompt information emotion groups includes a plurality of emotion prompt voices; the emotion prompt voice is an initial prompt voice that meets a preset emotion category; Performing voice quality detection on the emotional prompt voice in each of the prompt information emotional groups to determine the voice quality score of each of the emotional prompt voices; Selecting a second preset number of emotional prompt voices with the highest voice quality scores from all the emotional prompt voices as screening prompt voices; A large model prompt detection is performed on the screening prompt voice to determine the static quality value of each screening prompt voice.

3. The method for selecting large model prompt information for speech synthesis scenarios according to claim 2, characterized in that: The performing emotion classification processing on the initial prompt voice to determine a plurality of prompt information emotion groups includes: Acquire pitch feature information of each of the initial prompt voices, and determine a pitch average value and a pitch variance value of each of the initial prompt voices based on the pitch feature information; Clustering all initial prompt voices based on the pitch average value and the pitch variance value to determine multiple initial emotion groups; Determine the emotion average value of the initial emotion group based on the pitch average value of all initial prompt voices in the initial emotion group, and determine the variance average value of the initial emotion group based on the pitch variance value of all initial prompt voices in the initial emotion group; Acquire a strong emotion threshold group and a weak emotion threshold group; the strong emotion threshold group includes a first average threshold and a first variance threshold, and the weak emotion threshold group includes a second average threshold and a second variance threshold; The initial emotion group whose average emotion value is greater than the first average threshold and whose average variance value is greater than the first variance threshold is determined as the prompt information emotion group, and the initial emotion group whose average emotion value is less than the second average threshold and whose average variance value is less than the second variance threshold is determined as the prompt information emotion group.

4. The method for selecting large model prompt information for speech synthesis scenarios according to claim 2, characterized in that: The step of performing voice quality detection on the emotion prompt voice in each of the prompt information emotion groups to determine the voice quality score of each of the emotion prompt voices includes: Inputting the emotion prompt speech into a preset audio quality assessment model to determine a first quality score of each emotion prompt speech through the preset audio quality assessment model; Acquire a preset text quality assessment model, and determine a second quality score of each of the emotion prompt speech by using the preset text quality assessment model; The sum of the first quality score and the second quality score is determined as the speech quality score of the emotion prompt speech.

5. The method for selecting large model prompt information for speech synthesis scenarios according to claim 4, characterized in that: The step of obtaining a preset text quality assessment model and determining the second quality score of each of the emotion prompting speech by using the preset text quality assessment model includes: Acquire a voice prompt text corresponding to the emotion prompt voice, and determine first emotion category information corresponding to the emotion prompt voice based on the prompt information emotion group corresponding to the emotion prompt voice; The voice prompt text and the first emotion category information are input into the preset text quality assessment model to determine a second quality score corresponding to the emotion prompt voice through the preset text quality assessment model.

6. The method for selecting large model prompt information for speech synthesis scenarios according to claim 2, characterized in that: The step of performing large model prompt detection on the screening prompt voice to determine the static quality value of each screening prompt voice includes: Get the preset large language model and training speech text; Inputting the training speech text and the screening prompt speech into the preset large language model, so as to generate a training synthetic speech corresponding to the training speech text based on the screening prompt speech through the preset large language model; Based on the screening prompt voice and the corresponding training synthesized voice, a static quality value of each screening prompt voice is determined.

7. The method for selecting large model prompt information for speech synthesis scenarios according to claim 6, characterized in that: The step of determining the static quality value of each of the screening prompt voices based on the screening prompt voices and the corresponding training synthesized voices includes: Based on the training speech text, determining a character error rate of the training synthesized speech; Extracting speaker features from the screening prompt voice and the corresponding training synthesized voice, and determining feature similarity between the speaker features of the screening prompt voice and the speaker features of the training synthesized voice; Determine the first emotion category information corresponding to the screening prompt voice based on the prompt information emotion group corresponding to the screening prompt voice, and at the same time perform emotion classification processing on the training synthesized voice to determine the second emotion category information of the training synthesized voice; Determining the emotional similarity between the screening prompt voice and the training synthesized voice based on the first emotional category information and the second emotional category information; Based on the character error rate, feature similarity and emotion similarity corresponding to the same screening prompt voice, the static quality value of each screening prompt voice is determined.

8. A large model prompt information screening device for a speech synthesis scene, characterized in that: include: A voice library acquisition module is used to acquire an initial prompt voice library; The initial prompt voice library includes multiple initial prompt voices; A static quality detection module, used to perform static quality detection on the initial prompt voices to determine a static quality value corresponding to each of the initial prompt voices; A first prompt voice screening module, used for selecting a first preset number of initial prompt voices with the highest static quality values ​​from all the initial prompt voices as candidate prompt voices; A similarity detection module, configured to receive a speech synthesis instruction including a speech synthesis text, and determine a semantic similarity between the speech synthesis text and each of the candidate prompt speech; The second prompt voice screening module is used to determine the candidate prompt voice with the highest semantic similarity as the target prompt voice of the speech synthesis text.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the large model prompt information screening method for the speech synthesis scenario as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for screening large model prompt information for a speech synthesis scenario as described in any one of claims 1 to 7 is implemented.

Images