Voice style conversion method and device, equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of research and development design and audio processing, is applied to conversion processing scenes of different styles of voices, and relates to a voice style conversion method, device and equipment and a storage medium thereof. Inputting the style prompt text into a preset semantic analysis model, and obtaining target style information through semantic analysis; performing text conversion processing on the source voice to obtain a voice text sequence; screening out exclusive acoustic features of the target style information from a pre-trained style encoder; and inputting the voice text sequence and the exclusive acoustic features into a pre-trained voice decoder to obtain converted voice of the target style information. And voice conversion is carried out in combination with the style prompt text, and the source voice is converted into the conversion voice of the target style information, so that the style conversion result is more accurate and personalized. The method is applied to the field of financial service virtual voice conversion, and voice services of different styles can be provided for clients.

Description

Technical Field

[0001] The present application relates to the field of R&D design and audio processing technology, and is applied to conversion and processing scenarios of different styles of speech, and in particular to a speech style conversion method, device, equipment and storage medium thereof. Background Art

[0002] In recent years, existing neural vocoder technology has made significant progress in fields such as speech synthesis and speech conversion. However, the current technology still has corresponding shortcomings.

[0003] This is especially true in the field of speech conversion, especially converting source speech into target style speech, for example, converting the style of a news report into the style of a chat conversation. Existing speech style conversion solutions are mainly based on reference speech, which requires reference to speech samples of the target style. Once the more styles are integrated, the more likely it is that the speech style conversion will be inaccurate, resulting in the inability to provide users with accurate and personalized speech conversion services. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a speech style conversion method, apparatus, device and storage medium thereof to solve the problem that when performing existing speech style conversion, the more style patterns are integrated, the more likely it is that the speech style conversion will be inaccurate, resulting in the inability to provide users with accurate and personalized speech conversion services.

[0005] In order to solve the above technical problems, the present application embodiment provides a method for voice style conversion, which adopts the following technical solution:

[0006] A method for voice style conversion comprises the following steps:

[0007] Obtaining source speech and style prompt text to be converted into speech;

[0008] Inputting the style hint text into a preset semantic parsing model to obtain target style information through semantic parsing;

[0009] Performing text conversion on the source speech to obtain a speech-to-text sequence of the source speech;

[0010] Filtering the exclusive acoustic features of the target style information from the pre-trained style encoder;

[0011] The speech text sequence and the exclusive acoustic features are input into a pre-trained speech decoder, and speech synthesis decoding processing is performed to obtain the converted speech of the target style information.

[0012] Furthermore, the preset semantic parsing model includes a text semantic recognition model based on BERT, and the step of inputting the style hint text into the preset semantic parsing model and obtaining the target style information through semantic parsing specifically includes:

[0013] Inputting the style hint text into the BERT-based text semantic recognition model;

[0014] By performing semantic analysis on the style hint text, outputting all candidate style information hit by the style hint text;

[0015] Feeding back all the candidate style information to the input acquisition interface corresponding to the style prompt text;

[0016] Based on the confirmation instruction of the input acquisition interface, target style information is identified from all the candidate style information.

[0017] Furthermore, the step of converting the source speech into text to obtain a speech-to-text sequence of the source speech specifically includes:

[0018] Using speech recognition technology or speech-to-text technology to convert the source speech into text, and obtain all text contents after the source speech is converted;

[0019] According to the pronunciation order of all the text contents in the source speech, all the text contents are serialized to obtain a speech-text sequence of the source speech.

[0020] Furthermore, before performing the step of filtering out the exclusive acoustic features of the target style information from the pre-trained style encoder, the method further includes:

[0021] According to the preset speech style categories, batch reference speech of different speech style categories are obtained respectively;

[0022] Using batches of reference speech of different speech style categories as training samples, and inputting them into a style encoder to be trained, to obtain a trained style encoder, wherein the style encoder to be trained includes a style encoder based on a GSTs tagging method;

[0023] The trained style encoder is used as the pre-trained style encoder.

[0024] Furthermore, the step of inputting batches of reference speech of different speech style categories as training samples into the style encoder to be trained to obtain a trained style encoder specifically includes:

[0025] Using the style encoder based on the GSTs tagging method to extract acoustic features from the batch reference speech in the training sample;

[0026] According to the speech style categories respectively corresponding to the batch of reference speech, clustering processing is performed on the acoustic feature extraction results of different speech style categories;

[0027] According to the clustering processing result, obtaining the first common acoustic features corresponding to the same speech style category and the second common acoustic features corresponding to all speech style categories;

[0028] According to the first common acoustic feature and the second common acoustic feature, obtaining exclusive acoustic features corresponding to different speech style categories;

[0029] The exclusive acoustic features corresponding to the different speech style categories are respectively deployed as encoding knowledge into the style encoder to be trained to obtain a trained style encoder.

[0030] Furthermore, the step of obtaining the exclusive acoustic features corresponding to different speech style categories according to the first common acoustic features and the second common acoustic features specifically includes:

[0031] Step 701, sequentially screening different speech style categories as current speech style categories;

[0032] Step 702, deleting the second common acoustic feature from the first common acoustic feature corresponding to the current voice style category to obtain the exclusive acoustic feature of the current voice style category;

[0033] Step 703, repeating steps 701 to 702 until all speech style categories obtain corresponding exclusive acoustic features.

[0034] Furthermore, the step of selecting the exclusive acoustic features of the target style information from the pre-trained style encoder specifically includes:

[0035] Identifying the speech style category corresponding to the target style information;

[0036] According to the speech style category, selecting corresponding exclusive acoustic features from the pre-trained style encoder;

[0037] Using the exclusive acoustic feature as the exclusive acoustic feature of the target style information;

[0038] Before executing the step of inputting the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder to perform speech synthesis decoding processing to obtain the converted speech of the target style information, the method further includes:

[0039] deploying the second common acoustic feature as encoding knowledge into the speech decoder;

[0040] The step of inputting the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder, performing speech synthesis decoding processing, and obtaining the converted speech of the target style information specifically includes:

[0041] Performing preliminary synthesis processing on the speech text sequence according to the second common acoustic feature to obtain a preliminary synthesized speech signal;

[0042] Embedding the exclusive acoustic feature as an embedded feature into the preliminary synthesized speech signal to obtain a final synthesized speech signal;

[0043] The final synthesized speech signal is decoded to obtain the converted speech of the target style information.

[0044] In order to solve the above technical problems, the embodiment of the present application also provides a speech style conversion device, which adopts the following technical solution:

[0045] A speech style conversion device, comprising:

[0046] A voice conversion related acquisition module is used to obtain the source voice to be converted and the style prompt text;

[0047] A target style information parsing module, used for inputting the style hint text into a preset semantic parsing model to obtain target style information through semantic parsing;

[0048] A speech-to-text sequence acquisition module, used to convert the source speech into text and acquire a speech-to-text sequence of the source speech;

[0049] A dedicated acoustic feature screening module, used to screen out the dedicated acoustic features of the target style information from the pre-trained style encoder;

[0050] The speech synthesis decoding processing module is used to input the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder to perform speech synthesis decoding processing to obtain the converted speech of the target style information.

[0051] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0052] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned speech style conversion method when executing the computer-readable instructions.

[0053] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0054] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the voice style conversion method as described above.

[0055] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0056] The speech style conversion method described in the embodiment of the present application is as follows: the source speech to be speech converted and the style prompt text are obtained; the style prompt text is input into a preset semantic analysis model, and the target style information is obtained through semantic analysis; the source speech is converted into text to obtain the speech-to-text sequence of the source speech; the exclusive acoustic features of the target style information are screened out from the pre-trained style encoder; the speech-to-text sequence and the exclusive acoustic features are input into the pre-trained speech decoder, and speech synthesis decoding is performed to obtain the converted speech of the target style information. The speech conversion is combined with the style prompt text to convert the source speech into the converted speech of the target style information, which ensures that the style conversion result is more accurate and personalized. The application of this application to the field of virtual speech conversion in financial services can provide customers with speech services of different styles. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0059] Figure 2 is a flow chart of an embodiment of a method for voice style conversion according to the present application;

[0060] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 is shown;

[0061] Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 is shown;

[0062] Figure 5is a flowchart of a specific embodiment of pre-training a style encoder in the speech style conversion method described in the present application;

[0063] Figure 6 yes Figure 5 A flowchart of a specific embodiment of step 502 is shown;

[0064] Figure 7 yes Figure 6 A flowchart of a specific embodiment of step 604 is shown;

[0065] Figure 8 yes Figure 2 A flowchart of a specific embodiment of step 204 is shown;

[0066] Fig. 9 yes Figure 2 A flowchart of a specific embodiment of step 205 is shown;

[0067] Fig.10 is a structural schematic diagram of an embodiment of a speech style conversion device according to the present application;

[0068] Fig.11 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0070] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0071] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0072] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0073] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0074] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0075] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0076] It should be noted that the speech style conversion method provided in the embodiment of the present application is generally executed by a server, and accordingly, the speech style conversion device is generally arranged in the server.

[0077] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0078] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for voice style conversion according to the present application. The method for voice style conversion comprises the following steps:

[0079] Step 201, obtaining source speech to be converted into speech and style prompt text.

[0080] In this embodiment, the source voice includes the voice to be converted recorded by the user through the voice input device. The style prompt text includes a text that describes the voice style intention to be converted. The source voice can be a recitation content, a piece of music content, etc., and of course, it can also be a business conversation voice between the customer service and the customer. By converting the voice style of the source voice through the voice style conversion method described in this application, it is possible to provide customers with more personalized and style-diverse voice services in customer voice service scenarios.

[0081] Step 202: input the style hint text into a preset semantic parsing model, and obtain target style information through semantic parsing.

[0082] Specifically, the target style information can be obtained by semantically parsing the style hint text, so that when there are too many style patterns integrated in the speech conversion component, the target speech style can be accurately screened out for speech style conversion, thereby improving the accuracy of personalized speech conversion.

[0083] Step 203: perform text conversion on the source speech to obtain a speech-to-text sequence of the source speech.

[0084] The purpose of obtaining a speech-to-text sequence of the source speech by converting the source speech into text is to retain the speech-to-text content in the source speech so as to facilitate subsequent speech conversion processing based on the speech-to-text content.

[0085] Step 204 : Filter out the exclusive acoustic features of the target style information from the pre-trained style encoder.

[0086] By pre-training the style encoder, the exclusive acoustic features of different styles are retained in the style encoder in advance, which is convenient for the subsequent speech style conversion. Exclusive acoustic features can be directly combined with the target style information for screening, and the speech style conversion can be mapped to the screening and searching process, thereby improving the efficiency and accuracy of speech style conversion.

[0087] Step 205: input the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder, perform speech synthesis decoding processing, and obtain the converted speech of the target style information.

[0088] In this embodiment, the source speech to be converted into speech and the style hint text are obtained; the style hint text is input into a preset semantic parsing model, and the target style information is obtained through semantic parsing; the source speech is converted into text to obtain the speech-text sequence of the source speech; the exclusive acoustic features of the target style information are screened out from the pre-trained style encoder; the speech-text sequence and the exclusive acoustic features are input into the pre-trained speech decoder, and speech synthesis decoding is performed to obtain the converted speech of the target style information. The speech conversion is combined with the style hint text to convert the source speech into the converted speech of the target style information, which ensures that the style conversion result is more accurate and personalized. The application of this application to the field of virtual speech conversion in financial services can provide customers with speech services of different styles.

[0089] In this embodiment, the preset semantic parsing model includes a text semantic recognition model based on BERT.

[0090] Continue to refer Figure 3 , Figure 3 yes Figure 2 The flowchart of a specific embodiment of step 202 shown includes the following steps:

[0091] Step 301, inputting the style hint text into the BERT-based text semantic recognition model;

[0092] In this embodiment, the BERT-based text semantic recognition model is a pre-trained text semantic recognition model, which can at least recognize the speech style category to be converted based on the style hint text.

[0093] Step 302, outputting all candidate style information matched by the style prompt text by semantically parsing the style prompt text;

[0094] Step 303, feeding back all the candidate style information to the input acquisition interface corresponding to the style prompt text;

[0095] In this embodiment, in order to avoid ambiguity in the style prompt text input by the user, a corresponding feedback interface is provided for the user to make independent selection when non-unique style information appears, which is more in line with humanized needs.

[0096] Step 304 : identifying target style information from all the candidate style information based on the confirmation instruction of the input acquisition interface.

[0097] Specifically, the target style information is identified according to the style information finally confirmed by the user among all the candidate style information, and a style confirmation is performed based on the style prompt text input by the user, thereby ensuring the accuracy of the speech style conversion.

[0098] Continue to refer Figure 4 , Figure 4 yes Figure 2 The flowchart of a specific embodiment of step 203 shown includes the following steps:

[0099] Step 401, using speech recognition technology or speech-to-text technology to convert the source speech into text, and obtaining all text contents after the source speech is converted;

[0100] In this embodiment, the speech recognition technology or speech-to-text technology is used to convert the source speech into text to obtain all text content after the source speech is converted. A speech recognition plug-in provided by a third-party platform can be used, or a self-developed speech recognition plug-in can be used.

[0101] Step 402: serialize all the text contents according to the order of pronunciation of all the text contents in the source speech to obtain a speech-text sequence of the source speech.

[0102] By combining the pronunciation order of all text contents in the source voice, all text contents are serialized so as to ensure that the voice-text output order is consistent with the source voice during subsequent voice style conversion.

[0103] Continue to refer Figure 5 In some optional implementations, a step of pre-training the style encoder is also included before step 204. Figure 5 This is a flowchart of a specific embodiment of pre-training a style encoder in the speech style conversion method described in the present application, comprising the following steps:

[0104] Step 501, obtaining batches of reference speech of different speech style categories respectively according to the preset speech style categories;

[0105] In this embodiment, the pre-set speech style categories, such as conversation style, recitation style, reading style, etc., also include style types customized according to differences in acoustic features.

[0106] Step 502: using batches of reference speech of different speech style categories as training samples, and inputting them into a style encoder to be trained, to obtain a trained style encoder, wherein the style encoder to be trained includes a style encoder based on a GSTs (Global Style Tokens) marking method;

[0107] Step 503: Use the trained style encoder as the pre-trained style encoder.

[0108] By pre-training the style encoder, the exclusive acoustic features of different styles are retained in the style encoder in advance, which is convenient for the subsequent speech style conversion. Exclusive acoustic features can be directly combined with the target style information for screening, and the speech style conversion can be mapped to the screening and searching process, thereby improving the efficiency and accuracy of speech style conversion.

[0109] Continue to refer Figure 6 , Figure 6 yes Figure 5 The flowchart of a specific embodiment of step 502 shown includes the following steps:

[0110] Step 601, using the style encoder based on the GSTs tagging method to extract acoustic features from the batch reference speech in the training sample;

[0111] Step 602, clustering the acoustic feature extraction results of different speech style categories according to the speech style categories corresponding to the batch of reference speech;

[0112] Step 603, obtaining, according to the clustering processing result, the first common acoustic features corresponding to the same speech style category and the second common acoustic features corresponding to all speech style categories;

[0113] Specifically, assuming that after the clustering processing in step 602, clustering processing results corresponding to 10 speech style categories are obtained, that is, 10 acoustic feature clustering processing results are obtained. At this time, the acoustic features in the same clustering processing result are screened for common acoustic features to obtain the first common acoustic features. For example, the acoustic features possessed by more than 80% of the reference speeches in the same clustering processing result are set as the first common acoustic features; and the second common acoustic features corresponding to all speech style categories are screened out as the second common acoustic features by performing a secondary screening on the first common acoustic features corresponding to all clustering processing results.

[0114] Step 604, obtaining exclusive acoustic features corresponding to different speech style categories according to the first common acoustic features and the second common acoustic features;

[0115] It should be understood that the first common acoustic feature and the second common acoustic feature include audio waveforms of common phase and frequency in the Mel spectrum.

[0116] Step 605: deploy the exclusive acoustic features corresponding to the different speech style categories as coding knowledge into the style encoder to be trained to obtain a trained style encoder.

[0117] Correspondingly, the exclusive acoustic features corresponding to the different speech style categories include exclusive audio waveforms corresponding to the different speech style categories.

[0118] Continue to refer Figure 7 , Figure 7 yes Figure 6 The flowchart of a specific embodiment of step 604 shown includes the following steps:

[0119] Step 701, sequentially screening different speech style categories as current speech style categories;

[0120] Step 702, deleting the second common acoustic feature from the first common acoustic feature corresponding to the current voice style category to obtain the exclusive acoustic feature of the current voice style category;

[0121] Step 703, repeating steps 701 to 702 until all speech style categories obtain corresponding exclusive acoustic features.

[0122] By pre-training the style encoder, the exclusive acoustic features corresponding to the different speech style categories are deployed as encoding knowledge in the style encoder to be trained, so that the exclusive acoustic features of different styles are pre-retained in the style encoder, which is convenient for the subsequent speech style conversion. Exclusive acoustic features can be directly combined with the target style information for screening, and the speech style conversion is mapped to the screening and searching process, thereby improving the efficiency and accuracy of speech style conversion.

[0123] Continue to refer Figure 8 , Figure 8 yes Figure 2 The flowchart of a specific embodiment of step 204 shown includes the following steps:

[0124] Step 801, identifying the speech style category corresponding to the target style information;

[0125] Step 802, according to the speech style category, selecting corresponding exclusive acoustic features from the pre-trained style encoder;

[0126] Step 803: Use the exclusive acoustic feature as the exclusive acoustic feature of the target style information.

[0127] In this embodiment, before executing the step of inputting the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder, performing speech synthesis decoding processing, and obtaining the converted speech of the target style information, the method also includes: deploying the second common acoustic features as encoding knowledge into the speech decoder.

[0128] By deploying the second common acoustic feature as coding knowledge into the speech decoder, the speech decoder includes universal common acoustic features, which is convenient for subsequent speech synthesis.

[0129] Continue to refer Fig. 9 , Fig. 9 yes Figure 2 The flowchart of a specific embodiment of step 205 shown includes the following steps:

[0130] Step 901, performing preliminary synthesis processing on the speech text sequence according to the second common acoustic feature to obtain a preliminary synthesized speech signal;

[0131] Step 902, embedding the exclusive acoustic feature as an embedded feature into the preliminary synthesized speech signal to obtain a final synthesized speech signal;

[0132] Step 903: Decode the final synthesized speech signal to obtain the converted speech of the target style information.

[0133] Specifically, the universal second common acoustic features are first used to perform preliminary synthesis of the speech signal. Then, the exclusive acoustic features corresponding to the target speech style category are used as embedded features and embedded into the preliminary synthesized speech signal to obtain the final synthesized speech signal. Finally, decoding processing is performed to obtain the converted speech of the target style information.

[0134] This application obtains the source speech to be converted and the style hint text; inputs the style hint text into a preset semantic parsing model, and obtains the target style information through semantic parsing; performs text conversion on the source speech to obtain the speech-to-text sequence of the source speech; selects the exclusive acoustic features of the target style information from the pre-trained style encoder; inputs the speech-to-text sequence and the exclusive acoustic features into the pre-trained speech decoder, performs speech synthesis decoding, and obtains the converted speech of the target style information. The speech conversion is combined with the style hint text to convert the source speech into the converted speech of the target style information, ensuring that the style conversion result is more accurate and personalized. This application is applied to the field of virtual speech conversion in financial services, and can provide customers with speech services of different styles.

[0135] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0136] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0137] In an embodiment of the present application, a source speech to be converted into speech and a style hint text are obtained; the style hint text is input into a preset semantic parsing model, and the target style information is obtained through semantic parsing; the source speech is converted into text to obtain a speech-to-text sequence of the source speech; the exclusive acoustic features of the target style information are screened out from a pre-trained style encoder; the speech-to-text sequence and the exclusive acoustic features are input into a pre-trained speech decoder, and speech synthesis decoding is performed to obtain the converted speech of the target style information. The speech conversion is performed in combination with the style hint text, and the source speech is converted into the converted speech of the target style information, which ensures that the style conversion result is more accurate and personalized. The present application is applied to the field of virtual speech conversion in financial services, and can provide customers with speech services of different styles.

[0138] Further references Fig.10 , as a response to the above Figure 2 The present application provides an embodiment of a speech style conversion device, and the device embodiment is similar to Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0139] like Fig.10 As shown, the speech style conversion device 10 described in this embodiment includes: a speech conversion related acquisition module 10a, a target style information analysis module 10b, a speech text sequence acquisition module 10c, a dedicated acoustic feature screening module 10d and a speech synthesis decoding processing module 10e. Among them:

[0140] A voice conversion related acquisition module 10a is used to acquire the source voice to be converted and the style prompt text;

[0141] A target style information parsing module 10b is used to input the style hint text into a preset semantic parsing model and obtain the target style information through semantic parsing;

[0142] A speech-to-text sequence acquisition module 10c is used to convert the source speech into text to obtain a speech-to-text sequence of the source speech;

[0143] A dedicated acoustic feature screening module 10d, configured to screen out dedicated acoustic features of the target style information from the pre-trained style encoder;

[0144] The speech synthesis decoding processing module 10e is used to input the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder to perform speech synthesis decoding processing to obtain the converted speech of the target style information.

[0145] This application obtains the source speech to be converted and the style hint text; inputs the style hint text into a preset semantic parsing model, and obtains the target style information through semantic parsing; performs text conversion on the source speech to obtain the speech-to-text sequence of the source speech; selects the exclusive acoustic features of the target style information from the pre-trained style encoder; inputs the speech-to-text sequence and the exclusive acoustic features into the pre-trained speech decoder, performs speech synthesis decoding, and obtains the converted speech of the target style information. The speech conversion is combined with the style hint text to convert the source speech into the converted speech of the target style information, ensuring that the style conversion result is more accurate and personalized. This application is applied to the field of virtual speech conversion in financial services, and can provide customers with speech services of different styles.

[0146] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0147] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0148] To solve the above technical problems, the present application also provides a computer device. Fig.11 , Fig.11 This is a basic structural block diagram of the computer device in this embodiment.

[0149] The computer device 11 includes a memory 11a, a processor 11b, and a network interface 11c which are interconnected through a system bus. It should be noted that: Fig.11 Only a computer device 11 having a component memory 11a, a processor 11b, and a network interface 11c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0150] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0151] The memory 11a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11a can be an internal storage unit of the computer device 11, such as a hard disk or memory of the computer device 11. In other embodiments, the memory 11a can also be an external storage device of the computer device 11, such as a plug-in hard disk, smart memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card), etc. equipped on the computer device 11. Of course, the memory 11a can also include both the internal storage unit of the computer device 11 and its external storage device. In this embodiment, the memory 11a is generally used to store the operating system and various application software installed on the computer device 11, such as a computer-readable instruction of a speech style conversion method, etc. In addition, the memory 11a can also be used to temporarily store various types of data that have been output or are to be output.

[0152] The processor 11b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 11b is generally used to control the overall operation of the computer device 11. In this embodiment, the processor 11b is used to run the computer-readable instructions stored in the memory 11a or process data, such as computer-readable instructions for running the speech style conversion method.

[0153] The network interface 11c may include a wireless network interface or a wired network interface. The network interface 11c is generally used to establish a communication connection between the computer device 11 and other electronic devices.

[0154] The computer device proposed in this embodiment belongs to the field of R&D design and audio processing technology, and is applied to conversion processing scenarios of different styles of speech. This application obtains the source speech to be converted and the style prompt text; inputs the style prompt text into a preset semantic parsing model, and obtains the target style information through semantic parsing; performs text conversion processing on the source speech to obtain the speech-to-text sequence of the source speech; selects the exclusive acoustic features of the target style information from the pre-trained style encoder; inputs the speech-to-text sequence and the exclusive acoustic features into the pre-trained speech decoder, performs speech synthesis decoding processing, and obtains the converted speech of the target style information. The style prompt text is combined for speech conversion, and the source speech is converted into the converted speech of the target style information, which ensures that the style conversion result is more accurate and personalized. This application is applied to the field of virtual speech conversion in financial services, and can provide customers with voice services of different styles.

[0155] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the speech style conversion method as described above.

[0156] The computer-readable storage medium proposed in this embodiment belongs to the field of research and development design and audio processing technology, and is applied to the conversion processing scenarios of different styles of speech. This application obtains the source speech to be converted and the style prompt text; inputs the style prompt text into a preset semantic parsing model, and obtains the target style information through semantic parsing; converts the source speech into text to obtain the speech-to-text sequence of the source speech; selects the exclusive acoustic features of the target style information from the pre-trained style encoder; inputs the speech-to-text sequence and the exclusive acoustic features into the pre-trained speech decoder, performs speech synthesis decoding processing, and obtains the converted speech of the target style information. The style prompt text is combined for speech conversion, and the source speech is converted into the converted speech of the target style information, which ensures that the style conversion result is more accurate and personalized. This application is applied to the field of virtual speech conversion in financial services, and can provide customers with voice services of different styles.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0158] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for voice style conversion, characterized in that: The steps include: Obtaining source speech and style prompt text to be converted into speech; Inputting the style hint text into a preset semantic parsing model to obtain target style information through semantic parsing; Performing text conversion on the source speech to obtain a speech-to-text sequence of the source speech; Filtering the exclusive acoustic features of the target style information from the pre-trained style encoder; The speech text sequence and the exclusive acoustic features are input into a pre-trained speech decoder, and speech synthesis decoding processing is performed to obtain the converted speech of the target style information.

2. The method for voice style conversion according to claim 1, characterized in that: The preset semantic parsing model includes a text semantic recognition model based on BERT. The step of inputting the style hint text into the preset semantic parsing model and obtaining the target style information through semantic parsing specifically includes: Inputting the style hint text into the BERT-based text semantic recognition model; By performing semantic analysis on the style hint text, outputting all candidate style information hit by the style hint text; Feeding back all the candidate style information to the input acquisition interface corresponding to the style prompt text; Based on the confirmation instruction of the input acquisition interface, target style information is identified from all the candidate style information.

3. The method for voice style conversion according to claim 1, characterized in that: The step of converting the source speech into text to obtain a speech-to-text sequence of the source speech specifically includes: Using speech recognition technology or speech-to-text technology to convert the source speech into text, and obtain all text contents after the source speech is converted; According to the pronunciation order of all the text contents in the source speech, all the text contents are serialized to obtain a speech-text sequence of the source speech.

4. The method for voice style conversion according to any one of claims 1 to 3, characterized in that: Before performing the step of filtering out the exclusive acoustic features of the target style information from the pre-trained style encoder, the method further includes: According to the preset speech style categories, batch reference speech of different speech style categories are obtained respectively; Using batches of reference speech of different speech style categories as training samples, and inputting them into a style encoder to be trained, to obtain a trained style encoder, wherein the style encoder to be trained includes a style encoder based on a GSTs tagging method; The trained style encoder is used as the pre-trained style encoder.

5. The method for voice style conversion according to claim 4, characterized in that: The step of inputting batches of reference speech of different speech style categories as training samples into the style encoder to be trained to obtain a trained style encoder specifically includes: Using the style encoder based on the GSTs tagging method to extract acoustic features from the batch reference speech in the training sample; According to the speech style categories respectively corresponding to the batch of reference speech, clustering processing is performed on the acoustic feature extraction results of different speech style categories; According to the clustering processing result, obtaining the first common acoustic features corresponding to the same speech style category and the second common acoustic features corresponding to all speech style categories; According to the first common acoustic feature and the second common acoustic feature, obtaining exclusive acoustic features corresponding to different speech style categories; The exclusive acoustic features corresponding to the different speech style categories are respectively deployed as encoding knowledge into the style encoder to be trained to obtain a trained style encoder.

6. The method for voice style conversion according to claim 5, characterized in that: The step of obtaining the exclusive acoustic features corresponding to different speech style categories according to the first common acoustic features and the second common acoustic features specifically includes: Step 701, sequentially screening different speech style categories as current speech style categories; Step 702, deleting the second common acoustic feature from the first common acoustic feature corresponding to the current voice style category to obtain the exclusive acoustic feature of the current voice style category; Step 703, repeating steps 701 to 702 until all speech style categories obtain corresponding exclusive acoustic features.

7. The method for voice style conversion according to claim 5 or 6, characterized in that: The step of selecting the exclusive acoustic features of the target style information from the pre-trained style encoder specifically includes: Identifying the speech style category corresponding to the target style information; According to the speech style category, selecting corresponding exclusive acoustic features from the pre-trained style encoder; Using the exclusive acoustic feature as the exclusive acoustic feature of the target style information; Before executing the step of inputting the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder to perform speech synthesis decoding processing to obtain the converted speech of the target style information, the method further includes: deploying the second common acoustic feature as encoding knowledge into the speech decoder; The step of inputting the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder, performing speech synthesis decoding processing, and obtaining the converted speech of the target style information specifically includes: Performing preliminary synthesis processing on the speech text sequence according to the second common acoustic feature to obtain a preliminary synthesized speech signal; Embedding the exclusive acoustic feature as an embedded feature into the preliminary synthesized speech signal to obtain a final synthesized speech signal; The final synthesized speech signal is decoded to obtain the converted speech of the target style information.

8. A speech style conversion device, characterized in that: include: A voice conversion related acquisition module is used to obtain the source voice to be converted and the style prompt text; A target style information parsing module, used for inputting the style hint text into a preset semantic parsing model to obtain target style information through semantic parsing; A speech-to-text sequence acquisition module, used to convert the source speech into text and acquire a speech-to-text sequence of the source speech; A dedicated acoustic feature screening module, used to screen out the dedicated acoustic features of the target style information from the pre-trained style encoder; The speech synthesis decoding processing module is used to input the speech text sequence and the exclusive acoustic features into a pre-trained speech decoder to perform speech synthesis decoding processing to obtain the converted speech of the target style information.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the speech style conversion method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the speech style conversion method according to any one of claims 1 to 7 are implemented.

Images