A speech synthesis method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the field of data processing technology, and provides a speech synthesis method, apparatus, terminal device and storage medium, the method comprising: first obtaining the acoustic features of the text to be processed, processing the acoustic features using a parallel computing model, and obtaining the first audio information of the text to be processed, the first audio information including the audio corresponding to each sampling moment; processing the acoustic features and the first audio information using a serial computing model, and obtaining the residual value of the audio corresponding to each sampling moment; based on the first audio information and the residual value corresponding to each sampling moment, obtaining the synthesized audio of the text to be processed. The present application first processes the text using a parallel computing model, and then processes the first audio information using a serial computing model to obtain the residual value, and obtains the final audio information using the residual value and the first audio information. The present application processes the text faster and ensures the quality of the synthesized audio information.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a speech synthesis method, apparatus, terminal device and storage medium. Background Art

[0002] Speech synthesis is a technology that produces artificial speech through mechanical and electronic methods. Speech synthesis is a technology that converts text information generated by the computer itself or input from the outside into understandable and fluent speech information output.

[0003] At present, autoregressive models or non-autoregressive models are mostly used for speech synthesis. The non-autoregressive model synthesizes speech quickly, but the synthesized audio quality is poor. Although the audio quality synthesized by the autoregressive model is higher than that synthesized by the non-autoregressive model, the speed of the autoregressive model synthesizing speech is slower than that of the non-autoregressive model. Therefore, a speech synthesis method with fast speed and high audio quality is needed. Summary of the invention

[0004] The embodiments of the present application provide a speech synthesis method, apparatus, terminal device and storage medium, which can solve the problems of slow speech synthesis speed and poor audio quality.

[0005] In a first aspect, an embodiment of the present application provides a speech synthesis method, comprising:

[0006] Obtaining acoustic features of the text to be processed;

[0007] Processing the acoustic features using a parallel computing model to obtain first audio information of the text to be processed, wherein the first audio information includes audio corresponding to each sampling moment;

[0008] Processing the acoustic features and the first audio information using a serial computing model to obtain residual values ​​of the first audio information corresponding to each sampling moment;

[0009] Based on the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment, the second audio information corresponding to the i-th sampling moment is obtained, and the synthesized audio of the text to be processed includes each second audio information, where i=1, 2...n, and n is the total number of the sampling moments.

[0010] In a second aspect, an embodiment of the present application provides a speech synthesis device, including:

[0011] A data acquisition module, used to acquire acoustic features of the text to be processed;

[0012] A first model processing module, used for processing the acoustic features by using a parallel computing model to obtain first audio information of the text to be processed, wherein the first audio information includes audio corresponding to each sampling moment;

[0013] A second model processing module, used for processing the acoustic features and the first audio information by using a serial calculation model to obtain residual values ​​of the first audio information corresponding to each sampling moment;

[0014] An audio generation module is used to obtain second audio information corresponding to the i-th sampling moment based on the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment. The synthesized audio of the text to be processed includes each second audio information, wherein i=1, 2...n, and n is the total number of the sampling moments.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the speech synthesis method described in any one of the first aspects above when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program, when executed by a processor, implements the speech synthesis method described in any one of the first aspects above.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the speech synthesis method described in any one of the above-mentioned first aspects.

[0018] The beneficial effects of the first aspect embodiment of the present application compared with the prior art are: the present application first obtains the acoustic features of the text to be processed, processes the acoustic features using a parallel computing model, and obtains the first audio information of the text to be processed, the first audio information includes the audio corresponding to each sampling moment; the preliminary converted audio of the text to be processed is obtained using the parallel computing model, and the text to be processed is processed using the parallel computing model. Compared with the existing process of processing text using only a serial computing model, the present application is faster.

[0019] The acoustic features and the first audio information are processed using a serial computing model to obtain the residual value of the first audio information corresponding to each sampling moment; based on the first audio information and the residual value corresponding to each sampling moment, the synthesized audio of the text to be processed is obtained. This application will use a serial computing model to process the first audio information to obtain the residual value, and use the residual value and the first audio information to obtain the final audio information, so that the quality of the obtained audio information can be better than the existing audio information obtained only using a parallel computing model.

[0020] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 This is a schematic diagram of an application scenario of a speech synthesis method provided by an embodiment of the present application;

[0023] Figure 2 is a flowchart of a speech synthesis method provided by an embodiment of the present application;

[0024] Figure 3 is a flowchart of a method for calculating a residual value provided in an embodiment of the present application;

[0025] Figure 4 is a structural schematic diagram of a speech synthesis device provided in an embodiment of the present application;

[0026] Figure 5 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0028] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0029] It should also be understood that the term “and / or” used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0030] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0032] Currently, vocoders include parallel neural network models and autoregressive serial models. The parallel neural network model has a fast processing speed, but the audio quality is poor. The autoregressive serial model has a slow processing speed, but the audio quality is good.

[0033] This application first uses a parallel neural network model to process the text to obtain an audio with slightly poor quality, and then uses an autoregressive serial model to obtain the residual of the audio, and uses the residual to obtain audio with good quality. This application combines a parallel neural network model with an autoregressive serial model, and the quality of the audio generated is better than that generated by using only a parallel neural network model, and the speed of the audio generated is faster than that generated by using only an autoregressive serial model.

[0034] Specifically, the processing principle of text information in this application is as follows:

[0035]

[0036] Where X = [x1, x2, … x n-1 ,x n ] is the audio to be synthesized; is the estimated value of X predicted by the parallel model; m is the acoustic feature; is the conditional probability of taking the value of the i-th sampling point with only m as known, 0≤i≤n, based on the conditional independence assumption; Can be processed in parallel. Finally, sampling according to probability can obtain the estimated value of the audio, that is, the first audio information. The first audio information is obtained by using a parallel computing model.

[0037]

[0038] in, is the second audio information obtained by using the serial calculation model. (x [1:t-1] ,m) is the second audio information of the t-1th sampling point. The first audio information is a text message; is the probability of taking the value of the tth sampling point.

[0039] Figure 1 The present invention is a schematic diagram of an application scenario of the speech synthesis method provided in an embodiment of the present invention, wherein the speech synthesis method can be used to convert text information into speech information. The text generation device 10 is used to generate text information, and the speech synthesis device 20 is used to obtain text information from the text generation device 10, and process the text information through a parallel computing model and a serial computing model to obtain audio information.

[0040] Figure 2 A schematic flow chart of the speech synthesis method provided by the present application is shown, referring to Figure 2 , the method is described in detail as follows:

[0041] S101, obtaining acoustic features of the text to be processed.

[0042] In this embodiment, the electronic device can obtain the acoustic features of the text to be processed from an external device. The electronic device can also obtain the text to be processed from an external device, and the electronic device extracts the acoustic features from the obtained text to be processed. The electronic device can also obtain information input by the user, and generate the text to be processed according to the information input by the user. The electronic device can be a vocoder, a computer, etc.

[0043] In this embodiment, the electronic device can obtain the acoustic features of the text to be processed by using an acoustic feature extraction model. The acoustic feature extraction model can be a convolutional neural network model, a recursive neural network model, etc.

[0044] In this embodiment, the acoustic feature may include a Mel spectrum or a Mel cepstrum.

[0045] S102: Process the acoustic features using a parallel computing model to obtain first audio information of the text to be processed.

[0046] In this embodiment, the first audio information includes audio corresponding to each sampling moment. The sampling moments are equally spaced moments. The first audio information may be a sequence of audio sampling points of equal length.

[0047] In this embodiment, the parallel computing model may be a parallel neural network model, such as WaveGAN and WaveGlow.

[0048] S103: Process the acoustic features and the first audio information using a serial calculation model to obtain residual values ​​of the first audio information corresponding to each sampling moment.

[0049] In this embodiment, the serial computing model can be an autoregressive serial model, such as Wave Net and WaveRNN. Since the serial computing model is mainly used to calculate the residual value, the structure of the serial computing model is relatively simple and the processing speed is relatively fast. The serial computing model processes data step by step, and each step of the serial computing model requires the processing result of the previous step.

[0050] In this embodiment, the residual refers to the difference between the actual observed value and the estimated value (fitted value), and the residual can be regarded as the observed value of the error.

[0051] S104, based on the first audio information corresponding to the ith sampling moment and the residual value corresponding to the ith sampling moment, obtain the second audio information corresponding to the ith sampling moment, and the synthesized audio of the text to be processed includes each second audio information.

[0052] In this embodiment, i=1, 2...n, where n is the total number of sampling moments. The synthesized audio of the text to be processed includes n second audio information.

[0053] In this embodiment, after the second audio information is obtained, the second audio information may be sent to an audio playback device, and the second audio information may be played through the audio playback device.

[0054] In an embodiment of the present application, the acoustic features of the text to be processed are first obtained, and the acoustic features are processed using a parallel computing model to obtain the first audio information of the text to be processed, the first audio information including the audio corresponding to each sampling moment; the preliminary converted audio of the text to be processed is obtained using the parallel computing model, and the processing of the text to be processed using the parallel computing model is faster than the processing process using the serial computing model. The acoustic features and the first audio information are processed using a serial computing model to obtain the residual values ​​of the audio corresponding to each sampling moment; based on the first audio information and the residual values ​​corresponding to each sampling moment, the synthesized audio of the text to be processed is obtained. The present application uses a serial computing model to process the first audio information to obtain the residual value, and uses the residual value and the first audio information to obtain the final audio information. The audio information obtained by the present application is of better quality than the existing audio information obtained only using a parallel computing model.

[0055] like Figure 3 As shown, in a possible implementation manner, the implementation process of step S103 may include:

[0056] S1031, using a serial calculation model to process the first audio information corresponding to the first sampling moment, the acoustic features corresponding to the first sampling moment, and a preset residual value to obtain the residual value of the first audio information corresponding to the first sampling moment.

[0057] In this embodiment, the preset residual value can be set as needed, for example, the preset residual value can be set to 0, 1 or 2, etc.

[0058] Specifically, the first audio information corresponding to the first sampling moment, the acoustic features corresponding to the first sampling moment, and the preset residual value are input into the serial calculation model to obtain the residual value of the first audio information corresponding to the first sampling moment.

[0059] S1032, using a serial calculation model to process the first audio information corresponding to the j-th sampling moment, the acoustic features corresponding to the j-th sampling moment, and the residual value corresponding to the j-1-th sampling moment, to obtain the residual value of the first audio information corresponding to the j-th sampling moment.

[0060] In this embodiment, j=2, 3, ... n.

[0061] As an example, when j=3, the serial calculation model is used to process the first audio information corresponding to the third sampling moment, the acoustic features corresponding to the third sampling moment, and the residual value corresponding to the second sampling moment to obtain the residual value of the audio corresponding to the third sampling moment.

[0062] Specifically, the first audio information corresponding to the jth sampling moment, the acoustic feature corresponding to the jth sampling moment, and the residual value corresponding to the j-1th sampling moment are input into the serial calculation model to obtain the residual value of the first audio information corresponding to the jth sampling moment.

[0063] In the embodiment of the present application, the residual value at the previous sampling moment is used to infer the residual value at the current sampling moment, so that the residual value at the current moment can be more accurate.

[0064] In a possible implementation, the implementation process of step S104 may include:

[0065] The sum of the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment is calculated. The sum of the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment is used as the second audio information corresponding to the i-th sampling moment.

[0066] Specifically, the second audio information may be calculated by an audio calculation model. The audio calculation model includes:

[0067] T i =t i +c i ;

[0068] Among them, T i is the second audio information corresponding to the i-th sampling time, t i is the first audio information corresponding to the i-th sampling moment, c i is the residual value corresponding to the i-th sampling moment.

[0069] In a possible implementation, after step S101, the method may further include:

[0070] The acoustic features are sampled to obtain processed acoustic features.

[0071] In this embodiment, the sampling process includes upsampling and downsampling. Upsampling refers to the process of interpolating information of one type of pixel based on information of another type of pixel. Downsampling is a multi-rate digital signal processing technology or a process of reducing the signal sampling rate, which is usually used to reduce the data transmission rate or data size.

[0072] In this embodiment, the processed acoustic features are processed using a parallel computing model to obtain the first audio information of the text to be processed.

[0073] Specifically, if the sampling rate of the acoustic feature is lower than the preset sampling rate of the synthesized audio of the text to be processed, the acoustic feature is upsampled based on the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed to obtain the processed acoustic feature.

[0074] In this embodiment, the sampling rate of the synthesized audio of the text to be processed can be set as needed. The sampling rate of the acoustic feature can also be set as needed. Specifically, the acoustic feature is sampled according to a preset time window.

[0075] In this embodiment, if the sampling rate of the acoustic feature is lower than the preset sampling rate of the synthesized audio of the text to be processed, the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed is calculated, and upsampling is performed according to the ratio.

[0076] Specifically, if the sampling rate of the acoustic feature is greater than the preset sampling rate of the synthesized audio of the text to be processed, the acoustic feature is downsampled based on the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed to obtain the processed acoustic feature.

[0077] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0078] Corresponding to the speech synthesis method described in the above embodiment, Figure 4 A structural block diagram of a speech synthesis device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0079] Reference Figure 4 The device 200 may include: a data acquisition module 210, a first model processing module 220, a second model processing module 230 and an audio generation module 240.

[0080] The data acquisition module 210 is used to acquire acoustic features of the text to be processed;

[0081] A first model processing module 220 is used to process the acoustic features using a parallel computing model to obtain first audio information of the text to be processed, wherein the first audio information includes audio corresponding to each sampling moment;

[0082] A second model processing module 230, configured to process the acoustic features and the first audio information using a serial calculation model to obtain residual values ​​of the first audio information corresponding to each sampling moment;

[0083] The audio generation module 240 is used to obtain the second audio information corresponding to the i-th sampling moment based on the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment. The synthesized audio of the text to be processed includes each second audio information, where i=1, 2...n, and n is the total number of the sampling moments.

[0084] In a possible implementation, the data acquisition module 210 is also connected to:

[0085] A sampling module, used for sampling and processing the acoustic features to obtain processed acoustic features;

[0086] Accordingly, the first model processing module 220 is further used for:

[0087] The processed acoustic features are processed using a parallel computing model to obtain first audio information of the text to be processed.

[0088] In a possible implementation, the sampling module may be specifically used for:

[0089] If the sampling rate of the acoustic feature is lower than the preset sampling rate of the synthesized audio of the text to be processed, the acoustic feature is upsampled based on the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed to obtain the processed acoustic feature.

[0090] In a possible implementation, the second model processing module 230 may be specifically used to:

[0091] Using a serial calculation model, the first audio information corresponding to the first sampling moment, the acoustic feature corresponding to the first sampling moment, and a preset residual value are processed to obtain a residual value corresponding to the first sampling moment;

[0092] The first audio information corresponding to the jth sampling moment, the acoustic features corresponding to the jth sampling moment, and the residual value corresponding to the j-1th sampling moment are processed using a serial calculation model to obtain the residual value of the first audio information corresponding to the jth sampling moment, where j=2, 3...n.

[0093] In a possible implementation, the audio generation module 240 may be specifically used for:

[0094] Calculating the sum of the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment;

[0095] The sum of the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment is used as the second audio information corresponding to the i-th sampling moment.

[0096] In a possible implementation, the data acquisition module 210 may be specifically used for:

[0097] The text to be processed is input into an acoustic feature extraction model to obtain the acoustic features of the text to be processed.

[0098] In a possible implementation, the sampling module may be specifically used for:

[0099] If the sampling rate of the acoustic feature is greater than the preset sampling rate of the synthesized audio of the text to be processed, the acoustic feature is downsampled based on the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed to obtain the processed acoustic feature.

[0100] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0101] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0102] The present application also provides a terminal device, see Figure 5 The terminal device 400 may include: at least one processor 410, a memory 420, and a computer program stored in the memory 420 and executable on the at least one processor 410. When the processor 410 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented, for example Figure 2 Steps S101 to S104 in the illustrated embodiment. Alternatively, when the processor 410 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 4 The functions of modules 210 to 240 are shown.

[0103] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 420, and executed by the processor 410 to complete the present application. The one or more modules / units may be a series of computer program segments capable of completing specific functions, and the program segments are used to describe the execution process of the computer program in the terminal device 400.

[0104] Those skilled in the art will understand that Figure 5 It is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, buses, etc.

[0105] The processor 410 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0106] The memory 420 may be an internal storage unit of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 420 is used to store the computer program and other programs and data required by the terminal device. The memory 420 may also be used to temporarily store data that has been output or is to be output.

[0107] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0108] The speech synthesis method provided in the embodiment of the present application can be applied to terminal devices such as computers, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0109] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0110] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0111] In the embodiments provided in the present application, it should be understood that the disclosed terminal equipment, devices and methods can be implemented in other ways. For example, the terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by one or more processors, the steps of each of the above-mentioned method embodiments can be implemented.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by one or more processors, the steps of each of the above-mentioned method embodiments can be implemented.

[0116] Similarly, as a computer program product, when the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0117] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0118] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A speech synthesis method, characterized in that: include: Obtaining acoustic features of the text to be processed; Processing the acoustic features using a parallel computing model to obtain first audio information of the text to be processed, wherein the first audio information includes audio corresponding to each sampling moment; Processing the acoustic features and the first audio information using a serial computing model to obtain residual values ​​of the first audio information corresponding to each sampling moment; Based on the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment, obtaining the second audio information corresponding to the i-th sampling moment, the synthesized audio of the text to be processed includes each second audio information, wherein i=1, 2...n, n is the total number of the sampling moments; The using of a serial computing model to process the acoustic features and the first audio information to obtain residual values ​​of the first audio information corresponding to each sampling moment includes: Using a serial calculation model, the first audio information corresponding to the first sampling moment, the acoustic feature corresponding to the first sampling moment, and a preset residual value are processed to obtain the residual value of the first audio information corresponding to the first sampling moment; The first audio information corresponding to the jth sampling moment, the acoustic features corresponding to the jth sampling moment, and the residual value corresponding to the j-1th sampling moment are processed using a serial calculation model to obtain the residual value of the first audio information corresponding to the jth sampling moment, where j=2, 3...n.

2. The speech synthesis method according to claim 1, characterized in that: After obtaining the acoustic features of the text to be processed, the following steps are included: Performing sampling processing on the acoustic features to obtain processed acoustic features; Accordingly, the acoustic features are processed using a parallel computing model to obtain the first audio information of the text to be processed, including: The processed acoustic features are processed using a parallel computing model to obtain first audio information of the text to be processed.

3. The speech synthesis method according to claim 2, characterized in that: The sampling and processing of the acoustic features to obtain processed acoustic features includes: If the sampling rate of the acoustic feature is lower than the preset sampling rate of the synthesized audio of the text to be processed, the acoustic feature is upsampled based on the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed to obtain the processed acoustic feature.

4. The speech synthesis method according to any one of claims 1 to 3, characterized in that: The obtaining, based on the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment, the second audio information corresponding to the i-th sampling moment includes: Calculating the sum of the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment; The sum of the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment is used as the second audio information corresponding to the i-th sampling moment.

5. The speech synthesis method according to claim 1, wherein: The step of obtaining the acoustic features of the text to be processed includes: The text to be processed is input into an acoustic feature extraction model to obtain the acoustic features of the text to be processed.

6. The speech synthesis method according to claim 2, characterized in that: The sampling and processing of the acoustic features to obtain processed acoustic features includes: If the sampling rate of the acoustic feature is greater than the preset sampling rate of the synthesized audio of the text to be processed, the acoustic feature is downsampled based on the ratio of the sampling rate of the acoustic feature to the sampling rate of the synthesized audio of the text to be processed to obtain the processed acoustic feature.

7. A speech synthesis device, characterized in that: include: A data acquisition module, used to acquire acoustic features of the text to be processed; A first model processing module, used for processing the acoustic features by using a parallel computing model to obtain first audio information of the text to be processed, wherein the first audio information includes audio corresponding to each sampling moment; A second model processing module, used for processing the acoustic features and the first audio information by using a serial calculation model to obtain residual values ​​of the first audio information corresponding to each sampling moment; An audio generation module, configured to obtain second audio information corresponding to the i-th sampling moment based on the first audio information corresponding to the i-th sampling moment and the residual value corresponding to the i-th sampling moment, wherein the synthesized audio of the text to be processed includes each second audio information, wherein i=1, 2...n, and n is the total number of the sampling moments; The second model processing module is also used for: Using a serial calculation model, the first audio information corresponding to the first sampling moment, the acoustic feature corresponding to the first sampling moment, and a preset residual value are processed to obtain a residual value corresponding to the first sampling moment; The first audio information corresponding to the jth sampling moment, the acoustic features corresponding to the jth sampling moment, and the residual value corresponding to the j-1th sampling moment are processed using a serial calculation model to obtain the residual value of the first audio information corresponding to the jth sampling moment, where j=2, 3...n.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the speech synthesis method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech synthesis method according to any one of claims 1 to 6 is implemented.

Images