Speech synthesis output method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of speech synthesis, is applied to a scene of converting a text into speech, and relates to a speech synthesis output method and device, equipment and a storage medium thereof. Inputting the input text into an improved text encoder, and carrying out text serialization encoding processing; obtaining a text coding sequence output by the improved text encoder, and performing synthetic speech duration prediction on the text coding sequence through a preset duration predictor to obtain predicted duration; performing voice stream synthesis on the text coding sequence according to a predicted time length to obtain a synthesized target voice stream; and inputting the target voice stream into a preset voice decoder to complete target voice output. By optimizing the encoding and decoding processing process during speech synthesis output, under the condition that the quality of the output speech is not changed, the encoding and decoding parameters are reduced, the speech synthesis processing complexity is reduced, and the synthesis processing efficiency during speech synthesis is improved.

Description

Technical Field

[0001] The present application relates to the field of speech synthesis technology, and is applied to the scenario of converting text into speech, and in particular to a speech synthesis output method, apparatus, device and storage medium thereof. Background Art

[0002] Speech synthesis is an important area in the field of artificial intelligence. It is a natural ability for humans, but it represents a high level of progress for artificial intelligence. Speech synthesis has made great progress due to the development of technology. The neural codec language model shows strong capabilities in zero-shot speech synthesis, but the codec language model is limited by the autoregressive characteristics and the implicit alignment relationship between text and audio, which leads to frequent challenges in inference speed and stability.

[0003] The field of speech synthesis is currently dominated by end-to-end models, and the speech quality generated by end-to-end speech synthesis is comparable to that of real people. However, the current end-to-end speech synthesis method based on the Transformer encoder structure involves many processing parameters in the encoding and decoding part, such as many encoding parameters and decoding parameters. This results in a large number of processing parameters consuming a lot of processing resources during speech synthesis, which affects the speech synthesis efficiency to a certain extent. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a speech synthesis output method, device, equipment and storage medium thereof to solve the problem that when performing speech synthesis, a large number of processing parameters will consume a large amount of processing resources, which will affect the speech synthesis efficiency to a certain extent.

[0005] In order to solve the above technical problems, the present application embodiment provides a speech synthesis output method, which adopts the following technical solutions:

[0006] A speech synthesis output method comprises the following steps:

[0007] Get the input text to be synthesized;

[0008] Inputting the input text into the improved text encoder to perform text serialization encoding processing;

[0009] Obtaining a text encoding sequence output by the improved text encoder, and performing synthetic speech duration prediction on the text encoding sequence through a preset duration predictor to obtain a predicted duration;

[0010] Performing speech stream synthesis on the text encoding sequence according to the predicted duration to obtain a synthesized target speech stream;

[0011] The target voice stream is input into a preset voice decoder to complete the target voice output.

[0012] Furthermore, the improved text encoder and the preset speech decoder form an end-to-end codec architecture mode, and the improved text encoder is a text encoder based on a linear attention mechanism, and the sequence output control in the end-to-end codec architecture mode adopts a gating mechanism.

[0013] Furthermore, the step of inputting the input text into the improved text encoder to perform text serialization encoding processing specifically includes:

[0014] Inputting the input text into the text encoder based on the linear attention mechanism;

[0015] Acquire linearized low-dimensional text features corresponding to the input text according to the linear attention mechanism;

[0016] Based on the linearized low-dimensional text features, a gating mechanism is used to serialize and encode the input text for output, thereby obtaining a text encoding sequence corresponding to the input text.

[0017] Furthermore, the step of obtaining the text encoding sequence output by the improved text encoder and predicting the duration of synthesized speech on the text encoding sequence by a preset duration predictor to obtain the predicted duration specifically includes:

[0018] Identify all text fields contained in the text encoding sequence, and determine the pronunciation selection object corresponding to the text encoding sequence based on the selection instruction;

[0019] According to all text fields contained in the text encoding sequence and reference knowledge in the duration predictor, identifying all phoneme units contained in the text encoding sequence;

[0020] Serializing all the phoneme units to obtain a phoneme sequence corresponding to the text encoding sequence;

[0021] According to the pronunciation durations corresponding to all the phoneme units, the text encoding sequence and the phoneme sequence are aligned in terms of text fields and pronunciation durations to obtain the total pronunciation duration corresponding to the text encoding sequence after the alignment process;

[0022] The total pronunciation duration is used as the predicted duration.

[0023] Furthermore, before executing the step of obtaining the text encoding sequence output by the improved text encoder and predicting the duration of synthesized speech on the text encoding sequence by a preset duration predictor to obtain the predicted duration, the method further includes:

[0024] Acquire a target voice package, wherein the target voice package includes batch voice segments of N pronunciation selection objects, where N is a positive integer;

[0025] Taking a single object as an extraction unit, voice features are extracted from the batch of voice segments in the target voice package to obtain voice features corresponding to the N pronunciation selection objects respectively;

[0026] Inputting the speech features corresponding to the N pronunciation selection objects into a preset posterior encoder to obtain the corresponding relationship between the text field, phoneme unit and pronunciation duration corresponding to the N pronunciation selection objects;

[0027] The correspondence between the text fields, phoneme units and pronunciation durations respectively corresponding to the N pronunciation selection objects is deployed in the duration predictor as reference knowledge.

[0028] Furthermore, before executing the step of inputting the target voice stream into a preset voice decoder to complete the target voice output, the method further includes:

[0029] Obtaining a speech decoding sequence output by the speech decoder for the target speech stream;

[0030] Extracting speech features from the speech decoding sequence and the batch speech segments of the pronunciation selection object respectively to obtain speech features corresponding to the speech decoding sequence and speech features corresponding to the pronunciation selection object;

[0031] Performing speech waveform processing on the speech features corresponding to the speech decoding sequence to obtain a first speech waveform;

[0032] Performing speech waveform processing on the speech feature corresponding to the pronunciation selection object to obtain a second speech waveform;

[0033] By comparison, identifying whether the first speech waveform matches the second speech waveform;

[0034] If the first speech waveform does not conform to the second speech waveform, performing decoding parameter tuning processing on the speech decoder until the first speech waveform conforms to the second speech waveform, and stopping the decoding parameter tuning processing;

[0035] If the first speech waveform matches the second speech waveform, the speech decoding sequence is output as the target speech.

[0036] Furthermore, the sequence output control in the end-to-end codec architecture mode adopts a gating mechanism, which specifically includes:

[0037] When obtaining the text encoding sequence output by the improved text encoder, a gating mechanism is adopted; and,

[0038] When obtaining the phoneme sequence corresponding to the text encoding sequence, a gating mechanism is adopted; and,

[0039] A speech decoding sequence output by the speech decoder for the target speech stream is obtained, using a gating mechanism.

[0040] In order to solve the above technical problems, the embodiment of the present application also provides a speech synthesis output device, which adopts the following technical solution:

[0041] A speech synthesis output device, comprising:

[0042] An input text acquisition module is used to acquire input text to be synthesized into speech;

[0043] A text serialization encoding module, used for inputting the input text into the improved text encoder to perform text serialization encoding processing;

[0044] A synthetic speech duration prediction module, used to obtain a text encoding sequence output by the improved text encoder, and perform synthetic speech duration prediction on the text encoding sequence through a preset duration predictor to obtain a predicted duration;

[0045] A target speech stream synthesis module, used for performing speech stream synthesis on the text encoding sequence according to the predicted duration to obtain a synthesized target speech stream;

[0046] The voice stream decoding output module is used to input the target voice stream into a preset voice decoder to complete the target voice output.

[0047] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0048] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned speech synthesis output method when executing the computer-readable instructions.

[0049] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0050] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the speech synthesis output method as described above.

[0051] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0052] The speech synthesis output method described in the embodiment of the present application is as follows: obtaining the input text to be speech synthesized; inputting the input text into the improved text encoder to perform text serialization encoding processing; obtaining the text encoding sequence output by the improved text encoder, and predicting the synthesized speech duration of the text encoding sequence through a preset duration predictor to obtain the predicted duration; performing speech stream synthesis on the text encoding sequence according to the predicted duration to obtain the synthesized target speech stream; inputting the target speech stream into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding processing process during speech synthesis output, it is achieved to reduce the encoding and decoding parameters, reduce the complexity of speech synthesis processing, and improve the synthesis processing efficiency during speech synthesis without changing the output speech quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0055] Figure 2 is a flow chart of an embodiment of a speech synthesis output method according to the present application;

[0056] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 is shown;

[0057] Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 is shown;

[0058] Figure 5 It is a flowchart of a specific embodiment of acquiring and deploying duration prediction reference knowledge in the speech synthesis output method described in the present application;

[0059] Figure 6 It is a flowchart of a specific embodiment of tuning the decoding parameters of a speech decoder in the speech synthesis output method described in the present application;

[0060] Figure 7 is a structural schematic diagram of an embodiment of a speech synthesis output device according to the present application;

[0061] Figure 8 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0063] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0065] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0066] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0067] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0068] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0069] It should be noted that the speech synthesis output method provided in the embodiment of the present application is generally executed by a server, and accordingly, the speech synthesis output device is generally arranged in the server.

[0070] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0071] Continue to refer Figure 2 , shows a flow chart of an embodiment of a speech synthesis output method according to the present application. The speech synthesis output method comprises the following steps:

[0072] Step 201: Obtain input text to be subjected to speech synthesis.

[0073] Specifically, in this embodiment, the input text to be speech synthesized is, for example, a response communication text pre-organized for financial customer service services or banking customer service services.

[0074] Correspondingly, the acquisition of the input text to be speech synthesized includes acquiring a carrier document recording the input text to be speech synthesized, and finally acquiring the input text by parsing the content in the carrier document; it also includes directly optically recognizing the input text to be speech synthesized in the target carrier document, and acquiring the input text according to the optical recognition result.

[0075] By obtaining the input text to be speech synthesized and applying the speech synthesis output method to the financial customer service business or banking customer service business response communication scenario, it is possible to realize automated customer service response and reduce the workload of manual agents.

[0076] Step 202: input the input text into the improved text encoder to perform text serialization encoding processing.

[0077] In this embodiment, an end-to-end architecture mode is adopted when performing speech synthesis, that is, it is composed of a codec (part), wherein the encoder part uses a text encoder to encode the input text, and the decoder (part) decodes the speech stream synthesized according to the text encoding. Compared with the encoder structure of the conventional Transformer, the text encoder part of the improved text encoder no longer uses a multi-head (self) attention mechanism, but is improved to use a linear attention mechanism, and the output sequence control used in the codec structure is improved to a gating mechanism, and the output of the sequence is controlled by the gating mechanism. The multi-head (self) attention mechanism in the text encoder part is improved to use a linear attention mechanism, which reduces the number of parameters during encoding. At the same time, the output sequence control used in the codec structure is improved to a gating mechanism, and the output of the sequence is controlled by the gating mechanism. Compared with the use of word embedding, position encoding and self-attention mechanism to control the sequence output, the number of parameters is reduced to a certain extent, the encoding and decoding efficiency is improved, and the speech synthesis efficiency is improved.

[0078] Step 203, obtaining a text encoding sequence output by the improved text encoder, and performing synthetic speech duration prediction on the text encoding sequence through a preset duration predictor to obtain a predicted duration.

[0079] In this embodiment, the text coding sequence is predicted for the duration of the synthesized speech by a preset duration predictor to obtain the predicted duration, so as to ensure that the output speech duration is consistent with the output speech speed of the target person object and is more realistic.

[0080] Step 204: synthesize a speech stream for the text encoding sequence according to the predicted duration to obtain a synthesized target speech stream.

[0081] Step 205: input the target voice stream into a preset voice decoder to complete the target voice output.

[0082] In this embodiment, the input text to be synthesized is obtained; the input text is input into the improved text encoder to perform text serialization encoding processing; the text encoding sequence output by the improved text encoder is obtained, and the synthesized speech duration is predicted for the text encoding sequence by a preset duration predictor to obtain the predicted duration; the text encoding sequence is synthesized into a speech stream according to the predicted duration to obtain the synthesized target speech stream; the target speech stream is input into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding processing process during speech synthesis output, it is achieved to reduce encoding and decoding parameters, reduce the complexity of speech synthesis processing, and improve the synthesis processing efficiency during speech synthesis without changing the output speech quality.

[0083] In this embodiment, the improved text encoder and the preset speech decoder form an end-to-end codec architecture mode, and the improved text encoder is a text encoder based on a linear attention mechanism, and the sequence output control in the end-to-end codec architecture mode adopts a gating mechanism.

[0084] In this embodiment, compared with the encoder structure of the conventional Transformer, the text encoder part of the improved text encoder no longer adopts the multi-head (self) attention mechanism, but is improved to adopt the linear attention mechanism, and the output sequence control adopted in the encoding and decoding structure is improved to a gating mechanism, and the output of the sequence is controlled by the gating mechanism. The multi-head (self) attention mechanism in the text encoder part is improved to adopt the linear attention mechanism, which reduces the number of parameters during encoding. At the same time, the output sequence control adopted in the encoding and decoding structure is improved to a gating mechanism, and the output of the sequence is controlled by the gating mechanism. Compared with the use of word embedding, position encoding and self-attention mechanism to control the sequence output, the number of parameters is also reduced to a certain extent, and the encoding and decoding efficiency is improved, thereby improving the speech synthesis efficiency. It is ensured that the speech synthesis parameters are reduced and the speech synthesis efficiency is improved without changing the output speech quality.

[0085] Continue to refer Figure 3 , Figure 3 yes Figure 2 The flowchart of a specific embodiment of step 202 shown includes the following steps:

[0086] Step 301, inputting the input text into the text encoder based on the linear attention mechanism;

[0087] Step 302, obtaining a linearized low-dimensional text feature corresponding to the input text according to the linear attention mechanism;

[0088] Step 303: Based on the linearized low-dimensional text features, the input text is serialized and encoded and outputted using a gating mechanism to obtain a text encoding sequence corresponding to the input text.

[0089] Specifically, when performing text serialization encoding processing, in terms of text feature acquisition, there is no need to obtain higher-dimensional text features, only low-dimensional text features and position features. Therefore, it is considered to directly use the linear attention mechanism with fewer parameters during encoding processing, instead of using the previous multi-head (self) attention mechanism, which reduces the number of parameters during speech synthesis and improves the efficiency of speech synthesis.

[0090] Continue to refer Figure 4 , Figure 4 yes Figure 2 The flowchart of a specific embodiment of step 203 shown includes the following steps:

[0091] Step 401, identifying all text fields contained in the text encoding sequence, and determining the pronunciation selection object corresponding to the text encoding sequence based on the selection instruction;

[0092] Specifically, the pronunciation selection object refers to the selected target person object, and the text content is voice outputted in the voice of the target person object.

[0093] Step 402, identifying all phoneme units contained in the text encoding sequence according to all text fields contained in the text encoding sequence and reference knowledge in the duration predictor;

[0094] Specifically, the phoneme unit, for example, one character corresponds to one phoneme unit, one word corresponds to one phoneme unit, or one phrase corresponds to one phoneme unit. The specific phoneme unit is organized according to actual pronunciation requirements.

[0095] Step 403, serializing all the phoneme units to obtain a phoneme sequence corresponding to the text encoding sequence;

[0096] Step 404, performing text field and pronunciation duration alignment processing on the text encoding sequence and the phoneme sequence according to the pronunciation durations corresponding to all the phoneme units, and obtaining the total pronunciation duration corresponding to the text encoding sequence after the alignment processing;

[0097] Step 405: taking the total pronunciation duration as the predicted duration.

[0098] The synthesized speech duration of the text coding sequence is predicted by a preset duration predictor to obtain the predicted duration, and the text coding sequence and the phoneme sequence are aligned with the pronunciation duration to ensure that the output speech duration is consistent with the output speech speed of the target person object and is more realistic.

[0099] Continue to refer Figure 5 In some optional implementations, before step 203, a step of acquiring and deploying duration prediction reference knowledge is also included. Figure 5 It is a flowchart of a specific embodiment of acquiring and deploying duration prediction reference knowledge in the speech synthesis output method described in the present application, comprising the following steps:

[0100] Step 501, obtaining a target voice package, wherein the target voice package includes batch voice segments of N pronunciation selection objects, where N is a positive integer;

[0101] Step 502, extracting speech features of the batch speech segments in the target speech package respectively with a single object as an extraction unit, and obtaining speech features corresponding to the N pronunciation selection objects respectively;

[0102] It should be understood that the speech features include the correspondence between text fields, phoneme units, and pronunciation durations.

[0103] Step 503, inputting the speech features corresponding to the N pronunciation selection objects into a preset posterior encoder to obtain the corresponding relationship between the text field, phoneme unit and pronunciation duration corresponding to the N pronunciation selection objects;

[0104] Step 504: deploy the corresponding relationships among the text fields, phoneme units, and pronunciation durations respectively corresponding to the N pronunciation selection objects as reference knowledge into the duration predictor.

[0105] Through the target speech package, the correspondence between the text fields, phoneme units and pronunciation durations respectively corresponding to the N pronunciation selection objects is obtained, and the correspondence between the text fields, phoneme units and pronunciation durations respectively corresponding to the N pronunciation selection objects is deployed as reference knowledge into the duration predictor, so as to improve the duration prediction support for the duration predictor when actually performing speech synthesis, thereby ensuring the authenticity of the synthesized speech.

[0106] Continue to refer Figure 6 In some optional implementations, before step 205, a step of tuning the decoding parameters of the speech decoder is also included. Figure 6 This is a flowchart of a specific embodiment of tuning the decoding parameters of a speech decoder in the speech synthesis output method described in the present application, comprising the following steps:

[0107] Step 601, obtaining a speech decoding sequence output by the speech decoder for the target speech stream;

[0108] Step 602, performing speech feature extraction on the speech decoding sequence and the batch speech segments of the pronunciation selection object respectively, to obtain speech features corresponding to the speech decoding sequence and speech features corresponding to the pronunciation selection object;

[0109] Step 603, performing speech waveform processing on the speech features corresponding to the speech decoding sequence to obtain a first speech waveform;

[0110] Step 604, performing speech waveform processing on the speech feature corresponding to the pronunciation selection object to obtain a second speech waveform;

[0111] Step 605, identifying whether the first speech waveform matches the second speech waveform by comparison;

[0112] Specifically, whether the first speech waveform is consistent with the second speech waveform, for example: the first speech waveform is consistent with a specific waveform segment in the second speech waveform, or the first speech waveform is consistent with the second speech waveform in amplitude and frequency or the similarity reaches a preset threshold, then the first speech waveform is consistent with the second speech waveform, and the speech features are converted into speech waveforms for comparison, which is more intuitive and convenient for speech synthesis and understanding.

[0113] Step 606, if the first speech waveform does not conform to the second speech waveform, performing decoding parameter tuning processing on the speech decoder until the first speech waveform conforms to the second speech waveform, and stopping the decoding parameter tuning processing;

[0114] Step 607: If the first speech waveform matches the second speech waveform, output the speech decoding sequence as the target speech.

[0115] Specifically, in this embodiment, the tuning of decoding parameters of the speech decoder can be understood as training of decoding parameters of the speech decoder, the purpose of which is to ensure that the synthesized speech finally outputted is closer to the real pronunciation of the pronunciation selection object.

[0116] In this embodiment, the sequence output control in the end-to-end codec architecture mode adopts a gating mechanism, specifically including: when obtaining the text encoding sequence output by the improved text encoder, a gating mechanism is adopted; and, when obtaining the phoneme sequence corresponding to the text encoding sequence, a gating mechanism is adopted; and, when obtaining the speech decoding sequence output by the speech decoder for the target speech stream, a gating mechanism is adopted.

[0117] By adopting a gating mechanism when outputting a sequence, compared with combining word embedding, position encoding, and self-attention mechanisms to control sequence output, the number of parameters is reduced to a certain extent, the encoding and decoding efficiency is improved, and thus the speech synthesis efficiency is improved. This ensures that the speech synthesis parameters are reduced and the speech synthesis efficiency is improved without changing the output speech quality.

[0118] The present application obtains the input text to be synthesized; inputs the input text into an improved text encoder to perform text serialization encoding processing; obtains the text encoding sequence output by the improved text encoder, and predicts the duration of synthesized speech on the text encoding sequence through a preset duration predictor to obtain the predicted duration; synthesizes the text encoding sequence into a speech stream according to the predicted duration to obtain the synthesized target speech stream; inputs the target speech stream into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding process during speech synthesis output, it is achieved to reduce encoding and decoding parameters, reduce the complexity of speech synthesis processing, and improve the synthesis processing efficiency during speech synthesis without changing the output speech quality.

[0119] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0120] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0121] In the embodiment of the present application, the input text to be synthesized is obtained; the input text is input into the improved text encoder to perform text serialization encoding processing; the text encoding sequence output by the improved text encoder is obtained, and the synthesized speech duration is predicted for the text encoding sequence by a preset duration predictor to obtain the predicted duration; the text encoding sequence is synthesized into a speech stream according to the predicted duration to obtain the synthesized target speech stream; the target speech stream is input into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding process during speech synthesis output, it is achieved to reduce encoding and decoding parameters, reduce the complexity of speech synthesis processing, and improve the synthesis processing efficiency during speech synthesis without changing the output speech quality.

[0122] Further references Figure 7 , as a response to the above Figure 2 The present application provides an embodiment of a speech synthesis output device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0123] like Figure 7 As shown, the speech synthesis output device 700 described in this embodiment includes: an input text acquisition module 701, a text serialization encoding module 702, a synthesized speech duration prediction module 703, a target speech stream synthesis module 704 and a speech stream decoding output module 705. Among them:

[0124] An input text acquisition module 701 is used to acquire input text to be synthesized into speech;

[0125] A text serialization encoding module 702 is used to input the input text into the improved text encoder to perform text serialization encoding processing;

[0126] The synthesized speech duration prediction module 703 is used to obtain the text code sequence output by the improved text encoder, and perform synthesized speech duration prediction on the text code sequence through a preset duration predictor to obtain a predicted duration;

[0127] A target speech stream synthesis module 704 is used to perform speech stream synthesis on the text encoding sequence according to the predicted duration to obtain a synthesized target speech stream;

[0128] The voice stream decoding output module 705 is used to input the target voice stream into a preset voice decoder to complete the target voice output.

[0129] The present application obtains the input text to be synthesized; inputs the input text into an improved text encoder to perform text serialization encoding processing; obtains the text encoding sequence output by the improved text encoder, and predicts the duration of synthesized speech on the text encoding sequence through a preset duration predictor to obtain the predicted duration; synthesizes the text encoding sequence into a speech stream according to the predicted duration to obtain the synthesized target speech stream; inputs the target speech stream into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding process during speech synthesis output, it is achieved to reduce encoding and decoding parameters, reduce the complexity of speech synthesis processing, and improve the synthesis processing efficiency during speech synthesis without changing the output speech quality.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0131] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0132] To solve the above technical problems, the present application also provides a computer device. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0133] The computer device 8 includes a memory 8a, a processor 8b, and a network interface 8c which are interconnected through a system bus. It should be noted that: Figure 8 Only a computer device 8 having a component memory 8a, a processor 8b, and a network interface 8c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0134] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0135] The memory 8a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 8a can be an internal storage unit of the computer device 8, such as a hard disk or memory of the computer device 8. In other embodiments, the memory 8a can also be an external storage device of the computer device 8, such as a plug-in hard disk equipped on the computer device 8, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 8a can also include both the internal storage unit of the computer device 8 and its external storage device. In this embodiment, the memory 8a is generally used to store the operating system and various application software installed on the computer device 8, such as a computer-readable instruction of a speech synthesis output method, etc. In addition, the memory 8a can also be used to temporarily store various types of data that have been output or are to be output.

[0136] The processor 8b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 8b is generally used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is used to run the computer-readable instructions stored in the memory 8a or process data, such as computer-readable instructions for running the speech synthesis output method.

[0137] The network interface 8c may include a wireless network interface or a wired network interface. The network interface 8c is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0138] The computer device proposed in this embodiment belongs to the field of speech synthesis technology and is applied to the scenario of converting text into speech. The present application obtains the input text to be speech synthesized; inputs the input text into an improved text encoder to perform text serialization encoding processing; obtains the text encoding sequence output by the improved text encoder, and predicts the synthesized speech duration of the text encoding sequence through a preset duration predictor to obtain the predicted duration; performs speech stream synthesis on the text encoding sequence according to the predicted duration to obtain the synthesized target speech stream; inputs the target speech stream into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding process during speech synthesis output, it is achieved that the encoding and decoding parameters are reduced, the complexity of speech synthesis processing is reduced, and the synthesis processing efficiency during speech synthesis is improved without changing the output speech quality.

[0139] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the speech synthesis output method as described above.

[0140] The computer-readable storage medium proposed in this embodiment belongs to the field of speech synthesis technology and is applied to the scenario of converting text into speech. The present application obtains the input text to be speech synthesized; inputs the input text into an improved text encoder to perform text serialization encoding processing; obtains the text encoding sequence output by the improved text encoder, and predicts the synthesized speech duration of the text encoding sequence through a preset duration predictor to obtain the predicted duration; performs speech stream synthesis on the text encoding sequence according to the predicted duration to obtain the synthesized target speech stream; inputs the target speech stream into a preset speech decoder to complete the target speech output. By optimizing the encoding and decoding process during speech synthesis output, it is achieved that the encoding and decoding parameters are reduced, the complexity of speech synthesis processing is reduced, and the synthesis processing efficiency during speech synthesis is improved without changing the output speech quality.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0142] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A speech synthesis output method, characterized in that: The steps include: Get the input text to be synthesized; Inputting the input text into the improved text encoder to perform text serialization encoding processing; Obtaining a text encoding sequence output by the improved text encoder, and performing synthetic speech duration prediction on the text encoding sequence through a preset duration predictor to obtain a predicted duration; Performing speech stream synthesis on the text encoding sequence according to the predicted duration to obtain a synthesized target speech stream; The target voice stream is input into a preset voice decoder to complete the target voice output.

2. The speech synthesis output method according to claim 1, characterized in that: The improved text encoder and the preset speech decoder form an end-to-end codec architecture mode, and the improved text encoder is a text encoder based on a linear attention mechanism, and the sequence output control in the end-to-end codec architecture mode adopts a gating mechanism.

3. The speech synthesis output method according to claim 2, characterized in that: The step of inputting the input text into the improved text encoder to perform text serialization encoding processing specifically includes: Inputting the input text into the text encoder based on the linear attention mechanism; Acquire linearized low-dimensional text features corresponding to the input text according to the linear attention mechanism; Based on the linearized low-dimensional text features, a gating mechanism is used to serialize and encode the input text for output, thereby obtaining a text encoding sequence corresponding to the input text.

4. The speech synthesis output method according to claim 2 or 3, characterized in that: The step of obtaining the text encoding sequence output by the improved text encoder and predicting the duration of synthesized speech on the text encoding sequence by a preset duration predictor to obtain the predicted duration specifically includes: Identify all text fields contained in the text encoding sequence, and determine the pronunciation selection object corresponding to the text encoding sequence based on the selection instruction; According to all text fields contained in the text encoding sequence and reference knowledge in the duration predictor, identifying all phoneme units contained in the text encoding sequence; Serializing all the phoneme units to obtain a phoneme sequence corresponding to the text encoding sequence; According to the pronunciation durations corresponding to all the phoneme units, the text code sequence and the phoneme sequence are aligned in terms of text fields and pronunciation durations to obtain the total pronunciation duration corresponding to the text code sequence after the alignment process; The total pronunciation duration is used as the predicted duration.

5. The speech synthesis output method according to claim 4, characterized in that: Before executing the step of obtaining the text encoding sequence output by the improved text encoder and predicting the duration of synthesized speech on the text encoding sequence by a preset duration predictor to obtain the predicted duration, the method further includes: Acquire a target voice package, wherein the target voice package includes batch voice segments of N pronunciation selection objects, where N is a positive integer; Taking a single object as an extraction unit, voice features are extracted from the batch of voice segments in the target voice package to obtain voice features corresponding to the N pronunciation selection objects respectively; Inputting the speech features corresponding to the N pronunciation selection objects into a preset posterior encoder to obtain the corresponding relationship between the text field, phoneme unit and pronunciation duration corresponding to the N pronunciation selection objects; The correspondence between the text fields, phoneme units and pronunciation durations respectively corresponding to the N pronunciation selection objects is deployed in the duration predictor as reference knowledge.

6. The speech synthesis output method according to claim 4, characterized in that: Before executing the step of inputting the target voice stream into a preset voice decoder to complete the target voice output, the method further includes: Obtaining a speech decoding sequence output by the speech decoder for the target speech stream; Performing speech feature extraction on the speech decoding sequence and the batch speech segments of the pronunciation selection object respectively to obtain speech features corresponding to the speech decoding sequence and speech features corresponding to the pronunciation selection object; Performing speech waveform processing on the speech features corresponding to the speech decoding sequence to obtain a first speech waveform; Performing speech waveform processing on the speech feature corresponding to the pronunciation selection object to obtain a second speech waveform; By comparison, identifying whether the first speech waveform matches the second speech waveform; If the first speech waveform does not conform to the second speech waveform, performing decoding parameter tuning processing on the speech decoder until the first speech waveform conforms to the second speech waveform, and stopping the decoding parameter tuning processing; If the first speech waveform matches the second speech waveform, the speech decoding sequence is output as the target speech.

7. The speech synthesis output method according to claim 6, characterized in that: The sequence output control in the end-to-end codec architecture mode adopts a gating mechanism, which specifically includes: When obtaining the text encoding sequence output by the improved text encoder, a gating mechanism is adopted; and, When obtaining the phoneme sequence corresponding to the text encoding sequence, a gating mechanism is adopted; and, A speech decoding sequence output by the speech decoder for the target speech stream is obtained, using a gating mechanism.

8. A speech synthesis output device, characterized in that: include: An input text acquisition module is used to acquire input text to be synthesized into speech; A text serialization encoding module, used for inputting the input text into the improved text encoder to perform text serialization encoding processing; A synthetic speech duration prediction module is used to obtain a text encoding sequence output by the improved text encoder, and perform synthetic speech duration prediction on the text encoding sequence through a preset duration predictor to obtain a predicted duration; A target speech stream synthesis module, used for performing speech stream synthesis on the text encoding sequence according to the predicted duration to obtain a synthesized target speech stream; The voice stream decoding output module is used to input the target voice stream into a preset voice decoder to complete the target voice output.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the speech synthesis output method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the speech synthesis output method according to any one of claims 1 to 7 are implemented.

Images