Method and device for converting text into voice, equipment and storage medium

Abstract

The embodiment of the invention provides a method, device and equipment for converting a text into voice and a storage medium. The method comprises the steps of obtaining a target text, and determining semantic information corresponding to the target text; the semantic information is processed through a voice word segmentation device in a target large language model, a discrete voice token corresponding to the target text is obtained, and the discrete voice token comprises coding information corresponding to the semantic information; based on the coding information, determining position information corresponding to the discrete voice token; the discrete voice token is predicted through a vocoder in the target large language model, and a voice signal corresponding to the discrete voice token is obtained; and the sorting module is used for sorting the voice signals according to the position information to obtain a target voice stream corresponding to the target text. The embodiment of the invention aims to realize relatively strong semantic association in the process of converting the text into the voice, and improve the accuracy and efficiency of converting the text into the voice.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for converting text to speech, a device for converting text to speech, a computer device, and a computer-readable storage medium. Background Art

[0002] Speech synthesis refers to the process of converting text into speech audio. Currently, a speech synthesis model is usually used to perform the above-mentioned operation of synthesizing speech audio. In the related technology, the speech synthesis technology based on neural network is mainly used to realize text-to-speech conversion.

[0003] Speech synthesis systems based on neural networks are divided into two categories, namely autoregressive and non-autoregressive models. The autoregressive model has the problem of long audio synthesis time and poor effect, while the non-autoregressive model is limited by the inconsistency of input text and output length. The synthesized speech is difficult to align with the text, so it is impossible to achieve strong speech association, resulting in poor effect and low accuracy. Summary of the invention

[0004] The present application provides a text-to-speech method, a text-to-speech device, a computer device, and a computer-readable storage medium, which are intended to achieve stronger semantic associations in the text-to-speech process and improve the accuracy and efficiency of text-to-speech conversion.

[0005] To achieve the above purpose, the present application also provides a method for converting text to speech, comprising:

[0006] Acquire a target text, and determine semantic information corresponding to the target text;

[0007] Processing the semantic information through a speech segmenter in a target large language model to obtain discrete speech tokens corresponding to the target text, wherein the discrete speech tokens include encoding information corresponding to the semantic information;

[0008] Based on the coding information, determining the position information corresponding to the discrete speech token; and predicting the discrete speech token through the vocoder in the target large language model to obtain the speech signal corresponding to the discrete speech token;

[0009] Used to sort the voice signals according to the position information to obtain a target voice stream corresponding to the target text.

[0010] To achieve the above object, the present application also provides a text-to-speech device, comprising:

[0011] An acquisition module, used to acquire a target text and determine semantic information corresponding to the target text;

[0012] A determination module, configured to process the semantic information through a speech segmenter in a target large language model to obtain discrete speech tokens corresponding to the target text, wherein the discrete speech tokens include encoding information corresponding to the semantic information;

[0013] A determination module, configured to determine position information corresponding to the discrete speech token based on the encoding information; and predict the discrete speech token through a vocoder in the target large language model to obtain a speech signal corresponding to the discrete speech token;

[0014] The conversion module is used to sort the voice signals according to the position information to obtain a target voice stream corresponding to the target text.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of any one of the text-to-speech methods provided in the embodiments of the present application when executing the computer program.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the text-to-speech method provided in any one of the embodiments of the present application.

[0017] The present application discloses a method for converting text to speech, a device for converting text to speech, a computer device and a computer-readable storage medium. The method includes obtaining a target text and determining semantic information corresponding to the target text. Furthermore, the semantic information can be processed by a speech segmenter in a target large language model to obtain discrete speech tokens corresponding to the target text, wherein the discrete speech tokens include encoding information corresponding to the semantic information. Thus, the position information corresponding to the discrete speech tokens can be determined based on the encoding information; and the discrete speech tokens can be predicted by a vocoder in a target large language model to obtain speech signals corresponding to the discrete speech tokens. Finally, the speech signals can be sorted according to the position information to obtain a target speech stream corresponding to the target text. The present application embodiment can effectively process the semantic information of the target text and generate corresponding discrete speech tokens by introducing a speech segmenter and a vocoder in a target large language model, thereby ensuring accurate encoding and decoding of the semantic information. At the same time, by determining the position information of the discrete speech tokens based on the encoding information, and generating high-quality speech signals in combination with the prediction of the vocoder, a coherent and natural speech stream is further obtained by sorting, thereby improving the accuracy and fluency of text-to-speech conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a scenario diagram of a method for converting text to speech provided in an embodiment of the present application;

[0020] Figure 2 It is a flowchart of a method for converting text to speech provided in an embodiment of the present application;

[0021] Figure 3 is a schematic block diagram of a text-to-speech device provided in an embodiment of the present application;

[0022] Figure 4 It is a schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions. In addition, although the functional modules are divided in the device schematic, in some cases, the module division can be different from that in the device schematic.

[0025] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0027] like Figure 1 As shown, the text-to-speech method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. The application environment includes a terminal device 110 and a server 120, wherein the terminal device 110 can communicate with the server 120 through a network. Specifically, the server 120 can obtain the target text and determine the semantic information corresponding to the target text; process the semantic information through the speech segmenter in the target large language model to obtain the discrete speech token corresponding to the target text, wherein the discrete speech token includes the encoding information corresponding to the semantic information; determine the position information corresponding to the discrete speech token based on the encoding information; and predict the discrete speech token through the vocoder in the target large language model to obtain the speech signal corresponding to the discrete speech token; and sort the speech signal according to the position information to obtain the target speech stream corresponding to the target text, thereby sending the target speech stream to the terminal device 110. Among them, the server 120 can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0028] See also Figure 2 , Figure 2 1 is a flowchart of a method for converting text to speech provided in an embodiment of the present application. Figure 2 As shown, the method of converting text to speech can be implemented through steps S11 to S14.

[0029] Step S11: Acquire the target text and determine the semantic information corresponding to the target text.

[0030] For example, the target text refers to the input text that needs to be converted into speech, and may include any text data input by the user, such as a sentence, paragraph, or article. The text may include natural language descriptions, commands, or other formatted data.

[0031] For example, semantic information refers to the actual meaning or content implied in the target text, which is the deep information extracted from the literal content of the text. Semantic information not only focuses on the surface meaning of words, but also includes the context, sentence structure, emotional tendency, intention and other contents of the text.

[0032] For example, the target text can be converted into a semantic vector to capture its meaning through natural language processing techniques such as word embedding models and sentence vector models. The extraction of semantic information requires not only the analysis of the meaning of words, but also grammatical and syntactic analysis to understand the logic and context of text expression.

[0033] The above implementation method, by obtaining the input target text and extracting the semantic information of the text through natural language processing technology, provides an accurate semantic basis for subsequent speech generation, so that the generated speech not only contains the correct vocabulary pronunciation, but also accurately conveys the meaning and emotion of the text.

[0034] Step S12: Process the semantic information through the speech segmenter in the target large language model to obtain discrete speech tokens corresponding to the target text.

[0035] The discrete speech tokens include encoding information corresponding to the semantic information.

[0036] It should be noted that discrete speech tokens are the result of speech segmentation. Discrete speech tokens are used to represent the discrete representation of semantic information in the speech synthesis process, similar to the basic elements of speech such as phonemes, tones, and rhythms.

[0037] For example, discrete speech tokens also carry encoding information extracted from semantic information. The encoding information is used to represent important features in the original semantic information, such as emotional expression, such as happiness, anger, doubt, etc. Speech features: such as pitch, speaking speed, volume, etc., to ensure that the synthesized speech is fluent and natural. Context understanding, to ensure the coherence of speech generation between consecutive sentences.

[0038] For example, the speech segmenter can generate discrete speech tokens suitable for speech synthesis by segmenting and encoding semantic information. Next, the discrete speech tokens will be used to predict the actual speech signal and finally synthesize it into natural and fluent speech output.

[0039] The above implementation methods can use the speech segmenter to process complex semantic information into discrete speech tokens. Discrete speech tokens not only represent the pronunciation units in the semantic information, but also carry encoding information such as emotion, intonation, and speaking speed, providing a basis for the subsequent generation of speech signals, ensuring that the generated speech is not only accurate in content, but also can convey the emotions and tone contained in the text.

[0040] Step S13: based on the coding information, determining the position information corresponding to the discrete speech token; and predicting the discrete speech token through the vocoder in the target large language model to obtain the speech signal corresponding to the discrete speech token.

[0041] For example, discrete speech tokens have been processed by a speech segmenter and carry corresponding encoding information. The encoding information not only represents the content of the speech segment (such as phonemes, emotions, tones, etc.), but is also used to mark the relative positions of each speech segment to ensure that the generated speech maintains the same order and intonation as the original text.

[0042] For example, the position information refers to the order and position of each discrete speech token in the target speech stream. It is used to ensure that the synthesized speech signal is coherent and conforms to the natural pronunciation order of the target text. By encoding the information, the exact position of each discrete speech token in the speech sequence can be extracted.

[0043] It should be noted that the vocoder is a key module for generating the final speech signal. The vocoder can convert discrete speech tokens into continuous speech waveform signals. The vocoder generates high-quality and natural speech signals by predicting the input discrete speech tokens.

[0044] For example, the vocoder can process discrete speech tokens one by one in the order of position information and predict the corresponding speech waveform signal. The speech signal of each discrete speech token is gradually generated to form a continuous speech stream. The vocoder will process the details of the speech during the prediction process, such as smoothing the speech waveform, adjusting the pitch, controlling the speaking speed, etc., to ensure that the generated speech is natural and smooth. Through the prediction of the vocoder, each discrete speech token will generate a corresponding speech signal segment. These signal segments are then combined according to the position information to form a complete target speech stream.

[0045] It should be understood that by predicting all discrete speech tokens and sorting them by position information, the final generated speech stream corresponds to the semantic information of the target text and can accurately convey the content and emotion of the original text. This speech stream can be directly used in speech synthesis applications, such as voice assistants, navigation systems, audio content generation and other scenarios.

[0046] Step S14: Sort the voice signals according to the position information to obtain a target voice stream corresponding to the target text.

[0047] For example, according to the position information, all the voice segments are sorted in the order in which they should appear. For example, the segment with position information 1 should appear at the front, followed by the segment with position information 2, and so on, until the last voice segment.

[0048] For example, in addition to the order, the position information can also help determine the time interval between each speech segment. By adjusting the time length between each segment, the rhythm of the speech can be ensured to be natural and maintain a suitable speaking speed.

[0049] For example, after the speech segments are sorted, they are spliced ​​into a continuous speech stream. When splicing, the connection problem is handled to ensure that the transition between the speech segments is natural and avoids abrupt or incoherent situations.

[0050] In the above implementation, the discrete speech signal segments generated by the vocoder are sorted and spliced ​​according to the position information, and finally a complete and coherent target speech stream is obtained. Through the precise control of the position information, the generated speech can naturally reflect the text content and semantics, while maintaining the accuracy of the speech rhythm, speaking speed and emotional expression. This step ensures that the output of speech synthesis is of high quality and can meet various practical application requirements.

[0051] The method for converting text to speech disclosed in the embodiment of the present application includes obtaining a target text and determining the semantic information corresponding to the target text. Further, the semantic information can be processed by the speech segmenter in the target large language model to obtain discrete speech tokens corresponding to the target text, wherein the discrete speech tokens include encoding information corresponding to the semantic information. Thus, the position information corresponding to the discrete speech tokens can be determined based on the encoding information; and the discrete speech tokens can be predicted by the vocoder in the target large language model to obtain the speech signal corresponding to the discrete speech tokens. Finally, the speech signal can be sorted according to the position information to obtain the target speech stream corresponding to the target text. The embodiment of the present application can effectively process the semantic information of the target text and generate the corresponding discrete speech tokens by introducing the speech segmenter and vocoder in the target large language model, thereby ensuring the accurate encoding and decoding of the semantic information. At the same time, by determining the position information of the discrete speech tokens based on the encoding information, and generating high-quality speech signals in combination with the prediction of the vocoder, a coherent and natural speech stream is further obtained by sorting, thereby improving the accuracy and fluency of text-to-speech conversion.

[0052] Optionally, determining the semantic information corresponding to the target text includes: performing a preprocessing operation on the target text to obtain a preprocessed target text, wherein the preprocessing operation includes a word segmentation operation, a normalization operation, and a stop word removal operation; analyzing the text structure information of the preprocessed target text, and determining the semantic information corresponding to the target text based on the text structure information.

[0053] For example, the target text can be preprocessed first to more accurately extract semantic information. The preprocessing operation includes decomposing the text into vocabulary units (segmentation operation), unifying the text format (normalization operation), and removing stop words that do not contribute significantly to semantics. Furthermore, by analyzing the structural information of the preprocessed text (such as sentence composition and the relationship between words), the semantics of the text can be more fully understood, and the corresponding semantic information can be extracted, providing an accurate basis for subsequent speech generation.

[0054] The above embodiment ensures more accurate and complete extraction of semantic information through text preprocessing and structural analysis, and provides high-quality semantic input for subsequent speech synthesis.

[0055] Optionally, discrete speech tokens are obtained by the following formula:

[0056]

[0057] Among them, μ l is a discrete speech token; ||·| |2 is the L2 norm; VQ is the vector quantizer; h l is the hidden representation; C is the preset code book.

[0058] Optionally, the speech segmenter includes a vector quantizer and a convolutional encoder, and the semantic information is processed by the speech segmenter in the target large language model to obtain discrete speech tokens corresponding to the target text, including: obtaining a preset code book, and determining the target hidden representation corresponding to the semantic information through the convolutional encoder; determining the code vector with the smallest distance to the target hidden representation from the code vectors in the preset code book through the vector quantizer, and determining the index of the code vector with the smallest distance as the discrete speech token corresponding to the target text.

[0059] It should be noted that a codebook is a set of multiple code vectors that represent different speech or semantic features in the feature space. It can be understood as a discretized reference table, where each code vector represents a specific speech feature. When converting the semantic information of a text into discrete tokens, the codebook provides a standard reference, which enables the continuous semantic information to be mapped into a discrete, quantifiable form.

[0060] Furthermore, the task of the convolutional encoder is to encode the input semantic information and convert it into a target hidden representation suitable for subsequent quantization. The convolutional encoder can extract local features of semantic information, such as the relationship between words and context. The output of the convolutional encoder is the target hidden representation, which is a continuous numerical vector that retains the core features of the semantic information. These feature vectors have not yet been discretized and need further processing.

[0061] In addition, the task of the vector quantizer is to discretize the target hidden representation and convert it into a specific speech token. The quantization process includes the following steps: Calculate the similarity, the vector quantizer selects a code vector from the code book that is closest to the target hidden representation. Specifically, it calculates the distance (such as Euclidean distance or cosine similarity) between each code vector in the code book and the target hidden representation; select the code vector with the smallest distance, find the code vector with the smallest distance, and regard it as the vector most similar to the target hidden representation. This closest code vector can best represent the characteristics of the current semantic information.

[0062] For example, after finding the code vector with the minimum distance, the index value of the code vector in the code book can be obtained. This index value is used as a discrete speech token, which is a quantized result of the current semantic information, which is convenient for further processing in the speech generation process.

[0063] It should be noted that discrete speech tokens represent the unique speech features of a certain semantic part of the target text. A sequence of multiple discrete speech tokens can be used for subsequent speech generation to ensure that the generated speech can accurately express the meaning of the original text. Discrete speech tokens can be input into a vocoder to predict specific speech signals and gradually realize the complete text-to-speech conversion process.

[0064] The above implementation method maps semantic information into discrete speech tokens through the cooperation of a convolutional encoder and a vector quantizer. The discrete speech tokens retain the characteristics of the semantic information and realize discretization representation through a code book, providing a unified discrete coding basis for subsequent speech generation.

[0065] Optionally, the target large language model is obtained in the following manner: obtaining a training set, the training set including several input texts; annotating the training set to obtain corresponding annotation results, wherein the training set annotation results include a speech stream corresponding to each input text; training the initial large language model through the training set and the annotation results to obtain the target large language model.

[0066] Specifically, the annotation results corresponding to the training set can be used as the label of the group of input data, and then each group of labeled training sets can be input into the initial diffusion model for supervised learning. When the training end conditions are met, such as the number of training times reaches the number threshold or the output accuracy of the model reaches the accuracy threshold, the training is terminated to obtain the trained target large language model.

[0067] In the embodiment of the present application, the training set and the annotation results can be input into the initial diffusion model for supervised learning, and then the target large language model can be trained to obtain the target large language model. Thus, the speech stream corresponding to the text can be output based on the target large language model.

[0068] Based on the above embodiment, the initial large language model is trained by the training set and the annotation results to obtain the target large language model, including: splicing the input text and the voice stream on the time axis to obtain the spliced ​​training data; training the initial large language model based on the spliced ​​training data to obtain the target large language model.

[0069] It should be understood that by splicing the input text and voice stream on the timeline, the changes and characteristics of the target object at different times can be more comprehensively reflected, increasing the diversity of data and helping to expand the generalization ability of the model. Furthermore, the spliced ​​annotation results combine the information of multiple time points, which can reduce the error of data annotation at a single moment, improve the accuracy of annotation, and thus improve the training effect of the model. It can also alleviate the noise interference that may exist in a single input text or voice stream to a certain extent, and improve the adaptability of the model to complex scenarios.

[0070] The above embodiments can enhance data quality and diversity during the training process of the target large language model, and improve the generalization ability and practical application effect of the model.

[0071] Optionally, after obtaining the target large language model, the method further includes: iteratively training the target large language model to extract data features and calculate a loss function; iteratively training the loss function using a preset method for the purpose of reducing the value of the loss function until an expected threshold requirement is met; and obtaining the iterative target large language model based on the loss function after iterative training.

[0072] It can be understood that in order to train a target large language model with higher accuracy, the target large language model can be repeatedly iteratively trained to continuously reduce the loss function until the loss function meets the expected threshold requirement, and then a more accurate speech stream corresponding to the text input can be obtained based on the iterated target large language model.

[0073] It should be noted that the present application does not limit the above-mentioned preset method and expected threshold. For example, the preset method may be a gradient descent algorithm, a batch gradient descent algorithm, a stochastic gradient descent algorithm, etc. The present application takes the gradient descent algorithm as an example for explanation.

[0074] The purpose of the gradient descent algorithm is to find the minimum value of the loss function by iteration, or to converge to the minimum value. In a geometric sense, the gradient descent algorithm means that the gradient decreases fastest in the direction opposite to the vector where the function changes the fastest, so it is easier to find the minimum value of the function. Based on this, in an embodiment of the present application, the target large language model can be repeatedly iterated and trained by the gradient descent algorithm so that the loss function is continuously reduced, thereby reducing the error of the calculation result.

[0075] In the embodiment of the present application, the denoising model is repeatedly trained by using a gradient descent algorithm so that the loss function is continuously reduced to obtain an iterative target large language model, and then a more accurate speech stream corresponding to the text input can be obtained based on the iterative target large language model.

[0076] See also Figure 3 , Figure 3The present invention provides a schematic block diagram of a text-to-speech device provided in an embodiment of the present invention. The text-to-speech device can be configured in a server to execute the above-mentioned text-to-speech method.

[0077] like Figure 4 As shown, the text-to-speech device 200 includes: an acquisition module 201 , a processing module 202 , a determination module 203 and a conversion module 204 .

[0078] The acquisition module 201 is used to acquire a target text and determine semantic information corresponding to the target text;

[0079] A processing module 202 is used to process the semantic information through a speech segmenter in a target large language model to obtain a discrete speech token corresponding to the target text, wherein the discrete speech token includes encoding information corresponding to the semantic information;

[0080] The determination module 203 is used to determine the position information corresponding to the discrete speech token based on the encoding information; and predict the discrete speech token through the vocoder in the target large language model to obtain the speech signal corresponding to the discrete speech token;

[0081] The conversion module 204 is used to sort the voice signals according to the position information to obtain a target voice stream corresponding to the target text.

[0082] The determination module 203 is also used to perform a preprocessing operation on the target text to obtain a preprocessed target text, wherein the preprocessing operation includes a word segmentation operation, a normalization operation, and a stop word removal operation; analyze the text structure information of the preprocessed target text, and determine the semantic information corresponding to the target text based on the text structure information.

[0083] The processing module 202 is also used to obtain a preset code book, and determine the target hidden representation corresponding to the semantic information through the convolution encoder; determine the code vector with the smallest distance from the target hidden representation from the code vectors in the preset code book through the vector quantizer, and determine the index of the code vector with the smallest distance as the discrete speech token corresponding to the target text.

[0084] The acquisition module 201 is also used to obtain a training set, which includes a number of input texts; annotate the training set to obtain corresponding annotation results, wherein the training set annotation results include a speech stream corresponding to each of the input texts; and train an initial large language model through the training set and the annotation results to obtain the target large language model.

[0085] The acquisition module 201 is further used to splice the input text and the voice stream on the time axis to obtain spliced ​​training data; and train the initial large language model based on the spliced ​​training data to obtain the target large language model.

[0086] The acquisition module 201 is also used to iteratively train the target large language model to extract data features and calculate the loss function; iteratively train the loss function using a preset method for the purpose of reducing the value of the loss function until the expected threshold requirement is met; based on the loss function after iterative training, obtain the iterative target large language model.

[0087] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0088] The method and apparatus of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0089] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in Figure 4 Runs on the computer device shown.

[0090] See also Figure 4 , Figure 4 1 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0091] like Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a volatile storage medium, a non-volatile storage medium and an internal memory.

[0092] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any text-to-speech method.

[0093] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0094] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any text-to-speech method.

[0095] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that the structure of the computer device is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0096] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0097] Among them, in some embodiments, the processor is used to obtain a target text and determine the semantic information corresponding to the target text; process the semantic information through a speech segmenter in a target large language model to obtain a discrete speech token corresponding to the target text, wherein the discrete speech token includes encoding information corresponding to the semantic information; based on the encoding information, determine the position information corresponding to the discrete speech token; and predict the discrete speech token through a vocoder in the target large language model to obtain a speech signal corresponding to the discrete speech token; sort the speech signal according to the position information to obtain a target speech stream corresponding to the target text.

[0098] In some embodiments, the processor is also used to perform a preprocessing operation on the target text to obtain a preprocessed target text, wherein the preprocessing operation includes a word segmentation operation, a normalization operation, and a stop word removal operation; analyze the text structure information of the preprocessed target text, and determine the semantic information corresponding to the target text based on the text structure information.

[0099] In some embodiments, the processor is further used to obtain a training set, which includes several input texts; annotate the training set to obtain corresponding annotation results, wherein the training set annotation results include a speech stream corresponding to each of the input texts; and train an initial large language model using the training set and the annotation results to obtain the target large language model.

[0100] In some embodiments, the processor is further used to splice the input text and the voice stream on a time axis to obtain spliced ​​training data; and train the initial large language model based on the spliced ​​training data to obtain the target large language model.

[0101] In some embodiments, the processor is also used to iteratively train the target large language model to extract data features and calculate a loss function; iteratively train the loss function using a preset method for the purpose of reducing the value of the loss function until an expected threshold requirement is met; and based on the loss function after iterative training, obtain the iterative target large language model.

[0102] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any one of the text-to-speech conversion methods provided in the embodiments of the present application is implemented.

[0103] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0104] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like.

[0105] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for converting text to speech, characterized in that: The method comprises: Acquire a target text, and determine semantic information corresponding to the target text; Processing the semantic information through a speech segmenter in a target large language model to obtain discrete speech tokens corresponding to the target text, wherein the discrete speech tokens include encoding information corresponding to the semantic information; Based on the coding information, determining the position information corresponding to the discrete speech token; and predicting the discrete speech token through the vocoder in the target large language model to obtain the speech signal corresponding to the discrete speech token; Used to sort the voice signals according to the position information to obtain a target voice stream corresponding to the target text.

2. The method according to claim 1, characterized in that The determining of semantic information corresponding to the target text includes: Performing a preprocessing operation on the target text to obtain a preprocessed target text, wherein the preprocessing operation includes a word segmentation operation, a normalization operation, and a stop word removal operation; The text structure information of the preprocessed target text is analyzed, and semantic information corresponding to the target text is determined based on the text structure information.

3. The method according to claim 1, characterized in that The discrete speech tokens are obtained by the following formula: Among them, μ l is the discrete speech token; ||·| |2 is the L2 norm; VQ is the vector quantizer; h l is the hidden representation; C is the preset code book.

4. The method according to claim 1, characterized in that: The speech word segmenter includes a vector quantizer and a convolutional encoder, and the semantic information is processed by the speech word segmenter in the target large language model to obtain discrete speech tokens corresponding to the target text, including: Obtaining a preset code book, and determining a target hidden representation corresponding to the semantic information through the convolutional encoder; The vector quantizer determines the code vector with the smallest distance from the target hidden representation from the code vectors in the preset code book, and determines the index of the code vector with the smallest distance as the discrete speech token corresponding to the target text.

5. The method according to claim 1, characterized in that The target large language model is obtained in the following way: Obtaining a training set, wherein the training set includes a plurality of input texts; Annotating the training set to obtain corresponding annotation results, wherein the training set annotation results include a speech stream corresponding to each of the input texts; The initial large language model is trained using the training set and the annotation results to obtain the target large language model.

6. The method according to claim 5, characterized in that The training of the initial large language model by using the training set and the annotation result to obtain the target large language model includes: Splicing the input text and the voice stream on the time axis to obtain spliced ​​training data; The initial large language model is trained based on the concatenated training data to obtain the target large language model.

7. The method according to claim 5, characterized in that After obtaining the target large language model, the method further includes: Iteratively training the target large language model to extract data features, and calculating a loss function; Iteratively training the loss function using a preset method for the purpose of reducing the value of the loss function until the expected threshold value is met; Based on the loss function after iterative training, the target large language model after iteration is obtained.

8. A device for converting text to speech, characterized in that: The device comprises: An acquisition module, used to acquire a target text and determine semantic information corresponding to the target text; A determination module, configured to process the semantic information through a speech segmenter in a target large language model to obtain discrete speech tokens corresponding to the target text, wherein the discrete speech tokens include encoding information corresponding to the semantic information; A determination module, configured to determine position information corresponding to the discrete speech token based on the encoding information; and predict the discrete speech token through a vocoder in the target large language model to obtain a speech signal corresponding to the discrete speech token; The conversion module is used to sort the voice signals according to the position information to obtain a target voice stream corresponding to the target text.

9. A computer device, characterized in that: include: A memory and a processor; wherein the memory is connected to the processor and is used to store a program; the processor is used to implement the steps of the text-to-speech method as described in any one of claims 1-7 by running the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the text-to-speech method according to any one of claims 1 to 7.

Images