Text-to-speech method, device, terminal equipment and storage medium

Abstract

The application belongs to the field of voice playing, and particularly relates to a text-to-speech method and device, a terminal device and a storage medium. The method comprises the following steps: obtaining an initial text in an offline state; determining a target text according to the initial text in a preset offline voice package, wherein the offline voice package comprises a preset text set and a corresponding offline voice set, and the target text is a text in the text set that has a similarity greater than a preset threshold with the initial text; determining a voice corresponding to the target text from the offline voice set according to the target text, and playing the voice corresponding to the target text. That is, the application obtains an offline voice set according to an online voice, determines a target text in the offline voice set, and determines a voice corresponding to the target text from the offline voice set according to the target text. The voice corresponding to the target text can be played in real time, the gap between offline TTS and online TTS is reduced, and the playing effect of offline TTS is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of voice playing, and particularly relates to a text-to-speech method and device, a terminal device and a storage medium. BACKGROUND

[0002] Text-to-speech (TTS) is a technology of intelligently converting text into natural speech.

[0003] Currently, offline TTS is usually implemented by a local terminal device through a speech synthesis algorithm to synthesize text into speech. However, due to the computing power of the central processing unit (CPU), graphics processing unit (GPU), neural network processing unit (NPU) and other computing units of the mobile terminal, the terminal device cannot achieve real-time performance while ensuring the effect, thereby resulting in poor effect of offline TTS, which cannot reach the effect of online TTS. SUMMARY

[0004] The embodiments of the present application provide a text-to-speech method, device, terminal device and storage medium, which can reduce the gap between offline TTS and online TTS and improve the playing effect of offline TTS.

[0005] In a first aspect, the embodiments of the present application provide a text-to-speech method, comprising:

[0006] obtaining an initial text at an offline time;

[0007] determining a target text according to the initial text in a preset offline speech package, the offline speech package comprising a preset text set and a corresponding offline speech set, the offline speech set being obtained according to online speech, and the target text being a text in the text set having a similarity greater than a preset threshold with the initial text;

[0008] determining speech corresponding to the target text from the offline speech set according to the target text, and playing the speech corresponding to the target text.

[0009] In a possible implementation manner of the first aspect, before the obtaining of the initial text at the offline time, the method further comprises:

[0010] obtaining the text set, the text set comprising a target text;

[0011] converting the text set by an online text-to-speech tool, and obtaining the offline speech package according to a conversion result;

[0012] or;

[0013] The text set is searched in the online voice set, and the offline voice package is obtained based on the search results.

[0014] The step of determining the target text based on the initial text includes:

[0015] Calculate the similarity between the initial text and each text in the text set;

[0016] The text corresponding to the similarity scores that are greater than a preset threshold among the various similarity scores is determined as the target text.

[0017] The calculation of the similarity between the initial text and each text in the text set includes:

[0018] Obtain the text vector of the initial text and the text vector of each text in the text set;

[0019] Calculate the distance between the text vector of the initial text and the text vectors of each text in the text set, and determine the distance as the similarity between the initial text and each text in the text set.

[0020] The step of calculating the distance between the text vector of the initial text and the text vectors of each text in the text set includes:

[0021] The distance between the text vector of the initial text and the text vectors of each text in the text set is calculated using the following formula:

[0022]

[0023] Where, d i x represents the distance between the text vector of the initial text and the text vectors of all texts in the text set. 11 and x 12 The text vector x represents the initial text. i1 and x i2 This represents the text vector of each text in the text set.

[0024] The calculation of the similarity between the initial text and each text in the text set includes:

[0025] Obtain the word segmentation vector of the initial text and the word segmentation vector of each text in the text set;

[0026] Calculate the similarity coefficient between the word segmentation vector of the initial text and the word segmentation vector of each text in the text set, and determine the similarity coefficient as the similarity between the initial text and each text in the text set.

[0027] The similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set is calculated, including:

[0028] The similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set is calculated according to the following formula:

[0029]

[0030] Wherein, j i represents the similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set, A represents the word segmentation set vector of each text in the text set, and B i represents the word segmentation set vector of the initial text.

[0031] In a second aspect, an embodiment of the present application provides a text-to-speech device, comprising:

[0032] An acquisition module is configured to acquire an initial text in an offline state.

[0033] A first determination module is configured to determine a target text from a preset offline speech package according to the initial text, the target text being a text in the offline speech package having a similarity greater than a preset threshold with the initial text, and the offline speech package being obtained according to an online speech.

[0034] A second determination module is configured to determine a speech corresponding to the target text from the offline speech set according to the target text, and play the speech corresponding to the target text.

[0035] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method of any one of the first aspect when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the method of any one of the first aspect.

[0037] Compared with the prior art, the embodiment of the present application has the beneficial effects that: the technical scheme of the present application obtains the initial text when offline; in the preset offline voice package, the target text is determined according to the initial text, the offline voice package includes a preset text set and a corresponding offline voice set, the offline voice set is obtained according to the online voice, and the target text is a text in the text set with a similarity greater than a preset threshold to the initial text; according to the target text, the voice corresponding to the target text is determined from the offline voice set, and the voice corresponding to the target text is played. That is, the present application obtains the offline voice set according to the online voice, determines the target text in the offline voice set, and determines the voice corresponding to the target text from the offline voice set according to the target text, so that the voice corresponding to the target text can be played in real time, the gap between offline TTS and online TTS is reduced, and the playing effect of offline TTS is improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0039] Figure 1 is an application scenario diagram of a text-to-speech method provided by an embodiment of the present application;

[0040] Figure 2 is a schematic flowchart of a text-to-speech method provided by an embodiment of the present application;

[0041] Figure 3 is a schematic flowchart of a method for obtaining an offline voice package provided by an embodiment of the present application;

[0042] Figure 4 is a schematic flowchart of another method for obtaining an offline voice package provided by an embodiment of the present application;

[0043] Figure 5 is a schematic flowchart of a method for determining a target text provided by an embodiment of the present application;

[0044] Figure 6 is a schematic flowchart of a method for calculating the similarity between an initial text and each text in a text set provided by an embodiment of the present application;

[0045] Figure 7 is a schematic flowchart of another method for calculating the similarity between an initial text and each text in a text set provided by an embodiment of the present application;

[0046] Figure 8is a structural schematic diagram of a text-to-speech device provided by an embodiment of the present application;

[0047] Figure 9 is a structural schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, for the purposes of explanation, numerous specific details are set forth in order to thoroughly describe the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, and apparatuses are omitted so as not to obscure the description of the present application with unnecessary detail. In other instances, well-known structures have not been described in detail in order to avoid obscuring the present application.

[0049] It should be understood that the term "comprises" when used in this specification and the appended claims, specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0050] It should also be understood that the term "and / or" when used in this specification and the appended claims, means any one of the associated listed items, or a combination of any combination of at least one of the associated listed items.

[0051] In this specification, the phrase "one or more embodiments of the present application" or "some embodiments of the present application" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one or more embodiments of the present application" or "in some embodiments of the present application" in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specified. The terms "comprise," "comprises," "comprising," "include," "includes," "including," "contain," "contains," "containing," and the like mean "including but not limited to."

[0052] In addition, in the description of the specification and the appended claims, the terms "first," "second," and the like are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0053] Currently, offline TTS is usually realized by local terminals through voice synthesis algorithm to synthesize voice from text. However, due to the computing power of CPU, GPU and NPU of mobile terminals, the terminal device cannot guarantee the effect while achieving real-time, thus leading to poor effect of offline TTS, which cannot reach the effect of online TTS.

[0054] To solve the above-mentioned defects, the inventive concept of the present application is:

[0055] According to the online voice, the offline voice set is obtained, the target text is determined in the offline voice set, and the voice corresponding to the target text is determined from the offline voice set according to the target text. The voice corresponding to the target text can be played in real time, the gap between offline TTS and online TTS is reduced, and the playing effect of offline TTS is improved.

[0056] In order to illustrate the technical scheme of the present application, the following will be described by specific embodiments.

[0057] Please refer to Figure 1 , Figure 1 is an application scenario diagram of an offline voice playing method provided by an embodiment of the present application. For the convenience of description, only the part related to the present application is shown. The application scenario includes a terminal device 10 and a carrier 20, and the carrier 20 contains text 201.

[0058] The terminal device 10 includes a scanning pen, a translation pen, a point-and-read pen, a server, a computer, a smart phone and a tablet computer, etc. The specific type of the terminal device is not limited by the embodiments of the present application.

[0059] The terminal device 10 is used to identify the text 201 in the carrier 20 according to the optical image recognition technology, and to play the identified text 201 according to the digital voice technology.

[0060] The carrier 20 includes an audio book, and the audio book is configured with an invisible optical identification code. The terminal device 10 scans the invisible optical identification code to identify the text in the audio book and play the identified text.

[0061] The text 201 includes characters, images, videos, etc. The type of the text 201 is not limited by the embodiments of the present application.

[0062] The terminal device 20 stores an offline voice package, which is used to determine a target text in the offline voice package, determine a voice corresponding to the target text from the offline voice set according to the target text, and play the voice corresponding to the target text.

[0063] Please refer to Figure 2 , Figure 2is a schematic flow chart of a text-to-speech method provided by an embodiment of the present application. Figure 2 The execution subject of the method in the embodiment of the present application can be a terminal device. Figure 2 As shown in the embodiment of the present application, the method comprises S201-S203.

[0064] S201, the terminal device acquires initial text in offline state.

[0065] The purpose of the embodiment of the present application is to reduce the difference between the effect of speech playing in online state and the effect of speech playing in offline state, so the text to be recognized needs to be processed in offline state.

[0066] Specifically, offline is a state, which means that the terminal device interrupts the connection with the Internet, and can also be called offline or offline.

[0067] The initial text refers to the text on the carrier to be recognized.

[0068] In offline state of the terminal device, the initial text in offline state can be acquired by scanning the text to be recognized on the carrier through the terminal device.

[0069] S202, the terminal device determines target text according to the initial text in a preset offline speech package.

[0070] Specifically, the offline speech package comprises a preset text set and a corresponding offline speech set, and the offline speech set is obtained according to online speech.

[0071] The offline speech package in the embodiment of the present application is pre-stored in the terminal device, and the method for making the offline speech package is described in the embodiment of the present application. Figure 3 , Figure 3 is a schematic flow chart of a method for obtaining an offline speech package provided by an embodiment of the present application. Figure 3 The execution subject of the method in the embodiment of the present application can be a terminal device. Figure 3 As shown in the embodiment of the present application, the method comprises S301-S302.

[0072] S301, acquire a text set.

[0073] Specifically, the text set refers to a collection of all texts contained in the offline speech package. The more texts contained in the text set, the higher the speed and accuracy of speech recognition, and the higher the speed and accuracy of speech playing.

[0074] Exemplarily, the text set is {text 1, text 2, text 3, …, text n-1, text n}.

[0075] The text set includes target texts. When identifying the initial text, the initial text needs to be matched with the texts in the text set in terms of similarity. The text in the text set that has a similarity greater than a preset threshold with the initial text is referred to as a target text in the embodiment of the present application.

[0076] In S302, the text set is converted by an online text-to-speech tool, and an offline speech package is obtained according to the conversion result.

[0077] In order to reduce the gap between offline TTS and online TTS, in the embodiment of the present application, when the offline speech package is made, the text set in the offline speech package is converted by an online TTS tool to obtain an offline speech set corresponding to the text set, and then the offline speech package can be obtained. The offline speech package includes the text set and the corresponding offline speech set.

[0078] The online TTS tool includes a neural network model and a vocoder. The neural network model is trained using texts and speech corresponding to the texts. The neural network model is used to extract acoustic features of the texts. The vocoder is used to convert the extracted acoustic features into a waveform to obtain speech corresponding to the texts.

[0079] The acoustic model includes a spectrum prediction model Tacotron2, a speech synthesis model based on an attention mechanism Transformer TTS, a fast high-quality speech synthesis model SpeedySpeech, etc. The vocoder includes a vocoder based on a regression model and a flow model WaveFlow, a fast lightweight vocoder Parallel WaveGAN, Multi Band MelGAN, a MelGAN vocoder Style MelGAN, an efficient vocoder HiFiGAN with multi-scale and multi-period discriminators, etc.

[0080] For example, the text set is {text1, text2, text3, …, textn-1, textn}, which is converted by the online TTS tool to obtain {speech1, speech2, speech3, …, speechn-1, speechn}. The {speech1, speech2, speech3, …, speechn-1, speechn} is referred to as an offline speech set in the embodiment of the present application. The texts in the text set correspond one-to-one to the speech in the offline speech set.

[0081] In the embodiment of the present application, since the offline speech package includes the text set and the offline speech set, and the offline speech set is generated by converting the online TTS tool, when offline, the speech corresponding to the target text can be played by using the offline speech package, the gap between offline TTS and online TTS can be reduced while the speech corresponding to the target text is played in real time, and the playing effect of offline TTS is improved.

[0082] Please refer to Figure 4 , Figure 4is a schematic flowchart of another method for obtaining an offline voice package provided in an embodiment of the present application. Figure 4 The execution subject of the method in the method can be a terminal device. As shown in Figure 4 The method includes S401-S402.

[0083] S401, obtaining a text set.

[0084] Specifically, the method of S401 is the same as that of S301, which will not be described here.

[0085] S402, searching the text set in an online voice set, and obtaining an offline voice package according to a search result.

[0086] Specifically, the online voice set includes a pre-recorded voice set.

[0087] For example, when user A uses a pen to read English words in an English textbook that have not been learned or mastered, user B (who has learned or mastered all the words in the English textbook) pre-records the voices corresponding to all the words in the English textbook. The pre-recorded voices corresponding to all the words in the English textbook are referred to as an online voice set in an embodiment of the present application. Assuming that there are 10 English words that user A has not learned or mastered, user B searches the 10 English words in the online voice set, obtains the voices corresponding to the 10 English words, and the collection of the voices corresponding to the 10 English words is referred to as an offline voice set in an embodiment of the present application.

[0088] The text set and the offline voice set are packaged to obtain an offline voice package.

[0089] In an embodiment of the present application, since the offline voice package includes the text set and the offline voice set, and the offline voice set is generated by searching in the online voice set, when offline, the voice corresponding to the target text can be played in real time by using the offline voice package to play the voice corresponding to the target text, the gap between offline TTS and online voice playing can be reduced, and the playing effect of offline TTS can be improved.

[0090] In an embodiment of the present application, the offline voice package can be obtained directly according to the text set.

[0091] For example, user B does not need to search the 10 English words in the online voice set, but directly records the voices corresponding to the 10 English words to generate an offline voice set. The text set and the offline voice set are packaged to obtain an offline voice package.

[0092] In the embodiment of the present application, since the offline voice package includes a text set and an offline voice set which is directly generated according to the text set, when offline, the voice corresponding to the target text is played by using the offline voice package, so that the voice corresponding to the target text can be played in real time, and the gap between offline TTS and online voice playing is reduced, and the playing effect of offline TTS is improved.

[0093] In the embodiment of the present application, the target text is a text in the offline voice package which has a similarity greater than a preset threshold with the initial text. Please refer to Figure 5 , Figure 5 is a schematic flowchart of a method for determining a target text provided by the embodiment of the present application. Figure 5 The execution subject of the method in the embodiment of the present application can be a terminal device. As shown in Figure 5 , the method includes S501 to S502.

[0094] S501, the terminal device calculates the similarity of the initial text and each text in the text set.

[0095] Specifically, the similarity refers to the similarity degree of two texts, including literal similarity, semantic similarity and style similarity, etc. Please refer to Figure 6 , Figure 6 is a schematic flowchart of a method for calculating the similarity of the initial text and each text in the text set provided by the embodiment of the present application. Figure 6 The execution subject of the method in the embodiment of the present application can be a terminal device. As shown in Figure 6 , the method includes S601 to S602.

[0096] S601, the terminal device obtains the text vector of the initial text and the text vector of each text in the text set.

[0097] Specifically, the method for obtaining the text vector of the initial text and the text vector of each text in the text set in the embodiment of the present application includes, but is not limited to, obtaining the text vector of the initial text and the text vector of each text in the text set through a shallow and double-layer neural network model word2vec which has been trained.

[0098] The word2vec model can map the word segmentation in each text to a vector.

[0099] In the embodiment of the present application, when obtaining the text vector of the initial text and the text vector of each text in the text set, first, the initial text and each text in the text set are subjected to word segmentation operation to obtain the word segmentation of the initial text and the word segmentation of each text in the text set. The word segmentation operation can be performed by using a dictionary-based word segmentation algorithm or a statistical-based machine learning algorithm, and the method for word segmentation operation is not limited in the embodiment of the present application.

[0100] Then, the word segmentation of the initial text and the word segmentation of each text in the text set are respectively input into the word2vec model to obtain the word segmentation vector of the initial text and the word segmentation vector of each text in the text set.

[0101] Finally, the word segmentation vectors of the initial text are summed to obtain the text vector of the initial text, and the word segmentation vectors of each text in the text set are summed to obtain the word segmentation vectors of each text in the text set. Specifically, when the word segmentation components are summed, the text vector can be obtained by weighted summation.

[0102] S602, the terminal device calculates the distance between the text vector of the initial text and the text vector of each text in the text set, and determines the distance as the similarity between the initial text and each text in the text set.

[0103] Specifically, the method for calculating the distance between the text vector of the initial text and the text vector of each text in the text set includes, but is not limited to, Euclidean distance, cosine distance, Manhattan distance, Chebyshev distance, Minkowski distance, Mahalanobis distance, standardized Euclidean distance, Hamming distance, Jaccard distance and correlation distance.

[0104] In the embodiment of the application, when the obtained text vector of the initial text and the text vector of each text in the text set are both two-dimensional vectors, the distance between the text vector of the initial text and the text vector of each text in the text set can be calculated according to the following formula:

[0105]

[0106] wherein d i represents the distance between the text vector of the initial text and the text vector of each text in the text set, x 11 and x 12 represent the text vector (two-dimensional vector) of the initial text, x i1 and x i2 represent the text vector (two-dimensional vector) of each text in the text set.

[0107] In other embodiments, when the obtained text vector of the initial text and the text vector of each text in the text set are both multi-dimensional vectors, the distance between the text vector of the initial text and the text vector of each text in the text set can be calculated according to the following formula:

[0108]

[0109] wherein d i represents the distance between the text vector of the initial text and the text vector of each text in the text set, x 11 , x 12 and x iN represent the text vector (multi-dimensional vector) of the initial text, x i1 , xi2 and x 1N represents the text vector (multi-dimensional vector) of each text in the text set, and N represents the dimension.

[0110] In the embodiment of the present application, the smaller the distance between the text vector of the initial text and the text vector in the text set, the more similar the text vector of the initial text and the text vector in the text set.

[0111] Generally, the value of the similarity is [0, 1], and the closer to 1, the higher the similarity. In order to make the value of the similarity obtained by the present application meet this concept, the distance d i is converted by the following formula:

[0112]

[0113] wherein D represents the similarity of the initial text and each text in the text set, and the greater the value of D, the more similar the initial text and the text in the text set.

[0114] Please refer to Figure 7 , Figure 7 is another schematic flowchart of a method for calculating the similarity of the initial text and each text in the text set provided by the embodiment of the present application. Figure 7 The execution subject of the method in Figure 7 may be a terminal device. As shown in the figure, the method comprises S701-S702.

[0115] S701, the terminal device obtains the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set.

[0116] Specifically, the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set can be obtained by referring to S601, which will not be described here. In the embodiment of the present application, the word segmentation set vector is a collection of word vectors.

[0117] S702, the terminal device calculates the similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set, and determines the similarity coefficient as the similarity of the initial text and each text in the text set.

[0118] Specifically, the similarity coefficient refers to the quantitative index of the degree of similarity between the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set.

[0119] In the embodiment of the present application, the similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set is calculated according to the following formula:

[0120]

[0121] wherein j iLet A represent the similarity coefficient between the word segmentation vector of the initial text and the word segmentation vectors of each text in the text set, and B represent the word segmentation vectors of each text in the text set. i This represents the word segmentation vector of the initial text.

[0122] In this embodiment of the application, the larger the similarity coefficient, the more similar the two word segmentation vectors are.

[0123] S502, The terminal device identifies the text corresponding to the similarity scores that are greater than a preset threshold as the target text.

[0124] Specifically, the preset threshold can be set according to different methods for calculating similarity. For example, based on... Figure 6 When calculating similarity using the method described in this application, the preset threshold can be set to (0.7, 0.9), for example, 0.8. In this embodiment, text with a similarity score greater than 0.8 can be identified as the target text.

[0125] For example, according to Figure 7 When calculating similarity using the method described in this application, the preset threshold can be set to (0.6, 0.8), for example, 0.7. In this embodiment, text with a similarity score greater than 0.7 can be identified as the target text.

[0126] In other embodiments, the method for determining the target text further includes: performing a weighted summation of the similarity between the initial text and each text in the text set based on the similarity between the initial text and each text in the text set and a preset weight, obtaining a weighted summation result, and determining the text corresponding to the similarity that is greater than a preset threshold as the target text.

[0127] For example, the terminal device pre-stores and utilizes Figure 6 The similarity weight d obtained by the method in [the text] i For example, ω1, pre-stored and utilized Figure 7 The similarity weights j obtained by the method in [the original text] i For example, ω2, then the weighted sum is obtained using the following formula:

[0128] M=ω1×d i +ω2×j i .

[0129] Where M is the weighted summation result.

[0130] The preset threshold can be set to (0.6, 0.8), for example, 0.7. In this embodiment, the text corresponding to values ​​greater than 0.7 in each weighted summation result can be identified as the target text.

[0131] S203. The terminal device determines the corresponding voice from the offline voice set based on the target text, and plays the voice corresponding to the target text.

[0132] Specifically, the terminal device determines the target text, and the offline speech set of the offline speech package stores the speech corresponding to the target text.

[0133] To sum up, in the technical scheme of the present application, the initial text at the offline time is obtained, the target text is determined according to the initial text in the preset offline speech package, the offline speech package includes a preset text set and a corresponding offline speech set, the offline speech set is obtained according to the online speech, the target text is a text in the text set with a similarity greater than a preset threshold to the initial text, the speech corresponding to the target text is determined from the offline speech set according to the target text, and the speech corresponding to the target text is played. That is, the offline speech set is obtained according to the online speech, the target text is determined in the offline speech set, and the speech corresponding to the target text is determined from the offline speech set according to the target text. The speech corresponding to the target text can be played in real time, the gap between the offline TTS and the online TTS is reduced, and the playing effect of the offline TTS is improved.

[0134] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0135] Please refer to Figure 8 , Figure 8 is a structural schematic diagram of a text-to-speech device provided by the embodiment of the present application, which comprises:

[0136] The acquisition module 81 is configured to acquire an initial text at an offline time.

[0137] The first determination module 82 is configured to determine a target text according to the initial text in a preset offline speech package, the offline speech package includes a preset text set and a corresponding offline speech set, the offline speech set is obtained according to the online speech, and the target text is a text in the text set with a similarity greater than a preset threshold to the initial text.

[0138] The second determination module 83 is configured to determine the speech corresponding to the target text from the offline speech set according to the target text, and play the speech corresponding to the target text.

[0139] The device further comprises an offline speech package acquisition module 84 configured to acquire a text set, the text set includes the target text.

[0140] The text set is converted by an online text-to-speech tool, and the offline speech package is obtained according to the conversion result.

[0141] Or;

[0142] The text set is searched in the online speech set, and an offline speech packet is obtained according to a search result.

[0143] The first determining module 82 is further configured to calculate the similarity of the initial text and each text in the text set.

[0144] The text corresponding to the similarity greater than the preset threshold in the similarities is determined as the target text.

[0145] The first determining module 82 is further configured to obtain the text vector of the initial text and the text vector of each text in the text set.

[0146] The distance between the text vector of the initial text and the text vector of each text in the text set is calculated, and the distance is determined as the similarity of the initial text and each text in the text set.

[0147] The first determining module 82 is further configured to calculate the distance between the text vector of the initial text and the text vector of each text in the text set according to the following formula:

[0148]

[0149] Wherein, d i represents the distance between the text vector of the initial text and the text vector of each text in the text set, x 11 represents the text vector of the initial text, x 12 represents the text vector of the initial text. i1 i2 represents the text vector of each text in the text set.

[0150] The first determining module 82 is further configured to obtain the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set.

[0151] The similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set is calculated, and the similarity coefficient is determined as the similarity of the initial text and each text in the text set.

[0152] The first determining module 82 is further configured to calculate the similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set according to the following formula:

[0153]

[0154] Wherein, j i represents the similarity coefficient of the word segmentation set vector of the initial text and the word segmentation set vector of each text in the text set, A represents the word segmentation set vector of each text in the text set, and B i represents the word segmentation set vector of the initial text.

[0155] ​Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the application. The specific working process of the unit and module in the system can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0156] As shown in Figure 9 The terminal device 200 provided by the embodiment of the application comprises a memory 21, a processor 22, and a computer program 23 stored in the memory 21 and executable on the processor 22. When the processor 22 executes the computer program 23, the method for converting text to speech in each of the above embodiments is implemented.

[0157] The processor 22 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0158] The memory 21 can be an internal storage unit of the terminal device 200. The memory 21 can also be an external storage device of the terminal device 200, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 21 can include both the internal storage unit and the external storage device of the terminal device 200. The memory 21 is used to store computer programs and other programs and data required by the terminal device 200. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0159] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the text-to-speech method of the above-mentioned embodiments.

[0160] The embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal is caused to implement the text-to-speech method of the above-mentioned embodiments.

[0161] The integrated units, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods by a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of the above-mentioned method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable storage medium can not be electrical carrier signal and telecommunication signal.

[0162] In the above-mentioned embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0163] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0164] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the application.

[0165] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of text-to-speech, characterized by, The method comprises the following steps: acquiring an initial text at an offline time; determining a target text according to the initial text in a preset offline voice package, wherein the offline voice package comprises a preset text set and a corresponding offline voice set, the offline voice set is obtained according to an online voice, the text set in the offline voice package is converted by an online TTS tool to obtain the offline voice set corresponding to the text set, and then the offline voice package is obtained; the online TTS tool comprises a neural network model and a vocoder, the neural network model is trained by using a text and a voice corresponding to the text, the neural network model is used to extract acoustic features of the text, and the vocoder is used to convert the extracted acoustic features into a waveform to obtain the voice corresponding to the text; the target text is a text in the text set, and a similarity between the initial text and the target text is greater than a preset threshold; determining a voice corresponding to the target text from the offline voice set according to the target text, and playing the voice corresponding to the target text.

2. The method of claim 1, wherein, Before the step of acquiring the initial text at the offline time, the method further comprises the following steps: searching the text set in an online voice set, and obtaining the offline voice package according to a searching result.

3. The method of claim 2, wherein, The step of determining the target text according to the initial text comprises the following steps: calculating a similarity between the initial text and each text in the text set; determining a text corresponding to a similarity greater than a preset threshold in each similarity as the target text.

4. The method of claim 3, wherein, The step of calculating the similarity between the initial text and each text in the text set comprises the following steps: acquiring a text vector of the initial text and text vectors of each text in the text set; calculating a distance between the text vector of the initial text and the text vectors of each text in the text set, and determining the distance as the similarity between the initial text and each text in the text set.

5. The method of claim 4, wherein, The step of calculating the distance between the text vector of the initial text and the text vectors of each text in the text set comprises the following step: calculating the distance between the text vector of the initial text and the text vectors of each text in the text set according to the following formula: wherein d i represents the distance between the text vector of the initial text and the text vector of each text in the text set, x 11 and x 12 represents the text vector of the initial text, x i1 and x i2 represents the text vector of each text in the text set.

6. The method of claim 3, wherein, The step of calculating the similarity between the initial text and each text in the text set comprises the following steps: acquiring a word segmentation set vector of the initial text and word segmentation set vectors of each text in the text set; calculating a similarity coefficient between the word segmentation set vector of the initial text and the word segmentation set vectors of each text in the text set, and determining the similarity coefficient as the similarity between the initial text and each text in the text set.

7. The method of claim 6, wherein, The step of calculating the similarity coefficient between the word segmentation set vector of the initial text and the word segmentation set vectors of each text in the text set comprises the following step: calculating the similarity coefficient between the word segmentation set vector of the initial text and the word segmentation set vectors of each text in the text set according to the following formula: wherein j i represents the similarity coefficient of the word set vector of the initial text and the word set vector of each text in the text set, A represents the word set vector of each text in the text set, B i represents the word set vector of the initial text.

8. An apparatus for text-to-speech, the apparatus comprising: The method comprises the following steps: an acquiring module, configured to acquire an initial text at an offline time; The first determining module is configured to determine a target text from the initial text in a preset offline voice package, wherein the offline voice package comprises a preset text set and a corresponding offline voice set, and the offline voice set is obtained from online voice; when the offline voice package is made, the text set in the offline voice package is converted by an online TTS tool to obtain the offline voice set corresponding to the text set, and then the offline voice package is obtained; the online TTS tool comprises a neural network model and a vocoder; the neural network model is trained by using text and voice corresponding to the text; the neural network model is used to extract acoustic features of the text; the vocoder is used to convert the extracted acoustic features into a waveform to obtain voice corresponding to the text; and the target text is a text in the text set, which has a similarity greater than a preset threshold to the initial text. The second determining module is configured to determine voice corresponding to the target text from the offline voice set according to the target text, and play the voice corresponding to the target text.

9. A terminal device, comprising: The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images