Method and device for converting text into voice
By checking and generating target speech files in the storage space of terminal devices and business servers, the problem of high resource consumption and low efficiency caused by frequent interaction with speech conversion servers in existing technologies is solved, and more efficient speech synthesis is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2026-03-31
Smart Images
Figure CN121768360A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of speech synthesis technology, and in particular to a method and apparatus for text-to-speech conversion. Background Technology
[0002] Text-to-speech (TTS) technology, also known as speech synthesis technology, is a technique that uses algorithms or models to analyze and process input text, thereby generating the corresponding speech. Speech synthesis technology has a wide range of applications, such as in audiobook production, where it can convert book content into speech to provide audiobooks; and in navigation services, it can convert text navigation information into speech to provide voice navigation prompts.
[0003] Currently, the commonly used speech synthesis process is as follows: First, the text to be converted into speech is sent to a speech conversion server. Then, the speech conversion server processes the text using algorithms or models to obtain the corresponding speech. Finally, it receives the speech corresponding to the text sent by the speech conversion server. If speech synthesis is performed using the current process, interaction with the speech conversion server is necessary every time speech conversion occurs. This interaction consumes significant resources and is time-consuming, thus the efficiency of speech synthesis needs to be improved. Summary of the Invention
[0004] In view of this, embodiments of this application provide a method and apparatus for text-to-speech conversion to improve the efficiency of speech synthesis.
[0005] To achieve the above objectives, the technical solutions provided in this application are as follows:
[0006] In a first aspect, embodiments of this application provide a text-to-speech method, including:
[0007] The system receives a voice conversion operation from a user on the first text input in the video editing interface. The voice conversion operation is used to convert the first text into speech based on pre-configured speech synthesis parameters.
[0008] Based on the pre-configured speech synthesis parameters and the first text, it is determined whether a target speech file is stored in the first storage space. The target speech file is obtained in advance by converting the first text into speech based on the pre-configured speech synthesis parameters. The first storage space is the storage space of the terminal device.
[0009] If so, the target audio file is retrieved from the first storage space;
[0010] If not, the target speech file is generated based on the pre-configured speech synthesis parameters and the first text, and the target speech file is saved to the first storage space;
[0011] Based on the target audio file, add a first audio segment within the time interval corresponding to the first text on the editing timeline.
[0012] As an optional implementation of this application, the step of generating the target speech file based on the pre-configured speech synthesis parameters and the first text includes:
[0013] A speech conversion request is sent to the business server; the speech conversion request carries the pre-configured speech synthesis parameters and the first text, and is used to request speech conversion of the first text based on the pre-configured speech synthesis parameters.
[0014] Receive the access address of the target audio file sent by the service server;
[0015] The target audio file is obtained based on the access address.
[0016] As an optional implementation of this application, the step of determining whether a target speech file is stored in the first storage space based on the pre-configured speech synthesis parameters and the first text includes:
[0017] A target file identifier is generated based on the pre-configured speech synthesis parameters and the first text;
[0018] Determine whether the target file identifier is included in the first file identifier list;
[0019] If the target file identifier is included in the first file identifier list, then it is determined that the target audio file is stored in the first storage space;
[0020] If the target file identifier is not included in the first file identifier list, it is determined that the target audio file is not stored in the first storage space;
[0021] After saving the target audio file to the first storage space, the method further includes adding the target file identifier to the first file identifier list.
[0022] As an optional implementation of this application, the pre-configured speech synthesis parameters include: timbre parameters and language parameters; the step of generating a target file identifier based on the pre-configured speech synthesis parameters and the first text includes:
[0023] The timbre parameters, the language parameters, and the identifier information of the first text are concatenated to obtain concatenation parameters;
[0024] The concatenation parameters are processed using a message digest algorithm to obtain the target file identifier.
[0025] As an optional implementation of this application, before saving the target audio file to the first storage space, the method further includes:
[0026] Determine whether the amount of data in the voice files stored in the first storage space is greater than a first threshold;
[0027] In response to the fact that the amount of data of the voice files stored in the first storage space is greater than the first threshold, the idle time of each voice file in the first storage space is obtained; the idle time of any voice file is the time elapsed since the last time the voice file was accessed.
[0028] The voice file with the longest idle time is deleted from the first storage space, and the data volume of the voice files stored in the first storage space is re-evaluated to see if it is greater than the first threshold, until the data volume of the voice files stored in the first storage space is less than or equal to the first threshold.
[0029] Secondly, this application provides a text-to-speech method, including:
[0030] The system receives a voice conversion request sent by a terminal device; the voice conversion request carries pre-configured voice synthesis parameters and a first text, and is used to request voice conversion of the first text based on the pre-configured voice synthesis parameters.
[0031] The second storage space is determined based on the pre-configured speech synthesis parameters and the first text to determine whether a target speech file is stored therein. The target speech file is obtained in advance by converting the first text into speech based on the pre-configured speech synthesis parameters. The second storage space is the storage space of the business server.
[0032] If the target audio file is stored in the second storage space, then the access address of the target audio file is generated according to the storage location of the target audio file;
[0033] If the target speech file is not saved in the second storage space, the target speech file is generated according to the pre-configured speech synthesis parameters and the first text, the target speech file is saved to the second storage space, and the access address of the target speech file is generated according to the storage location of the target speech file.
[0034] The access address is sent to the terminal device so that the terminal device can obtain the target audio file based on the access address.
[0035] As an optional implementation of this application, the step of generating the target speech file based on the pre-configured speech synthesis parameters and the first text includes:
[0036] Send the voice conversion request to the voice conversion server;
[0037] Receive the target audio file sent by the speech conversion server.
[0038] As an optional implementation of this application, the step of determining whether a target speech file is stored in the second storage space based on the pre-configured speech synthesis parameters and the first text includes:
[0039] A target file identifier is generated based on the pre-configured speech synthesis parameters and the first text;
[0040] Determine whether the target file identifier is included in the second file identifier list. If the target file identifier is included in the second file identifier list, determine that the target audio file is stored in the second storage space. If the target file identifier is not included in the second file identifier list, determine that the target audio file is not stored in the second storage space.
[0041] After saving the target audio file to the second storage space, the method further includes: adding the target file identifier to the second file identifier list.
[0042] As an optional implementation of this application, before saving the target audio file to the second storage space, the method further includes:
[0043] Determine whether the amount of data in the voice files stored in the second storage space is greater than the second threshold;
[0044] In response to the fact that the amount of data of the voice files stored in the second storage space is greater than the second threshold, the untouched duration of each voice file stored in the second storage space is obtained; the untouched duration of any voice file is the time elapsed since the last time the voice file was accessed.
[0045] The longest untouched audio file is deleted from the second storage space, and the data volume of the audio files stored in the second storage space is re-evaluated to see if it is greater than the second threshold, until the data volume of the audio files stored in the second storage space is less than or equal to the second threshold.
[0046] Thirdly, embodiments of this application provide a terminal device, including:
[0047] The user input unit is used to receive the user's voice conversion operation for the first text input in the video editing interface. The voice conversion operation is used to convert the first text into speech based on pre-configured speech synthesis parameters.
[0048] The processing unit is configured to determine whether a target speech file is stored in the first storage space based on the pre-configured speech synthesis parameters and the first text, wherein the target speech file is obtained in advance by converting the first text into speech based on the pre-configured speech synthesis parameters; the first storage space is the storage space of the terminal device.
[0049] The acquisition unit is configured to acquire the target speech file from the first storage space when the target speech file is stored in the first storage space, and generate the target speech file from the first text according to the pre-configured speech synthesis parameters and the first text when the target speech file is not stored in the first storage space, and save the target speech file to the first storage space.
[0050] The editing unit is used to add a first audio segment to the time interval corresponding to the first text on the editing timeline, based on the target audio file.
[0051] As an optional implementation of this application, the acquisition unit is specifically used to send a speech conversion request to the service server; the speech conversion request carries the pre-configured speech synthesis parameters and the first text, and is used to request speech conversion of the first text based on the pre-configured speech synthesis parameters; receive the access address of the target speech file sent by the service server; and acquire the target speech file according to the access address.
[0052] As an optional implementation of this application, the processing unit is specifically configured to generate a target file identifier based on the pre-configured speech synthesis parameters and the first text; determine whether the target file identifier is included in the first file identifier list; if the target file identifier is included in the first file identifier list, determine that the target speech file is stored in the first storage space; if the target file identifier is not included in the first file identifier list, determine that the target speech file is not stored in the first storage space.
[0053] The acquisition unit is further configured to add the target file identifier to the first file identifier list after saving the target audio file to the first storage space.
[0054] As an optional implementation of this application, the pre-configured speech synthesis parameters include: timbre parameters and language parameters; the processing unit is specifically used to concatenate the timbre parameters, the language parameters, and the identifier information of the first text to obtain concatenation parameters; and to process the concatenation parameters based on a message digest algorithm to obtain the target file identifier.
[0055] As an optional implementation of this application, the acquisition unit is further configured to determine whether the data volume of the voice files stored in the first storage space is greater than a first threshold before saving the target voice file to the first storage space; in response to the data volume of the voice files stored in the first storage space being greater than the first threshold, acquire the idle time of each voice file in the first storage space; the idle time of any voice file is the time elapsed since the last access to the voice file; delete the voice file with the longest idle time from the first storage space, and re-determine whether the data volume of the voice files stored in the first storage space is greater than the first threshold, until the data volume of the voice files stored in the first storage space is less than or equal to the first threshold.
[0056] Fourthly, embodiments of this application provide a service server, including:
[0057] A receiving unit is configured to receive a voice conversion request sent by a terminal device; the voice conversion request carries pre-configured voice synthesis parameters and a first text, and is configured to request voice conversion of the first text based on the pre-configured voice synthesis parameters;
[0058] The processing unit is configured to determine whether a target speech file is stored in the second storage space based on the pre-configured speech synthesis parameters and the first text, wherein the target speech file is obtained in advance by converting the first text into speech based on the pre-configured speech synthesis parameters; the second storage space is the storage space of the business server.
[0059] The generation unit is configured to, when the target speech file is stored in the second storage space, generate an access address for the target speech file based on the storage location of the target speech file; when the target speech file is not stored in the second storage space, generate the target speech file based on the pre-configured speech synthesis parameters and the first text, save the target speech file to the second storage space, and generate an access address for the target speech file based on the storage location of the target speech file.
[0060] The sending unit is configured to send the access address to the terminal device so that the terminal device can obtain the target audio file based on the access address.
[0061] As an optional implementation of this application, the generation unit is specifically used to send the voice conversion request to the voice conversion server and receive the target voice file sent by the voice conversion server.
[0062] As an optional implementation of this application, the processing unit is specifically used to generate a target file identifier based on the pre-configured speech synthesis parameters and the first text; determine whether the target file identifier is included in the second file identifier list; if the target file identifier is included in the second file identifier list, determine that the target speech file is stored in the second storage space; if the target file identifier is not included in the second file identifier list, determine that the target speech file is not stored in the second storage space.
[0063] The generation unit is further configured to add the target file identifier to the second file identifier list after saving the target audio file to the second storage space.
[0064] As an optional implementation of this application, the generation unit is further configured to, before saving the target voice file to the second storage space, determine whether the data volume of the voice file saved in the second storage space is greater than a second threshold; in response to the data volume of the voice file saved in the second storage space being greater than the second threshold, obtain the untouched duration of each voice file saved in the second storage space; the untouched duration of any voice file is the time elapsed since the last access to the voice file; delete the voice file with the longest untouched duration from the second storage space, and re-determine whether the data volume of the voice file saved in the second storage space is greater than the second threshold, until the data volume of the voice file saved in the second storage space is less than or equal to the second threshold.
[0065] Fifthly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program and the processor is used to cause the electronic device to implement the text-to-speech method described in any of the above embodiments when executing the computer program.
[0066] Sixthly, embodiments of this application provide a computer-readable storage medium that, when executed by a computing device, causes the computing device to implement the text-to-speech method described in any of the above embodiments.
[0067] Seventhly, embodiments of this application provide a computer program product that, when run on a computer, enables the computer to implement the text-to-speech method described in any of the above embodiments.
[0068] The text-to-speech method provided in this application, upon receiving a speech conversion operation for converting a first text to speech based on pre-configured speech synthesis parameters, first determines whether a target speech file, pre-converted based on the pre-configured speech synthesis parameters, is stored in a first storage space. If the target speech file is stored in the first storage space, it is directly retrieved from the first storage space. If the target speech file is not stored in the first storage space, the target speech file is generated based on the pre-configured speech synthesis parameters and the first text, and saved to the first storage space. Finally, a first audio segment is added to the time interval corresponding to the first text on the editing timeline based on the target speech file. Since the text-to-speech method provided in this application determines whether a target speech file, pre-converted based on the pre-configured speech synthesis parameters, is stored in the first storage space of the terminal device, the terminal device can retrieve the target speech file without interacting with the server when the target speech file is stored in the first storage space of the terminal device. Therefore, this application embodiment can improve the efficiency of speech synthesis. Attached Figure Description
[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0070] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings that need to be called in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0071] Figure 1 This is one of the flowcharts of the text-to-speech method provided in the embodiments of this application;
[0072] Figure 2 The second flowchart of the text-to-speech method provided in the embodiments of this application;
[0073] Figure 3 The third flowchart of the text-to-speech method provided in the embodiments of this application;
[0074] Figure 4 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;
[0075] Figure 5This is a schematic diagram of the structure of the service server provided in an embodiment of this application;
[0076] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0077] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0078] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.
[0079] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Furthermore, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.
[0080] This application provides a text-to-speech method, which can be executed by terminal devices such as mobile phones, personal computers, PDAs, and in-vehicle devices. (Refer to...) Figure 1 As shown, the text-to-speech method includes the following steps:
[0081] S101, Receive the user's voice conversion operation for the first text input in the video editing interface.
[0082] The speech conversion operation is used to convert the first text into speech based on pre-configured speech synthesis parameters.
[0083] In some embodiments, the pre-configured speech synthesis parameters may include timbre parameters and language parameters. For example, if the timbre parameter is male and the language parameter is Chinese, and the text content of the first text is "The weather is really nice today", then the speech conversion operation is used to instruct the text "The weather is really nice today" to be converted into a Chinese male voice audio file.
[0084] In some embodiments, the terminal device may display optional timbres and languages, and determine the pre-configured speech synthesis parameters based on the user's selected operation or default timbre and language parameters.
[0085] In some embodiments, a new business service interface can be added to the terminal device to call the text-to-speech method provided in this application embodiment. This business service interface and different vendors' text-to-speech engines can be encapsulated using a text-to-speech protocol (BaseTextToAudioSpeechProtocol). During speech conversion, the business service interface is called first. If the target speech file cannot be obtained through the business service interface, the corresponding text-to-speech engine is called to obtain the target speech file.
[0086] In some embodiments, receiving a voice conversion operation can be a click operation on the voice conversion control corresponding to the first text, a voice command input to the first text, or one or more specific gestures. The specific gesture can be any one of a single-click gesture, a swipe gesture, a pressure-recognition gesture, a long-press gesture, an area-change gesture, a double-press gesture, or a double-tap gesture.
[0087] S102. Determine whether the target speech file is stored in the first storage space based on the pre-configured speech synthesis parameters and the first text.
[0088] The target speech file is obtained by pre-converting the first text into speech based on the pre-configured speech synthesis parameters; the first storage space is the storage space of the terminal device.
[0089] That is, the terminal device first checks whether the local storage space contains a previously cached target speech file obtained by converting the first text into speech based on the pre-configured speech synthesis parameters and the first text.
[0090] In step S102 above, if the target audio file is stored in the first storage space, then step S103 is executed as follows:
[0091] S103. Obtain the target audio file from the first storage space.
[0092] That is, if the terminal device's first storage space contains a target speech file that has been pre-converted from the first text based on the pre-configured speech synthesis parameters, then the target speech file is directly obtained from the first storage space, without going through the speech conversion server to obtain the target speech file obtained from the first text based on the pre-configured speech synthesis parameters.
[0093] In some embodiments, the target audio file includes: audio data corresponding to the first text and metadata of the audio data corresponding to the first text.
[0094] For example, the metadata of the audio data corresponding to the first text may include the time period corresponding to each character in the first text. For instance, if the text content of the first text is "The weather is really nice today," then the metadata of the audio data corresponding to the first text may be as follows:
[0095] (00ms, 02ms]->"Today";
[0096] (03ms, 04ms) -> "day";
[0097] (06ms, 08ms) -> "day";
[0098] (08ms, 09ms) -> "Qi";
[0099] (10ms, 12ms) -> "True";
[0100] (13ms, 13ms) -> "OK".
[0101] In step S102 above, if the target audio file is not saved in the first storage space, then step S104 is executed as follows:
[0102] S104. Generate the target speech file according to the pre-configured speech synthesis parameters and the first text, and save the target speech file to the first storage space.
[0103] After obtaining the target speech file by performing the above steps S103 or S104, the following step S105 is further performed:
[0104] S105. Based on the target audio file, add a first audio segment within the time interval corresponding to the first text on the editing timeline.
[0105] In some embodiments, the first audio segment is the entire audio file in the target speech file.
[0106] In other embodiments, the first audio segment is a portion of the audio in the target speech file.
[0107] For example, if the time interval corresponding to the first text on the editing timeline is [t1, t2], then the first audio segment is added to the time interval [t1, t2] according to the target audio file.
[0108] The text-to-speech method provided in this application, upon receiving a speech conversion operation for converting a first text to speech based on pre-configured speech synthesis parameters, first determines whether a target speech file, pre-converted based on the pre-configured speech synthesis parameters, is stored in a first storage space. If the target speech file is stored in the first storage space, it is directly retrieved from the first storage space. If the target speech file is not stored in the first storage space, the target speech file is generated based on the pre-configured speech synthesis parameters and the first text, and saved to the first storage space. Finally, a first audio segment is added to the time interval corresponding to the first text on the editing timeline based on the target speech file. Since the text-to-speech method provided in this application determines whether a target speech file, pre-converted based on the pre-configured speech synthesis parameters, is stored in the first storage space of the terminal device, the terminal device can retrieve the target speech file without interacting with the server when the target speech file is stored in the first storage space of the terminal device. Therefore, this application embodiment can improve the efficiency of speech synthesis.
[0109] This application provides another text-to-speech method, referring to... Figure 2 As shown, the text-to-speech method includes the following steps:
[0110] S201. The terminal device receives the user's voice conversion operation for the first text input in the video editing interface.
[0111] The speech conversion operation is used to convert the first text into speech based on pre-configured speech synthesis parameters.
[0112] S202, The terminal device generates a target file identifier based on the pre-configured speech synthesis parameters and the first text.
[0113] In this embodiment, the implementation method of generating a target file identifier based on the pre-configured speech synthesis parameters and the first text is not limited. The implementation method of generating a file identifier based on speech synthesis parameters and text can make the file identifier generated based on different speech synthesis parameters and / or different texts different, while the file identifier generated based on the same speech synthesis parameters and text is unique.
[0114] In some embodiments, the pre-configured speech synthesis parameters include: timbre parameters and language parameters; generating a target file identifier based on the pre-configured speech synthesis parameters and the first text includes: concatenating the timbre parameters, the language parameters, and the identifier information of the first text to obtain concatenation parameters, and processing the concatenation parameters based on a message digest algorithm to obtain the target file identifier.
[0115] For example, if the timbre parameter is aaaaa, the language parameter is bbbbb, and the identifier information of the first text is ccccc, then the concatenation parameter obtained by concatenating the timbre parameter, the language parameter, and the identifier information of the first text is aaaaabbbbbccccc.
[0116] In some embodiments, the splicing parameters can be processed using the MD5 (Match 5 Message Digest) algorithm to obtain the target file identifier.
[0117] S203. The terminal device determines whether the target file identifier is included in the first file identifier list.
[0118] In some embodiments, the first file identifier list includes file identifiers of various voice files stored in the first storage space.
[0119] In step S203 above, if the terminal device determines that the target file identifier is included in the first file identifier list, then the following steps S204 to S206 are executed.
[0120] S204. The terminal device determines that the target voice file is stored in the first storage space.
[0121] The first storage space is the storage space of the terminal device.
[0122] S205. The terminal device retrieves the target voice file from the first storage space.
[0123] In some embodiments, the first file identifier list further includes the storage location corresponding to each voice text stored in the first storage space, and obtaining the target voice file from the first storage space includes: obtaining the target voice file from the first storage space according to the storage address of the voice file.
[0124] For example, the first file identifier list is shown in Table 1 below:
[0125] First document identifier list Table 1
[0126]
[0127] When the target file is identified as file identifier 2, it can be determined that the target audio file is stored in the first storage space, and the target audio file can be obtained from the first storage space according to storage address 2.
[0128] In step S203 above, if the terminal device determines that the target file identifier is not included in the first file identifier list, then the following steps are performed:
[0129] S206. The terminal device determines that the target voice file is not saved in the first storage space.
[0130] That is, if it is determined that the target speech file is not stored in the local storage space of the terminal device, the target speech file cannot be obtained directly from the local storage space, and the target speech file needs to be obtained through other means.
[0131] S207. The terminal device sends a voice conversion request to the service server.
[0132] Correspondingly, the business server receives voice conversion requests sent by the terminal devices.
[0133] The speech conversion request carries the pre-configured speech synthesis parameters and the first text, and the speech conversion request is used to request speech conversion of the first text based on the pre-configured speech synthesis parameters.
[0134] In some embodiments, the terminal device can send a voice conversion request to the service server through a communication link established with the service server.
[0135] S208, The business server obtains the target file identifier.
[0136] The target file identifier is generated based on the pre-configured speech synthesis parameters and the first text.
[0137] In some embodiments, the voice conversion request also carries the target file identifier, and the service server obtains the target file identifier by: obtaining the target file identifier from the voice conversion request.
[0138] In some embodiments, the service server obtains the target file identifier by: generating the target file identifier based on the pre-configured speech synthesis parameters and the first text.
[0139] The implementation method for generating a target file identifier based on the pre-configured speech synthesis parameters and the first text may include: concatenating the timbre parameters, the language parameters, and the identifier information of the first text to obtain concatenation parameters; and processing the concatenation parameters based on a message digest algorithm to obtain the target file identifier.
[0140] It should be noted that the implementation method of the business server generating the target file identifier based on the pre-configured speech synthesis parameters and the first text can be the same as the implementation method of the terminal device generating the target file identifier based on the pre-configured speech synthesis parameters and the first text, or it can be different from the implementation method of the terminal device generating the target file identifier based on the pre-configured speech synthesis parameters and the first text.
[0141] S209. The service server determines whether the target audio file is stored in the second storage space based on the target file identifier.
[0142] The target speech file is obtained by pre-converting the first text into speech based on the pre-configured speech synthesis parameters; the second storage space is the storage space of the business server.
[0143] In step S209 above, if the target audio file is stored in the second storage space, then step S210 is executed as follows:
[0144] S210. The service server generates the access address of the target audio file based on the storage location of the target audio file.
[0145] In some embodiments, the access address of the target audio file is the Uniform Resource Locator (URL) of the target audio file.
[0146] In step S209 above, if the target audio file is not saved in the second storage space, then step S211 is executed as follows:
[0147] S211. The service server generates the target speech file according to the pre-configured speech synthesis parameters and the first text, saves the target speech file to the second storage space, and generates the access address of the target speech file according to the storage location of the target speech file.
[0148] After generating the access address of the target audio file based on any of the above methods, the service server further performs the following steps:
[0149] S212. The service server sends the access address to the terminal device.
[0150] Accordingly, the terminal device receives the access address of the target audio file sent by the service server.
[0151] S213. The terminal device obtains the target audio file according to the access address.
[0152] After the terminal device obtains the target audio file using any of the above methods, it further performs the following steps:
[0153] S214. The terminal device adds a first audio segment to the time interval corresponding to the first text on the editing timeline according to the target audio file.
[0154] This application provides another text-to-speech method, referring to... Figure 3 As shown, the text-to-speech method includes the following steps:
[0155] S301, The terminal device receives the user's voice conversion operation for the first text input in the video editing interface.
[0156] The speech conversion operation is used to convert the first text into speech based on pre-configured speech synthesis parameters.
[0157] S302. The terminal device combines the timbre parameters, the language parameters, and the identifier information of the first text to obtain the splicing parameters.
[0158] S303. The terminal device processes the splicing parameters based on a message digest algorithm to obtain the target file identifier.
[0159] S304. The terminal device determines whether the target file identifier is included in the first file identifier list.
[0160] In step S304 above, if the terminal device determines that the target file identifier is included in the first file identifier list, then the following steps S305 to S306 are executed:
[0161] S305. The terminal device determines that the target voice file is stored in the first storage space.
[0162] The first storage space is the storage space of the terminal device.
[0163] S306. The terminal device retrieves the target voice file from the first storage space.
[0164] In step S304 above, if the terminal device determines that the target file identifier is not included in the first file identifier list, then the following steps are performed:
[0165] S307. The terminal device determines that the target voice file is not saved in the first storage space.
[0166] S308, The terminal device sends a voice conversion request to the service server.
[0167] Correspondingly, the business server receives voice conversion requests sent by the terminal devices.
[0168] The speech conversion request carries the pre-configured speech synthesis parameters and the first text, and the speech conversion request is used to request speech conversion of the first text based on the pre-configured speech synthesis parameters.
[0169] S309. The business server obtains the target file identifier.
[0170] The target file identifier is generated based on the pre-configured speech synthesis parameters and the first text.
[0171] S310, The service server determines whether the target file identifier is included in the second file identifier list.
[0172] In some embodiments, the second file identifier list may include file identifiers of various voice files stored in the second storage space.
[0173] In step S310 above, if the service server determines that the target file identifier is included in the second file identifier list, then the following steps are performed:
[0174] S311. The service server determines that the target voice file is stored in the second storage space.
[0175] S312. The service server generates the access address of the target audio file based on the storage location of the target audio file.
[0176] In step S310 above, if the service server determines that the target file identifier is included in the second file identifier list, then the following steps are performed:
[0177] S313. The service server determines that the target voice file is not saved in the second storage space.
[0178] S314. The service server sends the voice conversion request to the voice conversion server.
[0179] In response, the voice conversion server receives the voice conversion request sent by the service server.
[0180] S315. The speech conversion server performs speech conversion on the first text based on the pre-configured speech synthesis parameters to obtain the target speech file.
[0181] S316. The voice conversion server sends the target voice file to the service server.
[0182] Accordingly, the service server receives the target audio file sent by the voice conversion server.
[0183] S317. The service server determines whether the amount of data in the voice file stored in the second storage space is greater than the second threshold.
[0184] In step S317 above, if the amount of data in the voice file stored in the second storage space is less than or equal to the second threshold, then step S318 is executed as follows:
[0185] S318. The service server saves the target audio file to the second storage space, adds the target file identifier to the second file identifier list, and generates the access address of the target audio file according to the storage location of the target audio file.
[0186] In step S317 above, if the amount of data in the voice file stored in the second storage space is greater than the second threshold, then the following steps are performed:
[0187] S319. The service server obtains the untouched duration of each voice message stored in the second storage space.
[0188] The untouched duration of any audio file is the time elapsed since the last access to that audio file.
[0189] S320, the service server deletes the voice file with the longest untouched duration from the second storage space, removes the file identifier corresponding to the voice file with the longest untouched duration from the second file identifier list, and returns to step S317.
[0190] That is, the business server deletes the voice file with the longest untouched duration from the second storage space, removes the file identifier corresponding to the voice file with the longest untouched duration from the second file identifier list, and re-determines whether the data volume of the voice file stored in the second storage space is greater than the second threshold. After the data volume of the voice file stored in the second storage space is less than or equal to the second threshold, step S318 is executed to save the target voice file to the second storage space, add the target file identifier to the second file identifier list, and generate the access address of the target voice file according to the storage location of the target voice file.
[0191] After the business server generates the access address of the target audio file based on any of the above implementation methods, the business server further executes the following step S321.
[0192] S321. The service server sends the access address to the terminal device.
[0193] In response, the terminal device accepts the access address sent by the service server.
[0194] S323. The terminal device obtains the target audio file according to the access address.
[0195] After the terminal device obtains the target audio file based on any of the above implementation methods, the following steps are performed:
[0196] S324. The terminal device adds a first audio segment to the time interval corresponding to the first text on the editing timeline according to the target audio file.
[0197] If the target audio file is not stored in the first storage space, the terminal device performs the following steps after acquiring the target audio file:
[0198] S325. The terminal device determines whether the amount of data in the voice file stored in the first storage space is greater than the first threshold.
[0199] For example, when the first threshold is 50MB, the terminal device determines whether the data size of the voice file stored in the first storage space is greater than 50MB.
[0200] In step S325 above, if the terminal device determines that the amount of data in the voice file stored in the first storage space is less than or equal to the first threshold, then the following step S326 is executed:
[0201] S326. The terminal device adds the target file identifier to the first file identifier list and saves the target voice file to the first storage space.
[0202] In step S325 above, if the terminal device determines that the amount of data in the voice file stored in the first storage space is less than or equal to the first threshold, then the following steps are performed:
[0203] S327. The terminal device obtains the idle time of each voice file in the first storage space.
[0204] The idle time of any audio file is the time elapsed since the last access to that audio file.
[0205] S328. The terminal device deletes the voice file with the longest idle time from the first storage space, deletes the file identifier corresponding to the voice file with the longest idle time from the first file identifier list, and returns to step S325.
[0206] That is, the voice file with the longest idle time is deleted from the first storage space, and the file identifier corresponding to the voice file with the longest idle time is deleted from the first file identifier list. Then, it is re-determined whether the data volume of the voice file stored in the first storage space is greater than the first threshold. After the data volume of the voice file stored in the first storage space is less than or equal to the first threshold, the above step S326 is executed to add the target file identifier to the first file identifier list and save the target voice file to the first storage space.
[0207] Based on the same inventive concept, as an implementation of the above method, this application embodiment also provides a terminal device and a business server. This embodiment corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the terminal device and business server in this embodiment can correspondingly implement all the contents of the aforementioned method embodiment.
[0208] This application provides a terminal device. Figure 4 This is a schematic diagram of the terminal device, such as... Figure 4 As shown, the terminal device 400 includes:
[0209] User input unit 41 is used to receive a user's voice conversion operation on the first text input in the video editing interface. The voice conversion operation is used to convert the first text into speech based on pre-configured speech synthesis parameters.
[0210] Processing unit 42 is configured to determine whether a target speech file is stored in the first storage space based on the pre-configured speech synthesis parameters and the first text, wherein the target speech file is obtained in advance by converting the first text into speech based on the pre-configured speech synthesis parameters; the first storage space is the storage space of the terminal device.
[0211] The acquisition unit 43 is configured to acquire the target speech file from the first storage space when the target speech file is stored in the first storage space, and generate the target speech file from the first storage space according to the pre-configured speech synthesis parameters and the first text when the target speech file is not stored in the first storage space, and save the target speech file to the first storage space.
[0212] Editing unit 44 is used to add a first audio segment to the time interval corresponding to the first text on the editing timeline according to the target audio file.
[0213] As an optional implementation of this application, the acquisition unit 43 is specifically used to send a speech conversion request to the service server; the speech conversion request carries the pre-configured speech synthesis parameters and the first text, and is used to request speech conversion of the first text based on the pre-configured speech synthesis parameters; receive the access address of the target speech file sent by the service server; and acquire the target speech file according to the access address.
[0214] As an optional implementation of this application, the processing unit 42 is specifically used to generate a target file identifier based on the pre-configured speech synthesis parameters and the first text; determine whether the target file identifier is included in the first file identifier list; if the target file identifier is included in the first file identifier list, determine that the target speech file is stored in the first storage space; if the target file identifier is not included in the first file identifier list, determine that the target speech file is not stored in the first storage space.
[0215] The acquisition unit 43 is further configured to add the target file identifier to the first file identifier list after saving the target audio file to the first storage space.
[0216] As an optional implementation of this application, the pre-configured speech synthesis parameters include: timbre parameters and language parameters; the processing unit 42 is specifically used to concatenate the timbre parameters, the language parameters, and the identification information of the first text to obtain concatenation parameters; and to process the concatenation parameters based on a message digest algorithm to obtain the target file identifier.
[0217] As an optional implementation of this application, the acquisition unit 43 is further configured to determine whether the data volume of the voice files stored in the first storage space is greater than a first threshold before saving the target voice file to the first storage space; in response to the data volume of the voice files stored in the first storage space being greater than the first threshold, acquire the idle time of each voice file in the first storage space; the idle time of any voice file is the time elapsed since the last access to the voice file; delete the voice file with the longest idle time from the first storage space, and re-determine whether the data volume of the voice files stored in the first storage space is greater than the first threshold, until the data volume of the voice files stored in the first storage space is less than or equal to the first threshold.
[0218] This application provides a service server. Figure 5 This is a schematic diagram of the structure of the service server, as shown below. Figure 5 As shown, the service server 500 includes:
[0219] The receiving unit 51 is used to receive a voice conversion request sent by the terminal device; the voice conversion request carries pre-configured voice synthesis parameters and a first text, and is used to request voice conversion of the first text based on the pre-configured voice synthesis parameters;
[0220] Processing unit 52 is used to determine whether a target speech file is stored in the second storage space based on the pre-configured speech synthesis parameters and the first text. The target speech file is obtained in advance by converting the first text into speech based on the pre-configured speech synthesis parameters. The second storage space is the storage space of the business server.
[0221] The generation unit 53 is configured to generate an access address for the target speech file based on the storage location of the target speech file when the target speech file is stored in the second storage space; and to generate the target speech file based on the pre-configured speech synthesis parameters and the first text when the target speech file is not stored in the second storage space, save the target speech file to the second storage space, and generate an access address for the target speech file based on the storage location of the target speech file.
[0222] The sending unit 54 is used to send the access address to the terminal device so that the terminal device can obtain the target voice file according to the access address.
[0223] As an optional implementation of this application, the generation unit 53 is specifically used to send the voice conversion request to the voice conversion server and receive the target voice file sent by the voice conversion server.
[0224] As an optional implementation of this application, the processing unit 52 is specifically used to generate a target file identifier based on the pre-configured speech synthesis parameters and the first text; determine whether the target file identifier is included in the second file identifier list; if the target file identifier is included in the second file identifier list, determine that the target speech file is stored in the second storage space; if the target file identifier is not included in the second file identifier list, determine that the target speech file is not stored in the second storage space.
[0225] The generation unit 53 is further configured to add the target file identifier to the second file identifier list after saving the target audio file to the second storage space.
[0226] As an optional implementation of this application, the generation unit 53 is further configured to, before saving the target voice file to the second storage space, determine whether the data volume of the voice file saved in the second storage space is greater than a second threshold; in response to the data volume of the voice file saved in the second storage space being greater than the second threshold, obtain the untouched duration of each voice file saved in the second storage space; the untouched duration of any voice file is the duration from the last time the voice file was accessed to the current time; delete the voice file with the longest untouched duration from the second storage space, and re-determine whether the data volume of the voice file saved in the second storage space is greater than the second threshold, until the data volume of the voice file saved in the second storage space is less than or equal to the second threshold.
[0227] The terminal device and service server provided in this application embodiment can execute the text-to-speech method provided in any of the above embodiments. The implementation principle and technical effect are similar, and will not be described again here.
[0228] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, the electronic device provided in this embodiment includes a memory 601 and a processor 602. The memory 601 is used to store a computer program, and the processor 602 is used to execute the text-to-speech method provided in the above embodiment when executing the computer program.
[0229] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the computing device to implement the text-to-speech method provided in the above embodiments.
[0230] Based on the same inventive concept, this application also provides a computer program product that, when run on a computer, enables the computing device to implement the text-to-speech method provided in the above embodiments.
[0231] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.
[0232] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), or dedicated injection circuits.
[0233] Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0234] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0235] Computer-readable media include both permanent and non-permanent, removable and non-removable storage media. Storage media can store information using any method or technology; the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media do not include transient computer-readable media, such as modulated data signals and carrier waves.
[0236] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A method of text-to-speech, characterized by, The method comprises the following steps: receiving a voice conversion operation of a user on a first text input in a video editing interface, the voice conversion operation being used for voice conversion of the first text based on preconfigured voice synthesis parameters; determining whether a target voice file is saved in a first storage space according to the preconfigured voice synthesis parameters and the first text, the target voice file being obtained by pre-converting the first text into voice based on the preconfigured voice synthesis parameters; the first storage space being a storage space of a terminal device; if yes, obtaining the target voice file from the first storage space; if no, generating the target voice file according to the preconfigured voice synthesis parameters and the first text, and saving the target voice file into the first storage space; adding a first audio segment in a time interval corresponding to the first text on an editing timeline according to the target voice file.
2. The method of claim 1, wherein, The generating of the target voice file according to the preconfigured voice synthesis parameters and the first text comprises the following steps: sending a voice conversion request to a service server, the voice conversion request carrying the preconfigured voice synthesis parameters and the first text, and being used for requesting the voice conversion of the first text based on the preconfigured voice synthesis parameters; receiving an access address of the target voice file sent by the service server; obtaining the target voice file according to the access address.
3. The method of claim 1, wherein, The determining of whether the target voice file is saved in the first storage space according to the preconfigured voice synthesis parameters and the first text comprises the following steps: generating a target file identifier according to the preconfigured voice synthesis parameters and the first text; determining whether the target file identifier is included in a first file identifier list; if the target file identifier is included in the first file identifier list, it is determined that the target voice file is saved in the first storage space; if the target file identifier is not included in the first file identifier list, it is determined that the target voice file is not saved in the first storage space; After saving the target voice file into the first storage space, the method further comprises adding the target file identifier into the first file identifier list.
4. The method of claim 3, wherein, The preconfigured voice synthesis parameters comprise tone parameters and language parameters; the generating of the target file identifier according to the preconfigured voice synthesis parameters and the first text comprises the following steps: splicing the tone parameters, the language parameters and identifier information of the first text to obtain splicing parameters; processing the splicing parameters based on a message digest algorithm to obtain the target file identifier.
5. The method of claim 1, wherein, Before saving the target voice file into the first storage space, the method further comprises the following steps: determining whether a data amount of voice files saved in the first storage space is greater than a first threshold value; in response to the data amount of the voice files saved in the first storage space being greater than the first threshold value, obtaining an idle duration of each voice file in the first storage space; the idle duration of any voice file being a duration from a time point of last accessing the voice file to a current time point. delete the voice file with the longest idle time from the first storage space, and repeatedly determine whether the data amount of the voice files stored in the first storage space is greater than the first threshold value until the data amount of the voice files stored in the first storage space is less than or equal to the first threshold value.
6. A method of text-to-speech, characterized by, Comprise: receiving a voice conversion request sent by a terminal device; the voice conversion request carries preconfigured voice synthesis parameters and a first text, and is used to request voice conversion of the first text based on the preconfigured voice synthesis parameters; determining whether a target voice file is stored in a second storage space according to the preconfigured voice synthesis parameters and the first text, the target voice file being obtained by previously performing voice conversion on the first text based on the preconfigured voice synthesis parameters; the second storage space being a storage space of a service server; if the target voice file is stored in the second storage space, generating an access address of the target voice file according to a storage location of the target voice file; if the target voice file is not stored in the second storage space, generating the target voice file according to the preconfigured voice synthesis parameters and the first text, storing the target voice file into the second storage space, and generating an access address of the target voice file according to a storage location of the target voice file; sending the access address to the terminal device, so that the terminal device acquires the target voice file according to the access address.
7. The method of claim 6, wherein, the generating of the target voice file according to the preconfigured voice synthesis parameters and the first text comprises: sending the voice conversion request to a voice conversion server; receiving the target voice file sent by the voice conversion server.
8. The method of claim 6, wherein, the determining of whether the target voice file is stored in the second storage space according to the preconfigured voice synthesis parameters and the first text comprises: generating a target file identifier according to the preconfigured voice synthesis parameters and the first text; determining whether the target file identifier is included in a second file identifier list, and in a case where the target file identifier is included in the second file identifier list, determining that the target voice file is stored in the second storage space, and in a case where the target file identifier is not included in the second file identifier list, determining that the target voice file is not stored in the second storage space; after storing the target voice file into the second storage space, the method further comprises: adding the target file identifier into the second file identifier list.
9. The method of claim 6, wherein, before storing the target voice file into the second storage space, the method further comprises: determining whether a data amount of voice files stored in the second storage space is greater than a second threshold value; in response to the data amount of the voice files stored in the second storage space being greater than the second threshold value, acquiring an untouched time length of each voice stored in the second storage space; the untouched time length of any voice file being a time length from a time point of last access of the voice file to a current time point. Delete the voice file with the longest time length from the second storage space, and repeatedly determine whether the data amount of the voice files stored in the second storage space is greater than the second threshold value until the data amount of the voice files stored in the second storage space is less than or equal to the second threshold value.
10. A terminal device, comprising: The method comprises the following steps: The user input unit is configured to receive a voice conversion operation of a user on a first text input in a video editing interface, the voice conversion operation being configured to perform voice conversion on the first text based on preconfigured voice synthesis parameters. The processing unit is configured to determine whether a target voice file is stored in a first storage space according to the preconfigured voice synthesis parameters and the first text, the target voice file being obtained by performing voice conversion on the first text based on the preconfigured voice synthesis parameters in advance; and the first storage space being a storage space of a terminal device. The obtaining unit is configured to obtain the target voice file from the first storage space in a case where the target voice file is stored in the first storage space, and generate the target voice file according to the preconfigured voice synthesis parameters and the first text and store the target voice file into the first storage space in a case where the target voice file is not stored in the first storage space. The editing unit is configured to add a first audio segment in a time interval corresponding to the first text on an editing timeline according to the target voice file.
11. A service server, characterized by The receiving unit is configured to receive a voice conversion request sent by a terminal device. The voice conversion request carries preconfigured voice synthesis parameters and a first text, and is configured to request to perform voice conversion on the first text based on the preconfigured voice synthesis parameters. The processing unit is configured to determine whether a target voice file is stored in a second storage space according to the preconfigured voice synthesis parameters and the first text, the target voice file being obtained by performing voice conversion on the first text based on the preconfigured voice synthesis parameters in advance; and the second storage space being a storage space of a service server. The generating unit is configured to generate an access address of the target voice file according to a storage location of the target voice file in a case where the target voice file is stored in the second storage space. In a case where the target voice file is not stored in the second storage space, the generating unit is configured to generate the target voice file according to the preconfigured voice synthesis parameters and the first text, store the target voice file into the second storage space, and generate an access address of the target voice file according to a storage location of the target voice file. The sending unit is configured to send the access address to the terminal device, so that the terminal device obtains the target voice file according to the access address. The memory is configured to store a computer program, and the processor is configured to execute the computer program, so that the electronic device implements the text-to-speech method in any one of claims 1-9.
12. An electronic device, comprising: 13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the text-to-speech method in any one of claims 1-9.
14. A computer program product, characterised in that, The computer program product, when running on a computer, causes the computer to implement the text-to-speech method in any one of claims 1-9.