Text-to-voice processing method and device, medium and computer program product

Abstract

The embodiment of the invention relates to a text-to-voice processing method and device, a medium and a computer program product, in at least one embodiment of the invention, through obtaining at least one path of text-to-voice request, text splicing is carried out based on a first text of to-be-converted voice carried by the text-to-voice request to obtain a second text; and the second text is subjected to semantic segmentation to obtain the array structure corresponding to the second text, so that speech synthesis is performed based on the array structure to obtain the speech corresponding to the second text, speech synthesis by batch processing of a plurality of text-to-speech requests can be realized, and the speech synthesis efficiency is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a text-to-speech processing method, apparatus, medium, and computer program product. Background Technology

[0002] Currently, text-to-speech (TTS) processing is applied in conversational scenarios. TTS processing can be achieved through pre-trained TTS models, and the output speech can simulate the details of natural language such as speech rate, tone, and pauses.

[0003] However, current TTS models have low real-time factor (RTF) for synthesized speech, which cannot meet the needs of online speech synthesis. Furthermore, current TTS models do not support real-time synthesis of multiple audio streams, thus limiting the efficiency of speech synthesis. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a text-to-speech processing method, apparatus, medium, and computer program product.

[0005] In a first aspect, embodiments of this disclosure provide a text-to-speech processing method, the method comprising: Obtain at least one text-to-speech request, wherein the text-to-speech request carries a first text of the speech to be converted; Based on the first text carried by the at least one text-to-speech request, text concatenation is performed to obtain the second text; The second text is semantically segmented to obtain the array structure corresponding to the second text; Speech synthesis is performed based on the array structure to obtain the speech corresponding to the second text.

[0006] In some embodiments, the step of concatenating text based on the first text carried by the at least one text-to-speech request to obtain the second text includes: From the at least one text-to-speech request, select one or more requests as a request batch; Based on the first text carried by each request in the request batch, the text is concatenated to obtain the second text corresponding to the request batch.

[0007] In some embodiments, the step of semantically segmenting the second text to obtain the array structure corresponding to the second text includes: The second text corresponding to the request batch is semantically segmented to obtain the array structure corresponding to the request batch.

[0008] In some embodiments, after obtaining the array structure corresponding to the request batch, the method further includes: The request batch is dynamically updated to obtain an updated request batch, and one or more requests are added to the updated request batch. Based on the first text carried by each request in the updated request batch, text concatenation is performed to obtain the second text corresponding to the updated request batch.

[0009] In some embodiments, after obtaining the second text corresponding to the updated request batch, the method further includes: The second text corresponding to the updated request batch is semantically segmented to obtain an array structure corresponding to the updated request batch.

[0010] In some embodiments, dynamically updating the request batch includes: During the speech synthesis process based on the array structure corresponding to the request batch, after synthesizing the speech corresponding to the first text carried by the first request in the request batch, the first request in the request batch is deleted, and a second request is added to the request batch; the second request is one of the at least one text-to-speech requests.

[0011] In some embodiments, the text-to-speech request also carries timbre and / or speed; The process of synthesizing speech based on the array structure to obtain the speech corresponding to the second text includes: Based on the array structure and the timbre and / or speed corresponding to the array structure, speech synthesis is performed to obtain the speech corresponding to the second text.

[0012] Secondly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the text-to-speech processing method according to any embodiment of the first aspect.

[0013] Thirdly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program for performing the text-to-speech processing method described in any embodiment of the first aspect.

[0014] Fourthly, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the text-to-speech processing method described in any embodiment of the first aspect.

[0015] The technical solution provided in this disclosure obtains at least one text-to-speech request, then concatenates the first text of the speech to be converted carried by the text-to-speech request to obtain a second text; and performs semantic segmentation on the second text to obtain an array structure corresponding to the second text, thereby performing speech synthesis based on the array structure to obtain the speech corresponding to the second text. This can realize batch processing and speech synthesis of multiple text-to-speech requests, improving the efficiency of speech synthesis.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a text-to-speech processing method provided in this embodiment of the present disclosure; Figure 2 A flowchart illustrating another text-to-speech processing method provided in this embodiment of the present disclosure; Figure 3 A schematic diagram of the structure of a text-to-speech processing device provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0021] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0022] To address the issues that current TTS models (such as ChatTTS) neither support real-time online synthesis of multi-channel speech nor meet the real-time factor (RTF) requirements of online real-time dialogue scenarios, thus failing to achieve smooth performance, this disclosure optimizes the technical solution by transforming the original single-channel inference into simultaneous multi-channel inference, thereby improving the efficiency of synthesized speech. Simultaneously, step-by-step inference enables rapid return of the first packet of speech synthesis, consistently reducing the first packet return time to within 0.15 seconds, thus improving the RTF of the synthesized speech and meeting the needs of online speech synthesis.

[0023] Figure 1 This is a flowchart illustrating a text-to-speech processing method provided in an embodiment of this disclosure. The method can be executed by a text-to-speech processing device, which can be implemented in software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method includes, but is not limited to, steps 101 to 104: In step 101, at least one text-to-speech request is obtained, the text-to-speech request carrying the first text of the speech to be converted.

[0024] The text-to-speech request can also carry timbre and / or velocity. For example, a text-to-speech request may carry three parameters: the first text to be converted, the timbre, and the velocity. When a text-to-speech request is received, in response to the request, the pre-trained TTS model can synthesize speech from the first text to be converted carried in the request, according to the timbre and velocity carried in the request.

[0025] If the text-to-speech request does not include timbre and pitch, the pre-trained TTS model can synthesize the first text containing the speech to be converted in the request into speech according to the preset timbre and pitch.

[0026] In step 102, based on the first text carried by at least one text-to-speech request, text concatenation is performed to obtain the second text.

[0027] When multiple text-to-speech requests are received, the first text carried by each of the multiple requests can be concatenated to obtain a second text composed of multiple first texts.

[0028] The text concatenation rules can be preset. For example, the text concatenation rules are: add the prefix "r{serial number}" to the text of each request and separate them with newline characters, specifically including the following steps 1 to 4: Step 1: Iterate through the text content of each request in turn.

[0029] Step 2: Add a prefix r{i} to the i-th request (e.g., r1, r2).

[0030] Step 3: Concatenate the prefix with the original text and add a newline character after each record.

[0031] Step 4: Finally, remove any extra newline characters at the end to obtain the standardized output.

[0032] Example: The first request (denoted as r1) carries the following text: "Hello, I am a voice assistant." The second request (denoted as r2) carries the following text: "The weather is really nice today." The first text carried by each of these two requests is concatenated to obtain the second text. The text concatenation process is as follows: Steps 1 to 4: Step 1: Iterate through the text content of each request in turn.

[0033] Step 2: Add prefix r1 to the first request and prefix r2 to the second request.

[0034] Step 3: This step includes the following sections 3.1 to 3.3: 3.1: Concatenate the prefix r1 with the original text of the first request to obtain the following record: Hello r1, I am a voice assistant.

[0035] 3.2: Concatenate the prefix r2 with the original text of the second request to obtain the following record: r2 The weather is really nice today.

[0036] 3.3: Add a newline character after each record.

[0037] Step 4: Finally, remove the extra newline characters at the end, and the resulting second text is: Hello r1, I am a voice assistant.

[0038] r2 The weather is really nice today. In step 103, the second text is semantically segmented to obtain the array structure corresponding to the second text.

[0039] In this embodiment, the second text can be semantically segmented using a pre-trained large model encoder to obtain an array structure corresponding to the second text, wherein the array structure can be a token array structure.

[0040] A Token array is a set of tokens generated by splitting a string, text, or code according to specific rules. Each token represents a meaningful linguistic unit (such as keywords, identifiers, operators, etc.). Among them, the specific rules are a standardized process of splitting continuous text into discrete tokens through a tokenization algorithm, including text segmentation rules and encoding mapping rules, which are specifically as follows: 1. Text segmentation rules: Implement the conversion from the original text to semantic units.

[0041] In this embodiment, according to the segmentation granularity and algorithm logic, the text segmentation rules include at least one of the following rules (1) to (3): ‌(1) Basic symbol splitting rule.

[0042] First, split the text by explicit delimiters such as spaces and punctuation marks (such as commas, periods), forming preliminary segments. For example, in the sentence: "The weather today is really nice.", the punctuation mark "." will be used as an independent Token boundary.

[0043] ‌(2) Semantic unit recognition rule.

[0044] The semantic unit recognition rule includes at least one of the following (2.1) to (2.3):‌[[ID=第十八条]] ‌(2.1) Character splitting: Split the text into the smallest visual units (such as individual Chinese characters "今", "天"), which is applicable to languages without obvious lexical boundaries (such as Chinese).

[0045] ‌(2.2) Word splitting: Identify complete semantic words (such as "weather" as a whole Token) through dictionary matching or grammar rules, and avoid splitting words with independent meanings.

[0046] ‌(2.3) Sub-word splitting: When encountering out-of-vocabulary words (such as "真的很"), use algorithms such as Byte Pair Encoding (BPE), WordPiece, etc. to combine high-frequency character combinations into sub-word units (such as "真的", "很"), and keep low-frequency characters as independent Tokens to balance semantic integrity and vocabulary size.

[0047] ‌(3) Context-aware adjustment rule.

[0048] Combined with the context semantics, avoid splitting logical units that span sentences (such as "今天天气" as a whole semantic block), and ensure that the Token sequence conforms to human language logic.

[0049] 2. Encoding mapping rules: Implement the conversion from semantic units to array structures.

[0050] The segmented Tokens need to be converted into an array that can be processed by the model according to the following rules a and b: a: Vocabulary matching rule.

[0051] Each Token corresponds to a unique integer ID in the vocabulary (Vocabulary). For example, "today" corresponds to ID: 1001, "weather" corresponds to ID: 2002, and unmatched Tokens are represented by special symbols (such as [UNK]).

[0052] For example: The sentence "Today the weather is really nice." After segmentation, the Token array is [今, 天, 天, 气, 真, 的, 很, 不, 错, 。], and the corresponding ID array may be [5, 10, 10, 15, 22, 33, 45, 50, 66, 99].

[0053] b: Sequence normalization rule.

[0054] The order of Tokens strictly follows the original text word order to ensure semantic logic (e.g., the Token sequences of "the weather is nice" and "nice weather" are different).

[0055] Supplement special Tokens (such as starting with [CLS] and ending with [SEP]) to mark the text boundaries and assist the model in locating semantic key points.

[0056] Therefore, the essence of the specific rule is a three-stage process of "text segmentation", "semantic verification", and "encoding mapping": First, generate semantic units through multi-layer splitting of symbols, words, and sub-words; then map to an integer array through the vocabulary; finally, form a Token sequence that can be directly processed by the model.

[0057] For example, the second text is: r1 Hello, I am a voice assistant.

[0058] r2 Today the weather is really nice.

[0059] Performing semantic segmentation on the second text to obtain the Token array structure corresponding to the second text, including the following (1) to (3): (1) Word segmentation logic: Adopting the Chinese word segmentation standard, the sentence is disassembled into the smallest semantic units, including single Chinese characters and punctuation marks.

[0060] For the sentence of r1: "Hello, I am a voice assistant." After segmentation, the Token array is [你, 好, ,, 我, 是, 一, 个, 语, 音, 助, 手, 。].

[0061] For the sentence of r2: "Today the weather is really nice." After segmentation, the Token array is [今, 天, 天, 气, 真, 的, 很, 不, 错, 。] (2)Token Index: If r1 is sent at time t1, r1 is numbered sequentially starting from t1; if r2 is sent at time t2, r2 is numbered sequentially starting from t2; if r3 is sent at time t3, r3 is numbered sequentially starting from t3. And so on, if rn is sent at time tj (tj is a variable that can take any value), rn is numbered sequentially starting from tj. The complete sentence content is covered, and the columns not included are represented by "...". (The actual length of the Token array is subject to the actual number of tokens in the sentence).

[0062] For example, the Token array corresponding to r1 is [你, 好, ,, , 我, 是, 一,个, 语, 音, 助, 手, 。]. Numbered sequentially starting from t1, corresponding to [t1, t2, t3, t4, t5, t6, t7, t8, t9, t10, t11, t12].

[0063] For example, the Token array corresponding to r2 is [今, 天, 天, 气, 真, 的, 很, 不, 错, 。]. Numbered sequentially starting from t2, corresponding to [t2, t3, t4, t5, t6, t7, t8, t9, t10, t11].

[0064] ‌Combine the Token array corresponding to r1 and the Token array corresponding to r2 according to the Token Index‌, and the Token array structure corresponding to the second text is shown in the following table:

[0065] If the text-to-speech request can also carry timbre and / or speed of sound, then the timbre and / or speed of sound corresponding to the Token array structure can be determined. For example, r1 also carries timbre a2 and / or speed of sound a3, and r2 also carries timbre c2 and / or speed of sound c3, then the timbre and / or speed of sound corresponding to the Token array structure are: r1, a2, a3; r2, c2, c3.

[0066] In step 104, perform speech synthesis based on the array structure to obtain the speech corresponding to the second text.

[0067] Input the Token array structure into a pre-trained TTS model for inference. The TTS model performs speech synthesis according to the preset timbre and / or preset speed of sound, or according to the timbre and / or speed of sound corresponding to the Token array structure, to obtain the speech corresponding to the second text. <...

[0068] For example, if the timbre and / or speed corresponding to the Token array structure are: r1, a2, a3; r2, c2, c3, then the TTS model will convert the text corresponding to r1 in the Token array structure into speech according to the timbre a2 and / or speed a3; and convert the text corresponding to r2 in the Token array structure into speech according to the timbre c2 and / or speed c3.

[0069] As can be seen, in this embodiment, by acquiring at least one text-to-speech request, and then concatenating the first text of the speech to be converted carried by the text-to-speech request to obtain the second text; and performing semantic segmentation on the second text to obtain the array structure corresponding to the second text, and then performing speech synthesis based on the array structure to obtain the speech corresponding to the second text, the single-path inference is optimized to multi-path simultaneous inference, which can realize the batch processing and speech synthesis of multiple text-to-speech requests, thereby improving the efficiency of speech synthesis.

[0070] exist Figure 1 Based on the illustrated embodiment, this embodiment describes a scheme that supports parallel batch processing of synthesized speech for multiple requests, including the following steps 1 to 4: Step 1: Obtain N text-to-speech requests. Each text-to-speech request carries three parameters: the text to be converted, the timbre, and the speed of sound.

[0071] For example: At time t1, a request is made to r1 to send three parameters, including a1, a2, and a3, where a1, a2, and a3 are the text, timbre, and speed of the speech to be converted, respectively. Here, a1 is "Hello, I am a voice assistant." At time t2, a request is made to r2 to send three parameters, including c1, c2, and c3, where c1, c2, and c3 are the text, timbre, and speed of the speech to be converted, respectively. C1 is "The weather is really nice today." At time t3, r3 is requested to send three parameters, including d1, d2, and d3, where d1, d2, and d3 are the text, timbre, and speed of the speech to be converted, respectively. d1 is "Our country is prosperous and strong." ...; At time tj, rn is requested to send three parameters, including n1, n2, and n3, where n1, n2, and n3 are the text, timbre, and speed of the speech to be converted, respectively. Among them, n1 is "Have you finished your homework today?".

[0072] Step 2: Based on the text carried by the N-way text-to-speech request, perform text concatenation to obtain the second text.

[0073] The second text is: Hello r1, I am a voice assistant.

[0074] r2 The weather is really nice today.

[0075] r3 Our country is prosperous and strong.

[0076] ... Did you finish your homework today?

[0077] Step 3: Perform semantic segmentation on the second text to obtain the token array structure corresponding to the second text.

[0078] The token index is as follows: Since r1 is sent at time t1, r1 is numbered sequentially starting from t1; since r2 is sent at time t2, r2 is numbered sequentially starting from t2; since r3 is sent at time t3, r3 is numbered sequentially starting from t3. And so on, since rn is sent at time tj, rn is numbered sequentially starting from tj. The entire sentence content is covered; columns not included are indicated by "..." (the actual token array length is based on the actual number of tokens in the sentence).

[0079] In this embodiment, to facilitate the presentation of the Token array structure in tabular form, it is assumed that rn is sent at time t1 (i.e., tj=t1). Therefore, the second text is semantically segmented using a pre-trained large model encoder to obtain the Token array structure corresponding to the second text, as shown in the table below:

[0080] In step 104, speech synthesis is performed based on the Token array structure to obtain the speech corresponding to the second text.

[0081] In this embodiment, the token array structure is input into the pre-trained TTS model for inference to achieve batch processing of multiple text-to-speech requests to synthesize speech, thereby improving the efficiency of speech synthesis. For example, the throughput efficiency is improved by about 4 to 6 times (constrained by the size of the matrix that the graphics processing unit (GPU) can compute at the same time).

[0082] While batch processing of multiple text-to-speech requests to synthesize speech can improve the efficiency of speech synthesis, it does not solve the problem of low real-time factor (RTF). For example, the waiting time of the Nth request depends on the longest request being inferred, which is not suitable for real-time synthesis scenarios. Therefore, in some embodiments, the batch processing of multiple requests is adjusted to asynchronous batch processing to achieve step-by-step inference, reduce the return time of the first packet of speech synthesis, improve the RTF of synthesized speech, and meet the needs of online speech synthesis.

[0083] For example, in step 102, based on the first text carried by at least one text-to-speech request, text concatenation is performed to obtain the second text, including the following (1) and (2): (1) Select one or more requests from at least one text-to-speech request as a request batch.

[0084] In this embodiment, after receiving a text-to-speech request, the request can be stored in a waiting queue. That is, all new requests from users will first enter the waiting queue, where they are queued and await processing. Then, when the graphics processing unit (GPU) has idle resources, a pre-defined scheduler can select one or more requests from the waiting queue as a batch.

[0085] As can be seen, by selecting one or more requests as a request batch, one or more requests can be dynamically combined into a request batch, which can maximize the utilization of the graphics processing unit (GPU) and reduce the latency of streaming requests.

[0086] (2) Based on the first text carried by each request in the request batch, perform text concatenation to obtain the second text corresponding to the request batch.

[0087] During text concatenation, only the first text carried by each request in the request batch is concatenated to obtain the second text corresponding to the request batch. Then, in step 103, the second text is semantically segmented to obtain an array structure corresponding to the second text, implemented as follows: semantically segmenting the second text corresponding to the request batch to obtain a token array structure corresponding to the request batch. Therefore, in step 104, speech synthesis is performed based on the array structure to obtain the speech corresponding to the second text, implemented as follows: speech synthesis is performed based on the token array structure corresponding to the request batch.

[0088] In some embodiments, after obtaining the array structure corresponding to the request batch, the request batch can be dynamically updated to obtain an updated request batch, which may include one or more additional requests. Then, based on the first text carried by each request in the updated request batch, text concatenation is performed to obtain the second text corresponding to the updated request batch. Subsequently, semantic segmentation is performed on the second text corresponding to the updated request batch to obtain the token array structure corresponding to the updated request batch. Finally, speech synthesis is performed based on the token array structure corresponding to the updated request batch.

[0089] The dynamic update request batches include: During the speech synthesis process based on the array structure corresponding to the request batch, after synthesizing the speech corresponding to the first text carried by the first request in the request batch, the first request in the request batch is deleted, and a second request is added to the request batch; the second request is one of at least one text-to-speech request.

[0090] For example, the requests are denoted as r1, r2, and r3. That is, the waiting queue includes r1, r2, and r3. r1 and r2 are selected to form a request batch. During the speech synthesis process based on the array structure corresponding to the request batch, after synthesizing the speech corresponding to the first text carried by r1 in the request batch, r1 is deleted from the request batch, and r3 is added to the request batch. That is, the updated request batch includes r2 and r3.

[0091] Figure 2 This is a schematic diagram of another text-to-speech processing flow provided in this embodiment of the present disclosure. This embodiment supports adjusting the batch processing of multiple requests to asynchronous batch processing, realizing step-by-step inference. In this embodiment, dynamically combining one or more requests into request batches can maximize the utilization of the graphics processing unit (GPU) and reduce the latency of streaming requests. Figure 2 In Chinese, the text-to-speech processing flow includes the following steps 1 to 8: Step 1: New requests enter the waiting queue (New Request -> Waiting Queue).

[0092] In this embodiment, after receiving a text-to-speech request, the request can be stored in a waiting queue. That is, all new requests sent by users will first enter the waiting queue, where they will be queued and wait to be processed.

[0093] Step 2: The scheduler selects a request.

[0094] In this embodiment, a pre-defined scheduler continuously monitors the waiting queue and the status of the graphics processing unit (GPU). When the GPU has idle resources, the scheduler can select one or more requests from the waiting queue as a batch. The scheduler's request selection process is intelligent, prioritizing requests that can be processed efficiently.

[0095] Step 3: Create batches.

[0096] The pre-defined scheduler selects one or more requests from the waiting queue and moves the selected requests to the running queue to form a request batch.

[0097] Step 4: Model iterative running (Running Loop), scheduling GPU processing.

[0098] Requests entering the Running Queue are fed together into the GPU for forward propagation computation of the model, generating a token array structure. This process is iterative; each time a token array structure is generated, the scheduler re-evaluates the process, involving the following (1) and (2): (1) Dynamic addition (i.e.) Figure 2 Step 7: Continuous scheduling): If the GPU still has free capacity at this time (for example, a request has completed speech synthesis and released resources), the scheduler will immediately pull new requests from the waiting queue and add them to the current running queue without waiting for the entire batch to complete speech synthesis.

[0099] (2) Continuous operation (i.e.) Figure 2 Step 8: Dynamic Batch Processing): Requests that have not yet completed speech synthesis in the running queue will remain in the running queue for the next round of Token array structure generation.

[0100] Step 6: Move to the completed queue (Running -> Finished Queue).

[0101] When a request in the running queue completes speech synthesis (e.g., generates an end-of-speech symbol EOS, or reaches the maximum length), the request is removed from the running queue and moved to the finished queue. Finally, the result of the request is returned to the user from the finished queue.

[0102] This embodiment can reduce idle time caused by waiting for speech synthesis and batch combination, thereby improving the overall throughput.

[0103] Corresponding to the aforementioned text-to-speech processing method, this disclosure further provides a text-to-speech processing apparatus. Figure 3 This is a schematic diagram of a text-to-speech processing device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware, and is generally integrated into an electronic device, such as... Figure 3 As shown, the text-to-speech processing device includes: an acquisition unit 31, a splicing unit 32, a segmentation unit 33, and a synthesis unit 34. Detailed descriptions are as follows: Acquisition unit 31 is used to acquire at least one text-to-speech request, wherein the text-to-speech request carries the first text of the speech to be converted; The splicing unit 32 is used to splice text based on the first text carried by at least one text-to-speech request to obtain the second text; Segmentation unit 33 is used to semantically segment the second text to obtain the array structure corresponding to the second text; Synthesis unit 34 is used to perform speech synthesis based on an array structure to obtain the speech corresponding to the second text.

[0104] In some embodiments, the splicing unit 32 is used for: Select one or more requests from at least one text-to-speech request as a request batch; Based on the first text carried by each request in the request batch, the text is concatenated to obtain the second text corresponding to the request batch.

[0105] In some embodiments, the segmentation unit 33 is used to: perform semantic segmentation on the second text corresponding to the request batch to obtain an array structure corresponding to the request batch.

[0106] In some embodiments, the apparatus further includes: an update unit, configured to dynamically update the request batch after the segmentation unit 33 obtains the array structure corresponding to the request batch, to obtain an updated request batch, and to add one or more requests to the updated request batch; The splicing unit 32 is also used to: splice text based on the first text carried by each request in the updated request batch to obtain the second text corresponding to the updated request batch.

[0107] In some embodiments, the segmentation unit 33 is further configured to: after the splicing unit 32 obtains the second text corresponding to the updated request batch, perform semantic segmentation on the second text corresponding to the updated request batch to obtain an array structure corresponding to the updated request batch.

[0108] In some embodiments, the update unit is used for: After the speech corresponding to the first text carried by the first request in the synthesis request batch of the synthesis unit 34 is synthesized, the first request in the request batch is deleted, and a second request is added to the request batch; the second request is one of at least one text-to-speech request.

[0109] In some embodiments, the text-to-speech request also carries timbre and / or speed of sound; The synthesis unit 34 is used to: perform speech synthesis based on the array structure and the timbre and / or speed corresponding to the array structure to obtain the speech corresponding to the second text.

[0110] The text-to-speech processing apparatus provided in this disclosure can execute the text-to-speech processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device embodiments can be referred to the corresponding process in the method embodiments, and will not be repeated here.

[0112] This disclosure provides an electronic device, which includes: a storage device storing a computer program thereon; and a processing device for executing the computer program in the storage device to implement the steps of any method of this disclosure.

[0113] The following is for reference. Figure 4 This diagram illustrates a structural schematic of an electronic device 400 suitable for implementing embodiments of the present disclosure. The terminal devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0114] like Figure 4 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. Processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0115] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0116] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of the embodiments of this disclosure.

[0117] In addition to the methods and devices described above, embodiments of this disclosure can also be computer program products, comprising computer program instructions that, when executed by a processor, cause the processor to perform the methods provided in the embodiments of this disclosure. The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0118] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the text-to-speech processing method provided in embodiments of this disclosure.

[0119] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0120] This disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the text-to-speech processing method of this disclosure.

[0121] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0122] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0123] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0124] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0125] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0126] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text-to-speech processing method, characterized in that, The method includes: Obtain at least one text-to-speech request, wherein the text-to-speech request carries a first text of the speech to be converted; Based on the first text carried by the at least one text-to-speech request, text concatenation is performed to obtain the second text; The second text is semantically segmented to obtain the array structure corresponding to the second text; Speech synthesis is performed based on the array structure to obtain the speech corresponding to the second text.

2. The method according to claim 1, characterized in that, The process of concatenating the first text carried by the at least one text-to-speech request to obtain the second text includes: From the at least one text-to-speech request, select one or more requests as a request batch; Based on the first text carried by each request in the request batch, the text is concatenated to obtain the second text corresponding to the request batch.

3. The method according to claim 2, characterized in that, The step of semantically segmenting the second text to obtain the array structure corresponding to the second text includes: The second text corresponding to the request batch is semantically segmented to obtain the array structure corresponding to the request batch.

4. The method according to claim 3, characterized in that, After obtaining the array structure corresponding to the requested batch, the method further includes: The request batch is dynamically updated to obtain an updated request batch, and one or more requests are added to the updated request batch. Based on the first text carried by each request in the updated request batch, text concatenation is performed to obtain the second text corresponding to the updated request batch.

5. The method according to claim 4, characterized in that, After obtaining the second text corresponding to the updated request batch, the method further includes: The second text corresponding to the updated request batch is semantically segmented to obtain an array structure corresponding to the updated request batch.

6. The method according to claim 4, characterized in that, The dynamic updating of the request batch includes: During the speech synthesis process based on the array structure corresponding to the request batch, after synthesizing the speech corresponding to the first text carried by the first request in the request batch, the first request in the request batch is deleted, and a second request is added to the request batch; the second request is one of the at least one text-to-speech requests.

7. The method according to claim 1, characterized in that, The text-to-speech request also carries timbre and / or speed of sound; The process of synthesizing speech based on the array structure to obtain the speech corresponding to the second text includes: Based on the array structure and the timbre and / or speed corresponding to the array structure, speech synthesis is performed to obtain the speech corresponding to the second text.

8. An electronic device, characterized in that, The electronic device includes: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the text-to-speech processing method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the text-to-speech processing method according to any one of claims 1-7.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the text-to-speech processing method according to any one of claims 1-7.

Images