Speech synthesis method and device, computer equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a speech synthesis method and device, computer equipment and computer readable storage medium, for reducing the number of speech synthesis calls, improving the adaptive ability of cache and batch processing strategy, thereby reducing service pressure and improving overall processing efficiency under the premise of ensuring the continuity of playing. It includes: performing sentence processing on the to-be-processed text to obtain a first sentence set; performing hit checking on each sentence in the first sentence set in sequence according to a cache audio set; returning first audio data for the sentence that hits the cache audio set, and grouping and caching the sentence that does not hit the cache audio set to obtain target splicing text; performing speech synthesis on the target splicing text to obtain second audio data; and playing the first audio data and the second audio data in sequence.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, specifically to a speech synthesis method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] With the development of large language model technology, its application in intelligent interaction scenarios such as voice question answering and human-computer dialogue is becoming increasingly widespread. Related terminals typically receive text content output by the large language model in a streaming manner and convert the output into speech for playback using text-to-speech technology, thereby improving the real-time performance and naturalness of the interaction. In existing solutions, to balance response speed, a processing method that immediately initiates a speech synthesis request after detecting a complete sentence is often adopted, gradually forming a real-time voice broadcasting technology path oriented towards streaming output.

[0003] In the aforementioned technological development, existing streaming speech synthesis solutions generally adopt a sentence-by-sentence synthesis mode, meaning that for each complete sentence output by the model, the terminal initiates an independent call to the speech synthesis service. Since a single streaming response typically contains multiple sentences, this necessitates frequent, multiple synthesis requests during a single interaction. With a large number of concurrent users, the speech synthesis service needs to handle a large number of discrete requests simultaneously, easily leading to increased service pressure, higher response latency, and increased computational and invocation costs. Furthermore, existing technologies typically use the same request organization method for sentences of different lengths, lacking the ability to adaptively adjust synthesis batches, resulting in low resource utilization efficiency. In addition, existing technologies do not fully utilize the time window corresponding to the time during which the terminal plays the generated speech for caching and scheduling subsequent text, resulting in wasted buffer time; they also lack a mechanism for dynamically updating the caching strategy based on actual operational data, making it difficult to balance real-time performance and service load.

[0004] Therefore, there is an urgent need for a speech synthesis processing solution for streaming text output that can reduce the number of speech synthesis calls while ensuring playback continuity, improve the adaptability of caching and batch processing strategies, thereby reducing service pressure and improving overall processing efficiency. Summary of the Invention

[0005] This application provides a speech synthesis method, apparatus, computer device, and computer-readable storage medium, which reduces the number of speech synthesis calls and improves the adaptive capability of caching and batch processing strategies while ensuring playback continuity, thereby reducing service pressure and improving overall processing efficiency.

[0006] The technical solution adopted by this invention to solve the problem is as follows: Firstly, this application provides a speech synthesis method, including: The text to be processed is segmented into sentences to obtain the first set of sentences; Iterate through the cached audio collection and perform a hit check on each statement in the first statement collection in order; For statements that hit the cached audio set, return the first audio data; for statements that do not hit the cached audio set, group and cache them to obtain the target concatenated text. Speech synthesis is performed on the target concatenated text to obtain second audio data; Play the first audio data and the second audio data in sequence.

[0007] In some embodiments of this application, statements that do not match the cached audio set are grouped to obtain the target concatenated text, including: When the count of a statement that misses the cached audio set is less than or equal to a first threshold, the single statement is identified as the target concatenated text. When the count of statements that do not match the cached audio set is greater than the first threshold, the statements that do not match the cached audio set are cached based on the first preset sentence length to obtain the target concatenated text.

[0008] In some embodiments of this application, caching the statements that do not match the cached audio set based on a first preset sentence length to obtain the target concatenated text includes: The first preset sentence length is estimated online based on an exponentially weighted moving average algorithm; Based on the first preset sentence length, the statements that do not match the cached audio set are cached to obtain the target concatenated text.

[0009] In some embodiments of this application, the method further includes: Based on the first preset sentence length and target word count constraints, the target sentence length is dynamically calculated through a constraint optimization algorithm, and this target sentence length is applied to the next grouping cache.

[0010] In some embodiments of this application, after performing speech synthesis on the target concatenated text to obtain second audio data, the method further includes: The second audio data is segmented into multiple single-sentence audio segments; The multiple single-sentence audio clips are cached into this cached audio collection.

[0011] In some embodiments of this application, the second audio data is segmented into multiple single-sentence audio sequences, including: Obtain the start and end positions of each sentence in the target concatenated text and / or the global sentence number; The second audio data is segmented into multiple single-sentence audio segments based on the start and end positions and / or the global sentence number.

[0012] In some embodiments of this application, the method further includes: obtaining high-frequency statements; Initiate a speech synthesis request for the high-frequency statement to obtain third-party audio data; The third audio data is cached in the cached audio set.

[0013] Secondly, this application provides a speech synthesis device, comprising: The processing module is used to segment the text to be processed into sentences to obtain a first set of sentences; iterates through the cached audio set and checks each sentence in the first set in order for a match; returns the first audio data for sentences that match the cached audio set, and groups and caches the sentences that do not match the cached audio set to obtain the target concatenated text; and performs speech synthesis on the target concatenated text to obtain the second audio data. The playback module is used to play the first audio data and the second audio data in sequence.

[0014] Thirdly, this application also provides a computer device, which includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor to implement the law of any of the first aspects.

[0015] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the method of any one of the first aspects.

[0016] The beneficial effects of this invention are as follows: By employing a two-tiered processing strategy—prioritizing cache hits and then grouping and caching text for batch synthesis—it can balance cache reuse efficiency and batch processing performance during speech synthesis. Specifically, the first step is to query the cached audio, directly reusing previously synthesized identical or high-frequency text and speech, avoiding repeated calls to the synthesis service, thereby reducing computational resource consumption and interface call costs, and shortening response time. For texts not found in the cached audio set, grouping and batch synthesis are then performed, reducing scattered synthesis requests, increasing synthesis throughput, and simultaneously storing subsequent generated results as reusable grouped caches. This two-tiered strategy improves cache hit rate and resource utilization, while reducing initial packet latency and overall synthesis time in long text scenarios, thereby improving the continuity, stability, and user experience of speech playback. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the system architecture provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of an embodiment of the speech synthesis method provided by the present invention; Figure 3 This is a schematic flowchart of a speech synthesis method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a specific embodiment of the speech synthesis device provided in this invention. Figure 5 This is a schematic diagram of an embodiment of the computer device provided in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more features.

[0021] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0022] It should be noted that since the method in this application embodiment is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. Specific details will not be elaborated here.

[0023] With the development of large language model technology, its application in intelligent interaction scenarios such as voice question answering and human-computer dialogue is becoming increasingly widespread. Related terminals typically receive text content output by the large language model in a streaming manner and convert the output into speech for playback using text-to-speech technology, thereby improving the real-time performance and naturalness of the interaction. In existing solutions, to balance response speed, a processing method of initiating a speech synthesis request immediately after detecting a complete sentence is often adopted, gradually forming a real-time voice broadcasting technology path oriented towards streaming output. During the aforementioned technological development, existing streaming speech synthesis solutions generally adopt a sentence-by-sentence synthesis mode, that is, the terminal initiates an independent call to the speech synthesis service for each complete sentence output by the model. Since a single streaming response usually contains multiple sentences, multiple synthesis requests need to be frequently initiated during a single interaction. When there are many concurrent users, the speech synthesis service needs to process a large number of discrete requests simultaneously, easily leading to increased service pressure, increased response latency, and increased computational and invocation costs. At the same time, existing technologies typically use the same request organization method for sentences of different lengths, lacking the ability to adaptively adjust the synthesis batch, resulting in low resource utilization efficiency. Furthermore, existing technologies do not fully utilize the time window corresponding to the playback of generated speech on the terminal to cache and schedule subsequent text synthesis, resulting in wasted buffer time; they also lack a mechanism to dynamically update the caching strategy based on actual operating data, making it difficult to balance real-time performance and service load.

[0024] To address this technical problem, this application provides the following technical solution: The text to be processed is segmented into sentences to obtain a first set of sentences; the cached audio set is traversed, and each sentence in the first set is checked for a match in sequence; sentences that match the cached audio set are returned as first audio data, while sentences that do not match the cached audio set are grouped and cached to obtain the target concatenated text; the target concatenated text is then used for speech synthesis to obtain second audio data; and the first and second audio data are played sequentially. This two-level processing strategy, prioritizing cache hits followed by grouped caching and batch synthesis of the text to be processed, balances cache reuse efficiency and batch processing performance during speech synthesis. Specifically, firstly, the cached audio is queried, allowing direct reuse of previously synthesized identical or high-frequency text and speech, avoiding repeated calls to the synthesis service, thereby reducing computational resource consumption and interface call costs, and shortening response time; for texts that do not match the cached audio set, grouping caching and batch synthesis are then performed, reducing fragmented synthesis requests, increasing synthesis throughput, and simultaneously storing subsequent generated results as reusable grouped caches. This two-tiered strategy improves cache hit rate and resource utilization, while reducing first-packet latency and overall synthesis time in long text scenarios, thereby improving the continuity, stability, and user experience of voice playback.

[0025] This application provides a speech synthesis method, apparatus, computer device, and computer-readable storage medium, which reduces the number of speech synthesis calls and improves the adaptive capability of caching and batch processing strategies while ensuring playback continuity, thereby reducing service pressure and improving overall processing efficiency. The electronic device provided in this application can be implemented as various types of user terminals or as a server.

[0026] Electronic devices can reduce the number of speech synthesis calls and improve the adaptive capability of caching and batch processing strategies by running the speech synthesis method provided in the embodiments of this application, while ensuring playback continuity, thereby reducing service pressure and improving overall processing efficiency.

[0027] The technical solution of this application can be applied to voice interaction scenarios of terminals such as smart speakers, smart TVs, and smartphones, as well as application scenarios that require large language model (LLM) streaming speech synthesis (Text-to-Speech, TTS) such as online customer service, smart assistants, and audio reading.

[0028] In one exemplary solution, this speech synthesis method can be applied to the voice interaction scenario of a smart TV. For example, in a smart TV voice interaction scenario, when a user makes a request such as "play today's news," "read the movie synopsis," or "introduce the plot of this TV series" through the remote control or the TV's voice assistant, the smart TV first obtains the text content to be played and segments it into sentences according to rules such as punctuation and semantic pauses to obtain a first set of sentences. Subsequently, the TV or cloud-based voice service caches audio sets according to the time sequence of the sentences in the first set of sentences, checking each sentence to see if there is a corresponding audio cache. For frequently used sentences, fixed prompts, or previously played content that have been matched, the first audio data is directly read and returned. For new sentences such as news articles and program introductions that have not been matched, they are grouped and cached according to preset sentence length, target word count, or playback continuity requirements to form target concatenated text. The target concatenated text is then subjected to batch speech synthesis to obtain second audio data. Then, the system sorts, splices, or streams the first and second audio data according to the order of each sentence in the original text, so that users can quickly hear the voice corresponding to the cached sentences. At the same time, the background completes the batch synthesis of the missing text, thereby reducing the overhead of repeated synthesis, reducing the waiting time of the first segment, and ensuring that smart TVs have a smoother, more continuous and faster-responding voice interaction experience in scenarios such as long text reading, program recommendation, system prompts, and content search result broadcasting.

[0029] It should be understood that the above is only an exemplary application scenario of the speech synthesis method provided in this application. There may be many other specific application scenarios, which are not limited here.

[0030] The speech synthesis method provided in this application embodiment is applied to, for example, Figure 1 The system architecture diagram shown is for your reference. Figure 1 To support a speech synthesis method, the terminal device 100 connects to the server 300 via network 200, and the server 300 connects to the database 400. Network 200 can be a wide area network (WAN), a local area network (LAN), or a combination of both. The client used to implement the speech synthesis scheme is deployed on the terminal device 100, or it can run on the terminal device 100 as a standalone application. The specific form of the client is not limited here.

[0031] The server 300 involved in this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0032] Terminal equipment 100, also known as user equipment (UE), mobile station (MS), mobile terminal (MT), customer premises equipment (CPE), etc., can be a device that includes both receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such equipment can include cellular or other communication devices with single-line displays, multi-line displays, or no multi-line displays. Examples include handheld devices with wireless connectivity, vehicle-mounted devices, machine-type communication (MTC) terminals, etc. Currently, terminal devices 100 can include: mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving vehicles, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, etc. For example, wireless terminals in self-driving vehicles can be drones, helicopters, or airplanes. For example, wireless terminals in vehicle-to-everything (V2X) systems can be in-vehicle equipment, vehicle equipment, in-vehicle modules, vehicles, or ships, etc. Wireless terminals in industrial control can be cameras, robots, or robotic arms, etc. Wireless terminals in smart homes can be televisions, air conditioners, robot vacuums, speakers, or set-top boxes, etc.

[0033] It should be noted that the terminal device 100 may be a device or apparatus with a chip, or a device or apparatus with integrated circuitry, or a chip, module, or control unit in the device or apparatus shown above; this application does not impose any specific limitations. The solution provided in this application can be implemented by the terminal device 100 and the server 300 working together.

[0034] In short, a database can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, shared by multiple users, with minimal redundancy, and independent of application programs. A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or Extensible Markup Language (XML); or according to the type of computer they support, such as server clusters or mobile phones; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously. In this application, database 400 can be used to store data such as cached audio sets, text to be processed, and grouped queues.

[0035] Those skilled in the art will understand that Figure 1 The system architecture diagram shown is one possible system architecture for this application and does not constitute a limitation on the system architecture of this application. Other system architectures may include more advanced architectures. Figure 1 The number of more or fewer terminal devices or servers shown, for example Figure 1 The diagram shows one server. It is understood that the system architecture may also include one or more other terminal devices or servers, which are not limited here.

[0036] It should be noted that, Figure 1 The system architecture shown is an example. The servers and scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of servers and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0037] like Figure 2 The diagram shown is a flowchart of an embodiment of the speech synthesis method in this application. The following description, using a server as the execution entity, details the speech synthesis method, which may include the following steps 201-205: 201. Segment the text to be processed to obtain the first set of sentences.

[0038] In the application scenario of LLM streaming TTS, the server can obtain the text to be processed from the streaming output of the large model as needed, and trigger the operation of speech synthesis on the text to be processed.

[0039] Optionally, to improve the efficiency of subsequent processing, the following solution can also be provided in this embodiment: The server segments the text to be processed into sentences to obtain the first set of sentences.

[0040] In one possible implementation, the server can segment the text to be processed into sentences based on identifiers. These identifiers include at least one of a period, question mark, exclamation mark, semicolon, ellipsis, and newline character. The server scans the text sequentially according to the order of the characters. When an identifier is detected, its location is designated as a candidate sentence segmentation position. The server then verifies the candidate sentence segmentation position by combining the characters before and after the identifier to determine whether the identifier indicates the end of a sentence. If the verification result indicates that the identifier is a valid sentence end marker, the position of the identifier is used as the sentence boundary, and the continuous text preceding it is divided into a single sentence text unit. The scanning of the remaining text continues until all the text to be processed is segmented into single sentences, resulting in multiple semantically clear single sentence text units. The first sentence set is then generated based on these multiple single sentence text units.

[0041] In another possible implementation, the server can segment the text to be processed based on sentence segmentation rules and a model. For example, the server first identifies candidate sentence positions in the text to be processed based on preset sentence segmentation rules. These preset rules include determining candidate sentence boundaries based on identifiers such as periods, question marks, exclamation marks, ellipses, semicolons, and line breaks, and excluding identifiers that should not be used as sentence boundaries based on special text rules such as abbreviations, numbers, decimals, URLs, parentheses, and quotation marks. Subsequently, the server inputs the candidate sentence positions and the context text of a preset length before and after them into a pre-trained sentence boundary recognition model, which outputs a judgment result on whether the candidate sentence position is a real sentence boundary. When the judgment result indicates that the candidate sentence position is a real sentence boundary, the server segments the text to be processed based on the candidate sentence position, generating corresponding single-sentence text units, and generating the first sentence set based on these multiple single-sentence text units.

[0042] In this embodiment, by segmenting the text to be processed into sentences, continuous text can be divided into multiple semantically complete and clearly defined single-sentence text units. This allows subsequent speech synthesis to be performed based on smaller, more specific text units, thereby reducing interference caused by cross-sentence semantic mixing and improving the accuracy, stability, and interpretability of the text processing results. At the same time, by establishing numbers, position indexes, or correspondences for each single-sentence text unit, it is easy to quickly locate key information in the original text, realize the traceability between the processing results and the original text, and also allow for parallel processing of multiple single-sentence text units, thereby improving the overall text processing efficiency.

[0043] 202. Traverse the cached audio set and perform a hit check on each statement in the first statement set in order.

[0044] After obtaining the first set of statements, the server performs a hit check on the cached audio set in the order of the statements in the first set of statements in the text to be processed.

[0045] In this embodiment, each cached audio file in the cached audio set can be audio data cached at the sentence-by-sentence granularity. To facilitate hit checking, when storing each cached audio file in the cached audio set, the index information corresponding to each cached audio file can be extracted synchronously. This index information can include one or more of the following: audio name, text keywords corresponding to the audio content, business scenario identifier, audio type, speaker identifier, and language information (for example, the index information can be generated based on the hash value of the text content corresponding to the audio content, the speaker identifier (also known as the role identifier) ​​corresponding to the current audio content, and the language information (e.g., Chinese), which can also be called a cache key). The server establishes a mapping relationship between the storage address or audio identifier of the cached audio file and the above index information, and stores the mapping relationship in an index table, a key-value storage structure, or a cache management module.

[0046] During the match check, the server generates target index information based on the sentence to be matched or the target index information extracted from the sentence to be matched, such as the hash value of the text content corresponding to the audio content, the speaker identifier (also known as the role identifier) ​​and language information corresponding to the current audio content, and uses the target index information to query in the established mapping relationship; if a cached audio identifier that matches the target index information is found, it is determined that there is reusable cached audio in the cached audio set, and the cached audio is read for playback, return, or subsequent processing.

[0047] Using the above method, the system does not need to traverse and compare all audio files in the cached audio set one by one. Instead, it can quickly narrow down the search scope or directly locate the target cached audio based on the index information, thereby improving the efficiency of cache hit checking, reducing the amount of computation and response latency in the audio matching process, and facilitating the subsequent updating, deletion and unified management of cached audio.

[0048] 203. Return the first audio data for statements that hit the cached audio set, and group and cache statements that do not hit the cached audio set to obtain the target concatenated text.

[0049] In this embodiment, the server performs a hit check based on the hit check scheme described in step 202. For statements that hit the cached audio set, the server returns the corresponding first audio data. For statements that do not hit the cached audio set, the server groups and caches them to obtain the target concatenated text.

[0050] Optionally, to facilitate subsequent sequential playback, the first audio data can acquire global sentence annotations, whereby the global sentence annotations can be the sequential annotations of the corresponding sentences in the text to be processed.

[0051] In one possible implementation, the server can use the following technical solution when performing group caching: The server maintains a "missing sentence counter". It counts the missing sentences during the hit check of the text to be processed. When the count of the missing sentences in the cached audio set is less than or equal to a first threshold, the corresponding single sentence is identified as the target concatenated text and immediately submitted for speech synthesis. When the count of the missing sentences in the cached audio set is greater than the first threshold, the corresponding single sentence is cached, and the cache length threshold is set to a first preset sentence length. When the first preset sentence length is reached, the target concatenated text is output and submitted for speech synthesis.

[0052] For example, the first threshold is set to N, where N is a configurable instantaneous synthesis threshold, such as N being configured to 3. Assume that the first set of statements, according to global sentence annotations, includes the following statements: (sentence 1, annotation 1), (sentence 2, annotation 2), (sentence 3, annotation 3), (sentence 4, annotation 4), (sentence 5, annotation 5), (sentence 6, annotation 6), (sentence 7, annotation 7), (sentence 8, annotation 8). During the hit check, if sentence 1 matches audio 3 in the cached audio set, then audio 3 is returned and labeled "Label 1"; if sentence 2 does not match any audio in the cached audio set, the missing sentence counter value is 1, which is less than 3, and sentence 2 can be used as the target concatenated text and immediately submitted for speech synthesis; if sentence 3 matches audio 2 in the cached audio set, then audio 2 is returned and labeled "Label 3"; if sentence 4 does not match any audio in the cached audio set, the missing sentence counter value is 2, which is less than 3, and sentence 4 can be used as the target concatenated text and immediately submitted for speech synthesis; if sentence 5 does not match any audio in the cached audio set... If an audio in the audio set is matched with a sentence count of 3, then sentence 5 can be used as the target concatenated text and submitted immediately for speech synthesis. If sentence 6 matches audio 5 in the cached audio set, then audio 5 is returned and labeled "labeled 6". If sentence 7 does not match audio in the cached audio set, then the sentence count is 4, which is greater than 3, and sentence 6 can be added to the group queue for group caching. If sentence 7 does not match audio in the cached audio set, then the sentence count is 5, which is greater than 3, and sentence 7 can be added to the group queue for group caching. At this time, the concatenated text includes sentences 6 and 7. If the sentence length of the concatenated text including sentences 6 and 7 meets the first preset sentence length, then it can be determined as the target concatenated text and submitted for speech synthesis. After submission, the server can continue to check for hits on subsequent sentences. If sentence 8 does not match any audio in the cached audio set, the miss count will be 6. If the count is greater than 3, sentence 8 can enter a new grouping queue for caching. In this case, the new concatenated text includes sentence 8. If no new sentences follow, the server can directly submit sentence 8 for speech synthesis. If new sentences are added later, the server can continue to perform hit checks.

[0053] In some embodiments, the server may employ the following technical solution when obtaining the first preset sentence length: The server estimates the first preset sentence length online based on the Exponentially Weighted Moving Average (EWMA) algorithm. For example, for each sentence that is cache-missed and enters the group queue, the server uses the EWMA algorithm to estimate the sentence length sequence online and tracks the changing trend of the sentence length in real time.

[0054] That is, when the server groups sentences in the text to be processed, it maintains an exponentially weighted moving average sentence length estimate S to represent the current sentence length level. When the k-th sentence enters the grouping queue, the server obtains the character length Lk of that sentence and updates the sentence length estimate S according to the formula Sk=α×Lk+(1-α)×Sk-1, where α is a smoothing factor that satisfies 0<α<1, used to control the weight ratio of the current sentence length and the historical sentence length estimate in the update result. In an exemplary scheme, the value of α can range from 0.2 to 0.4, with a default value of 0.3, so that the sentence length estimate can respond to changes in sentence length while suppressing estimation fluctuations caused by abnormally long or short sentences. During the initialization phase, the server can set the initial sentence length estimate S0 to the length L1 of the first sentence entering the grouping queue, so that the sentence length estimate is calculated from the actual observation value, avoiding the deviation caused by using a fixed default value during the cold start phase. This provides a stable and reliable sentence length reference for subsequent sentence grouping, group length control, or batch text processing.

[0055] Optionally, in this embodiment, sentences that hit audio data in the cached audio set are not included in the EWMA sentence length estimation update because they will not enter batch synthesis, and their length should not affect the calculation of the grouping target. This ensures that the estimated value only reflects the length characteristics of sentences that actually need to be batch synthesized.

[0056] To better optimize the preset sentence length, in this embodiment, after completing one group caching, the server can also calculate the preset sentence length for the next group caching based on the first preset sentence length determined by this group caching.

[0057] In the exemplary scheme, the server can dynamically calculate the target sentence length based on the first preset sentence length and the target word count constraint using a constraint optimization algorithm, wherein the target sentence length can be applied to the next group caching.

[0058] Specifically, the above scheme can be designed as follows: First, let's assume the server is configured with the following parameters: T: Target Chars, representing the ideal total number of characters for each target concatenated text, with a default value of 80; Nmin: Minimum number of sentences in a batch (min Batch Sentences), the lower bound constraint for grouping, with a default value of 2; Nmax: Maximum number of batch sentences, an upper bound constraint for batches, with a default value of 10; Sk: Current EWMA sentence length estimate, for example, the first preset sentence length.

[0059] The calculation of the target sentence length can then be divided into the following two stages: Phase 1: Solving for the unconstrained optimal solution.

[0060] The optimization objective is to minimize the difference between the actual total number of words and the target total number of words, and to find the theoretically optimal number of sentences per group. Let the number of sentences per group be n, then the expected total number of words per group is n×Sk. The optimization objective is to minimize the deviation function: min|n×Sk-T|.

[0061] The solution yields the number of sentences with the unconstrained optimal grouping: n =round(T / Sk).

[0062] The `round(·)` function is used for rounding to the nearest integer. The physical meaning of this formula is: when the average sentence length estimated by EWMA is short, more sentences are needed to reach the target total word count, naturally increasing the group size; conversely, when the average sentence length is long, fewer sentences are needed to reach the target, naturally decreasing the group size. This achieves adaptive adjustment of group size based on sentence length characteristics.

[0063] Phase 2: Boundary constraint clipping.

[0064] The unconstrained optimal solution n Projecting onto the feasible region [Nmin, Nmax] yields the final grouping target N_batch: N_batch = clip(n , Nmin, Nmax) = max(Nmin, min(n (, Nmax)).

[0065] The clip(·) function ensures that the target group is not lower than the lower bound Nmin (to avoid excessive requests due to overly fragmented groups) and does not exceed the upper bound Nmax (to avoid excessive latency in a single synthesis due to overly large groups).

[0066] Numerical examples: Example 1: T=80, Sk=20→n =round(80 / 20)=4→N_batch=clip(4,2,10)=4 (Batch synthesis after accumulating 4 sentences).

[0067] Example 2: T=80, Sk=50→n =round(80 / 50)=2→N_batch=clip(2,2,10)=2 (batch synthesis after accumulating 2 sentences).

[0068] Example 3: T=80, Sk=10→n =round(80 / 10)=8→N_batch=clip(8,2,10)=8 (Short sentences are more common).

[0069] Example 4: T=80, Sk=100→n =round(80 / 100)=1→N_batch=clip(1,2,10)=2 (triggers lower bound protection).

[0070] The algorithm has a time complexity of O(1) and a space complexity of O(1). It only requires one division, one rounding, and one pruning operation to complete the calculation of the grouping target, which meets the strict real-time requirements of streaming processing scenarios.

[0071] Furthermore, when the EWMA estimate Sk has not yet been initialized (i.e., no sentences have entered the grouping queue), the server can employ a degradation strategy, directly setting the grouping objective to Nmin to ensure logical stability. As subsequent sentences arrive and the EWMA estimate is updated, the grouping objective will gradually converge to the optimal value.

[0072] Optionally, the server can also select an appropriate grouping caching method based on the actual situation. For example, for devices with lower performance, a fixed-size sliding window can be used to calculate the average sentence length or grouping can be done by a fixed number of sentences. For devices with higher performance, grouping can be based on semantic integrity.

[0073] 204. Perform speech synthesis on the target concatenated text to obtain the second audio data.

[0074] After the server completes one group caching operation, it can submit the target concatenated text to speech synthesis to obtain the second audio data.

[0075] Optionally, to increase the cached audio in the cached audio set, this embodiment may also provide a scheme for caching the second audio data. In an exemplary scheme, the server may perform speech segmentation on the second audio data to obtain multiple single-sentence audio; then, the multiple single-sentence audios are cached in the cached audio set.

[0076] In some embodiments, when performing speech segmentation on the second audio data, the server may obtain the start and end positions of each sentence in the target concatenated text and / or the global sentence number of each sentence in the text to be processed (which can also be understood as the aforementioned global sentence annotation); and then perform speech segmentation on the second audio data based on the start and end positions and / or the global sentence number to obtain multiple single-sentence audio.

[0077] In some embodiments, the server may first perform speech endpoint detection on the second audio data to determine the pause positions between adjacent sentences in the audio, and combine the text content, punctuation information or semantic boundaries obtained from speech recognition to divide the second audio data into single-sentence levels, thereby obtaining multiple single-sentence audios.

[0078] Optionally, when the server caches the multiple single-sentence audio, it can generate a corresponding audio identifier for each single-sentence audio, and generate corresponding index information based on the audio identifier and / or the text, keywords, and other information corresponding to the single-sentence audio. Then, the single-sentence audio and its index information are associated and stored in the cached audio set. When audio reuse or hit checking is required in the future, the server can quickly query the single-sentence audio in the cached audio set based on the text keywords or audio identifier, without having to reprocess the complete second audio data, thereby improving the audio cache utilization and subsequent matching efficiency.

[0079] Optionally, to reduce processing steps, the server can also perform overall caching when caching the second audio data.

[0080] In some embodiments, in order to improve the hit rate of each cached audio in the cached audio set, cache preheating and dynamic updates can also be performed on each cached audio in the cached audio set.

[0081] In one exemplary scheme, the server can obtain high-frequency statements; then submit a speech synthesis request for the high-frequency statements to obtain third audio data; and then cache the third audio data in the cached audio set at the sentence-by-sentence granularity.

[0082] Optionally, the server can acquire high-frequency sentences through data collection via embedded points. For example, the server can record information such as sentence text, angle identifier, language, and call time for each TTS call, generating a TTS call log; then, based on the TTS call log, it can perform aggregation analysis to count the frequency of each sentence, thereby generating high-frequency sentences.

[0083] For example, in an intelligent customer service voice broadcast scenario, the server can record information such as the sentence text, role identifier, language, call time, business scenario, and number of calls for each time the TTS service is called to generate voice, and generate a TTS call log. The server performs aggregate analysis on the TTS call log according to a preset statistical period (e.g., daily or hourly), grouping records with the same sentence text, role identifier, and language into the same category, counting the frequency of each type of sentence, and identifying sentences with a frequency exceeding a preset threshold as high-frequency sentences. For example, when the number of calls to the Chinese customer service opening phrase "Hello, how can I help you?" under the same role identifier exceeds the threshold, the server identifies it as a high-frequency sentence and can further pre-generate or add the TTS audio corresponding to the high-frequency sentence to the cached audio set so that the cached audio can be directly reused when the same or similar requests arrive later.

[0084] Optionally, the server can count the number of times the cached audio in the cached audio collection is called, and then clean up the low-frequency cached audio that has not been hit for a long time based on the statistical results, freeing up storage space and thus maintaining the efficient operation of the caching system.

[0085] Optionally, when storing the cached audio collection, the server can dynamically adjust the storage location of the cached audio collection based on actual conditions. For example, in scenarios with limited network conditions but sufficient local storage space, the cached audio collection can be stored in local storage. In scenarios with superior network performance and / or insufficient local storage space, the cached audio collection can be stored in cloud storage associated with the server.

[0086] Optionally, the server can also dynamically adjust high-frequency sentences and cached audio sets based on changes in user usage patterns. For example, in one period, if a user uses news broadcasts multiple times, the high-frequency sentence could be A; while in another period, if a user uses audiobook playback multiple times, the high-frequency sentence could be B.

[0087] 205. Play the first audio data and the second audio data in sequence.

[0088] In this embodiment, the server maintains a playback queue sorted by global sentence sequence numbers, which indicate the order of sentences within the text to be processed. After obtaining the audio data corresponding to each sentence, the server associates the audio data with the corresponding global sentence sequence number and adds it to the playback queue as an audio unit to be played. The sources of the audio data include the first audio data and the second audio data (i.e., including hit audio from the cached audio set, audio synthesized in real-time based on a single sentence, or single-sentence audio obtained by splitting batch-synthesized audio). The server receives... When any audio unit to be played is reached, it is first determined whether the global sentence number of the audio unit to be played is the current number to be played. If so, the audio unit to be played is pushed to the terminal for playback, and the current number to be played is incremented. At the same time, it continues to check whether there are audio units with subsequent consecutive numbers in the playback queue. If the global sentence number of the received audio unit to be played is later than the current number to be played, it is first cached in the playback queue and pushed in order after the missing preceding audio unit arrives. In this way, even if the audio from different sources takes different time to generate or obtain, it can ensure that the terminal plays the voice content in the original sentence order.

[0089] For example, the text to be processed is divided into three sentences, corresponding to global sentence numbers 1, 2, and 3 respectively. Sentence 1 hits the cached audio, sentence 2 needs to be synthesized in real time, and sentence 3 is obtained through batch synthesis and splitting. Since the cached audio of sentence 1 is obtained first, the audio sequence management module immediately pushes the audio of sentence number 1 to the terminal for playback and updates the current playback sequence number to 2. If the audio of sentence number 3 arrives before the audio of sentence number 2, the server does not immediately push the audio of sentence number 3, but temporarily stores it in the playback queue. When the audio of sentence number 2 is synthesized and arrives, the server first pushes the audio of sentence number 2, then checks that the audio of sentence number 3 already exists in the playback queue, and then continues to push the audio of sentence number 3, so that the user hears the voice content corresponding to sentence 1, sentence 2, and sentence 3 in sequence, avoiding playback order disorder caused by different audio sources or different generation completion times.

[0090] Optionally, in this embodiment, if the server detects an interruption signal, the server can immediately clear the concatenated text that has not yet been submitted in the current group cache queue, discard the cached text that has not been sent, and stop the playback of subsequent audio.

[0091] The following is based on Figure 3 The flowchart shown illustrates the technical solution provided in this application.

[0092] like Figure 3As shown, the server receives the text to be processed from the LLM streaming output; then it segments the text according to punctuation and extracts complete sentences; then it assigns a global sentence number to each complete sentence; for each individual sentence, it generates a cache key based on the text content and / or text keywords; it performs a hit check on the cached audio data based on this cache key; if a hit occurs, it returns the corresponding cached audio data; if a hit occurs, it counts the number of hits using a timer; when the count of hit sentences is less than or equal to N, it performs real-time synthesis and caches the real-time synthesized audio data into the cached audio data. The system collects and simultaneously plays the synthesized audio data in sequence. When the count of missed sentences is greater than N, it groups and caches sentences based on a preset sentence length calculated using the EWMA algorithm. Once the sentence length of the group cache reaches the preset sentence length, the concatenated text of the group cache is submitted for batch speech synthesis to obtain the corresponding audio data. The audio data is then split to obtain single-sentence audio. The single-sentence audio is cached in the cached audio set. Simultaneously, after the concatenated text of the previous group cache is submitted, a new group cache can be started. Finally, the cached audio set is updated based on the results of this TTS.

[0093] To better implement the speech synthesis method in the embodiments of this application, based on the speech synthesis method, the embodiments of this application also provide a speech synthesis device, such as... Figure 4 As shown, the speech synthesis device 400 includes: The processing module 401 is used to perform sentence segmentation on the text to be processed to obtain a first set of sentences; traverse the cached audio set and perform a hit check on each sentence in the first set of sentences in order; return the first audio data for sentences that hit the cached audio set, and group and cache the sentences that do not hit the cached audio set to obtain the target concatenated text; and perform speech synthesis on the target concatenated text to obtain the second audio data. The playback module 402 is used to play the first audio data and the second audio data in sequence.

[0094] In this embodiment, a two-level processing strategy—prioritizing cache hits and then grouping and caching text for batch synthesis—achieves a balance between cache reuse efficiency and batch processing performance during speech synthesis. Specifically, the system first queries cached audio, directly reusing previously synthesized identical or high-frequency text-to-speech data, avoiding repeated calls to the synthesis service, thus reducing computational resource consumption and API call costs, and shortening response time. For texts not found in the cached audio set, grouping and batch synthesis are then performed, reducing fragmented synthesis requests, increasing synthesis throughput, and simultaneously storing subsequent generated results as reusable grouped caches. This two-level strategy improves cache hit rate and resource utilization while reducing initial packet latency and overall synthesis time in long text scenarios, thereby improving the continuity, stability, and user experience of speech playback.

[0095] In some embodiments of this application, the processing module 401 is specifically used for: When the count of a statement that misses the cached audio set is less than or equal to a first threshold, the single statement is identified as the target concatenated text. When the count of statements that do not match the cached audio set is greater than the first threshold, the statements that do not match the cached audio set are cached based on the first preset sentence length to obtain the target concatenated text.

[0096] In this embodiment, by synthesizing single sentences of the first few missed texts and then batch synthesizing subsequent texts after grouping and caching, the overall synthesis efficiency can be improved while ensuring rapid output of the first segment of speech. Specifically, synthesizing the first few missed sentences one by one reduces user waiting time, allowing the system to quickly generate and play the initial speech even when the cache misses, thus improving the interactive response speed. Meanwhile, grouping and caching subsequent texts and then batch synthesizing reduces the number of synthesis requests and interface call overhead, lowers the computational cost of repetitive text, and improves the reuse efficiency after a cache hit. This approach balances real-time performance and throughput, avoiding both the excessive initial latency caused by synthesizing the entire text and the resource waste caused by sentence-by-sentence synthesis, thereby improving the playback continuity, processing efficiency, and user experience of the speech synthesis system in long text scenarios.

[0097] In some embodiments of this application, the processing module 401 is specifically used for: The first preset sentence length is estimated online based on an exponentially weighted moving average algorithm; Based on the first preset sentence length, the statements that do not match the cached audio set are cached to obtain the target concatenated text.

[0098] In this embodiment, online sentence length estimation based on the EWMA algorithm can dynamically predict the length changes of subsequent sentences during continuous text input or streaming processing, thereby providing a more stable decision-making basis for text grouping and batch speech synthesis. Based on the estimation results, the group size, cache granularity, and synthesis batch can be determined more reasonably, reducing the request overhead caused by excessively short groups and avoiding the first packet delay and playback wait caused by excessively long groups, thereby improving the real-time performance, continuity, resource utilization, and overall user experience in the speech synthesis process.

[0099] In some embodiments of this application, the processing module 401 is further configured to dynamically calculate the target sentence length based on the first preset sentence length and the target word count constraint using a constraint optimization algorithm, and the target sentence length is applied to the next grouping cache.

[0100] In this embodiment, by using a first preset sentence length as the basis for the current grouping experience and dynamically calculating the target sentence length in conjunction with the target word count constraint, the granularity of the next grouping cache can be adjusted in real time according to the text content and synthesis target. Compared with the fixed sentence length grouping method, the constraint optimization algorithm can comprehensively balance the number of sentences, text length, cache reuse rate, and speech synthesis latency while meeting the target word count range. This avoids low batch synthesis efficiency and many requests due to excessively short groups, and also avoids cache hit difficulties and increased first-packet waiting time due to excessively long groups. The resulting target sentence length is more in line with the actual length distribution of the input text and the system performance target, which can improve the stability and adaptability of grouping cache, improve the throughput efficiency, cache hit effect, and playback continuity of batch speech synthesis, thereby improving the overall processing efficiency and user experience in long text speech synthesis scenarios.

[0101] In some embodiments of this application, the processing module 401 is further configured to: The second audio data is segmented into multiple single-sentence audio segments; these multiple single-sentence audio segments are then cached into the cached audio set.

[0102] In this embodiment, the audio data obtained from speech synthesis is divided into single sentences, and the segmented single-sentence audio is written into a cached audio set. This improves the granularity of audio cache reuse and the flexibility of playback scheduling. Specifically, single-sentence audio is more likely to be hit and reused in subsequent identical or similar text scenarios than whole audio segments, thereby reducing the number of repeated speech synthesis attempts and lowering computational resource and interface call costs. Simultaneously, the segmented audio can be flexibly concatenated and played according to the text order, facilitating the resynthesis of only the changed sentences in scenarios involving long texts, streaming texts, or partial text changes, without needing to regenerate the entire audio segment. This method also supports finer-grained preloading, cache eviction, and breakpoint resume control, reducing playback waiting and stuttering, improving the continuity of speech playback, response speed, and system resource utilization, thereby improving the overall processing efficiency and user experience of the speech synthesis service.

[0103] In some embodiments of this application, the processing module 401 is further configured to: Obtain the start and end positions of each sentence in the target concatenated text and / or the global sentence number; The second audio data is segmented into multiple single-sentence audio segments based on the start and end positions and / or the global sentence number.

[0104] In this embodiment, single-sentence audio segmentation is performed based on the start and end positions of individual sentences in the concatenated text and / or the global sentence sequence number. This establishes a precise correspondence between text sentences and synthesized audio segments, thereby improving the accuracy of audio segmentation, cache writing, and subsequent reuse. Specifically, the start and end positions accurately identify the text range of each sentence in the concatenated text, avoiding audio segment matching errors caused by repeated occurrences of the same sentence, similar punctuation, or unclear sentence boundaries. The global sentence sequence number maintains the order of sentences within the entire text, facilitating subsequent audio concatenation, breakpoint resumption, and partial updates in the original text order. Segmenting single-sentence audio based on these identifiers reduces the risk of mismatches in the cached audio set, improves the retrieval efficiency and reuse reliability of single-sentence audio, and supports processing only the affected sentence audio when text is added, deleted, rearranged, or partially changed, thereby improving the cache management accuracy, playback continuity, and overall processing efficiency of the speech synthesis system.

[0105] In some embodiments of this application, the processing module 401 is further configured to: Retrieve frequently used statements; Initiate a speech synthesis request for the high-frequency statement to obtain third-party audio data; The third audio data is cached in the cached audio set.

[0106] In this embodiment, high-frequency statements are pre-synthesized and their corresponding audio is written to a cached audio set, which significantly improves the cache hit rate and response speed in scenarios with repetitive content. Since high-frequency statements are more likely to appear repeatedly in different texts, different user requests, or the same long text, pre-generating and caching their audio allows for direct playback or splicing of the cached audio when the same statement is encountered again, eliminating the need to repeatedly call the speech synthesis service. This reduces computational resource consumption, interface call costs, and synthesis queuing latency. Simultaneously, single-sentence audio snippets of high-frequency statements can be used as basic audio segments in long text splicing, improving audio reuse granularity and scheduling flexibility, reducing repeated synthesis caused by local repetitive text, and enhancing the continuity of speech playback, system throughput, and overall user experience.

[0107] This application also provides a computer device that integrates any of the speech synthesis devices provided in this application. The computer device includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor from the steps of the speech synthesis method in any of the embodiments described above.

[0108] This application also provides a computer device that integrates any of the speech synthesis devices provided in this application. For example... Figure 5 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically: The computer device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will understand that... Figure 5 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 501 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 501.

[0109] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0110] The computer equipment also includes a power supply 503 that supplies power to the various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0111] The computer device may also include an input unit 504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0112] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application programs stored in the memory 502 to realize various functions, as follows: The text to be processed is segmented into sentences to obtain a first set of sentences; the cached audio set is traversed and each sentence in the first set is checked for a match in sequence; the sentences that match the cached audio set are returned as first audio data, and the sentences that do not match the cached audio set are grouped and cached to obtain the target concatenated text; the target concatenated text is processed into speech to obtain second audio data; the first audio data and the second audio data are played in sequence.

[0113] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0114] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the speech synthesis methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps: The text to be processed is segmented into sentences to obtain a first set of sentences; the cached audio set is traversed and each sentence in the first set is checked for a match in sequence; the sentences that match the cached audio set are returned as first audio data, and the sentences that do not match the cached audio set are grouped and cached to obtain the target concatenated text; the target concatenated text is processed into speech to obtain second audio data; the first audio data and the second audio data are played in sequence.

[0115] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0116] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0117] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0118] The above provides a detailed description of a speech synthesis method, apparatus, computer device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A speech synthesis method, characterized in that, include: The text to be processed is segmented into sentences to obtain the first set of sentences; Iterate through the cached audio set and perform a hit check on each statement in the first statement set in sequence; For statements that hit the cached audio set, return the first audio data; for statements that do not hit the cached audio set, group and cache them to obtain the target concatenated text. The target concatenated text is subjected to speech synthesis to obtain second audio data; Play the first audio data and the second audio data in sequence.

2. The method according to claim 1, characterized in that, Statements that do not match the cached audio set are grouped to obtain the target concatenated text, which includes: When the count of statements that do not match the cached audio set is less than or equal to a first threshold, a single statement is determined as the target concatenated text; When the count of statements that do not match the cached audio set is greater than the first threshold, the statements that do not match the cached audio set are cached based on the first preset sentence length to obtain the target concatenated text.

3. The method according to claim 2, characterized in that, The step of caching the statements that do not match the cached audio set based on a first preset sentence length to obtain the target concatenated text includes: The first preset sentence length is estimated online based on an exponentially weighted moving average algorithm; Based on the first preset sentence length, the sentences that do not match the cached audio set are cached to obtain the target concatenated text.

4. The method according to claim 3, characterized in that, The method further includes: Based on the first preset sentence length and target word count constraints, the target sentence length is dynamically calculated using a constraint optimization algorithm, and the target sentence length is applied to the next grouping cache.

5. The method according to any one of claims 1 to 4, characterized in that, After performing speech synthesis on the target concatenated text to obtain second audio data, the method further includes: The second audio data is segmented into multiple single-sentence audio segments; The multiple single-sentence audio clips are cached into the cached audio set.

6. The method according to claim 5, characterized in that, The step of segmenting the second audio data into multiple single-sentence audio segments includes: Obtain the start and end positions and / or global sentence number of each sentence in the target concatenated text; The second audio data is segmented into multiple single-sentence audio segments based on the start and end positions and / or the global sentence sequence number.

7. The method according to any one of claims 1 to 4 and 6, characterized in that, The method further includes: Retrieve frequently used statements; A speech synthesis request is initiated for the high-frequency statement to obtain third audio data; The third audio data is cached in the cached audio set.

8. A speech synthesis device, characterized in that, include: The processing module is used to segment the text to be processed into sentences to obtain the first set of sentences; The cached audio set is traversed, and each statement in the first statement set is checked for a match in sequence. For statements that match the cached audio set, the first audio data is returned. For statements that do not match the cached audio set, they are grouped and cached to obtain the target concatenated text. The target concatenated text is subjected to speech synthesis to obtain second audio data; The playback module is used to play the first audio data and the second audio data in sequence.

9. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It contains a computer program that is loaded by a processor to perform the steps of the method according to any one of claims 1 to 7.

Images