Tts speech synthesis technology audio online-offline switching method and electronic device

Abstract

The application discloses a TTS speech synthesis technology audio online-offline switching method, an electronic device and a storage medium. The method comprises the following steps: determining whether an online TTS audio segment with successful integrity detection has been cached; if not, acquiring and playing an offline TTS audio segment, and downloading a preset threshold number of online TTS audio segments adjacent to the played offline TTS audio segment from a server; when the offline TTS audio segment is played to a preset position, performing integrity detection on the downloaded online TTS audio segment, if the integrity detection fails, acquiring and playing an offline TTS audio segment of the next segment, repeatedly performing the step according to the played offline TTS audio segment until the integrity detection is successful, and switching to play the downloaded online TTS audio segment; if yes, cyclically caching a plurality of online TTS audio segments according to a first caching rule, and playing the plurality of online TTS audio segments in sequence until the cached plurality of online TTS audio segments are played, and switching to play an offline TTS audio segment.

Description

Technical Field

[0001] This invention relates to the field of e-book technology, specifically to a method for switching audio between online and offline modes using TTS (Text-to-Speech) technology, an electronic device, and a storage medium. Background Technology

[0002] The combination of TTS (Text-to-Speech) and e-books provides users with a convenient audiobook experience, allowing them to listen and read in any situation. TTS audio includes online and offline TTS audio. Online TTS audio uses online speech synthesis engines, offering high-quality and natural-sounding audio. Offline TTS audio uses the client's local speech synthesis engine, resulting in generally lower quality and less natural-sounding audio compared to online TTS. Under good network conditions, online TTS audio provides a better listening experience. When network conditions are poor, users can switch to offline TTS audio to continue playback.

[0003] Current technology handles the switching between online and offline TTS audio rather abruptly. For example, if online TTS audio is being used and the network signal is poor, it will switch to offline TTS audio. However, network jitter and weak network conditions often cause the online-offline switching to be too frequent, resulting in stuttering and poor switching quality, which seriously affects the user experience. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a method, electronic device and storage medium for online and offline audio switching in TTS speech synthesis technology to overcome the above problems or at least partially solve the above problems.

[0005] According to one aspect of the present invention, a method for switching audio between online and offline modes in TTS speech synthesis technology is provided, comprising:

[0006] Determine if the online TTS audio segment that successfully passed the integrity check has been cached;

[0007] If not, acquire and play the offline TTS audio segment, and download the number of adjacent online TTS audio segments following the offline TTS audio segment from the server, which is a preset threshold number; when the offline TTS audio segment plays to a preset position, perform an integrity check on the downloaded online TTS audio segment; if the integrity check fails, acquire and play the next offline TTS audio segment, and repeat this step according to the played offline TTS audio segment until the integrity check is successful, then switch to playing the downloaded online TTS audio segment;

[0008] If so, cache multiple online TTS audio segments in a loop according to the first caching rule, and play multiple online TTS audio segments in sequence until the multiple cached online TTS audio segments have finished playing, then switch to play offline TTS audio segments.

[0009] According to another aspect of the present invention, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0010] The memory is used to store at least one executable instruction that causes the processor to perform the following operations:

[0011] Determine if the online TTS audio segment that successfully passed the integrity check has been cached;

[0012] If not, acquire and play the offline TTS audio segment, and download the number of adjacent online TTS audio segments following the offline TTS audio segment from the server, which is a preset threshold number; when the offline TTS audio segment plays to a preset position, perform an integrity check on the downloaded online TTS audio segment; if the integrity check fails, acquire and play the next offline TTS audio segment, and repeat this step according to the played offline TTS audio segment until the integrity check is successful, then switch to playing the downloaded online TTS audio segment;

[0013] If so, cache multiple online TTS audio segments in a loop according to the first caching rule, and play multiple online TTS audio segments in sequence until the multiple cached online TTS audio segments have finished playing, then switch to play offline TTS audio segments.

[0014] According to another aspect of the present invention, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, the executable instruction causing a processor to perform the operation corresponding to the TTS speech synthesis technology audio online / offline switching method described above.

[0015] According to the TTS speech synthesis technology audio online / offline switching method, electronic device, and storage medium provided by this invention, if an online TTS audio segment with successful integrity detection is not cached, an offline TTS audio segment is played. While offline, a preset threshold number of adjacent online TTS audio segments following the offline TTS audio segment are downloaded from the server. When the offline TTS audio segment reaches a preset position, an integrity detection is performed on the downloaded online TTS audio segment. If the integrity detection is successful, it indicates a good network connection quality, and online TTS audio can be switched to play. If the integrity detection fails, it indicates a poor network connection quality. In this case, switching to online will result in frequent switching due to network instability. Therefore, offline TTS audio segments continue to be played until the integrity detection is successful before switching, which can avoid network jitter and frequent switching in weak network conditions. If an online TTS audio segment with successful integrity detection is cached, multiple online TTS audio segments can be cached cyclically according to a first caching rule, and multiple online TTS audio segments can be played sequentially. The cached multiple online TTS audio segments can resolve network jitter for a period of time.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0018] Figure 1 A flowchart illustrating an online / offline audio switching method for TTS speech synthesis technology according to an embodiment of the present invention is shown.

[0019] Figure 2 A flowchart illustrating an online / offline audio switching method for TTS speech synthesis technology according to another embodiment of the present invention is shown;

[0020] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0021] Example 1

[0022] Figure 1A flowchart illustrating an online / offline audio switching method for TTS speech synthesis technology according to an embodiment of the present invention is shown, as follows: Figure 1 As shown, the method includes the following steps:

[0023] Step S101: Determine whether the online TTS audio segment that has successfully passed the integrity check has been cached.

[0024] When a user initiates the audiobook listening function using TTS audio, it can determine whether an online TTS audio segment with successfully cached integrity has been cached, in order to determine which TTS audio to use. Specifically, if it is determined that no online TTS audio segment has been cached, or that the cached online TTS audio segment is incomplete or cannot be played normally, causing its integrity detection to fail, the network connection quality is poor, and offline TTS audio is used, proceeding to step S102. If it is determined that an online TTS audio segment with successfully cached integrity has been cached, it indicates that the network connection quality is good, and a higher quality online TTS audio can be provided to the user, so online TTS audio can be used, proceeding to step S105.

[0025] Existing technologies lack standardized methods for detecting network connection quality. While network connection quality can be measured through metrics such as network speed, most speed measurements are single-point measurements, not continuous measurements, and they waste significant resources. Considering these issues, this embodiment uses the determination of whether an online TTS audio segment has successfully passed integrity checks to measure network connection quality. On one hand, online TTS audio segments are small, making the determination process simple, easy to operate, and resource-efficient. On the other hand, successfully passing integrity checks on online TTS audio segments indicates a fast download speed and good network connection quality, thus allowing the user to play the online TTS audio segment.

[0026] Step S102: Obtain and play offline TTS audio segments, and download a preset threshold number of online TTS audio segments adjacent to the offline TTS audio segments played from the server; when the offline TTS audio segments play to a preset position, perform an integrity check on the downloaded online TTS audio segments to determine whether the integrity check of the downloaded online TTS audio segments is successful.

[0027] To facilitate timely switching between online and offline TTS audio, this embodiment segments both online and offline TTS audio, playing each segment individually. When an online TTS audio segment fails the integrity check (indicating poor network connection quality), an offline TTS audio segment is synthesized locally on the client using an offline TTS engine for playback. Considering the challenges of weak network conditions, where network connection quality fluctuates, switching between channels during periods of good connection can lead to frequent switching and poor playback quality. Therefore, this embodiment employs a method where offline TTS audio segments are played while simultaneously downloading a preset threshold number of adjacent online TTS audio segments from the server. If the current network is intermittent and weak, the download speed will be slow due to these intermittent connections, potentially preventing the complete download of online TTS audio segments while playing offline ones. Conversely, if the network connection quality significantly improves, the download speed of online TTS audio segments will increase dramatically, exceeding the speed of offline playback. This ensures that the online TTS audio segments are downloaded completely when the offline ones are finished. The completeness of the downloaded online TTS audio segments allows for confirmation of improved network connection quality, preventing frequent switching based solely on temporary network connectivity during weak network conditions. Here, when downloading from the server, a preset threshold number of online TTS audio segments can be downloaded adjacent to the currently playing offline TTS audio segment. This preset threshold can be one or more, depending on the implementation. For example, if the online TTS audio segments are small, downloading only one online TTS audio segment with successful integrity checks may not be sufficient to determine if the download speed exceeds the playback speed or if the network connection quality has significantly improved. In such cases, the preset threshold can be set to multiple segments. When the offline TTS audio segment reaches a preset position, if all downloaded online TTS audio segments have successfully passed integrity checks, it indicates a significant improvement in network connection quality, allowing switching to play the online TTS audio segment and avoiding frequent switching between online and offline playback. The preset threshold number is set based on factors such as the playback speed of the offline TTS audio segments and the size of the online TTS audio segments; no specific limit is imposed here.

[0028] Specifically, when the offline TTS audio segment reaches a preset position, an integrity check is performed on the downloaded online TTS audio segment. The integrity check includes verifying whether the online TTS audio segment is complete and playable. If the integrity check is successful, it indicates improved network connection quality, and step S104 is executed. After playing the current offline TTS audio segment, playback can switch to the downloaded online TTS audio segment, and subsequent playback of online TTS audio segments can continue. If the integrity check fails, it indicates poor network connection quality, and step S103 is executed. The preset position can be set to a point before the end of the segment, one minute before the end, or even when playing to the middle, etc. The specific setting can be determined based on the implementation, such as the download speed of the online TTS audio segment, the playback speed of the offline TTS audio segment, and the size of the TTS audio segment; no limitations are imposed here.

[0029] Step S103: Obtain and play the offline TTS audio segment for the next segment, and repeat step S102 based on the played offline TTS audio segment.

[0030] If the integrity check of the downloaded online TTS audio segment fails, it indicates poor network connection quality. The system then continues to acquire and play the next offline TTS audio segment. Based on the currently playing offline TTS audio segment, step S102 is repeated: the offline TTS audio segment continues playing, and the system continues to download a preset threshold number of adjacent online TTS audio segments following the currently playing offline TTS audio segment. Integrity checks are performed on the downloaded online TTS audio segments to determine whether to switch to online TTS audio segment playback. If the integrity check fails, the offline TTS audio segment continues playing, and step S102 is repeated until the integrity check succeeds, indicating good network connection quality. After playing the current offline TTS audio segment, the system switches to playing the downloaded online TTS audio segment, completing the offline-to-online playback transition.

[0031] Step S104: Switch to playing the downloaded online TTS audio clip.

[0032] If the integrity check is successful, meaning the network connection quality is good, after playing the current offline TTS audio segment, switch to playing the downloaded online TTS audio segment, completing the offline to online playback transition.

[0033] Step S105: According to the first caching rule, multiple online TTS audio segments are cached in a loop, and multiple online TTS audio segments are played in sequence until the multiple cached online TTS audio segments have finished playing. Then, the offline TTS audio segments are switched to be played.

[0034] If the currently cached online TTS audio segment has successfully passed the integrity check, indicating good network connection quality, then a high-quality, natural-sounding online TTS audio segment can be provided to the user. To avoid network jitter, this embodiment, under good network connection conditions, cyclically caches multiple online TTS audio segments according to the first caching rule. This allows for sequential playback of cached online TTS audio segments even when network connection quality is poor, reducing the impact of network jitter. For example, cyclically caching multiple online TTS audio segments means playing one online TTS audio segment and then downloading and caching a new one. When a user enters a location with poor network connection quality, such as an elevator or subway entrance, the time spent in such locations is generally short, such as 3-5 minutes. In this case, sequentially playing multiple cached online TTS audio segments can support the user as they leave the location with poor network connection quality, avoiding switching between online and offline modes and providing the user with a high-quality, natural-sounding online TTS audio segment.

[0035] If a user is in an area with poor network connectivity for an extended period, multiple online TTS audio clips can be played sequentially until all cached online TTS audio clips have finished playing. Afterward, the user can switch to offline TTS audio clips, i.e., switch from online to offline playback. After playing offline TTS audio clips, the user can switch between online and offline playback according to the description in step S102.

[0036] According to the TTS speech synthesis technology audio online / offline switching method provided by this invention, if an online TTS audio segment with successful integrity detection is not cached, an offline TTS audio segment is played. While offline, a preset threshold number of adjacent online TTS audio segments following the offline TTS audio segment are downloaded from the server. When the offline TTS audio segment reaches a preset position, an integrity detection is performed on the downloaded online TTS audio segment. If the integrity detection is successful, it indicates a good network connection quality, and online TTS audio can be switched to play. If the integrity detection fails, it indicates a poor network connection quality. In this case, switching online will result in frequent switching due to network instability. Therefore, offline TTS audio segments continue to be played until the integrity detection is successful before switching, which avoids network jitter and frequent switching in weak network conditions. If an online TTS audio segment with successful integrity detection is cached, multiple online TTS audio segments can be cached cyclically according to a first caching rule, and multiple online TTS audio segments can be played sequentially. The cached multiple online TTS audio segments can resolve network jitter for a period of time.

[0037] Example 2

[0038] Figure 2 A flowchart illustrating a method for switching between online and offline audio using TTS speech synthesis technology according to another embodiment of the present invention is shown, as follows: Figure 2 As shown, the method includes the following steps:

[0039] Step S201: Determine whether the online TTS audio segment that has successfully passed the integrity check has been cached.

[0040] This step can be referred to in the description of step S101, and will not be repeated here.

[0041] When it is determined that the online TTS audio segment whose cache integrity has been successfully checked is not cached, step S202 is executed, that is, the offline state is adopted and the offline TTS audio segment is played; when it is determined that the online TTS audio segment whose cache integrity has been successfully checked is checked, step S206 is executed, that is, the online state is adopted so as to play the online TTS audio segment.

[0042] Step S202: Obtain and play offline TTS audio segments, and download a preset threshold number of online TTS audio segments adjacent to the offline TTS audio segments played from the server.

[0043] In this process, online and offline TTS audio segments can be processed using different threads. For example, the first thread can acquire and play offline TTS audio segments, while the second thread can download and play online TTS audio segments from the server, which are adjacent to a preset threshold number of segments. The two threads do not interfere with each other.

[0044] For acquiring and playing offline TTS audio segments, an audio file can be divided into multiple segments according to preset segmentation rules. An audio file can be divided into segments based on a preset size, such as 1MB of audio divided into 6 segments, each approximately 200KB. Alternatively, an audio file can be divided into chapters, with each chapter divided into 6 segments. These preset segmentation rules apply to both online and offline TTS audio segments, facilitating seamless switching between the two. For offline TTS audio segments, an offline TTS engine can be used to synthesize offline TTS audio segments, which can then be played.

[0045] Step S203: When the offline TTS audio segment plays to a preset position, perform an integrity check on the downloaded online TTS audio segment to determine whether the integrity check of the downloaded online TTS audio segment is successful.

[0046] When the offline TTS audio segment plays to a preset position, such as a position before the end of the audio segment, an integrity check is performed on the online audio segment downloaded from the server to determine whether the integrity check of the downloaded online TTS audio segment is successful. That is, to determine whether the downloaded online TTS audio segment is a complete audio segment and whether it can be played normally. If it is, the integrity check is considered successful and step S205 is executed; otherwise, the integrity check is considered to have failed and step S204 is executed.

[0047] Step S204: Obtain and play the offline TTS audio segment of the next segment, and download the number of adjacent online TTS audio segments after the played offline TTS audio segment, which is a preset threshold number. Repeat the integrity check of the downloaded online TTS audio segments.

[0048] If the integrity check of the downloaded online TTS audio segment fails, it indicates that the network connection quality is poor and cannot maintain an online state to play the online TTS audio segment. Therefore, it continues to maintain an offline state and obtains and plays the next offline TTS audio segment.

[0049] Based on the offline TTS audio segment being played, the server continues to download a preset threshold number of online TTS audio segments following the offline TTS audio segment. Step S203 is repeated to perform integrity checks on the downloaded online TTS audio segments until the integrity check of the downloaded online TTS audio segments is successful, indicating that the network connection quality has been greatly improved. At this point, the offline and online modes can be switched, and the downloaded online TTS audio segments can be switched to play.

[0050] Step S205: Backtrack and download the online TTS audio segments corresponding to the unplayed audio content of the currently playing offline TTS audio segment from the server until the backtracked online TTS audio segments contain the unplayed audio content of the currently playing offline TTS audio segment, then switch to playing the backtracked online TTS audio segments.

[0051] When switching from offline to online, you can wait for the currently playing offline TTS audio segment to finish playing before switching to play the downloaded online TTS audio segment, thus connecting it with the offline TTS audio segment.

[0052] To provide users with a better audiobook experience, when the network connection quality is good, after downloading online TTS audio segments and successfully performing integrity checks, the system can continue to download the corresponding online TTS audio segments from the server. For example, it can download from the end of the currently playing offline TTS audio segment, starting from the end of the currently playing offline TTS audio segment, and then download the online TTS audio segments corresponding to the unplayed audio content of the currently playing offline TTS audio segment. When the downloaded online TTS audio segments can connect with the content played in the currently playing offline TTS audio segment, that is, when the downloaded online TTS audio segments contain the unplayed audio content of the currently playing offline TTS audio segment, for example, if the current offline TTS audio segment has a total duration of 2 minutes, 1 minute has been played, and the downloaded online TTS audio segments contain the unplayed 1 minute of audio content, the playback can be switched directly, switching the currently playing offline TTS audio segment to the downloaded online TTS audio segment, completing the offline to online switch.

[0053] Step S206: Determine the caching rules for online TTS audio segments based on different scenarios.

[0054] When the online TTS audio segment has been successfully cached (indicating good connection quality), it can be played. Before playback, the caching rules for the online TTS audio segment can be determined based on different scenarios, allowing multiple online TTS audio segments to be cached on the client side. This facilitates continued playback using cached online TTS audio segments in situations such as network jitter or weak network conditions.

[0055] Caching rules include first caching rules, second caching rules, etc. First caching rules include ring caching rules, which contain a preset number of online TTS audio segments, such as 5-6 online TTS audio segments. Second caching rules include chapter-based caching rules, such as caching multiple chapters, such as 20 chapters of online audio segments. These cached online TTS audio segments can be played, providing users with a good audiobook experience. Different caching rules correspond to different sizes of cached online TTS audio segments, determined specifically according to the scenario. This embodiment mainly describes the first caching rule, addressing issues such as network jitter and weak network conditions.

[0056] Step S207: According to the first caching rule, multiple online TTS audio segments are cached in a loop, and multiple online TTS audio segments are played in sequence until the multiple cached online TTS audio segments are played. Then, the offline TTS audio segments are switched to be played.

[0057] The online TTS audio is segmented according to preset segmentation rules, which are the same as those for offline TTS audio segments, dividing a single TTS audio segment into multiple segments. The online TTS audio segments are obtained by segmenting preset unit audio data using the online TTS engine, and these multiple online TTS audio segments are then cached.

[0058] The first caching rule includes a ring caching rule. The ring caching contains a preset number of online TTS audio segments, such as caching 6 online TTS audio segments. After playing one online TTS audio segment, the next online TTS audio segment can be played in sequence. The new online TTS audio segments are cached in a loop according to the ring caching rule. For example, caching 1, 2, 3, 4, 5, and 6 online TTS audio segments at a time. After playing 1 online TTS audio segment, when playing 2 online TTS audio segment, 7 online TTS audio segments are cached in a loop, keeping 6 online TTS audio segments in the cache. This can support the playback of online TTS audio segments for a period of time. Even if the network connection quality is poor during this period, it will not affect the playback of online TTS audio segments, and can avoid switching between online and offline.

[0059] When multiple online TTS audio segments cached in the cache have finished playing, and there are no more online TTS audio segments cached in the cache (i.e., the network connection quality is poor and will not improve), and it is impossible to download more cached online TTS audio segments, then you can switch to playing offline TTS audio segments, completing the online to offline switch. After switching to playing offline TTS audio segments, you can switch back to online according to steps S202-S205, which will not be elaborated here.

[0060] According to the TTS speech synthesis technology audio online / offline switching method provided by this invention, when an online TTS audio segment whose integrity detection has not been successfully cached is played offline, an offline playback mode is adopted. While playing the offline TTS audio segment, a preset threshold number of adjacent online TTS audio segments following the offline TTS audio segment are downloaded from the server. This allows for assessment of network connection quality based on the download progress. If the integrity detection of the downloaded online TTS audio segment is successful, it indicates a good network connection quality, and the system switches to online playback. Furthermore, it can continue to download unplayed audio content of the currently playing offline TTS audio segment to provide the user with a better online TTS audio segment experience. If the integrity detection of the online TTS audio segment fails, it indicates that although the online TTS audio segment can be downloaded, the network connection quality is poor. Switching to online playback will still result in frequent switching due to network instability. By continuing to play the offline TTS audio segment, problems such as network jitter and frequent switching in weak network conditions can be avoided. If an online TTS audio segment has been successfully cached for integrity, it can be played online. Multiple online TTS audio segments can be cached in a loop according to the first caching rule. The cached online TTS audio channels can support the continued playback of online TTS audio segments even when the network connection quality is poor. This avoids the need to switch to offline playback when the network connection quality is poor for a short period of time.

[0061] Example 3

[0062] Embodiment 3 of this application provides a non-volatile computer storage medium storing at least one executable instruction that can execute the TTS speech synthesis technology audio online / offline switching method in any of the above method embodiments.

[0063] Executable instructions can specifically be used to cause the processor to perform the following operations:

[0064] Determine if an online TTS audio segment with successful integrity check has been cached. If not, acquire and play an offline TTS audio segment, and download a preset threshold number of adjacent online TTS audio segments following the offline TTS audio segment from the server. When the offline TTS audio segment reaches a preset position, perform an integrity check on the downloaded online TTS audio segment. If the integrity check fails, acquire and play the next offline TTS audio segment. Repeat this step based on the played offline TTS audio segments until the integrity check is successful, then switch to playing the downloaded online TTS audio segment. If yes, cache multiple online TTS audio segments in a loop according to the first caching rule, and play multiple online TTS audio segments sequentially until the cached multiple online TTS audio segments have finished playing, then switch to playing the offline TTS audio segment.

[0065] In one optional implementation, the executable instructions further cause the processor to perform the following operations: when an offline TTS audio segment is played to a preset position, perform an integrity check on the downloaded online TTS audio segment to determine whether the integrity check of the downloaded online TTS audio segment is successful; if not, acquire and play the next offline TTS audio segment, and download a preset threshold number of adjacent online TTS audio segments after the played offline TTS audio segment, repeat the integrity check on the downloaded online TTS audio segments until it is determined that the integrity check of the downloaded online TTS audio segments is successful, and switch to play the downloaded online TTS audio segments.

[0066] In one alternative implementation, the executable instructions further cause the processor to perform the following operations: perform an integrity check on the downloaded online TTS audio segment, and determine whether the integrity check of the downloaded online TTS audio segment is successful; if so, after playing the offline TTS audio segment, switch to playing the downloaded online TTS audio segment.

[0067] In one optional implementation, the executable instructions further cause the processor to perform the following operations: perform an integrity check on the downloaded online TTS audio segment, and determine whether the integrity check of the downloaded online TTS audio segment is successful; if so, back-download the online TTS audio segment corresponding to the unplayed audio content of the currently playing offline TTS audio segment from the server, until the back-downloaded online TTS audio segment contains the unplayed audio content of the currently playing offline TTS audio segment, and switch to play the back-downloaded online TTS audio segment.

[0068] In one alternative implementation, the executable instructions further cause the processor to perform the following operations: a first thread acquires and plays an offline TTS audio segment, and a second thread downloads from the server a preset threshold number of adjacent online TTS audio segments following the offline TTS audio segment being played.

[0069] In one alternative implementation, the executable instructions further cause the processor to perform the following operations: synthesize an offline TTS audio segment based on an offline TTS engine according to a preset segmentation rule, and play the synthesized offline TTS audio segment; and divide a preset unit audio segment into multiple online TTS audio segments based on an online TTS engine according to a preset segmentation rule, and cache the multiple online TTS audio segments.

[0070] In one optional implementation, the first caching rule includes a ring caching rule; the ring cache contains a preset number of online TTS audio segments; the executable instructions further cause the processor to perform the following operations: after playing an online TTS audio segment, play the next online TTS audio segment, and cyclically cache new online TTS audio segments according to the first caching rule.

[0071] In one alternative implementation, the executable instructions further cause the processor to perform the following operations: determine caching rules for online TTS audio segments based on different scenarios; the caching rules include a first caching rule and / or a second caching rule; the second caching rule includes caching rules based on chapters.

[0072] Example 4

[0073] Figure 3 The diagram shows a structural schematic of an electronic device according to Embodiment 4 of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the electronic device.

[0074] like Figure 3 As shown, the electronic device may include: a processor 302, a communications interface 304, a memory 306, and a communications bus 308.

[0075] in:

[0076] The processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308.

[0077] Communication interface 304 is used to communicate with other network elements such as clients or other servers.

[0078] The processor 302 is used to execute program 310, specifically to execute the relevant steps in the above-described embodiment of the online / offline audio switching method for TTS speech synthesis technology.

[0079] Specifically, program 310 may include program code that includes computer operation instructions.

[0080] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The server may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0081] Memory 306 is used to store program 310. Memory 306 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0082] Specifically, program 310 can be used to cause processor 302 to perform the following operations:

[0083] Determine if an online TTS audio segment with successful integrity check has been cached. If not, acquire and play an offline TTS audio segment, and download a preset threshold number of adjacent online TTS audio segments following the offline TTS audio segment from the server. When the offline TTS audio segment reaches a preset position, perform an integrity check on the downloaded online TTS audio segment. If the integrity check fails, acquire and play the next offline TTS audio segment. Repeat this step based on the played offline TTS audio segments until the integrity check is successful, then switch to playing the downloaded online TTS audio segment. If yes, cache multiple online TTS audio segments in a loop according to the first caching rule, and play multiple online TTS audio segments sequentially until the cached multiple online TTS audio segments have finished playing, then switch to playing the offline TTS audio segment.

[0084] In one optional implementation, program 310 is configured to cause processor 302 to perform an integrity check on the downloaded online TTS audio segment when the offline TTS audio segment is played to a preset position, and determine whether the integrity check of the downloaded online TTS audio segment is successful; if not, it acquires and plays the next offline TTS audio segment, and downloads the adjacent preset threshold number of online TTS audio segments after the played offline TTS audio segment, and repeats the integrity check on the downloaded online TTS audio segments until it is determined that the integrity check of the downloaded online TTS audio segments is successful, and then switches to play the downloaded online TTS audio segments.

[0085] In one optional implementation, program 310 is used to enable processor 302 to perform an integrity check on the downloaded online TTS audio segment, and determine whether the integrity check of the downloaded online TTS audio segment is successful; if so, after playing the offline TTS audio segment, switch to playing the downloaded online TTS audio segment.

[0086] In one optional implementation, program 310 is used to enable processor 302 to perform an integrity check on the downloaded online TTS audio segment, and determine whether the integrity check of the downloaded online TTS audio segment is successful; if so, back-download the online TTS audio segment corresponding to the unplayed audio content of the currently playing offline TTS audio segment from the server, until the back-downloaded online TTS audio segment contains the unplayed audio content of the currently playing offline TTS audio segment, and switch to play the back-downloaded online TTS audio segment.

[0087] In one alternative implementation, program 310 is configured to cause processor 302 to first thread acquire and play offline TTS audio segments, and second thread to download a preset threshold number of online TTS audio segments adjacent to the offline TTS audio segments played from the server.

[0088] In one optional implementation, program 310 is configured to cause processor 302 to synthesize offline TTS audio segments based on an offline TTS engine according to preset segmentation rules, and play the synthesized offline TTS audio segments; and to divide preset unit audio into multiple online TTS audio segments based on an online TTS engine according to preset segmentation rules, and cache multiple online TTS audio segments.

[0089] In one optional implementation, the first caching rule includes a ring caching rule; the ring cache contains a preset number of online TTS audio segments; the program 310 is used to cause the processor 302 to play the next online TTS audio segment after playing an online TTS audio segment, and to cyclically cache new online TTS audio segments according to the first caching rule.

[0090] In one alternative implementation, program 310 is used to enable processor 302 to determine caching rules for online TTS audio segments based on different scenarios; the caching rules include a first caching rule and / or a second caching rule; the second caching rule includes caching rules based on chapters.

[0091] The specific implementation of each step in program 310 can be found in the corresponding descriptions of the steps in the above-mentioned TTS speech synthesis technology audio online / offline switching embodiment, and will not be repeated here. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0092] The solution provided in this embodiment addresses the issue of online TTS audio segments failing integrity checks if they are cached. Instead, offline TTS audio segments are played. While offline, a preset threshold number of adjacent online TTS audio segments are downloaded from the server. When the offline TTS audio segment reaches a preset position, an integrity check is performed on the downloaded online TTS audio segments. If the integrity check is successful, it indicates a good network connection, allowing switching to online TTS audio segments. If the integrity check fails, it indicates a poor network connection. Switching to online TTS audio segments in this case would result in frequent switching due to network instability. Therefore, offline TTS audio segments continue playing until the integrity check is successful before switching, avoiding network jitter and frequent switching in weak network conditions. If online TTS audio segments with successful integrity checks are cached, multiple online TTS audio segments can be cached cyclically according to a first caching rule, and these segments can be played sequentially. The cached online TTS audio segments can mitigate network jitter over a period of time.

[0093] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0094] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0095] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0096] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0097] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for switching audio between online and offline modes using TTS speech synthesis technology, comprising: Determine if the online TTS audio segment that successfully passed the integrity check has been cached; If not, acquire and play the offline TTS audio segment, and download the number of adjacent online TTS audio segments following the offline TTS audio segment from the server, which is a preset threshold number; when the offline TTS audio segment plays to a preset position, perform an integrity check on the downloaded online TTS audio segment; if the integrity check fails, acquire and play the next offline TTS audio segment, and repeat this step according to the played offline TTS audio segment until the integrity check is successful, then switch to playing the downloaded online TTS audio segment; If so, cache multiple online TTS audio segments in a loop according to the first caching rule, and play the multiple online TTS audio segments in sequence until the multiple cached online TTS audio segments have finished playing, then switch to playing offline TTS audio segments.

2. The method according to claim 1, wherein, When the offline TTS audio segment reaches a preset position, an integrity check is performed on the downloaded online TTS audio segment. If the integrity check fails, the next offline TTS audio segment is acquired and played. This step is repeated based on the played offline TTS audio segment until the integrity check succeeds. Switching to play the downloaded online TTS audio segment further includes: When the offline TTS audio segment is played to a preset position, the integrity of the downloaded online TTS audio segment is checked to determine whether the integrity of the downloaded online TTS audio segment has been successfully checked. If not, then acquire and play the next offline TTS audio segment, and download the number of adjacent online TTS audio segments following the offline TTS audio segment that was played, which is a preset threshold number. Repeat the integrity check on the downloaded online TTS audio segments until the integrity check of the downloaded online TTS audio segments is successful, and then switch to play the downloaded online TTS audio segments.

3. The method according to claim 1 or 2, wherein, When the offline TTS audio segment reaches a preset position, an integrity check is performed on the downloaded online TTS audio segment. If the integrity check fails, the next offline TTS audio segment is acquired and played. This step is repeated based on the played offline TTS audio segment until the integrity check succeeds. Switching to play the downloaded online TTS audio segment further includes: Perform an integrity check on the downloaded online TTS audio clip to determine whether the integrity check was successful. If so, after playing the offline TTS audio segment, switch to playing the downloaded online TTS audio segment.

4. The method according to claim 1 or 2, wherein, When the offline TTS audio segment reaches a preset position, an integrity check is performed on the downloaded online TTS audio segment. If the integrity check fails, the next offline TTS audio segment is acquired and played. This step is repeated based on the played offline TTS audio segment until the integrity check succeeds. Switching to play the downloaded online TTS audio segment further includes: Perform an integrity check on the downloaded online TTS audio clip to determine whether the integrity check was successful. If so, download the online TTS audio segment corresponding to the unplayed audio content of the currently playing offline TTS audio segment from the server, until the downloaded online TTS audio segment contains the unplayed audio content of the currently playing offline TTS audio segment, then switch to playing the downloaded online TTS audio segment.

5. The method according to claim 1 or 2, wherein, The process of acquiring and playing offline TTS audio segments, and downloading and playing online TTS audio segments adjacent to a preset threshold number of segments from the server, specifically involves: a first thread acquiring and playing offline TTS audio segments, and a second thread downloading and playing online TTS audio segments adjacent to a preset threshold number of segments from the server.

6. The method according to claim 1 or 2, wherein, The specific steps of acquiring and playing offline TTS audio segments are as follows: according to preset segmentation rules, offline TTS audio segments are synthesized based on an offline TTS engine, and the synthesized offline TTS audio segments are played. The step of cyclically caching multiple online TTS audio segments according to the first caching rule specifically involves: according to a preset segmentation rule, dividing a preset unit audio segment based on the online TTS engine to obtain multiple online TTS audio segments, and caching multiple online TTS audio segments.

7. The method according to claim 1 or 2, wherein, The first caching rule includes a ring caching rule; the ring cache contains a preset number of online TTS audio segments; The step of cyclically caching multiple online TTS audio segments according to the first caching rule and sequentially playing the multiple online TTS audio segments further includes: After playing one online TTS audio segment, the next online TTS audio segment is played, and new online TTS audio segments are cached in a loop according to the first caching rule.

8. The method according to claim 1 or 2, wherein, After determining whether the online TTS audio segment whose integrity check was successful has been cached, the method further includes: Based on different scenarios, caching rules for online TTS audio segments are determined; the caching rules include a first caching rule and / or a second caching rule; the second caching rule includes caching rules based on chapters.

9. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the following operations: Determine if the online TTS audio segment that successfully passed the integrity check has been cached; If not, acquire and play the offline TTS audio segment, and download the number of adjacent online TTS audio segments following the offline TTS audio segment from the server, which is a preset threshold number; when the offline TTS audio segment plays to a preset position, perform an integrity check on the downloaded online TTS audio segment; if the integrity check fails, acquire and play the next offline TTS audio segment, and repeat this step according to the played offline TTS audio segment until the integrity check is successful, then switch to playing the downloaded online TTS audio segment; If so, cache multiple online TTS audio segments in a loop according to the first caching rule, and play the multiple online TTS audio segments in sequence until the multiple cached online TTS audio segments have finished playing, then switch to playing offline TTS audio segments.

10. The electronic device of claim 9, wherein the executable instructions further cause the processor to perform the following operations: When the offline TTS audio segment is played to a preset position, the integrity of the downloaded online TTS audio segment is checked to determine whether the integrity of the downloaded online TTS audio segment has been successfully checked. If not, then acquire and play the next offline TTS audio segment, and download the number of adjacent online TTS audio segments following the offline TTS audio segment that was played, which is a preset threshold number. Repeat the integrity check on the downloaded online TTS audio segments until the integrity check of the downloaded online TTS audio segments is successful, and then switch to play the downloaded online TTS audio segments.

11. The electronic device of claim 9 or 10, wherein the executable instructions further cause the processor to perform the following operations: Perform an integrity check on the downloaded online TTS audio clip to determine whether the integrity check was successful. If so, after playing the offline TTS audio segment, switch to playing the downloaded online TTS audio segment.

12. The electronic device according to claim 9 or 10, wherein the executable instructions further cause the processor to perform the following operations: Perform an integrity check on the downloaded online TTS audio clip to determine whether the integrity check was successful. If so, download the online TTS audio segment corresponding to the unplayed audio content of the currently playing offline TTS audio segment from the server, until the downloaded online TTS audio segment contains the unplayed audio content of the currently playing offline TTS audio segment, then switch to playing the downloaded online TTS audio segment.

13. The electronic device according to claim 9 or 10, wherein the executable instructions further cause the processor to perform the following operations: The first thread acquires and plays offline TTS audio segments, while the second thread downloads a preset threshold number of online TTS audio segments adjacent to the offline TTS audio segments played from the server.

14. The electronic device of claim 9 or 10, wherein the executable instructions further cause the processor to perform the following operations: According to the preset segmentation rules, offline TTS audio segments are synthesized based on the offline TTS engine, and the synthesized offline TTS audio segments are played. Based on preset segmentation rules, multiple online TTS audio segments are obtained by segmenting preset unit audio using an online TTS engine, and these segments are then cached.

15. The electronic device according to claim 9 or 10, wherein the first caching rule includes a ring caching rule; the ring cache contains a preset number of online TTS audio segments; the executable instructions further cause the processor to perform the following operations: After playing one online TTS audio segment, the next online TTS audio segment is played, and new online TTS audio segments are cached in a loop according to the first caching rule.

16. The electronic device of claim 9 or 10, wherein the executable instructions further cause the processor to perform the following operations: Based on different scenarios, caching rules for online TTS audio segments are determined; the caching rules include a first caching rule and / or a second caching rule; the second caching rule includes caching rules based on chapters.

17. A computer storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the online / offline audio switching method of TTS speech synthesis technology as described in any one of claims 1-8.

Images