A text-to-speech processing method, terminal and server

Abstract

A text-to-speech processing method, terminal and server are used to reduce the delay of terminal playing voice. The method and device of this invention can be applied to the fields of artificial intelligence, human-computer interaction, etc. Among them, the terminal detects an instruction for indicating text-to-speech conversion; sends the text to a server; downloads audio data converted from the text from the server; determines whether the first frame of audio data that can be played is downloaded within a preset time length; if the first frame of audio data that can be played is downloaded within the preset time length, continues to download the remaining audio data.

Description

Technical Field

[0001] The present application relates to the field of audio technology, and in particular to a text-to-speech processing method, terminal, and server. Background Art

[0002] At present, text to speech (TTS) technology has become an important technology for human-computer dialogue, achieving the goal of "making machines speak". TTS technology has a wide range of uses, including email reading, voice assistants, voice prompts, etc.

[0003] TTS technology mainly converts each text into a corresponding voice, and then splices (or synthesizes) the voice into an audio stream for playback. At present, speech synthesis mainly includes offline synthesis and online synthesis. Offline synthesis can be implemented by the terminal itself, but it requires the terminal to store the voice corresponding to a large amount of text. Online synthesis can be server synthesis. Specifically, the terminal connects to the server, sends the text to the server, and the server synthesizes the audio corresponding to the text into an audio stream, and then sends the audio stream to the terminal for playback.

[0004] However, during the online synthesis process, the time it takes for the server to return the audio stream cannot be controlled. For example, after the terminal connects to the server, the server synthesizes the audio stream slowly and takes a long time to return the audio stream, resulting in a long delay in the terminal playing the audio, affecting the user experience. Summary of the invention

[0005] The purpose of the present application is to provide a text-to-speech processing method, a terminal and a server, so as to reduce the delay of the terminal in playing the voice.

[0006] In a first aspect, a text-to-speech processing method is provided, which can be executed by a terminal or by a chip in the terminal. The method includes: detecting an instruction for instructing text-to-speech conversion; sending the text to a server; downloading audio data converted from the text from the server; determining whether the first frame of audio data that can be played is downloaded within a preset time; if the first frame of audio data that can be played is downloaded within the preset time, continuing to download the remaining audio data.

[0007] It should be understood that the network quality of the terminal or server changes dynamically. When the network quality is good, the terminal downloads faster, and when the network quality is poor, the download speed may be slower. In an embodiment of the present application, the terminal sends the text to the server to request the server to implement text-to-speech, and then the terminal downloads the voice data from the server. If the first frame of playable audio data is downloaded within a preset time, the remaining audio data continues to be downloaded. Otherwise, the audio signal corresponding to the text is determined in the offline database, and the audio signal is synthesized into audio data. Therefore, the delay of the terminal voice playback is shortened as much as possible to avoid the situation where the voice cannot be played after the user inputs the text-to-speech instruction.

[0008] The audio data that can be played in the first frame may be audio data with a preset length downloaded for the first time, or the audio data that can be played in the first frame may be data with at least one complete audio frame downloaded for the first time.

[0009] In an embodiment of the present application, the terminal and the server agree on a compression strategy, for example, it is predetermined that the data length of the audio data compressed by the server each time is the same and is a preset length; it can also be agreed that the audio data compressed by the server each time includes at least one complete frame of audio. In this way, the terminal downloads audio data of a preset length each time, and each downloaded audio data includes at least one complete frame of audio. In other words, when the terminal downloads audio data of a preset length for the first time within a preset time, or when the terminal downloads data including at least one complete frame of audio for the first time, it can continue to download the remaining audio data, otherwise an offline synthesis method is used. Therefore, the delay of the terminal voice playback is shortened as much as possible to avoid the situation where the voice cannot be played after the user inputs the text-to-speech instruction.

[0010] In a possible design, the terminal may also: store the downloaded audio data in a cache area; and when it is determined that the memory usage of the audio data in the cache area reaches a threshold, play the audio data that can be played in the first frame.

[0011] It should be understood that a certain amount of audio data stored in the cache area can ensure that after playing the audio data that can be played in the first frame, the next frame of audio data continues to be played to ensure that the first frame is played to completion without lag.

[0012] In some embodiments, the terminal plays the audio data that can be played in the first frame, including: passing the audio data that can be played in the first frame to a parser for parsing to obtain decompressed audio data; passing the audio data parsed by the parser to a decoder for decoding to obtain audio data in a preset audio format; passing the audio data decoded by the decoder to a player for playback.

[0013] It should be noted that the above is a process for the terminal to successfully play audio data after downloading the first frame of playable audio data from the server. In actual applications, it may include more or fewer of the above processes, and the embodiments of the present application are not limited thereto.

[0014] In some embodiments, the audio data downloaded from the server may be in, but is not limited to, Opus format, which is a low-latency, high-fidelity voice coding format with better fidelity than other coding formats.

[0015] In a second aspect, a text-to-speech processing method is also provided, which can be executed by a server or by a processing chip in the server. The method includes: receiving a text-to-speech request sent by a terminal, wherein the request carries text; converting the text into audio data; compressing the audio data, wherein the data length of the audio data obtained by each compression is the same; and sending the compressed voice data to the terminals respectively.

[0016] In an embodiment of the present application, the terminal and the server agree on a compression strategy, such as predetermining that the server uses a static compression mechanism, for example, the data length of the audio data compressed each time is the same. It can also be agreed that the data length of the audio data is a preset length; it can also be agreed that the audio data compressed by the server each time includes at least one complete frame of audio. In this way, the terminal downloads audio data of a preset length each time, and each downloaded audio data includes at least one complete frame of audio. In other words, when the terminal downloads audio data of a preset length for the first time within a preset time length, or when the terminal downloads data including at least one complete frame of audio for the first time, it can continue to download the remaining audio data, otherwise an offline synthesis method is used. Therefore, the delay of the terminal voice playback is shortened as much as possible to avoid the situation where the voice cannot be played after the user inputs the text-to-speech instruction.

[0017] In a possible design, the audio format of the audio data obtained by each compression may be, but is not limited to, the Opus format. The Opus format is a low-latency, high-fidelity voice coding format with better fidelity than other coding formats.

[0018] There are multiple ways to implement the compression strategy agreed upon by the terminal and the server, including but not limited to the following ways 1 to 3.

[0019] Mode 1: After the server determines to use a static compression mechanism and the data length of the audio data compressed each time, it can notify the terminal. For example, the server sends a first indication information to the terminal, for example, the first indication information is used to indicate that the static compression mechanism used by the server and the data length of the audio data compressed each time is a preset length, for example, 108 bytes.

[0020] Mode 2: The terminal may request the server to use a static compression mechanism and request that the data length of the audio data compressed each time be a preset length. For example, the terminal sends a second indication message to the server, the second indication message being used to instruct the server to use a static compression mechanism and instruct the server to use a preset length of the audio data compressed each time, for example, 108 bytes.

[0021] Mode 3: When the server manufacturer generates the server, the server is set to use a static compression mechanism by default, and the data length of the compressed audio data each time is a preset length, such as 108 bytes. When the terminal downloads the audio stream, the default download data length is an audio stream of the preset length.

[0022] The above method can make the compression strategy agreed upon by the terminal and the server more flexible and provide more options.

[0023] In the third aspect, an embodiment of the present application further provides a terminal, which includes modules / units for executing the first aspect or any possible design method of the first aspect; these modules / units can be implemented by hardware, or corresponding software can be implemented by hardware.

[0024] In a fourth aspect, an embodiment of the present application further provides a server, comprising a module / unit for executing the method of the second aspect or any possible design of the second aspect; these modules / units can be implemented by hardware, or corresponding software can be implemented by hardware.

[0025] In the fifth aspect, a terminal is also provided, comprising: a communication module for communicating with other devices; one or more processors; one or more memories; one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions, and when the instructions are executed by the one or more processors, the terminal executes the method steps provided in the first aspect above.

[0026] In the sixth aspect, a server is also provided, comprising: a communication module for communicating with other devices; one or more processors; one or more memories; one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions, and when the instructions are executed by the one or more processors, the server executes the method steps provided in the second aspect above.

[0027] In the seventh aspect, an embodiment of the present application provides a chip, which is coupled to a memory in a terminal and is used to execute the technical solution of the first aspect of the embodiment of the present application and any possible design of the first aspect; in the embodiment of the present application, "coupling" refers to the direct or indirect combination of two components with each other.

[0028] In an eighth aspect, an embodiment of the present application provides a chip, which is coupled to a memory in a server and is used to execute the technical solution of the second aspect of the embodiment of the present application and any possible design of the second aspect thereof; in the embodiment of the present application, "coupling" refers to the direct or indirect combination of two components with each other.

[0029] In a ninth aspect, a communication system is also provided, comprising a terminal as provided in the third aspect or the fifth aspect and a server as provided in the fourth aspect or the sixth aspect.

[0030] In the tenth aspect, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method provided in the first aspect above.

[0031] In the eleventh aspect, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method provided in the second aspect above.

[0032] In a twelfth aspect, a program product is also provided, wherein the program product stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the method provided in the first aspect above.

[0033] In a thirteenth aspect, a program product is also provided, wherein the program product stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method provided in the second aspect above.

[0034] For the technical effects that can be achieved in each of the third to thirteenth aspects, please refer to the effect descriptions of the corresponding designs in the first and second aspects, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of text-to-speech conversion in the prior art;

[0036] Figure 2 A schematic diagram of a system architecture provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0038] Figure 4 A flowchart of text-to-speech conversion provided in one embodiment of the present application;

[0039] Figure 5 A schematic diagram of a dynamic compression mechanism provided by an embodiment of the present application;

[0040] Figure 6 Another flowchart of text-to-speech conversion provided by an embodiment of the present application;

[0041] Figure 7 A schematic diagram of a cache queue provided in an embodiment of the present application;

[0042] Figure 8 A schematic diagram of the structure of a device provided in one embodiment of the present application;

[0043] Fig. 9 A functional block diagram of another device provided in one embodiment of the present application;

[0044] Fig.10 A schematic diagram of the structure of another device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the following embodiments of the present application.

[0046] Below, some of the terms involved in the embodiments of the present application are explained to facilitate easy understanding by those skilled in the art.

[0047] Terminals, such as mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MID), wearable devices (e.g., watches, bracelets, smart helmets, etc.), virtual reality (VR) devices, augmented reality (AR) devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The audio processing method provided in the embodiments of the present application can be applied to any of the above terminals.

[0048] In an embodiment of the present application, the device for implementing the terminal-side function of the embodiment of the present application may be a terminal, or may be a device capable of supporting the implementation of the function, such as a chip system, and the device may be installed in the terminal.

[0049] The server is used to realize the conversion from text to speech and synthesize speech into audio stream, which can be referred to as "TTS server".

[0050] In an embodiment of the present application, the device used to implement the server-side function of the embodiment of the present application may be a server, or may be a device that can support the implementation of the function, such as a chip system, which may be installed in the server.

[0051] The current online audio synthesis can be performed by a server. Specifically, the terminal is connected to the server, and the text is sent to the server. After the server synthesizes the audio corresponding to the text into an audio stream, the audio stream is sent to the terminal for playback.

[0052] Figure 1 The schematic diagram of the process of implementing text-to-speech conversion by a terminal in the prior art is shown. After the terminal is connected to a server (eg, a TTS server), audio playback is implemented through at least the following five steps.

[0053] 1. The terminal downloads the audio stream from the server. 2. The downloaded audio stream is cached in the cache area. The threshold of the cache area is 10M, that is, when the audio stream cached in the cache area is greater than 10M, the audio stream enters the next link. 3. The parser parses the audio stream, such as decompressing the audio stream, determining the number of frames included in the audio stream, etc. 4. The decoder decodes the parsed audio stream to obtain an audio stream in pulse code modulation (PCM) format. 5. The audio stream in PCM format is passed to the player for playback.

[0054] In the above process, the time it takes for the server to return the audio stream cannot be controlled. For example, after the terminal successfully connects to the server, the server synthesizes the audio stream slowly and takes a long time to return the audio stream, resulting in a long overall audio playback delay, affecting the user experience.

[0055] In view of this, an embodiment of the present application provides a text-to-speech processing method. The terminal requests the server to synthesize speech online. If the first frame of audio data that can be played is returned within the preset time, the online synthesis method can continue to be used. If the first frame of audio data that can be played is not returned within the preset time, the offline synthesis method is used to reduce the audio playback delay.

[0056] Figure 2 FIG. 1 is a functional block diagram showing an example of a system architecture provided by an embodiment of the present application. Figure 2As shown, the terminal includes an application side and a TTS service system. The application side includes multiple applications (apps for short), such as voice assistants, WeChat, emails, etc. The TTS service system is used to realize text-to-speech conversion.

[0057] An application on the application side requests the TTS service system to convert text to speech. The TTS service system can choose an offline synthesis method or an online synthesis method. For example, the TTS service system gives priority to the online synthesis method, that is, the TTS service system sends a connection request to the server side to request to establish a connection with the server side. If the server can be successfully connected within the preset time, the online synthesis method is used. If the server cannot be connected within the preset time, the offline synthesis method is used.

[0058] For another example, the TTS service system can determine the size of the text content. If the text content is small, an offline synthesis method can be used; if the text content is large, an online synthesis method can be used.

[0059] For another example, the TTS service system preferentially uses an offline synthesis method. If the speech corresponding to a certain text does not exist in the offline database in the terminal, the online synthesis method is used.

[0060] After the TTS service system successfully connects to the server, it can send text to the server. After the server determines the speech synthesis audio stream corresponding to the text, it sends the audio stream to the TTS service system. The TTS service system sends the audio stream to a player (such as a speaker) for playback. Alternatively, after the TTS service system determines the speech corresponding to all or part of the text in the offline database, it sends the determined speech to the server, which performs speech synthesis.

[0061] In some embodiments, during the online synthesis process of the server, if the server network is poor, the terminal can switch to an offline synthesis mode. For example, during the online synthesis process of the server, if the network is poor and the connection is interrupted, an error code indicating the network interruption can be sent to the terminal. After receiving the error code, the terminal can switch to an offline synthesis mode.

[0062] In other embodiments, there may be more than one server for speech synthesis. For example, two servers are used, one is a main server and the other is a backup server. When the terminal successfully connects to the main server within the first preset time, online synthesis is performed through the main server. If the terminal fails to connect to the main server within the first preset time, the backup server is connected. If the backup server is successfully connected within the second preset time, online synthesis is performed through the backup server. If the connection to the backup server fails within the second preset time, the offline synthesis method is used again.

[0063] Among them, an application on the application side requests the TTS service system to convert text to speech, which may include multiple situations. For example, taking the voice assistant as an example, the voice assistant receives a sound signal input by the user, and the sound signal is "open WeChat". The TTS service system determines the response signal corresponding to the sound signal. For example, the response signal is "WeChat has been opened for you", and the TTS service system synthesizes the text of "WeChat has been opened for you" into speech. For another example, take Toutiao today as an example. After the Toutiao application detects the instruction for indicating text-to-speech conversion, it requests the TTS service system to convert text to speech to realize voice playback of news.

[0064] In other embodiments, before the application on the application side requests the TTS service system to perform text-to-speech conversion, it can log in to the TTS service system. For example, the application sends login information to the TTS service system, and the login information may include the name of the application, the account number currently logged in by the application, the password, and other information. The TTS service system authenticates the application based on the login information, for example, determines whether the login information is consistent with the pre-stored login information. If they are consistent, it agrees to log in to the TTS service system and perform text-to-speech conversion; if they are inconsistent, it refuses to log in to the TTS service system. If the authentication fails, the application can switch to offline synthesis to ensure smooth execution of speech synthesis.

[0065] Figure 3 FIG. 1 is a schematic diagram showing an example of a terminal provided in an embodiment of the present application. Figure 3 As shown, the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0066] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the terminal 100. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.

[0067] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal 100, and can also be used to transmit data between the terminal 100 and peripheral devices. The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0068] The wireless communication function of the terminal 100 can be implemented by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0069] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied on the terminal 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0070] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0071] In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0072] The display screen 194 is used to display the display interface of the application, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0073] The terminal 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0074] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0075] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. In some examples, the 1 or N cameras 193 may include at least one wide-angle camera, which is used to capture images with a large field of view.

[0076] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0077] Video codecs are used to compress or decompress digital videos. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record videos in various coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0078] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0079] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and software code of at least one application (such as iQiyi application, WeChat application, etc.). The data storage area can store data (such as images, videos, etc.) generated during the use of the terminal 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0080] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as pictures and videos are stored in the external memory card.

[0081] The terminal 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0082] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0083] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be arranged on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of the terminal 100. In some embodiments, the angular velocity of the terminal 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B.

[0084] The gyroscope sensor 180B can be used for anti-shake shooting. The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation. The magnetic sensor 180D includes a Hall sensor. The terminal 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal 100 is a flip phone, the terminal 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip, the characteristics such as automatic unlocking of the flip are set. The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in various directions (generally three axes). When the terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal 100, and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0085] Distance sensor 180F, used to measure distance. Terminal 100 can measure distance by infrared or laser. In some embodiments, when shooting a scene, terminal 100 can use distance sensor 180F to measure distance to achieve fast focus. Proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. Terminal 100 emits infrared light outward through the light emitting diode. Terminal 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near terminal 100. When insufficient reflected light is detected, terminal 100 can determine that there is no object near terminal 100. Terminal 100 can use proximity light sensor 180G to detect that the user holds the terminal 100 close to the ear to talk, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used in leather case mode, pocket mode automatic unlocking and lock screen.

[0086] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal 100 is in a pocket to prevent accidental touches. The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc.

[0087] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal 100 heats the battery 142 to avoid abnormal shutdown of the terminal 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0088] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be arranged on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be arranged on the surface of the terminal 100, which is different from the position of the display screen 194.

[0089] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of vibrating bones of the human body. The bone conduction sensor 180M can also contact the human body's pulse to receive blood pressure beating signals.

[0090] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The terminal 100 can receive the button input and generate a key signal input related to the user settings and function control of the terminal 100. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the terminal 100.

[0091] Understandably, Figure 2 The components shown do not constitute a specific limitation on the terminal 100. The mobile phone may also include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. Figure 2 The combination / connection relationship between the components can also be adjusted and modified.

[0092] Combine the following Figure 2 The structure of the terminal 100 shown in the figure is used to elaborate on the embodiment of the present invention. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " is generally used to indicate that the objects associated before and after are in an "or" relationship.

[0093] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0094] As used in the above embodiments, the term "when..." or "after..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrase "when determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)", depending on the context. In addition, in the above embodiments, relational terms such as first and second are used to distinguish one entity from another, without limiting any actual relationship and order between the entities.

[0095] Figure 4 FIG. 1 is a flow chart showing an example of a text-to-speech processing method provided in an embodiment of the present application. Figure 4 As shown, the process of the method includes:

[0096] 400. The terminal and the server agree on a compression policy.

[0097] Among them, the compression strategy includes a static compression mechanism (constants bitrate, CBR). The so-called static compression mechanism means that the data length of the audio data compressed each time is the same; the data length can be understood as the number of bytes and bits occupied by the data, for example, the data length is 108 bytes.

[0098] The compression strategy may also include that each compressed audio data includes at least one complete audio frame. Here, a complete audio frame means that each compressed audio data includes a frame header and a frame tail corresponding to a complete audio frame.

[0099] The compression strategy may also include that the data length of the audio data compressed each time is a preset length. The preset length may be determined in advance, for example, by the following formula:

[0100] Data length = ((number of samples per frame / 8)*bit rate) / sampling rate;

[0101] Among them, the number of samples per frame is the data sampled from a complete frame of audio; the sampling rate, that is, the sampling frequency, can be understood as the data sampled per second. Generally, the higher the sampling frequency, the more realistic the sound restoration. The bit rate can be understood as the number of bits transmitted per second, also known as the data transmission rate. In some embodiments, the bit rate / sampling rate can be a preset value. Or, it is related to the sound quality, for example, the better the sound quality, the higher the bit rate / sampling rate, and the worse the sound quality, the lower the bit rate / sampling rate. Therefore, the terminal can determine the bit rate / sampling rate based on the sound quality of the player, and then determine the preset length, that is, the data length of each compressed audio data, through the above formula.

[0102] The "agreement" described here can be implemented in a variety of ways, including but not limited to Ways 1 to 3 described below.

[0103] Mode 1: After the server determines to use a static compression mechanism and the data length of the audio data compressed each time, it can notify the terminal. For example, the server sends a first indication information to the terminal, for example, the first indication information is used to indicate that the static compression mechanism used by the server and the data length of the audio data compressed each time is a preset length, for example, 108 bytes.

[0104] Mode 2: The terminal may request the server to use a static compression mechanism and request that the data length of the audio data compressed each time be a preset length. For example, the terminal sends a second indication message to the server, the second indication message being used to instruct the server to use a static compression mechanism and instruct the server to use a preset length of the audio data compressed each time, for example, 108 bytes.

[0105] Mode 3: When the server manufacturer generates the server, the server is set to use a static compression mechanism by default, and the data length of the compressed audio data each time is a preset length, such as 108 bytes. When the terminal downloads the audio stream, the default download data length is an audio stream of the preset length.

[0106] It is understandable that when the terminal plays the audio stream, it plays it in frames. Usually, the audio stream will identify the frame header (or terminal) and frame tail (or code tail) of each frame. The terminal parses the frame header and frame tail of each frame in the audio stream, and then plays it frame by frame.

[0107] The server and the terminal agree that the data length of the compressed audio data each time is the same and is a preset length, and the data length includes at least one complete frame of audio, that is, includes the frame header and frame tail corresponding to at least one frame of audio. Therefore, when the terminal downloads the audio stream, it can download it with the agreed data length.

[0108] For example, assuming that the agreed data length is 108 bytes, the data length of the audio data compressed by the server each time is 108 bytes, and the terminal downloads 108 bytes of audio data each time, and the 108 bytes of audio data include at least one complete frame of audio, that is, include a frame header and a frame tail corresponding to at least one frame of audio. Therefore, the 108 bytes of audio data downloaded by the terminal each time are all playable audio streams, because the 108 bytes of audio data include at least one complete frame of audio.

[0109] It should be noted that in the prior art, the server uses a dynamic compression mechanism (variable bitrate, VBR), and the so-called dynamic compression mechanism means that the data length of the audio data compressed each time is different. For example, the audio data compressed by the server includes 2 compressed packets, the length of the audio data of one compressed packet is 100K, and the length of the audio data of the other compressed packet is 60K. Since the length of the dynamically compressed audio data is not fixed, the audio data contained in a compressed packet may be less than one frame, for example, it does not include a complete frame of audio, for example, it only includes the frame header of one frame of audio but does not include the frame tail. Figure 5 The schematic diagram of the dynamic compression mechanism is shown. Assume that the audio data compressed by the server includes two compressed packets, the first compressed packet includes 100K, and the second compressed packet includes 60K. The first compressed packet and the first 40K in the second compressed packet belong to a complete audio frame. Then, when the terminal only receives the first compressed packet, it cannot play.

[0110] Therefore, in the prior art, after the terminal downloads the compressed package each time, it is necessary to analyze the compressed package by a parser to determine whether at least one frame of complete audio is included, and if so, play the audio data. Before the parser analyzes the compressed package, it is impossible to determine whether the compressed package can be played.

[0111] In an embodiment of the present application, the terminal and the server agree that the data length of the audio data compressed each time (i.e., the compression ratio obtained by each compression) is the same, and is a preset length (e.g., 108 bytes), and the data length includes at least one frame of complete audio, i.e., includes a frame header and a frame tail corresponding to at least one frame of audio. Therefore, the terminal downloads audio data of a preset length (e.g., 108 bytes) each time, so the audio data downloaded each time can be played, because the audio data downloaded each time includes at least one frame of complete audio. In other words, after the terminal downloads the compressed audio data each time, it can also be determined that the audio data can be played without waiting for the parser to parse it.

[0112] 401: The terminal detects a text-to-speech request.

[0113] Exemplarily, the terminal may include a TTS service system. Different applications in the terminal may call the TTS service system to request text-to-speech conversion. Among them, different applications may include voice assistants, WeChat, etc. Therefore, the above-mentioned request for initiating text-to-speech conversion may be a request for an application to call the TTS service system. For example, taking the voice assistant as an example, the user sends a sound signal of "Hello, Huawei, help me open Weibo". After the terminal detects the sound signal, it calls the TTS service system to realize the conversion of text to speech through the TTS service system, and then outputs the voice information of "Weibo has been opened".

[0114] 402. The terminal starts the timer and begins counting.

[0115] It should be understood that after the terminal detects the request for initiating text-to-speech conversion, it can start the timer.

[0116] 403: The terminal sends the text to be converted to the server.

[0117] 403 and 402 may be executed simultaneously or at different times, and the execution order of the two is not limited.

[0118] It is understandable that the process of connecting the terminal to the server may also be included before 403. The connection between the terminal and the server may be, for example, an IP connection, a TCP connection, etc., which is not limited in the embodiment of the present application.

[0119] 404. The server performs speech conversion based on the received text, and compresses the converted audio data based on an agreed compression strategy. The format of the audio data may be, for example, Opus format.

[0120] As mentioned above, the terminal and the server agree that the data length of the audio data compressed each time is the same, and the data length includes at least one complete frame of audio, that is, includes the frame header and frame tail corresponding to at least one frame of audio. For example, if the agreed data length is 108 bytes, the audio data compressed by the server each time includes 108 bytes, and the 108 bytes include at least one complete frame of audio.

[0121] 405. The terminal downloads the audio data.

[0122] Assuming that the agreed data length is 108 bytes, the terminal downloads 108 bytes of audio data each time, and the 108 bytes include at least one complete frame of audio.

[0123] 406, the terminal determines whether the first frame of playable audio data is downloaded before the timer reaches the threshold; if not, execute 407, and if so, execute 408.

[0124] The threshold value may be pre-set, such as a default setting before the terminal leaves the factory, or may be user-defined, such as 1.5s, which is not limited in the embodiments of the present application.

[0125] As mentioned above, the terminal and the server agree that the data length of the compressed audio data each time is the same and is a preset length, and the data length includes at least one complete frame of audio, that is, includes a frame header and a frame tail corresponding to at least one frame of audio. Therefore, the terminal downloads the audio data of the preset length each time and can play the audio data.

[0126] Therefore, the audio data that can be played in the first frame includes audio data of a preset length (e.g., 108 bytes) downloaded by the terminal for the first time. Alternatively, the audio data that can be played in the first frame is data downloaded for the first time including at least one complete audio frame, or the audio data that can be played in the first frame is data downloaded for the first time including a frame header and a frame tail corresponding to at least one complete audio frame.

[0127] Taking the preset length of 108 bytes as an example, the terminal may determine whether the first frame of playable audio data is downloaded, including: determining whether 108 bytes are downloaded. Assume that 108 bytes are successfully downloaded before the timer threshold (e.g., 1.5s) is reached, that is, the first frame of playable audio data is downloaded. Therefore, the terminal can continue to download the remaining audio data. Assume that 108 bytes are not successfully downloaded before 1.5s, the online synthesis method is abandoned and the offline synthesis method is used.

[0128] 407. The terminal adopts the offline synthesis mechanism.

[0129] If the terminal does not download the first frame of audio data that can be played before the timer threshold is reached, an offline synthesis mechanism can be used. The offline synthesis mechanism can be that the terminal determines the speech corresponding to the text from an offline database and then synthesizes the speech into audio data. The offline database can be stored in the terminal in advance.

[0130] 408: The terminal continues to download the audio data.

[0131] If the terminal has downloaded the first frame of playable audio data before the timer threshold is reached, the terminal can continue to download the audio data. At this time, the terminal can choose to turn off the timer.

[0132] Therefore, in the embodiment of the present application, after the terminal sends the text to the server, if the first frame of playable audio data is not downloaded within the preset time, offline synthesis is performed to reduce the terminal audio playback delay.

[0133] In some other embodiments, step 408 may also include the following steps: Figure 6 409-410 shown.

[0134] 409: The terminal caches the audio data.

[0135] 410. When the terminal determines that the buffered audio data has reached an upper limit, the terminal transfers the first frame of the downloaded audio data that can be played to the parser for parsing. The upper limit may be, for example, 1M, 5M, etc., which is not limited in the embodiment of the present application.

[0136] Figure 7 A schematic diagram of a cache queue provided in an embodiment of the present application is shown. The upper limit (or threshold) of the cache queue is 1M. When the upper limit is reached, the first frame of audio data that can be played (for example, 108 bytes of audio data downloaded for the first time) is passed to the parser. After that, the terminal continues to download audio data and stores it in the cache queue, which helps to play audio without lag.

[0137] It should be noted that the upper limit of the cache in the embodiment of the present application is set relatively low (less than 10M in the prior art), which can ensure that the audio data is delivered to the parser for parsing as soon as possible.

[0138] It can be understood that the process of the parser parsing the audio data includes, for example, decompressing the audio data (the audio data downloaded from the server is compressed audio data), parsing the frame headers and frame tails in the audio data, etc., so that the player can recognize the frame headers and frame tails.

[0139] 411, the decoder decodes the parsed audio data to obtain audio data in a preset audio format. It is understandable that the decoding process of the audio data by the decoder can be understood as a process of converting the format of the audio data, such as converting the format of the audio data into a playable audio format, such as PCM format.

[0140] 412, the terminal transmits the decoded audio data to the player for playing.

[0141] like Figure 8 As shown, some other embodiments of the present application disclose a device 800, which may be the terminal described above, and the device 800 may have an audio output interface 805, such as a speaker. The electronic device may also include a communication interface 801: one or more processors 802, one or more memories 803; one or more computer programs 804; the above devices may be connected via one or more communication buses 806. Of course, the electronic device may also include other devices such as a display screen.

[0142] One or more computer programs 804 are stored in the memory 803 and configured to be executed by the one or more processors 802. The one or more computer programs 804 include instructions. The instructions can be used to execute the following steps: Figure 4The corresponding steps of the terminal in the embodiment.

[0143] For example, Figure 8 The electronic device shown may be Figure 4 In this case, the one or more processors 802 may be the processor 110 , the audio output interface 805 may be a speaker, and the one or more memories 803 may be the internal memory 121 .

[0144] like Fig. 9 As shown, some other embodiments of the present application disclose a device 900, which can be the server mentioned above, or a device in the server, such as a chip system. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0145] The device 900 includes a processing unit 901 and a communication unit 902. The communication unit 902 is used to receive a text-to-speech request sent by a terminal, wherein the request carries text; the processing unit 901 is used to convert the text into audio data; the audio data is compressed, and the data length of the audio data obtained by each compression is the same; the communication unit 902 is also used to send the compressed voice data to the terminal respectively.

[0146] The data length of the audio data compressed each time by the processing unit 901 may be a preset length. Alternatively, the audio data compressed each time by the processing unit 901 may include at least one complete frame of audio. Optionally, the communication unit 902 may also notify the terminal of the data length of the audio data compressed each time by the server.

[0147] In addition, the audio format of the audio data compressed each time by the processing unit 901 may be, but is not limited to, the Opus format.

[0148] For the specific execution process of the processing unit 901 and the communication unit 902 and the technical effects that can be achieved by the scheme, please refer to the records in the above method embodiment. The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application can be integrated into a processor, or it can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0149] like Fig.10As shown, some other embodiments of the present application disclose a device 1000, which can be the server mentioned above, or a device in the server, such as a chip system. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0150] The apparatus 1000 may include a communication interface 1001: one or more processors 1002, one or more memories 1003; one or more computer programs 1004; the above-mentioned components may be connected via one or more communication buses 1006. The communication interface 1001 is used to communicate with other devices via a transmission medium. One or more computer programs 1004 are stored in the above-mentioned memories 1003 and are configured to be executed by the one or more processors 1002. The one or more computer programs 1004 include instructions, which may be used to execute the following steps: Figure 4 The corresponding steps of the server in the embodiment.

[0151] The communication interface 1001 can be a transceiver, circuit, bus, module or other type of communication interface. The processor 1002 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to perform, or the hardware and software modules in the processor are combined and performed. The memory 1003 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., and can also be a volatile memory (volatile memory), such as a random access memory (RAM). The memory is any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory in the embodiments of the present application can also be a circuit or any other device that can realize a storage function. The memory 1003 is coupled to the processor 1002. The coupling in the embodiment of the present application is an interval coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. As another implementation, the memory 1003 can also be located outside the device 1000. The processor 1002 can operate in conjunction with the memory 1003. The processor 1002 can execute program instructions stored in the memory 1003. At least one of the at least one memory 1003 can also be included in the processor 1002. The connection medium between the above-mentioned communication interface 1001, the processor 1002 and the memory 1003 is not limited in the embodiment of the present application. For example, in the embodiment of the present application, Fig.10 The memory 1003, the processor 1002 and the communication interface 1001 may be connected via a bus, which may be divided into an address bus, a data bus, a control bus and the like.

[0152] Understandably, the above Fig. 9 The device in the embodiment shown can be Fig.10 Specifically, the processing unit 901 may be implemented by the processor 1002 , and the communication unit 902 may be implemented by the communication interface 1001 .

[0153] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0154] Note: A portion of this patent application document contains material which is subject to copyright protection. The copyright owner reserves all rights reserved except for the production of copies of the material in the patent file or patent record in the Patent Office.

Claims

1. A text-to-speech processing method, characterized in that: Applied to a terminal, the method comprises: Agreeing with the server on a compression strategy for voice data after converting text to speech, wherein the compression strategy includes that the data length of the audio data compressed each time is a preset length, the audio data of the preset length includes at least one frame of complete audio, and the complete audio is playable audio; An instruction for instructing text-to-speech conversion is detected; Sending the text to a server; Downloading audio data converted from the text from the server; Determine whether the first audio data with a preset data length is downloaded within a preset time length, wherein the first audio data with a preset data length includes the first frame of audio data that can be played; If the first audio data of the preset length is downloaded within the preset time, continue downloading the remaining audio data; If the first audio data of the preset length is not downloaded within the preset time, the audio signal corresponding to the text is determined in the offline database, and the audio signal is synthesized into audio data.

2. The method according to claim 1, characterized in that Downloading audio data converted from the text from the server includes: Each time, audio data having a data length of a preset length is downloaded from the server.

3. The method according to claim 2, characterized in that The method further comprises: Storing the downloaded audio data in the cache area; When it is determined that the memory usage of the audio data in the buffer area reaches a threshold, the audio data that can be played in the first frame is played.

4. The method according to claim 3, characterized in that Play the audio data that can be played in the first frame, including: Passing the playable audio data of the first frame to a parser for parsing to obtain decompressed audio data; The audio data parsed by the parser is passed to the decoder for decoding to obtain audio data in a preset audio format; Pass the audio data decoded by the decoder to the player for playback.

5. The method according to claim 1, characterized in that The audio data downloaded from the server is in Opus format.

6. A text-to-speech processing method, characterized in that: Applied to a server, the method comprises: A compression strategy for voice data after converting text to speech is agreed upon with the terminal, wherein the compression strategy includes that the data length of the audio data compressed each time is a preset length, the audio data of the preset length includes at least one frame of complete audio, and the complete audio is playable audio; A text-to-speech request sent by a receiving terminal, wherein the request carries text; converting the text into audio data; Compressing the audio data, wherein the data length of the audio data obtained by each compression is the same as the preset length and includes at least one complete frame of audio, and the at least one complete frame of audio is playable audio; The compressed voice data is sent to the terminals respectively.

7. The method according to claim 6, characterized in that The audio format of the audio data obtained by each compression is the Opus format.

8. The method according to claim 6, characterized in that The method further comprises: The terminal is informed of the data length of the audio data compressed by the server each time.

9. A terminal, characterized in that: include: A communication module, used for communicating with other devices; one or more processors; one or more memories; One or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions. When the instructions are executed by the one or more processors, the terminal executes the method according to any one of claims 1 to 5.

10. A server, characterized in that: include: A communication module, used for communicating with other devices; one or more processors; one or more memories; One or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions. When the instructions are executed by the one or more processors, the server executes the method according to any one of claims 6 to 8.

11. A communication system, characterized in that: It comprises the terminal as claimed in claim 9 and the server as claimed in claim 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a computer, the computer executes the method according to any one of claims 1 to 5; or, when the program instructions are executed by a computer, the computer executes the method according to any one of claims 6 to 8.

13. A program product, characterized in that The program product stores a computer program, which includes program instructions. When the program instructions are executed by a computer, the computer executes the method according to any one of claims 1 to 5; or, when the program instructions are executed by a computer, the computer executes the method according to any one of claims 6 to 8.

Images