Text-to-speech synchronous broadcasting method and device

Abstract

The application relates to a character voice synchronous broadcast method and device, which are applied to the voice broadcast subfield in the terminal artificial intelligence (AI) field, and the method comprises the following steps: obtaining a first text and a first audio to be broadcast in the case of receiving a voice broadcast request, the first text comprises at least one text line; the first text is displayed, and the first audio is played, wherein when the first audio is played to a line break character in a text line of the first text, the first text is displayed in a line break mode, and the line break character represents the last voiced character in the text line. The character voice synchronous broadcast method and device provided in the application can keep the voice broadcast content consistent with the display content of the display screen in the scene with indefinite broadcast content and playing time.

Description

TECHNICAL FIELD

[0001] The present application relates to the sub-field of voice broadcasting in the field of terminal artificial intelligence (AI), and in particular to a text voice synchronous broadcasting method and device. BACKGROUND

[0002] In a scenario where the broadcasting content and the playing time are known, a pre-set timestamp can be used to realize the synchronous broadcasting of text voice, such as the synchronous broadcasting of movie lines and movie subtitles, or the synchronous broadcasting of songs and lyrics, etc.

[0003] In a scenario where the broadcasting content and the playing time are indefinite, it is impossible to pre-set the timestamp, and it is impossible to realize the synchronous broadcasting of text voice. For example, a user randomly asks questions to a screen-equipped sound box, the screen-equipped sound box obtains the answers, plays the voice generated based on the text-to-speech (TTS) technology, and synchronously displays the answers on the display screen, and scrolls the answers along with the playing of the voice. At present, in this scenario, the screen scrolling speed is too fast, the current voice broadcasting content is not displayed in the interface, and the user experience is poor. SUMMARY

[0004] Therefore, a text voice synchronous broadcasting method and device are provided, which can keep the voice broadcasting content consistent with the display content of the display screen in a scenario where the broadcasting content and the playing time are indefinite.

[0005] In a first aspect, an embodiment of the present application provides a text voice synchronous broadcasting method, which comprises: obtaining a first text and a first audio to be broadcast in a case where a voice broadcasting request is received, the first text comprising at least one text line; displaying the first text and playing the first audio, wherein when the first audio is played to a line break character in the first text, the first text is displayed by line break, and the line break character represents the last voiced character in the text line.

[0006] In the embodiment of the present application, the last voiced character in the text line is taken as the line break character, the moment of voice broadcasting line break character is connected with the moment of displaying line break character, the synchronization of voice playing and text scrolling is realized, and thus the voice broadcasting content is kept consistent with the display content of the display screen in a scenario where the broadcasting content and the playing time are indefinite.

[0007] In a possible implementation, the method further includes: recording a first system timestamp of starting to display the first text and a second system timestamp of starting to play the first audio; determining a first time corresponding to the line break character in the first audio; determining a line break time corresponding to the line break character in the display of the first text according to the first system timestamp, the second system timestamp, and the first time corresponding to the line break character in the first audio; and controlling the first text to display line by line when a display duration of the first text reaches the line break time.

[0008] In this way, the time axis of the display duration of the first text and the time axis of the playing progress of the first audio are aligned based on the first system timestamp and the second system timestamp, the line break time corresponding to the line break character in the display of the first text can be determined, and the first text is controlled to display line by line after the line break character is played, thereby realizing the synchronization of voice playing and text scrolling.

[0009] In a possible implementation, the determining of the first time corresponding to the line break character in the first audio includes: splitting a second audio from the first audio; determining a second time corresponding to the line break character in the second audio; and determining the first time corresponding to the line break character in the first audio according to the second time corresponding to the line break character in the second audio and a position of the second audio in the first audio.

[0010] In this way, the processing efficiency can be improved, the user waiting time can be reduced, and the user experience can be improved by segmenting the first audio.

[0011] In a possible implementation, the determining of the second time corresponding to the line break character in the second audio includes: generating at least one character sequence based on a non-pronounced character in the first text, the character sequence not including the non-pronounced character; determining a character sequence corresponding to the second audio in the at least one character sequence; adjusting an amplitude envelope of the second audio until a number of wave crests on the amplitude envelope of the second audio is the same as a number of characters in the character sequence corresponding to the second audio; determining a wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio according to a position of the line break character in the character sequence corresponding to the second audio; and determining a time corresponding to a wave trough adjacent to the wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio as the second time corresponding to the line break character in the second audio.

[0012] In this way, by adjusting the number of peaks on the amplitude envelope of the second audio to be the same as the number of characters in the character sequence corresponding to the second audio, the second time corresponding to the line feed character in the second audio can be accurately determined, and the line feed time corresponding to the line feed character in the first text display process can be accurately determined, thereby realizing synchronization of voice playing and text scrolling.

[0013] In a possible implementation, the splitting the second audio from the first audio comprises: determining the first audio as the second audio; and the generating the at least one character sequence based on the non-pronounced character in the first text comprises: removing the non-pronounced character in the first text to obtain a character sequence.

[0014] In a possible implementation, the non-pronounced character comprises an annotation and a punctuation mark.

[0015] In a possible implementation, the splitting the second audio from the first audio comprises: detecting an amplitude of the first audio; and in a case where the amplitude of the first audio is less than a first threshold value and a duration is greater than a second threshold value, splitting the second audio from the first audio; and the generating the at least one character sequence based on the non-pronounced character in the first text comprises: splitting the first text into a plurality of second texts based on punctuation marks in the first text; and removing annotations in the second texts to obtain a plurality of character sequences.

[0016] In this way, audio splitting and text splitting based on punctuation marks can be implemented, so that the second audio is accurately matched with the character sequence, and the second time corresponding to the line feed character in the second audio can be accurately determined.

[0017] In a possible implementation, the adjusting the amplitude envelope of the second audio until the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio comprises: performing low-pass filtering on the second audio based on a cutoff frequency to obtain third audio; when the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio, increasing the cutoff frequency; when the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold, decreasing the cutoff frequency, where the third threshold is greater than the number of characters in the character sequence corresponding to the second audio; when the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold, sequentially removing peaks on the amplitude envelope of the third audio in order of amplitude difference between the peaks and adjacent valleys from small to large, until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio; and when the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio, determining the amplitude envelope of the third audio as the adjusted amplitude envelope of the second audio.

[0018] In this way, the number of peaks on the amplitude envelope of the second audio can be adjusted to be the same as the number of characters in the character sequence corresponding to the second audio.

[0019] In a possible implementation, the obtaining, in response to receiving the voice broadcast request, the first text to be broadcast and the first audio comprises: obtaining, in response to receiving the voice broadcast request, a broadcast text from a cloud server; performing rendering processing on the broadcast text to obtain the first text; and performing voice synthesis processing on the broadcast text to obtain the first audio.

[0020] In this way, the voice playing of the broadcast text and the character scrolling can be synchronized.

[0021] In a possible implementation, the line break time corresponding to the line break character is recorded in the scrolling control sequence; after determining the line break time corresponding to the line break character, the method further comprises: adding the line break time corresponding to the line break character to the tail of the scrolling control sequence; and after controlling the first text to be displayed in a line break manner, the method further comprises: removing the line break time corresponding to the line break display of the first text from the head of the scrolling control sequence.

[0022] In this way, the processing and broadcasting can be implemented simultaneously, the efficiency is improved, the user experience is improved, and the adaptability to the scene with indefinite broadcasting content and indefinite playing time is improved.

[0023] In a possible implementation, the first text line display includes that the first text is scrolled by the text line to a text line where a first voiced character in the first text is located.

[0024] In this way, multiple lines can be skipped at a time, and the synchronization of voice playing and text scrolling is facilitated, so that the voice broadcasting content is consistent with the display content of the display screen in the scene with indefinite broadcasting content and indefinite playing time.

[0025] In a second aspect, an embodiment of the present application provides a broadcasting device, the device comprising: an acquisition module configured to obtain first text and first audio to be broadcasted when a voice broadcasting request is received, the first text comprising at least one text line; and a display module configured to display the first text and play the first audio, wherein the first text is displayed by line when the first audio is played to a line break character in the first text, and the line break character represents a last voiced character in the text line.

[0026] In the embodiment of the present application, the last voiced character in the text line is taken as the line break character, the time of voice broadcasting the line break character is associated with the time of displaying the line break character, the synchronization of voice playing and text scrolling is implemented, and thus the voice broadcasting content is consistent with the display content of the display screen in the scene with indefinite broadcasting content and indefinite playing time.

[0027] In a possible implementation, the device further comprises: a recording module configured to record a first system timestamp of starting to display the first text and a second system timestamp of starting to play the first audio; a first determination module configured to determine a first time corresponding to the line break character in the first audio; a second determination module configured to determine a line break time corresponding to the line break character in the display process of the first text according to the first system timestamp, the second system timestamp, and the first time corresponding to the line break character in the first audio; and a control module configured to control the first text to be displayed by line when a display time of the first text reaches the line break time.

[0028] In a possible implementation, the first determining module is further configured to: split a second audio from the first audio; determine a second time corresponding to the line feed character in the second audio; and determine the first time corresponding to the line feed character in the first audio according to the second time corresponding to the line feed character in the second audio and a position of the second audio in the first audio.

[0029] In a possible implementation, the determining of the second time corresponding to the line feed character in the second audio comprises: generating at least one character sequence based on non-pronounced characters in the first text, the non-pronounced characters not included in the character sequence; determining a character sequence corresponding to the second audio in the at least one character sequence; adjusting an amplitude envelope of the second audio until a number of peaks on the amplitude envelope of the second audio is the same as a number of characters in the character sequence corresponding to the second audio; determining a peak corresponding to the line feed character on the adjusted amplitude envelope of the second audio according to a position of the line feed character in the character sequence corresponding to the second audio; and determining a time corresponding to a valley adjacent to the peak corresponding to the line feed character on the adjusted amplitude envelope of the second audio as the second time corresponding to the line feed character in the second audio.

[0030] In a possible implementation, the splitting of the second audio from the first audio comprises: determining the first audio as the second audio; and the generating of the at least one character sequence based on the non-pronounced characters in the first text comprises: removing the non-pronounced characters in the first text to obtain a character sequence.

[0031] In a possible implementation, the non-pronounced characters comprise annotations and punctuation marks.

[0032] In a possible implementation, the splitting of the second audio from the first audio comprises: detecting an amplitude of the first audio; and splitting the second audio from the first audio in a case where the amplitude of the first audio is less than a first threshold value and a duration is greater than a second threshold value; and the generating of the at least one character sequence based on the non-pronounced characters in the first text comprises: splitting the first text into a plurality of second texts based on punctuation marks in the first text; and removing annotations in the second texts to obtain a plurality of character sequences.

[0033] In a possible implementation, the adjusting the amplitude envelope of the second audio until the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio comprises: performing low-pass filtering on the second audio based on a cutoff frequency to obtain third audio; when the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio, increasing the cutoff frequency; when the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold, decreasing the cutoff frequency, where the third threshold is greater than the number of characters in the character sequence corresponding to the second audio; when the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold, sequentially removing peaks in the order of amplitude difference between the peaks and adjacent troughs on the amplitude envelope of the third audio from small to large, until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio; and when the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio, determining the amplitude envelope of the third audio as the adjusted amplitude envelope of the second audio.

[0034] In a possible implementation, the obtaining module is further configured to: in a case where the voice broadcast request is received, obtain a broadcast text from a cloud server; perform rendering processing on the broadcast text to obtain the first text; and perform voice synthesis processing on the broadcast text to obtain the first audio.

[0035] In a possible implementation, the line break moment corresponding to the line break character is recorded in the scroll control sequence; and the device further includes: an adding module configured to add the line break moment corresponding to the line break character to the tail of the scroll control sequence after the line break moment corresponding to the line break character is determined; and a popping module configured to pop the line break moment corresponding to the line break character in the first text from the head of the scroll control sequence after the first text is displayed in a line break manner.

[0036] In a possible implementation, the line break display of the first text comprises: the text line is scrolled to a text line where a first character sounding after the text line in the first text.

[0037] In a third aspect, the embodiments of the present application provide a broadcasting device, which can execute the text-to-speech synchronization broadcasting method of the first aspect or one or more of the possible implementation manners of the first aspect.

[0038] In a fourth aspect, the embodiments of the present application provide a computer program product, which comprises computer readable code or a non-volatile computer readable storage medium carrying computer readable code, and when the computer readable code is run in an electronic device, a processor in the electronic device executes the text-to-speech synchronization broadcasting method of the first aspect or one or more of the possible implementation manners of the first aspect.

[0039] These and other aspects of the present application will become more fully understood from the following (several) detailed embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated in and form a part of the specification, illustrate the exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1a A structure schematic diagram of a broadcasting device provided by an embodiment of the present application is shown;

[0042] Figure 1b A process schematic diagram of text-to-speech synchronization broadcasting performed by a broadcasting device in an embodiment of the present application is shown;

[0043] Figure 2 A structure schematic diagram of a broadcasting device according to an embodiment of the present application is shown;

[0044] Figure 3 An application scenario schematic diagram of a text-to-speech synchronization broadcasting method provided by an embodiment of the present application is shown;

[0045] Figure 4 A flowchart of a text-to-speech synchronization broadcasting method provided by an embodiment of the present application is shown;

[0046] Figure 5a An exemplary schematic diagram of a first text in an embodiment of the present application is shown;

[0047] Figure 5b An exemplary schematic diagram of a line break mark in an embodiment of the present application is shown;

[0048] Figure 6 A flowchart of a text-to-speech synchronization broadcasting method provided by an embodiment of the present application is shown;

[0049] Figure 7a A schematic diagram of a first text and a first audio alignment timeline process in an embodiment of the present application is shown;

[0050] Figure 7b An exemplary schematic diagram showing the line breaking moment in the embodiment of the present application is shown.

[0051] Figure 8 A flow schematic diagram of the text and voice synchronous broadcasting method provided by the embodiment of the present application is shown.

[0052] Figure 9 A structure schematic diagram of the broadcasting device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference signs in the drawings represent functionally identical or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0054] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0055] In addition, in order to better illustrate the present application, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present application can also be implemented without some specific details. In some examples, methods, means, elements and circuits well known to those skilled in the art are not described in detail in order to highlight the main idea of the present application.

[0056] In the scenario where the broadcasting content and the playing time are indefinite, in the related art, there is a situation that the voice broadcasting content is inconsistent with the display content of the display screen, for example, the current voice broadcasting content is not displayed in the display screen due to too fast scrolling speed of the display screen.

[0057] In order to solve the above technical problem, the present application provides a text and voice synchronous broadcasting method, which can realize the synchronous scrolling of the display content of the display screen with the voice broadcasting content in the scenario where the broadcasting content and the playing time are indefinite, so as to keep the voice broadcasting content consistent with the display content of the display screen in the scenario where the broadcasting content and the playing time are indefinite.

[0058] The text and voice synchronous broadcasting method provided by the embodiment of the present application can be applied to a broadcasting device, which can refer to a device with a display screen and a sound producing device. Figure 1a A structure schematic diagram of the broadcasting device provided by the embodiment of the present application is shown. As shown in FIG. 1, the broadcasting device 1 comprises a display screen 10 and a sound producing device 20. Figure 1aAs shown, the broadcast device includes a display screen, a sound producing device, and a control unit. The display screen can be used to display text, pictures, moving pictures, or videos, etc. The sound producing device can be used to play audio. The control unit can control the content displayed by the display screen to synchronously scroll with the audio played by the sound producing device. In a possible implementation, the display screen, the sound producing device, and the control unit can be distributed in an electronic device, such as a screen-equipped sound box, a mobile phone, a notebook computer, etc. In a possible implementation, the display screen, the sound producing device, and the control unit can be distributed in different electronic devices, for example, the display screen and the control unit are distributed in a mobile phone, and the sound producing device is distributed in a Bluetooth sound box connected with the mobile phone. The embodiments of the present application do not limit the distribution of the display screen, the sound producing device, and the control unit in the broadcast device.

[0059] Figure 1b A process diagram in which the broadcast device in the embodiments of the present application performs text and voice synchronous broadcast is shown. As shown, Figure 1b When the sound producing device plays to "Huawei Technologies Co., Ltd. was founded in 1987," the broadcast device controls the text line "Huawei Technologies Co., Ltd. was founded in 1987," to scroll to the center of the display screen and is highlighted; when the sound producing device plays to "Headquartered in Longgang District, Shenzhen City, Guangdong Province. Huawei," the broadcast device controls the text line "Headquartered in Longgang District, Shenzhen City, Guangdong Province. Huawei" to scroll to the center of the display screen and is highlighted.

[0060] The broadcast device involved in the present application can have a wireless connection function, which means that it can be connected with other terminal devices (such as mobile phones, notebook computers, or computers, etc.) or cloud servers through wireless connection modes such as wifi and Bluetooth. The broadcast device of the present application can also have a wired connection function for communication. The broadcast device involved in the embodiments of the present application can be touch screen or non-touch screen. The touch screen can be operated by clicking, sliding, etc. on the display screen through fingers, styluses, etc. The non-touch screen device can be connected with a mouse, a keyboard, a touch panel, etc. input device to control the broadcast device. The broadcast device can also have a voice recognition function, and the user can control the broadcast device through voice instructions.

[0061] For example, the broadcast device of the present application can be a smart phone, a netbook, a tablet computer, a notebook computer, a wearable electronic device (such as a smart bracelet, a smart watch, etc.), a TV, a virtual reality device, a screen-equipped sound box, etc.

[0062] Figure 2 A structure diagram of the broadcast device according to the embodiments of the present application is shown. Taking the broadcast device as a screen-equipped sound box for example, Figure 2A structural diagram of the sound box with screen 200 is shown.

[0063] The sound box with screen 200 can include a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charge management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, a wireless communication module 252, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a sensor module 280, an indicator 292, a camera 293, a display screen 294, and the like. The sensor module 280 can include a touch sensor, a gyroscope sensor, an acceleration sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a pressure sensor, a distance sensor, an ambient light sensor, and the like, which are not shown in the figure.

[0064] The processor 210 can include one or more processing units, for example: the processor 210 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), and the like. Different processing units can be independent devices, or can be integrated in one or more processors. The controller can be the nerve center and command center of the sound box with screen 200. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0065] A memory can also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory can save instructions or data that have just been used or are repeatedly used by the processor 210. If the processor 210 needs to use the instructions or data again, it can be directly called from the memory, avoiding repeated access and reducing the waiting time of the processor 210, thus improving the efficiency of the system.

[0066] The processor 210 can run the text-to-speech synchronous broadcasting method provided by the embodiments of the present application, so that the screen sound box 200 can realize the consistency of the voice playing content and the screen display content. The processor 210 can include different devices, such as integrated CPU and GPU. When the CPU and the GPU cooperate to execute the text-to-speech synchronous broadcasting method provided by the embodiments of the present application, for example, part of the algorithm in the text-to-speech synchronous broadcasting method is executed by the CPU, and another part of the algorithm is executed by the GPU, so as to obtain faster processing efficiency.

[0067] The display screen 294 is configured to display images, videos, and the like. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), or the like. In some embodiments, the screen sound box 200 can include one or N display screens 294, where N is a positive integer greater than 1. The display screen 294 can be configured to display information input by a user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, the display 294 can display photos, videos, webpages, or files, and the like. For another example, the display 294 can display a graphical user interface. The graphical user interface includes a status bar, a hideable navigation bar, time and weather widgets, and icons of applications, such as a browser icon, and the like. The status bar includes a carrier name, a mobile network (e.g., 4G), time, and remaining power. The navigation bar includes a back key icon, a home key icon, and a forward key icon. In addition, it can be understood that, in some embodiments, the status bar can also include a Bluetooth icon, a Wi-Fi icon, an external device icon, and the like.

[0068] In the embodiments of the present application, the display screen 294 can be a flexible display screen as a whole, or can be a spliced display screen composed of two rigid screens and a flexible screen between the two rigid screens, and the like, which are not limited in the embodiments of the present application.

[0069] The camera 293 (front-facing camera or rear-facing camera, or a camera that can be used as both a front-facing camera and a rear-facing camera) is used to capture still images or videos. Generally, the camera 293 can include a light-sensing element such as a lens group and an image sensor, where the lens group includes a plurality of lenses (convex lenses or concave lenses) for collecting light signals reflected by an object to be photographed and transmitting the collected light signals to the image sensor. The image sensor generates a raw image of the object to be photographed according to the light signals.

[0070] The internal memory 221 can be used to store computer executable program codes including instructions. The processor 210 performs various function applications and data processing of the sound bar 200 by running the instructions stored in the internal memory 221. The internal memory 221 can include a program storage area and a data storage area. The program storage area can store an operating system, codes of application programs (such as a camera application and the like), and the like. The data storage area can store data (such as the broadcast text, the first text, the first audio, the character sequence, the second audio, the first system timestamp, the second system timestamp, the first time, the second time, and the line feed time in the embodiments of the present application and the like) created during the use of the sound bar 200, and the like.

[0071] The internal memory 221 can also store one or more computer programs 1310 corresponding to the method for cell search provided in the embodiments of the present application. The one or more computer programs 1304 are stored in the above-mentioned memory 221 and are configured to be executed by the one or more processors 210, and the one or more computer programs 1310 include instructions that can be used to perform the steps in the respective embodiments as described above. Figure 4 、 Figure 6 or Figure 8 the respective steps in the embodiments.

[0072] In addition, the internal memory 221 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0073] Of course, the code of the method for cell search provided in the embodiments of the present application can also be stored in an external memory. In this case, the processor 210 can run the code of the text-to-speech synchronization broadcast method stored in the external memory through the external memory interface 220.

[0074] The wireless communication function of the sound bar 200 can be realized through the antenna 1, the antenna 2, the mobile communication module 251, the wireless communication module 252, a modem processor, a baseband processor, and the like.

[0075] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the soundbar 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in combination with a tuning switch.

[0076] The mobile communication module 251 can provide a solution for wireless communication including 3G / 4G / 5G, etc. on the soundbar 200. The mobile communication module 251 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 251 can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed signals to the modem processor for demodulation. The mobile communication module 251 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves radiated by the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 251 can be arranged in the processor 210. In some embodiments, at least part of the functional modules of the mobile communication module 251 and at least part of the modules of the processor 210 can be arranged in the same device. In the embodiments of the present application, the mobile communication module 251 can also be used to interact with other terminal devices or cloud servers.

[0077] The modem processor can include a modulator and a demodulator. The modulator is used to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to the loudspeaker 270A and the microphone 270B, etc.), or displays images or videos through the display screen 294. In some embodiments, the modem processor can be an independent device. In other embodiments, the modem processor can be independent of the processor 210, and arranged in the same device as the mobile communication module 251 or other functional modules.

[0078] The wireless communication module 252 can provide solutions for wireless communication applications on the screened speaker 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 252 can be one or more devices integrating at least one communication processing module. The wireless communication module 252 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 210. The wireless communication module 252 can also receive signals to be transmitted from processor 210, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. In this embodiment, the wireless communication module 252 is used to transmit data with other terminal devices under the control of the processor 210. For example, when the processor 210 runs the cell search method provided in this embodiment, the processor can control the wireless communication module 252 to send service requests to other terminal devices, and can also receive service results provided by other terminal devices based on the service requests. For example, sending a broadcast text request to other terminal devices and receiving broadcast text provided by other terminal devices.

[0079] The speaker 200 with a screen can implement audio functions through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, and an application processor. For example, the playback of the first audio in this embodiment.

[0080] It should be understood that, in practical applications, a screened speaker 200 may include more than Figure 2 The number of more or fewer components shown is not limited in the embodiments of this application. The screened speaker 200 shown is merely an example, and the screened speaker 200 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0081] Figure 3 This diagram illustrates an application scenario of the text-to-speech synchronized broadcasting method provided in an embodiment of this application. For example... Figure 3As shown, the terminal device or the cloud server can provide the announcement text to the announcement device. The announcement device obtains the first text and the first audio based on the announcement text, displays the first text in the display screen, plays the first audio through the sound emitting device, and the first text synchronously scrolls with the first audio, realizing the synchronous announcement of the first text and the first audio.

[0082] In a possible implementation, the user issues a voice instruction of "introducing A scenic spot" to the announcement device. After receiving the voice instruction, the announcement device sends a text request to the cloud server to obtain the introduction information of the A scenic spot. After receiving the text request, the cloud server sends a text request to the content provider. After the content provider finds the introduction information of the A scenic spot, the content provider generates an announcement text according to the introduction information of the A scenic spot and returns the announcement text to the cloud server. The content provider can be one or more, and the announcement text provided by the content provider can include text, pictures, moving pictures, videos and other contents. In an example, the announcement text provided by the content provider can be news, a term, a novel or a paper. The cloud server forwards the received announcement text to the announcement device. The announcement device obtains the first text and the first audio based on the received announcement text, displays the first text in the display screen, plays the first audio through the sound emitting device, and the first text synchronously scrolls with the first audio, realizing the synchronous announcement of the first text and the first audio.

[0083] In another possible implementation, after the terminal device establishes a Bluetooth connection with the announcement device, the user selects an article in the terminal device to perform an announcement operation. The terminal device sends the article selected by the user as an announcement text to the announcement device in response to the announcement operation. The announcement device obtains the first text and the first audio based on the received announcement text, displays the first text in the display screen, plays the first audio through the sound emitting device, and the first text synchronously scrolls with the first audio, realizing the synchronous announcement of the first text and the first audio.

[0084] In the embodiments of the present application, the announcement device can also obtain the announcement text through other manners, which is not limited in the present application. The announcement device can first obtain the announcement text, and obtain the first text and the first audio to be synchronously announced based on the announcement text, or directly obtain the first text and the first audio to be synchronously announced from the cloud server or other terminal devices, which is not limited in the present application.

[0085] Figure 4 A flowchart of a text and voice synchronous announcement method provided by the embodiments of the present application is shown. The method can be applied to an announcement device including a display screen and a sound emitting device, such as the announcement device shown in Figure 1a 、 Figure 1b or Figure 2 . As shown in Figure 4As shown, the text and voice synchronous broadcasting method can include:

[0086] At step S401, in a case where a voice broadcasting request is received, a first text to be broadcast and a first audio are obtained.

[0087] The voice broadcasting request can be used to request the broadcasting device to broadcast the text and the audio synchronously. The first text can represent the text to be broadcast, and the first text can be displayed on a display screen. The first audio can represent the audio to be broadcast, and the first audio can be played on a sound producing device. That is, the voice broadcasting request can be used to request the broadcasting device to broadcast the first text and the first audio synchronously. The first text can include at least one text line, and in this application, the first text can be scrolled from one text line to another text line synchronously with the playing of the first audio.

[0088] In one example, the broadcasting device can generate the voice broadcasting request in response to a voice broadcasting operation (e.g., touching a voice broadcasting control on the display screen) of a user on the broadcasting device, or can generate the voice broadcasting request in response to a voice instruction (e.g., a voice instruction of “introducing A scenic spot”) of a user to the broadcasting device. Accordingly, step S401 can include: in a case where the voice broadcasting request is received, obtaining a broadcasting text from a cloud server; rendering processing the broadcasting text to obtain the first text; and voice synthesis processing the broadcasting text to obtain the first audio.

[0089] In another example, other terminal devices can generate the voice broadcasting request in response to a voice broadcasting operation or a voice instruction of a user, and send the voice broadcasting request to the broadcasting device. Accordingly, step S401 can include: in a case where the voice broadcasting request is received from the terminal device, obtaining a broadcasting text from the voice broadcasting request; rendering processing the broadcasting text to obtain the first text; and voice synthesis processing the broadcasting text to obtain the first audio.

[0090] In implementation, the rendering processing of the broadcasting text includes: loading a font, a font size, and a line spacing, and determining a layout of each element (including text, pictures, moving pictures, and videos, etc.) in the broadcasting text. After the rendering processing of the broadcasting text, the text in the broadcasting text is distributed in one or more text lines, and each text line can include one or more characters. The one or more text lines form the first text.

[0091] In implementation, the process of performing speech synthesis on the announcement text can include: obtaining a pure-text text from the announcement text, performing linguistic analysis on the text by using a neural network (performing lexical, grammatical and semantic analysis on each sentence to determine the low-level structure of the sentence and the uniform composition of each character, including text breaking, word segmentation, processing of multi-syllable characters, processing of numbers, processing of abbreviations, etc.), then extracting the single character or phrase corresponding to the processed text from a speech synthesis library, and converting the linguistic description into speech waveform, thereby obtaining the first audio.

[0092] It should be noted that the above is only an exemplary description of the rendering process and the speech synthesis process of the announcement text. In implementation, other ways can also be used to perform rendering and speech synthesis on the announcement text, which is not limited in the present application.

[0093] Step S402, display the first text and play the first audio, wherein when the first audio is played to a line break character in a text line in the first text, the first text is displayed line by line.

[0094] The first text includes at least one text line, and each text line includes a line break character. The line break character can represent the last voiced character in the text line.

[0095] In the embodiments of the present application, the characters in the first text are divided into voiced characters and unvoiced characters. It can be understood that the voiced characters represent characters that will make a sound when the user is narrating, and the unvoiced characters represent characters that will not make a sound when the user is narrating. For example, the unvoiced characters include annotations and punctuation marks. The annotations refer to the text that explains the words and sentences, and the user does not need to read out the content of the annotations during narration. The annotations can be placed in parentheses within the main text, or identified as "annotation" or "note". Punctuation marks are symbols that assist written language, and are components of written language, used to indicate pauses, tones, and the nature and role of words. Punctuation marks include full stops, question marks, commas, and other point marks, as well as quotation marks, parentheses, and ellipses. Similarly, the user does not need to read out the punctuation marks during narration. The above is only an exemplary description of unvoiced characters and does not limit unvoiced characters.

[0096] For each text line in the first text, the announcement device can determine the last unvoiced character in the text line as the line break character. Figure 5a An exemplary schematic diagram of the first text according to the embodiments of the present application is shown. As shown in FIG. 1, the first text includes three text lines, and the last unvoiced character in each text line is determined as the line break character. Figure 5aAs shown, in the first text, characters such as the comma ",", the semicolon "、", and the full stop "." are non - vocal characters. Based on this, the line - break characters for each text line of the first text can be determined as: "year", "is", "solve", "hold", "in", "solve", and "trend".

[0097] In implementation, the broadcast device can number each character in the first text and add a line - break marker to each character. Among them, a first line - break marker (for example, "T") is added to the line - break characters, and a second line - break marker (for example, "F") is added to non - line - break characters other than the line - break characters. Figure 5b An exemplary schematic diagram of the line - break marker in the embodiment of the present application is shown. As Figure 5b shown, for Figure 5a each character in the first text shown, the corresponding serial number of each character is obtained; for Figure 5a each character in the first text shown, a line - break marker is added to obtain the corresponding line - break marker for each character. In this way, based on the serial number corresponding to the character and the line - break marker corresponding to the character, the broadcast device can determine the serial number corresponding to each line - break character in the first text, that is, the position of each line - break character in the first text.

[0098] In implementation, the broadcast device can display the first text on the display screen and play the first audio through the sound - emitting device. When the first audio is played to the line - break character of the first text line, the first text on the display screen is displayed in a new line; when the first audio is played to the line - break character of the second text line in the first text, the first text on the display screen is displayed in a new line again; and so on, which will not be elaborated here. As Figure 1b shown, when the first audio played by the sound - emitting device is played to the character "year", the first text shown on the display screen scrolls from the text line "Huawei Technologies Co., Ltd., founded in 1987," to the text line "Headquartered in Longgang District, Shenzhen, Guangdong Province. Huawei", achieving the new - line display of the first text.

[0099] In the embodiment of the present application, the last vocal character in the text line is used as the line - break character. By linking the moment when the voice broadcasts the line - break character with the moment when the line - break character is displayed, the synchronization of voice playback and text scrolling is achieved, so that in a scenario where the broadcast content and playback duration are uncertain, the voice - broadcast content is consistent with the content displayed on the display screen.

[0100] In a possible implementation manner, the new - line display of the first text line in step S402 may include: the first text line scrolls from the text line to the text line where the first vocal character after the text line in the first text is located.

[0101] The first text can jump over the text lines that do not include the voiced character and directly scroll to the text line that includes the voiced character when the first text is displayed line by line. For example, the first text includes, in sequence from front to back, text line 1, text line 2, text line 3, text line 4 and text line 5. If text line 1, text line 2, text line 3 and text line 4 include the voiced character, when the first audio is played to the line break character of text line 1, the first text is scrolled from text line 1 to text line 2. If text line 1 and text line 3, text line 4 include the voiced character, and text line 2 does not include the voiced character, when the first audio is played to the line break character of text line 1, the first text is scrolled from text line 1 to text line 3. If text line 1 and text line 4 include the voiced character, and text line 2 and text line 3 do not include the voiced character, when the first audio is played to the line break character of text line 1, the first text is scrolled from text line 1 to text line 4.

[0102] In one example, the broadcasting device can also mark the number of scrolled lines (i.e., the number of lines that need to be line broken) when adding the line break mark to the line break character in the first text. For example, the line break mark of the line break character can be marked as T(n+1), where n is the number of skipped text lines, and n+1 is the number of scrolled lines. For example, if the text line where the line break character is located is the first text line, the number of skipped text lines is 0, and the number of scrolled lines is 1, i.e., the second text line is scrolled to. If the number of skipped text lines is 1, the number of scrolled lines is 2, i.e., the third text line is scrolled to. If the number of skipped text lines is 2, the number of scrolled lines is 3, i.e., the fourth text line is scrolled to.

[0103] In view of the possible delay between the time when the first text is displayed and the time when the first audio is played, in order to realize the letter and voice synchronous broadcasting, in one possible implementation, the time when the first text is displayed and the time when the first audio is played can be aligned. Figure 6 A flowchart of a letter and voice synchronous broadcasting method provided by an embodiment of the present application is shown. As shown in Figure 6 As shown in Figure 4 The step S402 can include steps S4021 to S4025.

[0104] In step S4021, the first text is displayed, and the first audio is played.

[0105] In step S4022, a first system timestamp when the first text is displayed and a second system timestamp when the first audio is played are recorded.

[0106] The first system timestamp can be used to represent a system time corresponding to a time when the first text is displayed, and the second system timestamp can be used to represent a system time corresponding to a time when the first audio is played. In an example, the broadcasting device can display the first text when the rendering processing is completed, and play the first audio when the speech synthesis processing is completed. Since the time used for the rendering processing and the time used for the speech synthesis processing can be different, there can be a deviation between the time when the first text is displayed and the time when the first audio is played. The first system timestamp can be before the second system timestamp, or the first system timestamp can be after the second system timestamp, which is not limited in this application. For example, the first system timestamp is 1604028726782 ms, and the second system timestamp is 1604028726556 ms, and the deviation between the two is 226 ms.

[0107] In step S4023, a line break character in a text line in the first text is determined to correspond to a first time in the first audio.

[0108] The first time can represent a time when the sound of the line break character ends in the playing process of the first audio. That is, when the playing time of the first audio reaches the first time, it indicates that the sounder device ends the pronunciation of the line break character.

[0109] In step S4024, according to the first system timestamp, the second system timestamp, and the first time corresponding to the line break character in the first audio, a line break time corresponding to the line break character in the display process of the first text is determined.

[0110] The line break time can represent a time when the display of the line break character ends in the display process of the first text. That is, when the display time of the first text reaches the line break time, it indicates that the first text displayed on the display screen needs to be scrolled. In an example, the line break time corresponding to the line break character=(second system timestamp-first system timestamp)+first time corresponding to the line break character.

[0111] In step S4025, when the display time of the first text reaches the line break time, the first text is controlled to be displayed by line break.

[0112] In the embodiments of the present application, by aligning the time when the first text is displayed with the time when the first audio is played, and linking the time when the playing of the line break character in the first audio ends with the time when the display of the line break character in the display process of the first text ends, the synchronization of voice playing and text scrolling is realized.

[0113] Figure 7a A schematic diagram showing the process of aligning the first text with the first audio in the time axis in the embodiments of the present application is shown. As Figure 7aAs shown, the broadcast device obtains the first text after rendering the broadcast text, and starts to display the first text. The broadcast device obtains the first audio after performing speech synthesis on the broadcast text, and starts to play the first audio. The time when the display of the first text starts is recorded as a first system timestamp, and the time when the playing of the first audio starts is recorded as a second system timestamp. The offset between the first text and the first audio = the second system timestamp - the first system timestamp.

[0114] In the embodiment of the present application, in order to make the playing progress of the first audio consistent with the display progress of the first text, the first time corresponding to the line break character in the first audio can be mapped to the line break time corresponding to the line break character in the display process of the first text. For example, Figure 7a As shown, the offset between the first text and the first audio = the line break time - the first time.

[0115] Therefore, the line break time corresponding to the line break character = (the second system timestamp - the first system timestamp) + the first time corresponding to the line break character. When the display duration of the first text reaches the line break time, the playing duration of the first audio also reaches the first time. At this time, the broadcast device can determine that the line break character has been played, and can control the line break display of the first text.

[0116] Figure 7b An exemplary schematic diagram of the line break time in the embodiment of the present application is shown. As shown, Figure 7b The characters with serial numbers 7, 13 and 24 in the first text are line break characters, and the first times corresponding to these line break characters are 801 ms, 1770 ms and 2659 ms respectively. The first system timestamp is 1604028726782 ms, and the second system timestamp is 1604028726556 ms. The offset between the two is -226 ms. Therefore, the line break time corresponding to the line break character with serial number 7 is ( -226 ms) + 801 ms = 575 ms, the line break time corresponding to the line break character with serial number 13 is ( -226 ms) + 1770 ms = 1544 ms, and the line break time corresponding to the line break character with serial number 24 is ( -226 ms) + 2659 ms = 2433 ms. When the display duration of the first text reaches 575 ms, it indicates that the playing progress of the first audio reaches 801 ms, and the line break character with serial number 7 has been played. At this time, the broadcast device can control the line break display of the first text. When the display duration of the first text reaches 1544 ms, it indicates that the playing progress of the first audio reaches 1770 ms, and the line break character with serial number 13 has been played. At this time, the broadcast device can control the line break display of the first text. When the display duration of the first text reaches 2433 ms, it indicates that the playing progress of the first audio reaches 2659 ms, and the line break character with serial number 24 has been played. At this time, the broadcast device can control the line break display of the first text.

[0117] In a possible implementation, the line break moment corresponding to the line break character is recorded in the scroll control sequence. After determining the line break moment corresponding to the line break character, the line break moment corresponding to the line break character can be added to the tail of the scroll control sequence; and after controlling the line break display of the first text, the line break moment corresponding to the first text is popped from the head of the scroll control sequence. In this way, the broadcasting device can control the line break display of the first text based on the scroll control sequence; meanwhile, the line break moment can be determined along with the processing progress of the first audio, that is, the progress of determining the first moment in the first audio, to realize parallel processing and playing, thereby saving time.

[0118] The process of determining the first moment in the first audio corresponding to the line break character of a text line in the first text in step S4023 is described below.

[0119] In a possible implementation, step S4023 can include: splitting the second audio from the first audio; determining the second moment in the second audio corresponding to the line break character; and determining the first moment in the first audio corresponding to the line break character according to the second moment in the second audio corresponding to the line break character and the position of the second audio in the first audio.

[0120] The second audio can represent an audio split from the first audio. The whole or part of the first audio can be the second audio. In an example, the broadcasting device can determine the first audio as the second audio. In another example, the broadcasting device can detect the amplitude of the first audio; and in a case where the amplitude of the first audio is less than a first threshold value and the duration is greater than a second threshold value, the second audio is split from the first audio. At this time, multiple second audios can be split from the first audio. The broadcasting device can process each second audio synchronously to realize segmented processing of the first audio, thereby improving the processing efficiency.

[0121] The second moment can represent a moment corresponding to the end of playing of the line break character in the second audio. When the playing progress of the second audio reaches the second moment, it indicates that the line break character corresponding to the second moment has been played and the first text can be controlled to be displayed in lines. According to the second moment corresponding to the line break character in the second audio and the position of the second audio in the first audio, the first moment corresponding to the line break character in the first audio can be determined. For example, the second audio is completely the same as the first audio, and the second moment corresponding to the line break character in the second audio is the same as the first moment corresponding to the line break character in the first audio. For another example, the second audio corresponds to an audio segment of 200 ms to 1500 ms in the first audio, and if the second moment corresponding to the line break character in the second audio is moment 1, the first moment corresponding to the line break character in the first audio is moment 1+200 ms.

[0122] The process of determining the second moment corresponding to the line break character in the second audio is described below.

[0123] In a possible implementation, the determining the second moment corresponding to the line break character in the second audio includes: generating at least one character sequence based on the non-sound character in the first text, the non-sound character not being included in the character sequence; determining a character sequence corresponding to the second audio in the at least one character sequence; adjusting the amplitude envelope of the second audio until the number of wave crests on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio; determining a wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio according to the position of the line break character in the character sequence corresponding to the second audio; and determining a moment corresponding to a wave trough adjacent to the wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio as the second moment corresponding to the line break character in the second audio.

[0124] In implementation, if all of the first audio is taken as the second audio, the playing device can eliminate the non-sound character in the first text to obtain a character sequence. Thus, a corresponding relationship between the second audio and the character sequence is established.

[0125] In implementation, if part of the first audio is taken as the second audio, the playing device can split the first text into a plurality of second texts based on the punctuation marks in the first text, and eliminate the annotations in the second texts to obtain a plurality of character sequences. Thus, for each second audio, a corresponding relationship between the second audio and a character sequence is established.

[0126] In consideration of the fact that the user will not make a sound and will pause when encountering a punctuation mark during the narration, the broadcasting device can determine that a punctuation mark is present when it detects that the amplitude of the first audio is less than a first threshold value and the duration is greater than a second threshold value. Based on this, the broadcasting device splits the first audio into a plurality of second audios. At the same time, the broadcasting device splits the first text into a plurality of second texts based on the punctuation marks in the first text. In this way, the split second audios and the split second texts form a corresponding relationship. The first threshold value can be set as needed. In an example, the first threshold value can be 0. The second threshold value can be set as needed. In an example, the second threshold value can be 500 ms.

[0127] In an embodiment of the present application, the character sequence does not include punctuation marks and non-sound characters such as annotations. After determining the correspondence between the second audio and the character sequence, the broadcasting device can adjust the amplitude envelope of the second audio until the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio.

[0128] It can be understood that the sound-emitting characters correspond to the peaks of the amplitude envelope. Therefore, when the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio, it can be determined that each character in the character sequence corresponds to a peak. Since the position of the line break character in the character sequence is known, the peak corresponding to the line break character in the character sequence can be determined. The next valley adjacent to the peak corresponding to the line break character on the amplitude envelope of the second audio is the time when the line break character finishes pronouncing. Therefore, the time corresponding to the next valley adjacent to the peak corresponding to the line break character on the amplitude envelope of the second audio can be determined as the second time corresponding to the line break character in the second audio.

[0129] The process of adjusting the amplitude envelope of the second audio is described below.

[0130] In a possible implementation, the adjusting the amplitude envelope of the second audio until the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio comprises: performing low-pass filtering on the second audio based on a cutoff frequency to obtain third audio; when the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio, increasing the cutoff frequency; when the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold, decreasing the cutoff frequency, where the third threshold is greater than the number of characters in the character sequence corresponding to the second audio; when the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold, sequentially removing peaks on the amplitude envelope of the third audio in order of the difference between the peaks and adjacent valleys from small to large, until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio; and when the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio, determining the amplitude envelope of the third audio as the adjusted amplitude envelope of the second audio.

[0131] Although the second audio is artificially synthesized audio, high-frequency components still exist. Therefore, in the embodiment of the present application, the second audio is low-pass filtered based on a cutoff frequency to obtain third audio. The cutoff frequency can be set and adjusted as needed.

[0132] When the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio, it indicates that some of the peaks corresponding to the characters are removed, and at this time the cutoff frequency can be increased to make the amplitude envelope of the third audio have more peaks.

[0133] When the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold, it indicates that the number of peaks on the amplitude envelope of the third audio is much greater than the number of corresponding characters, and at this time the cutoff frequency can be decreased to make the amplitude envelope of the third audio have fewer peaks. The third threshold is greater than the number of characters in the character sequence corresponding to the second audio.

[0134] When the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold value, it is indicated that, although the number of peaks on the amplitude envelope of the third audio is greater than the number of corresponding characters, the difference is small and within an acceptable range. At this time, the excessive peaks can be removed so that the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio.

[0135] In the process of removing the peaks, the peaks can be removed in order of the amplitude difference between the peaks and adjacent valleys on the amplitude envelope of the third audio from small to large, until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio. Since the greater the amplitude difference between the peaks and the valleys, the higher the probability that a peak corresponds to a character, therefore, removing the peaks in order of the amplitude difference between the peaks and adjacent valleys on the amplitude envelope of the third audio from small to large can reduce the possibility of mistakenly removing the peaks, thereby improving the accuracy of matching the peaks and the characters.

[0136] When the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio, it is indicated that the envelope adjustment has been completed, and at this time, the amplitude envelope of the third audio can be determined as the amplitude envelope of the adjusted second audio, and then the second time corresponding to each line break character in the second audio is determined.

[0137] Figure 8 A flowchart of a text and voice synchronous broadcasting method provided by an embodiment of the present application is shown. The text and voice synchronous broadcasting method can be applied to a control unit in a broadcasting device (for example, the broadcasting device shown in the figure). Figure 1a As shown in the figure, the text and voice synchronous broadcasting method can include the following steps. Figure 8

[0138] In step S501, the display control module controls the display screen to display the first text.

[0139] In step S502, the display control module sends a first system timestamp of starting to display the first text to the line break control module.

[0140] In step S503, the playing control module controls the sound emitting device to play the first audio.

[0141] In step S504, the playing control module sends a second system timestamp of starting to play the first audio to the line break control module.​

[0142] In the embodiments of the present application, the execution order of step S501 and step S503 is not limited. Step S501 can be executed before, after or at the same time as step S503. In an example, step S501 can be executed when the rendering process on the entire or part of the broadcast text is completed. In an example, step S503 can be executed when the speech synthesis process on the entire or part of the broadcast text is completed. It can be understood that step S501 is executed when the rendering process on part of the broadcast text is completed, and step S503 is executed when the speech synthesis process on part of the broadcast text is completed, which can realize processing while loading and is beneficial to improving efficiency.

[0143] In step S505, the line break control module determines the offset according to the first system timestamp and the second system timestamp.

[0144] In step S506, the display control module splits the first text into at least one character sequence and determines the line break character in the character sequence.

[0145] The character sequence does not include non-sound characters. The manner of obtaining the character sequence can refer to the foregoing, which will not be described here.

[0146] In step S507, the display control module sends the position of the character sequence in the first text and the position of the line break character in the character sequence to the line break control module.

[0147] The position of the character sequence in the first text can be represented by the sequence number of the character in the character sequence in the first text. Taking the first text shown in FIG. 8 as an example, the character sequences split from the first text include: “Huawei Technologies Co., Ltd.”, “founded in 1987”, “headquartered in Longgang District, Shenzhen City, Guangdong Province” and “Huawei is a global leading information and communication technology” and the like. The first character sequence “Huawei Technologies Co., Ltd.” in the first text can be represented by the sequence number 0, the sequence number 1 and the sequence number 2 to the sequence number 7, the second character sequence “founded in 1987” can be represented by the sequence number 8 to the sequence number 16. The position of the line break character in the character sequence can be represented by the sequence number corresponding to the line break character, for example, the line break character “year” can be represented by the sequence number 16. Figure 5a Figure 5b The position of the character sequence in the first text can be represented by the sequence number of the character in the character sequence in the first text. Taking the first text shown in FIG. 8 as an example, the character sequences split from the first text include: “Huawei Technologies Co., Ltd.”, “founded in 1987”, “headquartered in Longgang District, Shenzhen City, Guangdong Province” and “Huawei is a global leading information and communication technology” and the like. The first character sequence “Huawei Technologies Co., Ltd.” in the first text can be represented by the sequence number 0, the sequence number 1 and the sequence number 2 to the sequence number 7, the second character sequence “founded in 1987” can be represented by the sequence number 8 to the sequence number 16. The position of the line break character in the character sequence can be represented by the sequence number corresponding to the line break character, for example, the line break character “year” can be represented by the sequence number 16.

[0148] In step S508, the playing control module splits the second audio from the first audio.

[0149] In step S509, the playing control module sends the position of the second audio in the first audio to the line break control module.

[0150] ​In implementation, the position of the second audio in the first audio can be represented by the corresponding start time and end time of the second audio in the first audio.

[0151] In step S510, the line break control module determines the character sequence corresponding to the second audio according to the position of the character sequence in the first text and the position of the second audio in the first audio.

[0152] In one example, the position of the character sequence in the first text can be represented as: first character sequence, second character sequence, etc., and the position of the second audio in the first audio can be represented as: first audio segment, second audio segment, etc. In the embodiments of the present application, the first audio segment corresponds to the first character sequence, and the second audio segment corresponds to the second character sequence. In implementation, each time a second audio is split from the first audio, the character sequence corresponding to the second audio can be determined, and subsequent processing can be performed. In this way, on-the-fly processing can be achieved, and efficiency can be improved.

[0153] In step S511, the line break control module determines the second time corresponding to the line break character in the second audio according to the position of the line break character in the character sequence.

[0154] In step S512, the line break control module determines the first time corresponding to the line break character in the first audio according to the second time corresponding to the line break character in the second audio and the position of the second audio in the first audio.

[0155] In step S513, the line break control module determines the line break time corresponding to the line break character in the first text display process according to the offset and the first time corresponding to the line break character in the first audio.

[0156] In step S514, the line break control module adds the line break time corresponding to the line break character to the tail of the scrolling control sequence.

[0157] In one example, the scrolling control sequence can include: line break time 1, line break time 2, line break time 3. As the second audio is processed, line break times are constantly added to the scrolling control sequence, and are sequentially added to the tail of the scrolling control sequence.

[0158] In step S515, the line break control module takes the head of the scrolling control sequence.

[0159] In step S516, the line break control module determines whether the display duration of the first text reaches the line break time indicated by the head of the scrolling control sequence. If yes, the scrolling control sequence performs steps S517 and S518; otherwise, step S515 is performed.

[0160] In step S517, the line break control module sends a line break signal to the display control module.

[0161] Step S518, the line control module pops the first line break character of the line control sequence.

[0162] Step S519, the line control module determines whether the line control sequence is empty. If yes, the line control module ends the flow; otherwise, step S515 is performed.

[0163] Step S520, the display control module waits for a line break, and when receiving a line break signal, controls the display screen to display the first text in line.

[0164] Step S521, the playing control module determines whether the first audio is played completely. If yes, the playing control module ends the flow; otherwise, step S508 is performed.

[0165] In the embodiment of the present application, the time of playing the line break character by voice and the time of displaying the line break character are associated, so that the text can be scrolled synchronously with the voice.

[0166] Figure 9 A structure diagram of a playing device provided by an embodiment of the present application is shown. The device can be used to execute the method shown in Figure 4 , Figure 6 or Figure 8 . As shown in Figure 9 , the device 90 comprises: an acquisition module 91, configured to obtain a first text to be played and a first audio in the case of receiving a voice playing request, the first text comprising at least one text line; a display module 92, configured to display the first text obtained by the acquisition module 91 and play the first audio obtained by the acquisition module 91, wherein the first text is displayed in line when the first audio is played to a line break character in a text line of the first text, and the line break character represents the last voiced character in the text line.

[0167] In the embodiment of the present application, the last voiced character in a text line is taken as a line break character, the time of playing the line break character by voice and the time of displaying the line break character are associated, so that the voice playing and the text scrolling are synchronized, and thus in the scene of indefinite playing time and playing content, the voice playing content and the display content of the display screen are kept consistent.

[0168] In a possible implementation, the apparatus further includes: a recording module, configured to record a first system timestamp of starting to display the first text and a second system timestamp of starting to play the first audio; a first determining module, configured to determine a first time corresponding to the line break character in the first audio; a second determining module, configured to determine a line break time corresponding to the line break character in the display of the first text according to the first system timestamp, the second system timestamp, and the first time corresponding to the line break character in the first audio; and a control module, configured to control the line break display of the first text when a display duration of the first text reaches the line break time.

[0169] In a possible implementation, the first determining module is further configured to: split a second audio from the first audio; determine a second time corresponding to the line break character in the second audio; and determine the first time corresponding to the line break character in the first audio according to the second time corresponding to the line break character in the second audio and a position of the second audio in the first audio.

[0170] In a possible implementation, the determination of the second time corresponding to the line break character in the second audio includes: generating at least one character sequence based on a non-pronounced character in the first text, the character sequence not including the non-pronounced character; determining a character sequence corresponding to the second audio in the at least one character sequence; adjusting an amplitude envelope of the second audio until a number of peaks on the amplitude envelope of the second audio is the same as a number of characters in the character sequence corresponding to the second audio; determining a peak corresponding to the line break character on the adjusted amplitude envelope of the second audio according to a position of the line break character in the character sequence corresponding to the second audio; and determining, as the second time corresponding to the line break character in the second audio, a time corresponding to a valley adjacent to the peak corresponding to the line break character on the adjusted amplitude envelope of the second audio.

[0171] In a possible implementation, the splitting of the second audio from the first audio includes: determining the first audio as the second audio; and the generation of the at least one character sequence based on the non-pronounced character in the first text includes: removing the non-pronounced character in the first text to obtain a character sequence.

[0172] In a possible implementation, the non-pronounced character includes an annotation and a punctuation mark.

[0173] In a possible implementation, the splitting the second audio from the first audio comprises: detecting an amplitude of the first audio; splitting the second audio from the first audio in a case that the amplitude of the first audio is less than a first threshold and a duration is greater than a second threshold; the generating the at least one character sequence based on the non-sounding character in the first text comprises: splitting the first text into a plurality of second texts based on punctuation marks in the first text; and removing notes in the second texts to obtain the plurality of character sequences.

[0174] In a possible implementation, the adjusting the amplitude envelope of the second audio until a number of peaks on the amplitude envelope of the second audio is the same as a number of characters in the character sequence corresponding to the second audio comprises: performing low-pass filtering on the second audio based on a cutoff frequency to obtain a third audio; increasing the cutoff frequency in a case that the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio; decreasing the cutoff frequency in a case that the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and a difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold, where the third threshold is greater than the number of characters in the character sequence corresponding to the second audio; in a case that the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold, sequentially removing peaks on the amplitude envelope of the third audio in an order from small to large of amplitude differences between the peaks and adjacent valleys until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio; and determining the amplitude envelope of the third audio as the adjusted amplitude envelope of the second audio in a case that the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio.

[0175] In a possible implementation, the obtaining module is further configured to: obtain, from a cloud server, a broadcast text in a case that a voice broadcast request is received; perform rendering processing on the broadcast text to obtain the first text; and perform voice synthesis processing on the broadcast text to obtain the first audio.

[0176] In a possible implementation, the line break moment corresponding to the line break character is recorded in the scroll control sequence; the device further includes an adding module, configured to add the line break moment corresponding to the line break character to the tail of the scroll control sequence after determining the line break moment corresponding to the line break character; and a popping module, configured to pop the line break moment corresponding to the line break moment of the first text from the head of the scroll control sequence after controlling the first text to perform line break display.

[0177] In a possible implementation, the first text line break display includes that the first text is scrolled by the text line to a text line where a first voiced character after the text line in the first text is located.

[0178] Embodiments of the present application provide a reporting device, including a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions.

[0179] Embodiments of the present application provide a non-volatile computer-readable storage medium having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the above method.

[0180] Embodiments of the present application provide a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above method.

[0181] Computer readable storage media can be tangible storage devices that can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital video disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium can be any tangible storage device that can retain and store instructions for use by an instruction execution device.

[0182] Computer readable program instructions or code can be downloaded to a starting device, to another computer or device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A remote server can store software programs, code or programs designed to carry out the processes described herein and provide the software to the user device(s) either for processing or installing. A remote server can transmits the software programs, code or programs to the user device(s) over the network. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0183] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0184] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0185] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0186] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0187] The flow diagrams and the block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved.

[0188] It also is noted that each of the blocks of the flowchart, and / or combinations of blocks, can be implemented by hardware, by software including one or more computer program instructions, by firmware, or by a combination of hardware, software, and / or firmware. Also, the flowcharts and / or block diagrams can be implemented in hardware, in software, or in a combination of hardware and software.

[0189] While the application has been described in connection with various embodiments thereof, modifications can occur to those skilled in the art upon reading this description. For example, it can be noted that the application can be applied to any type of computer system or environment having any type of architecture. Some embodiments of the application can have different functionality than that described herein, but achieve the same endpoint. Many modifications can be made to carry out the application in its various aspects, and such are therefore, considered to be a part of the overall technology, nonetheless. None, therefore, should be taken to be a limitation on the overall scope of the application. Specifications and details can be deemed to be superfluous if modified or added in accordance with the given application. Accordingly, the applicant(s) intend that only such limitations be imposed on the application that are imposed by the law.

[0190] Having described various embodiments of the application, it is to be understood that the above description is meant not to limit and not to encompass all of the possible embodiments. Many modifications and variations of this application can be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. It is intended that the scope of the application be defined by the scope of the patent and by the claims as allowed by the patent office, which can include adaptations based on the description, equivalents, and / or substitutions of elements individually or collectively to the entire disclosure.

Claims

1. A method for simultaneous text and voice broadcasting, characterized in that, The method comprises: In the case of receiving a voice broadcast request, obtaining a first text to be broadcast and a first audio, the first text comprising at least one text line; Displaying the first text and playing the first audio, wherein when the first audio is played to a line break character in a text line in the first text, the first text is displayed line by line, and the line break character represents the last voiced character in the text line; The method further comprises: Recording a first system timestamp at which the first text is started to be displayed, and a second system timestamp at which the first audio is started to be played; Determining a first time corresponding to the line break character in the first audio; According to the first system timestamp, the second system timestamp, and the first time corresponding to the line break character in the first audio, determining a line break time corresponding to the line break character in the display process of the first text; When the display duration of the first text reaches the line break time, controlling the first text to be displayed line by line.

2. The method of claim 1, wherein, The determination of the first time corresponding to the line break character in the first audio comprises: Splitting a second audio from the first audio; Determining a second time corresponding to the line break character in the second audio; According to the second time corresponding to the line break character in the second audio and the position of the second audio in the first audio, determining the first time corresponding to the line break character in the first audio.

3. The method of claim 2, wherein, The determination of the second time corresponding to the line break character in the second audio comprises: Generating at least one character sequence based on the non-voiced characters in the first text, the character sequence not including the non-voiced characters; In the at least one character sequence, determining a character sequence corresponding to the second audio; Adjusting the amplitude envelope of the second audio until the number of wave crests on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio; According to the position of the line break character in the character sequence corresponding to the second audio, determining a wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio; Determining, as the second time corresponding to the line break character in the second audio, a time corresponding to a wave trough adjacent to the wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio.

4. The method of claim 3, wherein, The splitting of the second audio from the first audio comprises: Determining the first audio as the second audio. The generation of at least one character sequence based on the non-voiced characters in the first text comprises: Removing the non-voiced characters in the first text to obtain a character sequence.

5. The method according to claim 3 or 4, characterized in that, The non-voiced characters include annotations and punctuation marks.

6. The method of claim 3, wherein, The splitting of the second audio from the first audio comprises: Detecting the amplitude of the first audio; In the case that the amplitude of the first audio is less than a first threshold value and the duration is greater than a second threshold value, splitting the second audio from the first audio. The generation of at least one character sequence based on the non-voiced characters in the first text comprises: Splitting the first text into multiple second texts based on punctuation marks in the first text; The second text is pruned to obtain a plurality of character sequences.

7. The method of claim 3, wherein, The adjusting of the amplitude envelope of the second audio is performed until the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio, including: The second audio is low-pass filtered based on a cutoff frequency to obtain third audio; When the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio, the cutoff frequency is increased; When the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold, the cutoff frequency is decreased, wherein the third threshold is greater than the number of characters in the character sequence corresponding to the second audio; When the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold, the peaks are sequentially removed in order of amplitude difference between the peaks and adjacent valleys on the amplitude envelope of the third audio from small to large until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio; When the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio, the amplitude envelope of the third audio is determined as the adjusted amplitude envelope of the second audio.

8. The method of claim 1, wherein, The obtaining of the first text and the first audio in the case that the voice broadcast request is received includes: In the case that the voice broadcast request is received, the broadcast text is obtained from a cloud server; The broadcast text is rendered to obtain the first text; The broadcast text is synthesized to obtain the first audio.

9. The method of claim 1, wherein, The line break time corresponding to the line break character is recorded in the scroll control sequence; After determining the line break time corresponding to the line break character, the method further includes: The line break time corresponding to the line break character is added to the tail of the scroll control sequence; After controlling the first text to be displayed in lines, the method further includes: The line break time corresponding to the line break display of the first text is popped from the head of the scroll control sequence.

10. The method of claim 1, wherein, The first text is displayed in lines includes: The first text is scrolled by the text line to the text line where the first character that pronounces after the text line.

11. An announcement device characterized by comprising: The device includes: An obtaining module is configured to obtain a first text and a first audio in the case that a voice broadcast request is received, the first text including at least one text line. The display module is configured to display the first text and play the first audio, wherein when the first audio plays to a line break character in a text line of the first text, the first text is displayed by line break, and the line break character represents a last voiced character in the text line; The device further comprises: The recording module is configured to record a first system timestamp of starting to display the first text and a second system timestamp of starting to play the first audio; The first determining module is configured to determine a first time corresponding to the line break character in the first audio; The second determining module is configured to determine a line break time corresponding to the line break character in the display process of the first text according to the first system timestamp, the second system timestamp, and the first time corresponding to the line break character in the first audio; The control module is configured to control the first text to be displayed by line break when a display duration of the first text reaches the line break time.

12. The apparatus of claim 11, wherein, The first determining module is further configured to: split a second audio from the first audio; determine a second time corresponding to the line break character in the second audio; determine the first time corresponding to the line break character in the first audio according to the second time corresponding to the line break character in the second audio and a position of the second audio in the first audio.

13. The apparatus of claim 12, wherein, The determination of the second time corresponding to the line break character in the second audio comprises: generate at least one character sequence based on unvoiced characters in the first text, the character sequence not including the unvoiced characters; determine a character sequence corresponding to the second audio in the at least one character sequence; adjust an amplitude envelope of the second audio until a number of wave crests on the amplitude envelope of the second audio is the same as a number of characters in the character sequence corresponding to the second audio; determine a wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio according to a position of the line break character in the character sequence corresponding to the second audio; determine the second time corresponding to the line break character in the second audio as a time corresponding to a next wave trough adjacent to the wave crest corresponding to the line break character on the adjusted amplitude envelope of the second audio.

14. The apparatus of claim 13, wherein, The splitting of the second audio from the first audio comprises: determining the first audio as the second audio. The generation of the at least one character sequence based on the unvoiced characters in the first text comprises: obtaining a character sequence by removing the unvoiced characters in the first text.

15. The apparatus of claim 13 or 14, wherein, The unvoiced characters include annotations and punctuation marks.

16. The apparatus of claim 13, wherein, The splitting of the second audio from the first audio comprises: detecting an amplitude of the first audio; splitting the second audio from the first audio in a case where the amplitude of the first audio is less than a first threshold value and a duration is greater than a second threshold value. The generation of the at least one character sequence based on the unvoiced characters in the first text comprises: splitting the first text into a plurality of second texts based on punctuation marks in the first text; obtaining a plurality of character sequences by removing annotations in the second texts.

17. The apparatus of claim 13, wherein, The adjusting of the amplitude envelope of the second audio is performed until the number of peaks on the amplitude envelope of the second audio is the same as the number of characters in the character sequence corresponding to the second audio, including: performing low-pass filtering on the second audio based on a cutoff frequency to obtain third audio; when the number of peaks on the amplitude envelope of the third audio is less than the number of characters in the character sequence corresponding to the second audio, increasing the cutoff frequency; when the number of peaks on the amplitude envelope of the third audio is greater than or equal to the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is greater than a third threshold value, wherein the third threshold value is greater than the number of characters in the character sequence corresponding to the second audio, decreasing the cutoff frequency; when the number of peaks on the amplitude envelope of the third audio is greater than the number of characters in the character sequence corresponding to the second audio, and the difference between the number of peaks on the amplitude envelope of the third audio and the number of characters in the character sequence corresponding to the second audio is less than or equal to the third threshold value, sequentially removing peaks in order of amplitude difference between adjacent peaks and valleys on the amplitude envelope of the third audio from small to large until the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio; when the number of peaks on the amplitude envelope of the third audio is equal to the number of characters in the character sequence corresponding to the second audio, determining the amplitude envelope of the third audio as the adjusted amplitude envelope of the second audio.

18. The apparatus of claim 11, wherein, The obtaining module is further configured to: obtain the announcement text from the cloud server in a case where a voice announcement request is received; perform rendering processing on the announcement text to obtain the first text; perform voice synthesis processing on the announcement text to obtain the first audio.

19. The apparatus of claim 11, wherein, The line feed time corresponding to the line feed character is recorded in the scroll control sequence; The device further includes: an adding module configured to add the line feed time corresponding to the line feed character to the tail of the scroll control sequence after the line feed time corresponding to the line feed character is determined; a popping module configured to pop the line feed time corresponding to the line feed of the first text from the head of the scroll control sequence after the first text is controlled to be displayed in lines.

20. The apparatus of claim 11, wherein, The first text is displayed in lines, including: the text line is scrolled to a text line where a first character sounding after the text line in the first text.

21. An announcement device, characterized by including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method of any one of claims 1 to 10 when executing the instructions.

22. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 10.

23. A computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in an electronic device, a processor in the electronic device performs the method of any one of claims 1 to 10.

Images