Voice broadcast method and device based on streaming text, electronic equipment and medium

Abstract

The invention provides a voice broadcast method and device based on a streaming text, electronic equipment, a computer readable storage medium and a computer program product, and relates to the field of artificial intelligence, in particular to the technical field of natural language processing, digital human, voice broadcast, intelligent agents and generative search. According to the implementation scheme, one or more text blocks which are output in a streaming mode and are not subjected to voice broadcast are obtained, wherein the one or more text blocks are at least one part of a text; character recognition is carried out on the one or more text blocks in sequence to recognize preset punctuation marks in the text blocks; in response to the fact that the preset punctuation mark is recognized, it is determined that the one or more text blocks comprise special texts before the preset punctuation mark, first texts except the special texts in the one or more text blocks before the preset punctuation mark are obtained to serve as to-be-broadcast texts, and the special texts are preset texts which are not suitable for voice broadcast; and inputting the text to be broadcasted into a voice broadcaster for voice broadcasting.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the fields of natural language processing, digital humans, voice broadcasting, intelligent agents, and generative search technology, and specifically to a voice broadcasting method, device, electronic device, computer-readable storage medium, and computer program product based on streaming text. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] With the development of artificial intelligence technology, speech synthesis technology has been widely used in various scenarios, such as digital humans, smart assistants, voice navigation, etc. Traditional speech synthesis methods generate audio data based on text and play it. When processing long texts or real-time voice broadcasts, there are often delays and incoherence problems. Summary of the invention

[0004] The present disclosure provides a streaming text-based voice broadcast method, device, electronic device, computer-readable storage medium, and computer program product.

[0005] According to one aspect of the present disclosure, a method for voice broadcasting based on streaming text is provided, comprising: obtaining one or more text blocks of streaming output that have not yet been voice broadcasted, wherein the one or more text blocks are at least a part of a text; performing character recognition on the one or more text blocks in sequence to identify preset punctuation marks in the one or more text blocks; in response to identifying the preset punctuation marks and determining that the one or more text blocks include special text before the preset punctuation marks, obtaining a first text other than the special text before the preset punctuation marks in the one or more text blocks to use the first text as a text to be broadcasted, wherein the special text is a preset text that is not suitable for voice broadcasting; and inputting the text to be broadcasted into a voice broadcaster for voice broadcasting.

[0006] According to another aspect of the present disclosure, a speech broadcast device based on streaming text is provided, comprising: an acquisition unit, configured to acquire one or more text blocks that are streamed and have not yet been speech broadcast, wherein the one or more text blocks are at least a part of a text; a recognition unit, configured to perform character recognition on the one or more text blocks in sequence to recognize preset punctuation marks in the one or more text blocks; a first determination unit, configured to, in response to recognizing the preset punctuation marks and determining that the one or more text blocks include special text before the preset punctuation marks, acquire a first text other than the special text before the preset punctuation marks in the one or more text blocks, so as to use the first text as a text to be broadcast, wherein the special text is a preset text that is not suitable for speech broadcast; and a broadcast unit, configured to input the text to be broadcast into a speech announcer for speech broadcast.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method described in the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the method described in the present disclosure when executed by a processor.

[0010] According to one or more embodiments of the present disclosure, preset punctuation marks are recognized based on the currently received text to broadcast a whole sentence or half a sentence, thereby ensuring the real-time and fluency of the subsequent voice broadcast; and special text in the text to be broadcast is filtered to avoid the occurrence of "garbled" broadcasts.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0014] Figure 2 A flow chart of a voice broadcasting method based on streaming text according to an embodiment of the present disclosure is shown;

[0015] Figure 3 A structural block diagram of a speech broadcasting device based on streaming text according to an embodiment of the present disclosure is shown; and

[0016] Figure 4 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0018] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0019] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0020] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0021] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0022] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable the method of voice announcement based on streaming text to be performed.

[0023] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0024] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0025] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to make voice announcements. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0026] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0027] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0028] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0029] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0030] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0031] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0032] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0033] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0034] Figure 1The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0035] In generative AI (artificial intelligence) applications, large models return text in a streaming manner. Then, in the case of digital human automatic voice broadcast (automatic broadcast), that is, in the real-time broadcast scenario, in order to provide a good broadcast experience, it means that the streaming incremental text needs to be analyzed and segmented in real time to ensure that the broadcast effect is smooth and natural. Usually, the entire paragraph of text is spit out to the TTS (Text To Speech) broadcaster for the entire paragraph to be broadcast, which limits real-time performance. (1) Broadcasting according to the currently received text will cause pauses in the sentence; (2) There is no filtering capability for special characters, resulting in some special rule markdown syntax being broadcast.

[0036] Therefore, according to an embodiment of the present disclosure, a voice broadcast method based on streaming text is provided. Figure 2 A flow chart of a method for voice broadcasting based on streaming text according to an embodiment of the present disclosure is shown, as shown in FIG. Figure 2 As shown, method 200 includes: obtaining one or more text blocks that have been streamed and have not yet been voice-announced, wherein the one or more text blocks are at least a part of a text (step 210); performing character recognition on the one or more text blocks in sequence to identify preset punctuation marks in the one or more text blocks (step 220); in response to recognizing the preset punctuation marks and determining that the one or more text blocks include special text before the preset punctuation marks, obtaining a first text other than the special text before the preset punctuation marks in the one or more text blocks, so as to use the first text as the text to be announced, wherein the special text is a preset text that is not suitable for voice announcement (step 230); inputting the text to be announced into a voice announcer for voice announcement (step 240).

[0037] According to an embodiment of the present disclosure, preset punctuation marks are recognized based on the currently received text to broadcast a whole sentence or half a sentence, thereby ensuring the real-time and fluency of the subsequent voice broadcast; and special text in the broadcast text is filtered to avoid the occurrence of "garbled" broadcasts.

[0038] In this disclosure, streaming text generally refers to text generated based on streaming output. Streaming output is a data transmission method that allows data to be displayed gradually during the transmission process, rather than waiting for all data to be fully loaded before displaying it. This method is particularly suitable for scenarios such as big data processing and real-time data transmission, such as live webcasting, video streaming, real-time data analysis, etc. Streaming output can significantly improve the user experience because it allows users to start using data without completely waiting for all data to be loaded.

[0039] In the present disclosure, the lost text is generated sequentially in a reading order. Therefore, character recognition is performed sequentially on the one or more text blocks, that is, character recognition is performed sequentially on the one or more text blocks based on the reading order.

[0040] In streaming text processing, the text outputted in streaming mode is usually in the form of text blocks (i.e., text data blocks), that is, the entire text is divided into text data blocks according to certain rules or sizes and output in sequence. In data processing and storage, a chunk usually refers to a smaller part or block of a larger data. For example, when processing a large amount of text or data, it may be divided into smaller chunks to facilitate processing block by block.

[0041] It is understandable that in the present disclosure, text blocks may be divided and streamed in any suitable manner, and no limitation is made herein.

[0042] In some embodiments, the text returned by the large model can be voiced using the method described in the embodiments of the present disclosure.

[0043] It is understandable that when a large model returns a lot of text, it will take a long time to return the complete result. If you wait for the large model to form a complete answer before showing it to the user, it will obviously give the user a bad experience. Therefore, most AI applications display results to users in the form of streaming output. In this way, the model does not output all the results at once when generating content, but generates output step by step and continuously. This method is similar to the thinking process of humans when communicating orally or writing, that is, thinking while expressing. Streaming output is more flexible and interactive, and can dynamically adjust content in long text generation, while allowing users to provide feedback or modify guidance directions instantly during the generation process.

[0044] In the present disclosure, special text is a preset text that is not suitable for voice broadcast. For example, when voice broadcasts text such as formulas and codes, it is easy to produce a garbled broadcast effect to the user, thereby affecting the user experience. According to an embodiment of the present disclosure, by identifying and filtering special text in the voice broadcast text block, the "garbled" broadcast is effectively avoided, and the fluency of the voice broadcast is improved.

[0045] Therefore, according to some embodiments, the special text includes at least one of the following items: markup language text, formula. In some examples, the special text in the one or more text blocks can be identified by identifying a corresponding identifier.

[0046] A markup language is a special computer text encoding that uses specific tags or markers to identify different parts of a text, thereby defining the purpose and relationship of these parts. Markup languages ​​provide semantic information for text content, allowing computers to understand the structure and meaning of the text. Typical examples of markup languages ​​include HTML, XML, and Markdown.

[0047] Markdown is a lightweight markup language that is commonly used to format text. Markdown uses simple symbols to represent the format of text, such as bold, italics, lists, and links. By recognizing Markdown syntax (for example, headers, lists, and code blocks), content can be intelligently divided according to its structure and hierarchy, resulting in more semantically coherent blocks.

[0048] HTML elements supported by Markdown and tags not covered by Markdown can be written directly in HTML in the document.

[0049] In some examples, Markdown can also be used with LaTex to represent mathematical formulas. In this case, the special text in the form of the formula is also a markup language text. For example, when a mathematical formula needs to be inserted, two dollar signs "$$" can be used to wrap the mathematical formula in Tex or LaTex format. In this case, the formula is identified, for example, by identifying the dollar sign "$$".

[0050] HTML (Hypertext Markup Language) is the basic markup language used to create web pages. It uses tags to describe various elements on a web page, such as text, images, links, and tables. HTML tags provide semantic information for each element, allowing the browser to correctly render the web page content. XML (Extensible Markup Language) is a markup language used to store and exchange data. Similar to HTML, XML uses tags to define data structures and is commonly used for data exchange, configuration files, and metadata management. For example, through the HTML markup language text " <img src＝’http: xxx.jpg’> " to mark this as a picture. At this time, for example, the identifier " " to identify the HTML markup language text.

[0051] In an embodiment of the present disclosure, after the text to be broadcast is determined in the one or more text blocks, the text after the text to be broadcast in the one or more text blocks is used as a text block that has not yet been voice broadcast and together with the subsequently received new text block as a new text block that has not yet been voice broadcast, so as to determine the new text to be broadcast in the new text block that has not yet been voice broadcast.

[0052] For example, when a text "From the snowy plateaus in the north to the tropical rain forests in southern Xinjiang, the mountains and rivers are magnificent, and the rivers are surging, which gave birth to the splendid Chinese civilization for five thousand years" is obtained, the multiple text blocks corresponding to the part of the text "From the snowy plateaus in the north to the tropical rain forests in southern Xinjiang, the mountains and rivers are magnificent, and the rivers are surging, which gave birth to" are obtained. For example, the preset punctuation mark is "comma", and the text before the "comma" is recognized as "From the snowy plateaus in the north to the tropical rain forests in southern Xinjiang, the mountains and rivers are magnificent, and the rivers are surging", then the text can be used as the text to be broadcast. In addition, the text "bred" that has not been broadcasted in the multiple text blocks is used together with the text blocks received subsequently as the text blocks that have not been voice broadcast in the next round, and the preset punctuation marks are recognized in turn.

[0053] It can be understood that in the above example, in some embodiments, when recognizing preset punctuation marks, the first "comma" can be recognized, that is, "from the snowy plateaus in the north to the tropical rain forests in southern Xinjiang" is used as the text to be broadcast in this round, and "magnificent mountains and rivers, rushing rivers, gave birth to" together with the subsequently received text blocks as the next round of text blocks that have not yet been voice broadcast, and the preset punctuation marks are recognized in turn.

[0054] Therefore, in some embodiments, when it is recognized that the one or more text blocks include multiple preset punctuation marks, the text to be broadcast can be determined based on any text before the recognized preset punctuation mark, as long as the determined text to be broadcast does not exceed the maximum number of input characters of the voice announcer.

[0055] Preferably, in some examples, when it is recognized that the one or more text blocks include multiple preset punctuation marks, and it does not exceed the maximum number of input characters of the voice announcer, the text before the last preset punctuation mark recognized can be determined as the text to be announced, so as to announce enough text that has been received, thereby ensuring the continuity of the voice announcement.

[0056] According to some embodiments, performing character recognition on the one or more text blocks to identify preset punctuation marks in the one or more text blocks includes: in response to determining the first sentence in the text to be voice-announced, identifying preset punctuation marks representing the entire sentence in the one or more text blocks.

[0057] Through this embodiment, when the product starts broadcasting (starting to broadcast the first sentence), a complete sentence is broadcast first, which ensures the integrity of the voice broadcast and the coherence of the broadcast tone, and improves the user experience.

[0058] According to some embodiments, the preset punctuation marks representing the entire sentence include at least one of the following items: a period, an exclamation mark, and a question mark.

[0059] For example, in an embodiment of voice broadcasting the text returned by the large model, when a text block returned by the large model in a streaming manner is received, the text block can be cached. When the first sentence in a text to be voice broadcasted is determined, punctuation marks representing the entire sentence in the cached text block are identified, such as a period, an exclamation mark, and a question mark.

[0060] Therefore, in some examples, in a Chinese broadcast scenario, the punctuation mark representing the whole sentence may be: ".!?"; in an English broadcast scenario, the punctuation mark representing the whole sentence may be: ".!?". In particular, in an alphabetic text broadcast scenario such as English, in order to prevent the decimal point from being accidentally damaged, such as the number: 3.14, the punctuation mark "." representing the whole sentence is determined by identifying ". plus a space".

[0061] In some examples, in order to ensure the start speed, the voice broadcast is started for the user as soon as possible, and the first sentence in the text is sent to the voice broadcaster immediately. Therefore, when determining the first sentence in the text to be voice broadcasted, the first punctuation mark representing the whole sentence in the one or more text blocks can be identified, so that the text before the first punctuation mark representing the whole sentence in the one or more text blocks is used as the text to be broadcasted. Thereby, the waiting time of the user is shortened as much as possible.

[0062] According to some embodiments, the method according to the present disclosure also includes: when determining the first sentence in the text to be voice broadcast, in response to not recognizing the preset punctuation mark in the one or more text blocks and recognizing that the one or more text blocks include the special text, obtaining a second text before the special text in the one or more text blocks, so as to use the second text as the text to be broadcast.

[0063] Specifically, when determining the first sentence in the text to be voice broadcasted, if the special text is recognized before the preset punctuation mark is recognized, appropriate degradation can be performed, that is, the broadcast of the first sentence is not guaranteed to be a complete sentence.

[0064] For example, suppose the text replied by the large model is: "The following is a quick sort algorithm written in C++:\n\n```\nc++\n......\n```\n, you can also try other ways of asking questions to provide you with more detailed answers." And the text block output by the large model is as follows:

[0065]

[0066] In the first round of voice broadcast, after accumulating multiple text blocks corresponding to the text "The following is a quick sorting algorithm written in C++:\n\n```", when the identifier "\n\n```" detects that the subsequent special text is a piece of code, the text "The following is a quick sorting algorithm written in C++:" before the identifier "\n\n```" (that is, before the special text) can be directly used as the content of the first round of voice broadcast. After accumulating multiple text blocks corresponding to the text "The following is a quick sorting algorithm written in C++:\n\n```\nc++\n......\n```\n, you can also try other ways of asking questions, which is", the text that is not voice broadcasted at this time is "You can also try other ways of asking questions, which is" (that is, the special text has been filtered out), then the subsequent preset punctuation recognition and the determination of the next round of text to be broadcast can be performed. If the preset punctuation marks to be recognized in the next round include a comma “,”, the text “You can also try other ways of asking questions” in the text “You can also try other ways of asking questions” that is not voice-announced at this time can be used as the second round of text to be announced.

[0067] According to some embodiments, performing character recognition on the one or more text blocks to identify preset punctuation marks in the one or more text blocks includes: in response to determining that a sentence other than the first sentence in the text to be voice-announced, identifying preset punctuation marks representing half a sentence in the one or more text blocks.

[0068] In this embodiment, starting from the second sentence of the text, it is acceptable to broadcast in half-sentence units (half-sentences formed by punctuation marks) to ensure the fluency of subsequent broadcasting.

[0069] In some embodiments, based on the principle of capturing as much text as possible, when performing the preset punctuation mark recognition representing half a sentence, the last punctuation mark representing half a sentence in the acquired one or more text blocks can be recognized. If the length of the text before the last punctuation mark representing half a sentence exceeds the maximum input character length of the voice player, then the previous (i.e., the second to last) punctuation mark representing half a sentence can be tried to be found... until the maximum input character length requirement of the voice player is met.

[0070] According to some embodiments, the preset punctuation marks representing half a sentence include at least one of the following items: a period, an exclamation mark, a question mark, a semicolon, and a comma.

[0071] In some examples, in a Chinese broadcast scenario, the punctuation mark representing a half sentence may be: “,;.!?”; in an English broadcast scenario, the punctuation mark representing a half sentence may be: “,.;!?”. It can be seen that a half sentence may include a whole sentence, and a half sentence divides the text in a finer granularity.

[0072] For example, suppose the text replied by the big model is: "First, Sun Wukong is a fearless man who dares to make trouble in heaven and enter the underworld; second, Sun Wukong is a wise and brave sword of justice who sweeps away the demons. Fearlessness and wisdom and courage make this character shine, which is also the value and essence of this book. This fearless spirit of daring to challenge all authority and daring to despise all sacredness is the basic feature of the artistic image of Sun Wukong, and it is also the personality spirit that the author strongly praises. Because Sun Wukong is wise and brave, he can see through all disguises and understand all the truth." And the text block output by the big model is as follows:

[0073]

[0074] In the first round of voice broadcast, after obtaining multiple text blocks corresponding to the text "First, Sun Wukong is a fearless man who dares to make trouble in heaven and enter the underworld; secondly, Sun Wukong is a wise and brave sword of justice that sweeps away the demons.", that is, after identifying the first punctuation mark "." in the text that represents the whole sentence, the sentence can be sent to the voice announcer as the text to be broadcast.

[0075] In the subsequent second round of voice broadcast, assuming that enough text blocks have been received in the process of broadcasting the first sentence, in order to ensure the continuity of the tone (that is, based on the principle of capturing as much text as possible), find the last punctuation mark representing half a sentence, and determine the new round of text to be broadcast based on the text before the last punctuation mark representing half a sentence. For example, when the second round of voice text to be broadcast is determined, assuming that the text "First, Sun Wukong is a fearless man who dares to make trouble in heaven and dares to enter the underworld; secondly, Sun Wukong is a wise and brave sword of justice that sweeps away the demons. Fearlessness and wisdom and courage make this character shine, which is also the value and essence of this book. This kind of fearless spirit that dares to challenge all authority and dares to despise all sacredness is the basic characteristic of the artistic image of Sun Wukong, and it is also" corresponding to the text block has been received cumulatively, and the text that has not been voice broadcasted at this time is "Fearlessness and wisdom and courage make this character shine, which is also the value and essence of this book. This kind of fearless spirit that dares to challenge all authority and dares to despise all sacredness is the basic characteristic of the artistic image of Sun Wukong, and it is also" (that is, special text has been filtered out), then subsequent preset punctuation recognition and determination of the next round of text to be broadcast can be performed. If the preset punctuation marks to be recognized in the next round include a comma "," then "Fearlessness and wisdom and courage make this character shine, which is also the value and essence of this book. This kind of fearless spirit of daring to challenge all authority and daring to despise all sacredness is the basic characteristic of the artistic image of Sun Wukong" can be used as the second round of text to be broadcast. Until the text is finished.

[0076] According to some embodiments, the text to be broadcast further includes: a template text used to replace the identified special text, wherein the template text is used to identify the special text.

[0077] In some examples, when a special text is identified, the special text can be filtered out, so that the text after the special text is removed is used as the text to be played. Alternatively, the special text is replaced with a preset template text to form a new text as the text to be played.

[0078] In some examples, the template text may correspond to the type of the special text to reflect the content identified by the special text.

[0079] For example, suppose the text of the big model's reply is: "According to your needs, an image has been generated for you: <img src＝’https: newapp-prompt.cdn.bcebos.com bgimg 2024-10 ba3b55742326bfada7d355f5c26e3835.png’> .” And the text block of the large model streaming output is as follows:

[0080]

[0081] When performing voice broadcast, the text to be played can be "According to your needs, a picture has been generated for you: here is the picture", and the template text "here is the picture" is used to indicate that this corresponds to a special text representing a picture. In this way, the situation of "garbled" broadcast is avoided, and the user can clearly understand the content.

[0082] According to the embodiments of the present disclosure, Figure 3 As shown, a speech broadcasting device 300 based on streaming text is also provided, including: an acquisition unit 310, configured to acquire one or more text blocks that are streamed and have not yet been voice broadcast, wherein the one or more text blocks are at least a part of a text; a recognition unit 320, configured to perform character recognition on the one or more text blocks in sequence to recognize preset punctuation marks in the one or more text blocks; a first determination unit 330, configured to respond to recognizing the preset punctuation marks and determining that the one or more text blocks include special text before the preset punctuation marks, to acquire a first text other than the special text before the preset punctuation marks in the one or more text blocks, so as to use the first text as the text to be broadcast, wherein the special text is a preset text that is not suitable for speech broadcast; a broadcasting unit 340, configured to input the text to be broadcast into a speech broadcaster for speech broadcasting.

[0083] Here, the operations of the above-mentioned units 310-340 of the streaming text-based voice broadcasting device 300 are similar to the operations of the above-described steps 210-240, and are not described in detail here.

[0084] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0085] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0086] refer to Figure 4 , a block diagram of an electronic device 400 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0087] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0088] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information to the electronic device 400. The input unit 406 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 407 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include but is not limited to a disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0089] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as method 200. For example, in some embodiments, the method 200 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the method 200 in any other appropriate manner (e.g., by means of firmware).

[0090] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0092] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0095] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0097] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A voice broadcasting method based on streaming text, comprising: Obtaining one or more text blocks that are streamed and have not yet been voice broadcast, wherein the one or more text blocks are at least a part of a text; Performing character recognition on the one or more text blocks in sequence to identify preset punctuation marks in the one or more text blocks; In response to identifying the preset punctuation mark and determining that the one or more text blocks include special text before the preset punctuation mark, obtaining a first text other than the special text before the preset punctuation mark in the one or more text blocks, so as to use the first text as the text to be broadcast, wherein the special text is a preset text that is not suitable for voice broadcast; The text to be broadcasted is input into a voice broadcaster for voice broadcasting.

2. The method of claim 1, wherein: Performing character recognition on the one or more text blocks to identify preset punctuation marks in the one or more text blocks includes: In response to determining that the first sentence in the text segment is to be voiced, preset punctuation marks representing the entire sentence in the one or more text blocks are identified.

3. The method of claim 2, further comprising: When determining the first sentence in the text to be voice-announced, in response to not recognizing the preset punctuation mark in the one or more text blocks and recognizing that the one or more text blocks include the special text, obtain a second text before the special text in the one or more text blocks to use the second text as the text to be announced.

4. The method according to any one of claims 1 to 3, wherein: Performing character recognition on the one or more text blocks to identify preset punctuation marks in the one or more text blocks includes: In response to determining that a sentence other than the first sentence in the text to be voiced is to be announced, a preset punctuation mark representing a half sentence in the one or more text blocks is identified.

5. The method according to claim 1 or 3, wherein: The text to be broadcast also includes: a template text used to replace the identified special text, wherein the template text is used to identify the special text.

6. The method of claim 2, wherein: The preset punctuation marks representing the whole sentence include at least one of the following items: a period, an exclamation mark, and a question mark.

7. The method of claim 4, wherein: The preset punctuation marks representing half a sentence include at least one of the following items: a period, an exclamation mark, a question mark, a semicolon, and a comma.

8. The method according to any one of claims 1 to 7, wherein: The special text includes at least one of the following: a markup language text, a formula, and wherein the special text in the one or more text blocks is identified by identifying a corresponding identifier.

9. A speech broadcasting device based on streaming text, comprising: An acquisition unit, configured to acquire one or more text blocks that are streamed and have not yet been voice broadcast, wherein the one or more text blocks are at least a part of a text; a recognition unit configured to perform character recognition on the one or more text blocks in sequence to recognize preset punctuation marks in the one or more text blocks; A first determining unit is configured to, in response to identifying the preset punctuation mark and determining that the one or more text blocks include special text before the preset punctuation mark, obtain a first text other than the special text before the preset punctuation mark in the one or more text blocks, so as to use the first text as the text to be broadcasted, wherein the special text is a preset text that is not suitable for voice broadcasting; The broadcasting unit is configured to input the text to be broadcast into the voice broadcaster for voice broadcasting.

10. The device of claim 9, wherein: The identification unit comprises: A unit for identifying preset punctuation marks representing a whole sentence in the one or more text blocks in response to determining that a first sentence in the text is to be voiced.

11. The apparatus of claim 10, further comprising: The second determination unit is configured to, when determining the first sentence in the text to be voice-announced, obtain a second text before the special text in the one or more text blocks in response to not recognizing the preset punctuation mark in the one or more text blocks and recognizing that the one or more text blocks include the special text, so as to use the second text as the text to be announced.

12. The device according to any one of claims 9 to 11, wherein: The identification unit comprises: A unit for identifying preset punctuation marks representing half sentences in the one or more text blocks in response to determining that a sentence other than the first sentence in the text to be voiced out is to be recognized.

13. The device according to claim 9 or 11, wherein: The text to be broadcast also includes: a template text used to replace the identified special text, wherein the template text is used to identify the special text.

14. The device of claim 10, wherein: The preset punctuation marks representing the whole sentence include at least one of the following items: a period, an exclamation mark, and a question mark.

15. The device of claim 12, wherein: The preset punctuation marks representing half a sentence include at least one of the following items: a period, an exclamation mark, a question mark, a semicolon, and a comma.

16. The device according to any one of claims 9 to 15, wherein: The special text includes at least one of the following: a markup language text, a formula, and wherein the special text in the one or more text blocks is identified by identifying a corresponding identifier.

17. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images