Announcing method, electronic device, storage medium and computer program product

Abstract

Embodiments of the present application disclose a kind of broadcast method, electronic equipment, computer readable storage medium and computer program product, the method includes: obtaining to be broadcast content, to be broadcast content includes first text content and second text content;To be broadcast content is handled, obtains to be broadcast text, to be broadcast text includes third text content, third text content is the text content obtained by rewriting second text content into first text content;To be broadcast text is converted into audio data, audio data includes the first audio corresponding to third text content, and first audio is broadcasted into first text content by second text content;Broadcast audio data.The embodiments of the present application are rewritten into first text content by second text content first, then to be broadcast text is converted into audio data, to be broadcast into first text content by second text content through first audio, so that the information of the broadcast voice can be more accurately conveyed original broadcast content.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a broadcasting method, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Text-to-speech (TTS) is a technology that converts text content into audio data and is widely used in scenarios such as audiobooks, notification broadcasts, and screen reading.

[0003] In related technologies, after acquiring articles, news, and notifications to be broadcast, electronic devices input the content to be broadcast into a TTS module, which then converts the content into audio data. Finally, the audio data is played to broadcast the articles, news, and notifications.

[0004] However, the voice broadcasting of related technologies cannot accurately convey the information of the original broadcast content, resulting in a poor human-computer interaction experience for users. Summary of the Invention

[0005] This application provides a broadcasting method, electronic device, computer-readable storage medium, and computer program product, which can solve the problem that voice broadcasting cannot accurately convey the information of the original broadcast content.

[0006] Firstly, embodiments of this application provide a broadcasting method applied to an electronic device. In the broadcasting method, the electronic device acquires content to be broadcast, which includes first text content and second text content; processes the content to be broadcast to obtain text to be broadcast, which includes third text content obtained by rewriting the second text content into the first text content; converts the text to be broadcast into audio data, which includes a first audio corresponding to the third text content, the first audio broadcasting the second text content as the first text content; and broadcasts the audio data.

[0007] As can be seen from the above, when the content to be broadcast includes a first text content and a second text content, this embodiment first rewrites the second text content into the first text content to obtain the text to be broadcast, then converts the text to be broadcast into audio data, and broadcasts the audio data. Thus, when performing voice broadcast based on the audio data, the second text content in the content to be broadcast can be read as the first text content through the first audio. Here, the first text content is annotation information, the second text content is the annotated text, and the annotated text is the text annotated by the annotation information; or, the first text content is a portion of the second text content.

[0008] In this way, when the first text content is annotation information and the second text content is the annotated information, the annotated text in the content to be broadcast can be read as the annotation information through the first audio, so that the annotated text will not be misread, and the repeated reading phenomenon of reading both the annotated text and the annotation information will not occur, further enabling the broadcast voice to convey the information of the original broadcast content more accurately.

[0009] When the first text content is a part of the text of the second text content, since the number of characters of the first text content is less than the number of characters of the second text content, reading the second text content as the first text content through the first audio can not only make the broadcast voice convey the information of the original broadcast content more accurately, but also reduce the complexity of the voice broadcast information.

[0010] In comparison, in the related art, the content to be broadcast including annotation information is directly input into the TTS module, and voice broadcast is performed according to the audio data output by the TTS module, which will cause misreading of the annotated text and the repeated reading phenomenon of reading both the annotated text and the annotation information, so that the broadcast voice cannot accurately convey the information of the original broadcast content. In the embodiments of the present application, after the annotated text in the content to be broadcast is replaced and corrected according to the annotation information, text-to-speech processing is performed, so that the broadcast voice can accurately convey the information of the original broadcast content. For example, in the related art, the phrase "无夸(奈)之举" is broadcast as "无夸奈之举". In the embodiments of the present application, the phrase "无夸(奈)之举" is broadcast as "无奈之举".

[0011] In the related art, directly broadcasting the second text content completely will lead to high complexity of the voice broadcast information, which is not conducive to accurately conveying the information of the original broadcast content. In the embodiments of the present application, the second text content is rewritten into the first text content and then voice broadcast is performed, which reduces the complexity of the voice broadcast information, and the broadcast voice can accurately convey the information of the original broadcast content.

[0012] In some possible implementations of the first aspect, the annotation information includes pinyin annotations and / or text annotations. The text annotation may be text located in parentheses and used to correct the preceding text; the pinyin annotation may be pinyin used to annotate the preceding text.

[0013] In some possible implementations of the first aspect, when the first text content is annotation information and the second text content is the annotated text, in the process of processing the content to be broadcast to obtain the text to be broadcast, the electronic device may acquire a fourth text content in the content to be broadcast, where the fourth text content is text content containing annotation information and the annotated text; the annotation information in the fourth text content is removed, the annotated text in the fourth text content is replaced with the annotation information to obtain a first replacement text, and the third text content includes the first replacement text.

[0014] In some possible implementations of the first aspect, during the process of processing the content to be broadcast and obtaining the text to be broadcast, the electronic device not only replaces the annotated text in the fourth text content with annotation information to obtain the first replacement text, but also obtains the fifth text content in the content to be broadcast. The fifth text content matches the fourth text content and is text content that includes some or all of the annotated text but does not include the annotation information. The annotation information is then used to replace the annotated text in the fifth text content to obtain the second replacement text, and the third text content also includes the second replacement text.

[0015] In other words, electronic devices can replace only the annotated text in the text containing the annotation information, or they can replace both the annotated text in the text containing the annotation information and the annotated text in other positions of the content to be broadcast.

[0016] In some possible implementations of the first aspect, the annotation information includes text annotations. At this time, the electronic device processes the content to be broadcast. During the process of obtaining the text to be broadcast, it can obtain the sixth text content within the content to be broadcast, which is text containing parentheses; remove the parentheses and the text within the parentheses from the sixth text content to obtain the first comparison text; remove the parentheses and the text within the parentheses from the sixth text content, and replace the preceding text of the parentheses in the sixth text content with the text within the parentheses to obtain the second comparison text; score the first comparison text and the second comparison text based on semantic coherence and / or text correctness to obtain the scores of the first comparison text and the second comparison text; obtain the target comparison text from the first comparison text and the second comparison text, where the target comparison text is the text with the higher score between the first and second comparison texts; the third text content includes the target comparison text.

[0017] In some possible implementations of the first aspect, when the first text content is a part of the second text content, the second text content can be a long text or a long number string. The long text consists of numbers and letters, and the long number string consists of numbers. The number of characters in the long text or long number string is M, where M is greater than or equal to a first threshold. The first text content can be N characters in the long text or long number string, where N is less than M.

[0018] At this time, while processing the content to be broadcast and obtaining the text to be broadcast, the electronic device can also extract N characters from the long text or long number string; by replacing the long text or long number string in the content to be broadcast with N characters, the text to be broadcast can be obtained.

[0019] The existing technology directly broadcasts long texts and long number strings in their entirety, resulting in high complexity of the voice broadcast information, increasing the user's cognitive load, and hindering the user's ability to quickly and accurately understand the information conveyed by the voice broadcast. In this implementation, long texts and long number strings are simplified before being broadcast, reducing the complexity of the voice broadcast information, decreasing the user's cognitive load, and making it easier for the user to quickly and accurately understand the information conveyed by the voice broadcast.

[0020] In some possible implementations of the first aspect, the N characters are the last N characters of a long text or a long string of numbers; the long text is a license plate number, and the long string of numbers is a mobile phone number or telephone number. That is, when broadcasting license plate numbers, mobile phone numbers, and telephone numbers, only the last digits are broadcast, which can simplify the complexity of the voice broadcast information and accurately convey the information of the original broadcast content.

[0021] In some possible implementations of the first aspect, when the content to be broadcast includes elements with at least two semantics, the electronic device processes the content to be broadcast and, in the process of obtaining the text to be broadcast, can also obtain a seventh text content in the content to be broadcast, which is text content containing elements with at least two semantics; by converting the elements in the seventh text content into text with a target semantic, an eighth text content is obtained, where the target semantic is one of at least two semantics, and the text to be broadcast includes the eighth text content.

[0022] In this implementation, elements with multiple semantic meanings are rewritten into text content with the target semantic meaning before being input into the TTS module. That is, the semantics of elements with multiple semantic meanings are already determined, and these elements are rewritten into text content with the target semantic meaning, without the TTS module rewriting punctuation marks with multiple semantic meanings according to its built-in rules. This reduces the probability of misreading or omitting the semantic meaning of elements due to incorrect rewriting of elements with multiple semantic meanings by the TTS module, allowing the audio data output by the TTS module to more accurately convey the information of the original broadcast content.

[0023] In some possible implementations of the first aspect, the electronic device, in the process of obtaining the eighth text content by converting the elements in the seventh text content into text with the target semantics, can determine the semantic usage of the elements in the seventh text content; determine the target rewriting rule corresponding to the semantic usage of the elements based on the mapping relationship between the semantic usage and the rewriting rule; and use the target rewriting rule to convert the elements in the seventh text content into text with the target semantics to obtain the eighth text content.

[0024] In some possible implementations of the first aspect, in the process of obtaining the eighth text content by converting the elements in the seventh text content into text with the target semantics, the electronic device can, according to the rewriting rules corresponding to each semantic usage of the elements, convert the elements in the seventh text content into text with the corresponding semantics, obtain candidate text content corresponding to each semantic usage; convert the seventh text content into the original semantic vector; convert each candidate text content into a candidate semantic vector; and obtain the eighth text content from each candidate text content based on the similarity between the original semantic vector and each candidate semantic vector, wherein the eighth text content is the text content with the highest similarity among the candidate text contents.

[0025] In some possible implementations of the first aspect, elements having at least two semantics include punctuation marks and / or phrases consisting of numbers and letters.

[0026] In some possible implementations of the first aspect, when the content to be broadcast includes a first time text described using absolute time, the electronic device, in the process of processing the content to be broadcast and obtaining the text to be broadcast, can convert the first time text into a second time text based on the time difference between the time described by the first time text and the current time, and / or the user's time description habits; wherein the second time text uses a relative time description, or conforms to the user's time description habits, or includes text described using absolute time and text described using relative time, or uses a relative time description and conforms to the user's time description habits, or includes text described using absolute time and text described using relative time, and conforms to the user's time description habits; the text to be broadcast includes the second time text.

[0027] In this implementation, the time text described using absolute time is rewritten into a relative time description and / or a time text that conforms to the user's time description habits before being input into the TTS module. Thus, when the audio data output by the TTS module is used for voice playback, the absolute time is not played in its entirety; instead, relative time and / or voice text that conforms to the user's time description habits are played. This reduces the complexity of the voice playback information, decreases the user's cognitive load, and facilitates the user's quick and accurate understanding of the information conveyed by the voice playback.

[0028] In some possible implementations of the first aspect, the user's time description habits are determined based on the target speech input by the user to the electronic device, and / or the system time settings of the electronic device; the target speech is speech that includes time information.

[0029] In some possible implementations of the first aspect, when the content to be played includes a first image, the electronic device, while processing the content to be played and obtaining the text to be played, can obtain the image position of the first image within the content to be played; parse the first image according to its image position to obtain its image content; if the image content of the first image is not expressed in the text content of the content to be played, then merge part or all of the image content of the first image with the text content of the content to be played to form a ninth text content; the text to be played includes the ninth text content. The audio data includes a second audio corresponding to the ninth text content, the second audio being the audio used to play part or all of the image content of the first image.

[0030] In this implementation, when the content of the first image is not expressed in the text content, part or all of the image content of the first image is integrated with the text content to form a semantically coherent text expression, and then the TTS module is used for speech synthesis. This allows for the speech recitation of the first image content by playing a second audio file while the audio data output by the TTS module is being used. The image content of the first image is not omitted during the speech recitation, ensuring that the recitation accurately conveys the information of the original content.

[0031] In some possible implementations of the first aspect, the content to be broadcast also includes a second image, the content of which is already expressed in the text content of the content to be broadcast. In this case, during the process of merging part or all of the image content of the first image and the text content of the content to be broadcast into the ninth text content, if it is determined that a text segment with a similarity to the image content of the second image exceeding a second threshold has a preset expression structure, the electronic device can merge the text content of the content to be broadcast and part or all of the image content of the first image into the ninth text content according to the preset expression structure; the ninth text content has a preset expression structure.

[0032] In this implementation, the image content and text content of the first image are integrated into semantically coherent text by using the expressive structure of the image content of the second image, which can further improve the semantic coherence of the preceding and following text, and thus further improve the semantic coherence of the broadcast speech.

[0033] In some possible implementations of the first aspect, when the content to be broadcast includes annotation information and / or paralinguistic information, the electronic device, in the process of processing the content to be broadcast and obtaining the text to be broadcast, can obtain the tenth text content in the content to be broadcast, which is text content containing annotation information and / or paralinguistic information; integrate the annotation information and / or paralinguistic information in the tenth text content into the text content to obtain a semantically coherent eleventh text content; the text to be broadcast includes the eleventh text content; the annotation information is information located in parentheses for supplementary description of the preceding content; the paralinguistic information is non-linguistic components other than spoken language.

[0034] In this implementation, the annotations and / or paralinguistic information in the content to be broadcast are semantically and coherently integrated into the text content to obtain semantically coherent text content that includes the annotations and / or paralinguistic information. Then, a TTS module is used for speech synthesis. In this way, the broadcast speech not only reflects the paralinguistic and / or annotation information but also improves the coherence of the broadcast information, thereby enabling the broadcast speech to accurately convey the information of the original broadcast content.

[0035] In some possible implementations of the first aspect, when the content to be broadcast includes a table in text form, the electronic device can extract the content of the table while processing the content to be broadcast and obtaining the text to be broadcast; based on the content of the table and the text content of the content to be broadcast, the text to be broadcast is obtained, and the text to be broadcast includes part or all of the table content.

[0036] In this implementation, the table content and other text content are integrated into a semantically coherent text, and then the TTS module is used for text-to-speech conversion. This results in better semantic coherence in the speech delivery, ensuring that the delivered speech accurately conveys the information of the original content.

[0037] In some possible implementations of the first aspect, when the content to be broadcast includes captions and / or superscripts, the electronic device may remove the captions and / or superscripts from the content to be broadcast during the process of processing the content to be broadcast and obtaining the text to be broadcast, thereby obtaining the text to be broadcast, wherein the text to be broadcast does not include captions and / or superscripts.

[0038] In this implementation, the captions and / or superscripts in the content to be broadcast are removed before the voice broadcast, which avoids the interference of captions and superscripts on the voice broadcast, and ensures that the broadcast voice can accurately convey the information of the original broadcast content.

[0039] In some possible implementations of the first aspect, the content to be broadcast includes message source description information, which describes the source of the notification message. In this case, during the processing of the content to be broadcast and the acquisition of the text to be broadcast, the electronic device can remove the preset type information from the message source description information to obtain simplified message source description information; the text to be broadcast includes the simplified message source description information.

[0040] In this implementation, the message source description is simplified before being broadcast aloud. This way, the simplified description is broadcast instead of the complete one, reducing the complexity of the message and lessening the cognitive load on the user. Furthermore, in simplifying the source description, redundant information is removed, retaining only the key and easily identifiable details, making it easier for users to distinguish the source of the notification message through voice broadcast.

[0041] In some possible implementations of the first aspect, when the content to be broadcast does not include target information, the electronic device, in the process of processing the content to be broadcast and obtaining the text to be broadcast, can obtain supplementary information based on the source of the content to be broadcast. The supplementary information includes historical broadcast content and / or information related to the content to be broadcast within the application. Based on the supplementary information and the content to be broadcast, a content summary text is generated, which includes target information. The text to be broadcast is the content summary text.

[0042] In this implementation, the missing target information of the content to be broadcast is supplemented before being input into the TTS module, thereby avoiding the ambiguity of the information conveyed by the voice broadcast.

[0043] In some possible implementations of the first aspect, during the process of processing the content to be played and obtaining the text to be played, if the text content of the text to be played includes at least one feature text, tags can be assigned to each feature text to obtain tagged text content; the text to be played includes tagged text content. Different feature texts have different tags; the tags are used to determine the sound parameters for playing the feature text.

[0044] Accordingly, in the process of converting the text to be broadcast into audio data, the electronic device can determine the audio parameters corresponding to each tagged text content according to the mapping relationship between tags and audio parameters; input the text to be broadcast and the audio parameters into the text-to-speech module to obtain the audio data output by the text-to-speech module.

[0045] The text to be broadcast includes text content with a first tag and text content with a second tag; the audio data includes a third audio corresponding to the text content with the first tag and a fourth audio corresponding to the text content with the second tag; the third audio and the fourth audio have different sound parameters.

[0046] In this implementation, different labels are assigned to different feature texts in the content to be broadcast in order to adjust the sound expression of different feature texts so that the sound parameters of different feature texts are different, thereby enabling the broadcast speech to accurately convey the information of the original broadcast content.

[0047] In some possible implementations of the first aspect, the feature text includes at least one of the following: first-level heading text, body text, dialogue text, subheading text, body text with a first font attribute, text containing sub-language, and voice-over text; the plural font attribute of the body text is a second font attribute, and the first font attribute is different from the second font attribute.

[0048] Secondly, embodiments of this application provide a broadcasting method applied to an electronic device. In the broadcasting method, the electronic device can acquire content to be broadcast, which includes long text or a long string of numbers. The long text consists of numbers and letters, and the long string of numbers consists of numbers. The number of characters in the long text or long string of numbers is M, where M is greater than or equal to a first threshold. N characters are extracted from the long text or long string of numbers, where N is less than M. The long text or long string of numbers in the content to be broadcast is replaced with the N characters to obtain the text to be broadcast. The text to be broadcast is converted into audio data. The audio data is then broadcast.

[0049] This application simplifies long texts and long number strings before voice broadcasting, reducing the complexity of the voice broadcast information, reducing the cognitive load on users, and making it easier for users to quickly and accurately understand the information conveyed by the voice broadcast.

[0050] In some possible implementations of the second aspect, the N characters are the last N characters of a long text or a long string of numbers.

[0051] In some possible implementations of the second aspect, the long text is a license plate number, and the long string of numbers is a mobile phone number or telephone number.

[0052] Thirdly, embodiments of this application provide a broadcasting method applied to an electronic device. In the broadcasting method, the electronic device acquires content to be broadcast, which includes first-time text, described using absolute time; processes the content to be broadcast to obtain second-time text, which is converted from the first-time text and includes text described using relative time and / or a time description conforming to user time description habits; converts the second-time text into audio data, which includes audio corresponding to the second-time text, and the audio corresponding to the second-time text broadcasts the first-time text as second-time text; and broadcasts the audio data.

[0053] In this application embodiment, time text described using absolute time is rewritten into time text described using relative time and / or time text conforming to the user's time description habits, and then the text to be broadcast is converted into audio data. Thus, when broadcasting voice based on audio data, the absolute time is not broadcast in its entirety; instead, relative time and / or voice conforming to the user's time description habits are broadcast using the audio corresponding to the second time text. This reduces the complexity of the voice broadcast information, reduces the user's cognitive load, and facilitates the user's quick and accurate understanding of the information conveyed by the original broadcast content through voice. Thirdly, this application embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in any of the first or second aspects above.

[0054] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in either the first or second aspect above.

[0055] Fifthly, embodiments of this application provide a chip system including a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the method described in either the first or second aspect above. The chip system may be a single chip or a chip module composed of multiple chips.

[0056] Sixthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in either the first or second aspect above.

[0057] It is understood that the beneficial effects of the third to sixth aspects mentioned above can be found in the relevant descriptions in the first and second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0058] Figure 1A This is a schematic diagram of an article broadcasting scenario provided in an embodiment of this application;

[0059] Figure 1B This is a schematic diagram of a screen reading scenario provided in an embodiment of this application;

[0060] Figure 1C This is a schematic diagram of a notification broadcast scenario provided in an embodiment of this application;

[0061] Figure 2 A schematic block diagram illustrating the broadcasting process provided in an embodiment of this application;

[0062] Figure 3 Another schematic block diagram of the broadcasting process provided in the embodiments of this application;

[0063] Figure 4 This is a schematic block diagram of the structure of the electronic device 400 provided in the embodiments of this application;

[0064] Figure 5A This is a schematic diagram illustrating semantic misreading of punctuation marks provided in the embodiments of this application;

[0065] Figure 5B A schematic diagram illustrating the semantics of unread punctuation marks provided in the embodiments of this application;

[0066] Figure 6 A schematic diagram illustrating the punctuation rewriting process provided in this application embodiment;

[0067] Figure 7 This application provides an intentional representation of the semantic usage and rewriting rule mapping relationship in its embodiments.

[0068] Figure 8A A schematic diagram illustrating the rewriting of punctuation marks provided in an embodiment of this application;

[0069] Figure 8B Another schematic diagram illustrating the rewriting of punctuation marks provided in the embodiments of this application;

[0070] Figure 9 A schematic diagram illustrating punctuation rewriting based on an LLM model, provided in an embodiment of this application;

[0071] Figure 10 A schematic diagram illustrating the punctuation rewriting process provided in this application embodiment;

[0072] Figure 11 A schematic diagram illustrating punctuation rewriting based on a semantic coding model provided in this application embodiment;

[0073] Figure 12 A schematic diagram illustrating the process of rewriting punctuation marks based on a semantic coding model provided in this application embodiment;

[0074] Figure 13 A schematic diagram illustrating a word rewriting process consisting of numbers and letters provided in an embodiment of this application;

[0075] Figure 14 A schematic diagram illustrating the rewriting of phrases with multiple semantic meanings based on an LLM model, as provided in this application embodiment;

[0076] Figure 15 This is a schematic diagram illustrating the pinyin and text annotations provided in the embodiments of this application;

[0077] Figure 16 This application provides a flowchart illustrating a process for correcting and replacing relevant text based on pinyin annotations in an embodiment of the present application.

[0078] Figure 17 A comparative illustration showing the difference between before and after correcting and replacing relevant text based on pinyin annotations, provided for embodiments of this application;

[0079] Figure 18 This application provides a flowchart illustrating a process for correcting and replacing relevant text based on text annotations in an embodiment of the present application.

[0080] Figure 19 This application provides a schematic diagram illustrating an implementation of text correction and replacement based on annotations using an LLM model.

[0081] Figure 20 A flowchart illustrating the process of rewriting absolute time into relative time is provided for an embodiment of this application.

[0082] Figure 21 This application provides a schematic diagram of a process for rewriting absolute time according to user time description habits in an embodiment of the present application.

[0083] Figure 22 A flowchart illustrating a simplified long text or long string provided in this application embodiment;

[0084] Figure 23 This is a schematic diagram illustrating the description of the last digit of a license plate provided in an embodiment of this application;

[0085] Figure 24 A flowchart illustrating supplementary key information provided for embodiments of this application;

[0086] Figure 25A Supplementary schematic diagram for key information of the notification message provided in the embodiments of this application;

[0087] Figure 25B Supplementary schematic diagram for key information of the notification message provided in the embodiments of this application;

[0088] Figure 26A flowchart illustrating supplementary key information provided for embodiments of this application;

[0089] Figure 27 A flowchart illustrating supplementary key information provided for embodiments of this application;

[0090] Figure 28A This is a schematic diagram illustrating the message source description information provided in the embodiments of this application;

[0091] Figure 28B A flowchart illustrating a simplified message source description provided in an embodiment of this application;

[0092] Figure 28C A schematic diagram of the broadcast text provided in the embodiments of this application;

[0093] Figure 29 A flowchart illustrating the integration of remarks and sub-language information into text content, provided for an embodiment of this application;

[0094] Figure 30 A schematic diagram illustrating the rewriting of remarks information and / or sub-language information based on an LLM model, as provided in an embodiment of this application;

[0095] Figure 31 A schematic diagram of news to be broadcast provided in an embodiment of this application;

[0096] Figure 32 A flowchart illustrating the integration of image and text content provided in an embodiment of this application;

[0097] Figure 33 Schematic diagrams of text-supplementary images and table-supplementary images provided for embodiments of this application;

[0098] Figure 34A This is a schematic diagram illustrating the integration of supplementary images provided in the embodiments of this application;

[0099] Figure 34B This is a schematic diagram illustrating the integration of text-supplementary images provided in the embodiments of this application;

[0100] Figure 34C This is a schematic diagram of the integrated images of the supplementary table provided in the embodiments of this application;

[0101] Figure 35 Another flowchart illustrating the integration of image and text content provided in this application embodiment;

[0102] Figure 36 A schematic diagram illustrating text integration for the expressive structure of the reference-matching image provided in the embodiments of this application;

[0103] Figure 37A flowchart illustrating the integration of table content and text content provided in an embodiment of this application;

[0104] Figure 38 This is a schematic diagram illustrating the integration of table content provided in an embodiment of this application.

[0105] Figure 39 A schematic diagram illustrating text rewriting based on a multimodal LLM model provided in this application embodiment;

[0106] Figure 40 This is a schematic flowchart illustrating the deletion of superscripts and captions provided in an embodiment of this application.

[0107] Figure 41 This is a schematic diagram of the judgment process provided in the embodiments of this application;

[0108] Figure 42 A schematic diagram of the broadcast text provided in the embodiments of this application;

[0109] Figure 43 A flowchart illustrating an embodiment of this application for adjusting the audio expression of different broadcast content based on text and semantic features;

[0110] Figure 44 A schematic diagram of the labels and parameters provided in the embodiments of this application;

[0111] Figure 45 A schematic diagram of the layout features of the broadcast text provided in the embodiments of this application;

[0112] Figure 46 A flowchart illustrating a method for adjusting the sound expression of different content structures based on page layout features, provided in an embodiment of this application.

[0113] Figure 47 A schematic diagram of the labels and parameters provided in the embodiments of this application;

[0114] Figure 48 This is a schematic diagram illustrating the acquisition of audio data based on the content to be broadcast, provided in an embodiment of this application.

[0115] Figure 49 A flowchart illustrating a broadcasting method provided in an embodiment of this application;

[0116] Figure 50 A flowchart illustrating a broadcasting method provided in an embodiment of this application;

[0117] Figure 51 Another flowchart illustrating the broadcasting method provided in this application embodiment;

[0118] Figure 52 This is another flowchart illustrating the broadcasting method provided in the embodiments of this application. Detailed Implementation

[0119] In the following description, specific details such as particular system architectures and technologies are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application.

[0120] The following is combined Figures 1A to 1C The following provides an exemplary description of application scenarios that may be involved in the embodiments of this application. Figure 1A This is a schematic diagram of an article broadcasting scenario provided in an embodiment of this application. Figure 1B This is a schematic diagram of a screen reading scenario provided in an embodiment of this application. Figure 1C This is a schematic diagram of a notification broadcast scenario provided in an embodiment of this application.

[0121] like Figure 1A As shown, the browser interface 111 of mobile phone 100 displays text content about "Li Shiqi" and a text-to-speech control 110. Users can click the text-to-speech control 110 to have mobile phone 100 read aloud the text content about "Li Shiqi". The text-to-speech control 110 can be a control from a third-party application or an operating system control; no limitation is made here.

[0122] In addition to supporting article reading, the text-to-speech control 110 can also support reading text content such as news, movie reviews, and e-books (or audiobooks). For example, users can click the text-to-speech control 110 on the news interface to make the phone 100 read the news; and click the text-to-speech control 110 on the e-book interface to make the phone 100 read the e-book.

[0123] like Figure 1B As shown, mobile phone 100 displays a screen reading settings interface 112. The screen reading settings interface 112 includes a screen reading switch control 113. The screen reading switch control 113 is used to turn the screen reading function of mobile phone 100 on and off. Figure 1B The text-to-speech function is enabled.

[0124] After enabling the screen reading function, the phone 100 can convert screen content into audio data and read it aloud by playing the audio data. The phone 100 can read the entire screen content or only the content selected by the user. For example, when the user selects certain text content on the screen, a "Read Aloud" button pops up; after the user clicks the "Read Aloud" button, the phone 100 reads the selected text content.

[0125] like Figure 1C As shown, phone 100 displays the Quick Reminder settings interface 114. The Quick Reminder settings interface 114 includes Quick Reminder settings options, Smart Assistant reminder broadcast settings options, and Application Notification Broadcast settings options.

[0126] The Quick Notification feature can be enabled or disabled via the Quick Notification settings. When enabled, if the phone 100 has a communication connection with audio output devices such as smart glasses or smart headphones, and the user is wearing the smart glasses or smart headphones, the phone 100 will automatically announce important notifications. In other words, when the phone 100 receives an important notification, it will announce the notification through the smart glasses or similar devices.

[0127] The smart assistant's reminder broadcast settings allow you to configure whether to broadcast reminders for flights, high-speed rail trips, and incoming calls via voice.

[0128] The application notification broadcast settings allow you to configure whether or not to broadcast application notifications via voice. For example, Figure 1C The notifications from Application 1 and Application 2 will be read aloud via voice, but notifications from Application 3 will not be read aloud via voice.

[0129] When the notification broadcast function is enabled on mobile phone 100, after receiving a notification, the notification is converted into audio data and then broadcast.

[0130] It should be noted that the application scenarios involved in this application embodiment include not only notification broadcasts, screen reading, and news article broadcasts, but also AI voice assistants and voice navigation. Therefore, the application scenarios of this application embodiment are not limited here. This application embodiment can also be applied to text-to-speech scenarios on web pages or application platforms. In this scenario, the web page or application platform has text-to-speech functionality, allowing text content to be input into the web page or application platform, and the web page or application platform to output audio data based on the input text content.

[0131] The technical solutions provided in this application can be applied to various scenarios involving the conversion of text information into auditory information.

[0132] See Figure 2 The diagram shown is a schematic block diagram of a broadcasting process provided in an embodiment of this application. For example... Figure 2 As shown, in related technologies, after obtaining the content to be played, the content is directly input into the TTS module to obtain the audio data output by the TTS module. That is, no text optimization processing or sound optimization processing is performed on the content to be played; instead, the content is directly input into the TTS module.

[0133] Directly inputting the content to be broadcast into the TTS module can lead to inaccurate transmission of information in the output audio data. For example, the TTS module might output misreadings or omissions; the original content might include both images and text, but the TTS module might only output audio for the text, excluding images. In other words, the voice broadcast might only read the text, ignoring images; or the TTS module might output only one voice expression, mechanically reciting the entire content. All of these situations result in the broadcasted audio failing to accurately convey the original information.

[0134] In response to the aforementioned issues, this application embodiment optimizes the text and / or audio of the content to be broadcast before inputting the optimized text content into the TTS module, so that the audio data output by the TTS module can accurately convey the information of the original broadcast content.

[0135] For example, see Figure 3 The diagram shown is another schematic block diagram of the broadcasting process provided in this application embodiment. After obtaining the content to be broadcast, this application embodiment first optimizes the text and / or the sound of the content to be broadcast, then inputs the optimized text content into the TTS module, and performs voice broadcasting based on the audio data output by the TTS module.

[0136] The embodiments of this application may perform text optimization processing only on the content to be broadcast; may perform audio optimization processing only on the content to be broadcast; or may perform both text optimization processing and audio optimization processing on the content to be broadcast.

[0137] It should be noted that text optimization and / or audio optimization can be implemented by an optimization module, which can be set as an independent front-end module between the content to be played and the TTS module; of course, the optimization module can also be integrated into the TTS module.

[0138] The technical solutions provided in this application can be applied to electronic devices such as mobile phones, tablets, laptops, vehicle terminals, and wearable devices. No specific type or structure of the electronic device is limited herein.

[0139] For example, see Figure 4 This is a schematic block diagram of the electronic device 400 provided in an embodiment of this application. The electronic device 400 may include, but is not limited to, a processor 410, a memory 420, and a display screen 430. Optionally, the electronic device 400 may also include an audio module 440, a speaker 450, and a microphone 460.

[0140] The illustrated structures in this application do not constitute a specific limitation on the electronic device 400. In other embodiments of this application, the electronic device 400 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. For example, the electronic device 400 may also include one or more of the following: a camera, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a receiver, a headphone jack, a sensor, buttons, a motor, an indicator, and a subscriber identification module (SIM) card interface.

[0141] Processor 410 may include one or more processing units, such as an application processor (AP), a graphics processing unit (GPU), and a controller. These different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0142] In some embodiments, the processor 410 may include one or more interfaces. Interfaces may include an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), and a general-purpose input / output (GPIO) interface, etc.

[0143] The I2S interface can be used for audio communication. In some embodiments, the processor 410 may include multiple I2S buses. The processor 410 can be coupled to the audio module 440 via the I2S bus to realize communication between the processor 410 and the audio module 440. In some embodiments, the audio module 440 can transmit audio signals to the wireless communication module via the I2S interface to realize the function of playing audio through Bluetooth headsets.

[0144] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 440 and the wireless communication module can be coupled via the PCM bus interface. In some embodiments, the audio module 440 can also transmit audio signals to the wireless communication module via the PCM interface, enabling audio playback via Bluetooth headsets. Both the I2S interface and the PCM interface can be used for audio communication.

[0145] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the audio module 440 can transmit audio signals to the wireless communication module via the UART interface, enabling audio playback through Bluetooth headphones.

[0146] The MIPI interface can be used to connect the processor 410 to peripheral devices such as the display screen 430. The MIPI interface includes a display serial interface (DSI). In some embodiments, the processor 410 and the display screen 430 communicate via the DSI interface to realize the display function of the electronic device 400.

[0147] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 410 to the display 430, audio module 440, etc. The GPIO interface can also be configured as an I2S interface, UART interface, and MIPI interface, etc.

[0148] The interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 400. In other embodiments of this application, the electronic device 400 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0149] The memory 420 can be used to store computer executable program code, which includes instructions. The memory 420 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback), etc. The data storage area may store data created during the use of the electronic device 400. Furthermore, the memory 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. The processor 410 executes various functional applications and data processing of the electronic device 400 by running instructions stored in the memory 420 and / or instructions stored in memory disposed in the processor. The processor 410 implements the flow steps of the various embodiments of the broadcasting method provided in this application by executing the computer program in the memory 420.

[0150] Electronic device 400 implements display functions through a GPU, display screen 430, and application processor. The GPU is a microprocessor for image processing, connecting the display screen 430 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 410 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0151] Display screen 430 is used to display images or videos. Display screen 430 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 400 may include one or N displays 430, where N is a positive integer greater than 1.

[0152] Electronic device 400 can implement audio functions through audio module 440, speaker 450, microphone 460, headphone jack, and application processor, such as music playback, recording, and voice broadcasting.

[0153] The audio module 440 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. The audio module 440 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 440 may be located in the processor 410, or some functional modules of the audio module 440 may be located in the processor 410.

[0154] Speaker 450, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Electronic device 400 can output voice broadcasts via the speaker. Of course, voice broadcasts can also be output via smart headphones, smart glasses, etc.

[0155] Microphone 460, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When sending voice information, the user can speak by bringing their mouth close to microphone 460, inputting the sound signal into microphone 460. Electronic device 400 may be equipped with at least one microphone. In this embodiment, microphone 460 can receive user input voice, and based on the user input voice, determine the user's time description habits. Thus, when rewriting time text using absolute time description, the time text using absolute time description can be rewritten to conform to the user's time description habits.

[0156] The following uses electronic device 400 as an example to introduce the technical solution provided in the embodiments of this application.

[0157] In the embodiments of this application, text optimization processing may include at least one of the following: rewriting punctuation marks with multiple semantic meanings, rewriting phrases with multiple semantic meanings (composed of numbers and letters), correcting and replacing relevant text according to pinyin annotations and / or text annotations, rewriting time text described using absolute time, simplifying long text or long number strings composed of numbers and letters, supplementing key information, rewriting message source description information, semantically integrating notes and / or paralinguistic information into the text content, integrating image content with text content, integrating table content with text content, and deleting superscripts and captions.

[0158] Sound optimization processing may include at least one of the following: adjusting the sound expression of different content structures according to layout features, and adjusting the sound expression of different broadcast content according to text and semantic features.

[0159] The following sections will introduce and explain the various aspects of text optimization and audio optimization.

[0160] Rewrite punctuation marks with multiple semantic meanings:

[0161] Some punctuation marks have only one semantic meaning, that is, the semantic meaning of the punctuation mark is the same in different contexts; while some punctuation marks have multiple semantic meanings, that is, the semantic meaning of the punctuation mark may be different in different contexts.

[0162] When the content to be broadcast includes words and phrases containing punctuation marks with multiple semantic meanings, in the related art, the content to be broadcast is directly input into the TTS module, which determines the semantic meaning of the punctuation mark according to built-in rules and outputs audio data based on the determined semantic meaning of the punctuation mark. However, the built-in rules of the TTS module usually cannot accurately understand the semantic meaning of punctuation marks in different contexts, resulting in misreading the semantic meaning of the punctuation marks.

[0163] For example, refer to Figure 5A which shows the schematic diagram of misreading of punctuation mark semantics provided in the embodiment of the present application, corresponding text content is displayed on the first news interface 510 to the fourth news interface 513, and the text content of the first news interface 510 to the fourth news interface 513 can be voice broadcast through the TTS module.

[0164] In the first news interface 510, the displayed text content is "No. 7.6-7.7 #3x3 Golden League# 25th season opening match". The "." and "-" in this text content are punctuation marks with multiple semantic meanings.

[0165] Under normal circumstances, in the context of the first news interface 510, the semantic usage of "." is for date, and the semantic usage of "-" is for range. Therefore, "." should be rewritten as "day", "-" should be rewritten as "to", and "7.6-7.7" should be read as "July 6th to July 7th".

[0166] However, when the related art performs voice broadcast on the text content of the first news interface 510, it misreads "7.6-7.7" as "7 point 6 dash 7 point 7". That is, "." is incorrectly rewritten as "point", and "-" is incorrectly rewritten as "dash".

[0167] In the second news interface 511, the displayed text content includes "the two sides tied 0-0", "23-Jack", "13-Romiger", etc. The "-" in this text content is a punctuation mark with multiple semantic meanings.

[0168] Under normal circumstances, in the context of the second news interface 511, the "-" in "0-0" should be rewritten as "to", that is, "0-0" should be read as "zero to zero"; the "-" in "23-Jack" should be rewritten as "No.", that is, "23-Jack" should be read as "No. 23 Jack"; the "-" in "13-Romiger" should be rewritten as "No.", that is, "13-Romiger" should be read as "No. 13 Romiger".

[0169] When the related art performs voice broadcast on the content of the second news interface 511, it misreads "0-0" as "0 to 0", that is, "-" is incorrectly converted to "to"; it misreads "23-Jack" as "23 minus Jack", that is, "-" is incorrectly converted to "minus"; it misreads "13-Romiger" as "13 gang Romiger", that is, "-" is incorrectly converted to "gang".

[0170] In the third news interface 512, the displayed text content includes "2000-5" and "2000-9". The "-" in the text content is a punctuation mark with multiple semantics.

[0171] Under normal circumstances, in the context of the third news interface 512, the "-" in "2000-5" and "2000-9" should be converted to "gang", "2000-5" should be read as "two thousand gang 5", and "2000-9" should be read as "two thousand gang 9".

[0172] When the related art performs voice broadcast on the content of the third news interface 512, it misreads "2000-5" as "two thousand to five", and misreads "2000-9" as "two thousand to nine". That is, "-" is incorrectly converted to "to".

[0173] In the fourth news interface 513, the displayed text content includes "Men's Basketball Team A VS Men's Basketball Team B-00:00" and "Men's Basketball Team C VS Men's Basketball Team D-03:30". The "-" in the text content is a punctuation mark with multiple semantics.

[0174] Under normal circumstances, the "-" in "-00:00" and "-03:30" should be omitted from reading. "-00:00" should be read as "zero o'clock", and "-03:30" should be read as "three thirty".

[0175] When the related art performs voice broadcast on the content of the fourth news interface 513, it misreads "-00:00" as "negative zero o'clock", and misreads "-03:30" as "negative three thirty". That is, "-" is incorrectly converted to "negative".

[0176] In the related art, when the content to be broadcast includes words or phrases containing a punctuation mark with multiple semantics, in addition to misreading the semantics of the punctuation mark, the semantics of the punctuation mark may also not be read.

[0177] For example, in Figure 5A the fourth news interface 513, the displayed text content includes "Men's Basketball 1 / 4 Final". The " / " in the text content is a punctuation mark with multiple semantics. When the related art performs voice broadcast on the content of the fourth news interface 513, it reads "Men's Basketball 1 / 4 Final" as "one four final", that is, omits " / " and does not read the semantics of " / ".

[0178] For another example, refer to Figure 5B which shows a schematic diagram of not reading the semantics of punctuation marks provided by the embodiment of the present application, corresponding text contents are displayed on each of the fifth news interface 514 to the eighth news interface 517, and the text contents on the fifth news interface 514 to the eighth news interface 517 can be voice-broadcasted through a TTS module.

[0179] The text content displayed on the fifth news interface 514 includes "0-4→4-4→5-5→7-5", wherein "-" and "→" in the text content are punctuation marks with multiple semantics. In the related art, when performing voice broadcasting on the content of the fifth news interface 514, "0-4→4-4→

[0180] 5-5→7-5" is read as "0 to 4 4 to 5 5 to 5 7 to 5". That is, only the semantics of "-" are read, and the semantics of "→" are not read. In addition, the punctuation mark "-" in "0-4", "4-4", "5-5" and "7-5" is also misread as "to", which should actually be read as "ratio".

[0181] The text content displayed on the sixth news interface 515 includes "13-13 / 15-19 / 18-20 / 22-19" and "13-13 draw", wherein "-" and " / " in the text content are punctuation marks with multiple semantics. In the related art, when performing voice broadcasting on the content of the sixth news interface 515, "13-13 / 15-19 / 18-20 / 22-19" is read as "one three one three one five one nine one eight two zero two two one nine". That is, the semantics of "-" are not read; 13-13 in "13-13 draw" is misread as "thirteen to thirteen", which should be read as "13 ratio 13".

[0182] The text content displayed on the seventh news interface 516 includes "86-78 win", wherein "-" in the text content is a punctuation mark with multiple semantics. In the related art, when performing voice broadcasting on the content of the seventh news interface 516, "86-78 win" is read as "eighty-six seventy-eight win". That is, the semantics of "-" are not read.

[0183] The text content displayed on the eighth news interface 517 includes "Open Gallery—Edit—AI Photo Retouch—Image Expansion", wherein "—" in the text content is a punctuation mark with multiple semantics. In the related art, when performing voice broadcasting on the content of the eighth news interface 517, "Open Gallery—Edit—AI Photo Retouch—Image Expansion" is read as "Open Gallery Edit AI Photo Retouch Image Expansion". That is, the semantics of "—" are not read. For punctuation marks with multiple semantics, in the related art, not reading and misreading the semantics of the punctuation marks during voice broadcasting will make the broadcast voice unable to accurately convey the information of the original broadcast content, resulting in user misunderstanding or comprehension barriers, and affecting the user's human-computer interaction experience.

[0184] In other words, in related technologies, when TTS modules rewrite punctuation marks with multiple semantic meanings according to built-in rules, they are prone to rewriting these marks with incorrect semantic meanings, or failing to rewrite them at all, resulting in low rewriting accuracy. Low rewriting accuracy for punctuation marks with multiple semantic meanings leads to inaccurate speech delivery, meaning the delivered speech cannot accurately convey the information of the original content.

[0185] In this embodiment, punctuation marks with multiple semantic meanings are rewritten into text content with corresponding semantic meanings before being input into the TTS module. That is, the semantic meaning of the punctuation marks is determined, and they are rewritten into text content with corresponding semantic meanings, without the TTS module rewriting punctuation marks with multiple semantic meanings according to its built-in rules. This reduces the probability of misreading or omitting the semantic meaning of punctuation marks due to incorrect rewriting by the TTS module, allowing the audio data output by the TTS module to more accurately convey the information of the original broadcast content.

[0186] In some embodiments, for text containing punctuation marks with multiple semantic meanings, the electronic device 400 can first identify the semantic usage of the punctuation marks with multiple semantic meanings based on the context, and then rewrite the punctuation marks with multiple semantic meanings into text with the target semantic meaning according to the rewriting rules corresponding to the semantic usage. The target semantic meaning is one of the multiple semantic meanings. The electronic device 400 can determine the semantic usage of the punctuation mark in the current context based on the context.

[0187] For example, see Figure 6 The illustration shows a schematic diagram of a punctuation rewriting process provided in an embodiment of this application. The method may include the following steps:

[0188] Step S601: Electronic device 400 acquires the content to be broadcast.

[0189] The content to be broadcast refers to the content that needs to be broadcast aloud, which may include at least one of text, images, and tables.

[0190] For example, in Figure 1A In the article broadcasting scenario shown, mobile phone 100 can respond to the user's click operation on the reading control 110 and obtain the content to be broadcast based on the news or article displayed on the browser interface 111. At this time, the content to be broadcast can include the content of the news or article.

[0191] For example, in Figure 1BIn the screen reading scenario shown, the mobile phone 100 can respond to user operations (such as operations to instruct the playback of the entire screen content, or operations to instruct the playback of content selected by the user), taking the entire screen content as the content to be played, or the content selected by the user as the content to be played, and thus obtaining the content to be played.

[0192] For example, in Figure 1C In the notification broadcast scenario shown, after receiving a notification message that needs to be broadcast, mobile phone 100 can use the message content of the notification message as the content to be broadcast, and thus obtain the content to be broadcast.

[0193] For example, electronic device 400 can identify screenshots or long screenshots to obtain the content to be read aloud. At this time, the user can input a screenshot operation or a long screenshot operation to electronic device 400 and instruct electronic device 400 to read aloud the content in the screenshot or long screenshot.

[0194] This application does not limit the method by which the electronic device 400 obtains the content to be broadcast, nor does it limit the content to be broadcast.

[0195] Step S602: Electronic device 400 determines whether the content to be broadcast contains text with punctuation marks that have multiple semantic meanings. If so, proceed to step S603.

[0196] If a segment of text (such as a word or a sentence) in the content to be broadcast contains punctuation marks with multiple semantic meanings, then the content to be broadcast is considered to contain text with punctuation marks with multiple semantic meanings; otherwise, the content to be broadcast is considered not to contain text with punctuation marks with multiple semantic meanings.

[0197] If the content to be broadcast does not contain punctuation marks with multiple semantic meanings, and no audio optimization or other text optimization operations are required (such as correcting or replacing related text based on annotations), the electronic device 400 can input the content to be broadcast into the TTS module for speech synthesis and output audio data. However, if audio optimization and other text optimization operations are required, these operations will be performed accordingly before speech synthesis via the TTS module.

[0198] It should be noted that text containing punctuation marks with multiple semantic meanings can be the original text in the content to be broadcast, or it can be obtained by converting non-text content (such as structured content like images and tables) in the content to be broadcast into text content. There is no limitation here.

[0199] Step S603: The electronic device 400 determines the semantic usage of a punctuation mark with multiple semantics.

[0200] Punctuation marks with multiple semantics have different semantic usages in different contexts, that is, their semantics vary in different contexts. The electronic device 400 may determine the semantic usage of the punctuation mark with multiple semantics in the current context in combination with the context of the punctuation mark with multiple semantics. For example, the electronic device 400 may input the text containing the punctuation mark with multiple semantics into a semantic model, so that the semantic model can understand the semantic usage of the punctuation mark with multiple semantics in the current context.

[0201] Step S604: For the text containing the punctuation mark with multiple semantics, the electronic device 400 uses the rewriting rule corresponding to the semantic usage of the punctuation mark to rewrite the punctuation mark with multiple semantics into text of the target semantics.

[0202] For each punctuation mark with multiple semantics, each semantic usage corresponds to a rewriting rule, which is used to rewrite the punctuation mark into text of the corresponding semantics. In practical applications, a mapping relationship table between semantic usages and rewriting rules may be preset, and after the semantic usage is determined, the corresponding rewriting rule may be found from the mapping relationship table between semantic usages and rewriting rules.

[0203] For example, refer to Figure 7 which is a schematic diagram of a mapping relationship table between semantic usages and transcription rules (hereinafter referred to as rewriting rules) provided in the embodiments of the present application. This mapping relationship table exemplarily shows punctuation marks with multiple semantics such as hyphen, dot, colon, separator, special symbol, greater than sign, multiplication sign and plus sign, different semantic usages of the same punctuation mark, and the rewriting rules corresponding to different semantic usages.

[0204] As shown in Figure 7 , for the punctuation mark "-", if the semantic usage is date, the rewriting rule is: rewrite to the corresponding Chinese date (that is, year, month, day or date). That is, rewrite the punctuation mark "-" to "year", "month", "day" or "date". If the semantic usage is negative value, the rewriting rule is: negative. That is, rewrite the punctuation mark "-" to "negative". If the semantic usage is temperature, the rewriting rule is: below zero. That is, rewrite the punctuation mark "-" to "below zero". If the semantic usage is score, the rewriting rule is: to. That is, rewrite the punctuation mark "-" to "to".

[0205] For the punctuation mark ":", if the semantic usage is score or ratio, the rewriting rule is: to. That is, rewrite the punctuation mark ":" to "to". If the semantic usage is time, the rewriting rule is: rewrite the time to the corresponding time format.

[0206] For the punctuation mark "+", if the semantic usage is addition, the rewriting rule is: add. That is, rewrite the punctuation mark "+" as "add". If the semantic usage is a positive number or chemical valence, the rewriting rule is: positive. That is, rewrite the punctuation mark "+" as "positive".

[0207] For details on the rewriting rules corresponding to the various semantic uses of punctuation marks with multiple meanings, please refer to [link / reference needed]. Figure 7 I will not go into details here.

[0208] according to Figure 7 As shown in the mapping table, after recognizing the semantic usage of punctuation marks with multiple semantic meanings, electronic device 400 can find the corresponding rewriting rule based on the semantic usage of the punctuation mark, and then rewrite the punctuation mark into text with the target semantic meaning according to the rewriting rule. For example, for the punctuation mark "-", if the semantic usage is a negative value, the target semantic text is "negative".

[0209] It should be noted that, Figure 7 The mapping table shown is merely an example and does not limit the punctuation marks, semantic usages, and rewriting rules with multiple semantics in the embodiments of this application. For example, punctuation marks with multiple semantics include... Figure 7 Other types of punctuation marks may precede the connectors, dots, and colons shown. Furthermore, the rewriting rules corresponding to the same semantic usage are not limited to... Figure 7 As shown, any rewritten rule can accurately express the semantics of a punctuation mark in its current usage. For example, for the punctuation mark "-", when the semantic usage is temperature, Figure 7 The rewriting rule shown is "below zero"; however, in other embodiments, the rewriting rule can also be "negative". Both rewriting rules can accurately express the semantics of the punctuation mark "-" in the current semantic usage.

[0210] Understandably, in text containing punctuation marks with multiple semantic meanings, electronic device 400 uses rewriting rules corresponding to the semantic usage to rewrite the punctuation marks with multiple semantic meanings into text with the target semantic meaning, thereby obtaining the rewritten text. The rewritten text includes the text with the target semantic meaning.

[0211] The content to be broadcast can consist only of text containing punctuation marks with multiple semantic meanings, or it can include other content and text containing punctuation marks with multiple semantic meanings. After rewriting the punctuation marks with multiple semantic meanings, the rewritten content to be broadcast includes the rewritten text. The rewritten content to be broadcast can be input into the TTS module to perform speech synthesis and obtain audio data.

[0212] For example, see Figure 8Ais a schematic diagram of punctuation rewriting shown, where the first content 81 to be rewritten is specifically "Sports News · The basketball open tournament from 7.6 to 7.7, player Zhang San scored 17.5 points · Juventus won 1-0 last night, breaking the record set in 2000-9". Among them, the punctuation marks "·", "." and "-" are punctuation marks with multiple semantics. The first content 81 to be rewritten is a text containing punctuation marks with multiple semantics.

[0213] A semantic model is used to identify that the semantic usage of the punctuation mark "·" in the first content 81 to be rewritten is a project serial number; according to the semantic usage "project serial number", from Figure 7 the corresponding rewriting rule is found, and the two "·" in the first content 81 to be rewritten are rewritten into "One" and "Two" respectively using the rewriting rule.

[0214] The semantic model is used to identify that the semantic usage of "." in "7.6-7.7" is a date; according to the semantic usage "date", from Figure 7 the corresponding rewriting rule is found; the "." in "7.6-7.7" is rewritten as "month" using the rewriting rule.

[0215] Similarly, the semantic model is used to identify that the semantic usage of "-" in "7.6-7.7" is variation or range; according to the semantic usage "variation or range", from Figure 7 the corresponding rewriting rule is found, and the "-" is rewritten as "to" using the rewriting rule. In addition, according to the context, "7.6-7.7" is finally rewritten into the text "July 6th to July 7th". Certainly, it can also be rewritten into the text "June 7 to July 7", as long as the rewritten text has smooth semantics.

[0216] The semantic model is used in rewriting to identify that the semantic usage of "." in "17.5 points" is a numerical value; according to the semantic usage "numerical value", from Figure 7 the corresponding rewriting rule is found, and the "." in "17.5 points" is rewritten as "point" using the rewriting rule.

[0217] The semantic model is used to identify that the semantic usage of "-" in "1-0" is a score; according to the semantic usage "score", from Figure 7 the corresponding rewriting rule is found, and the "-" in "1-0" is rewritten as "to" using the rewriting rule.

[0218] The semantic model is used to identify that the semantic usage of "-" in "2000-9" is a date; according to the semantic usage "date", from Figure 7 the corresponding rewriting rule is found, and the "-" in "2000-9" is rewritten as "year" using the rewriting rule.

[0219] The first set of content to be rewritten, 81, contains punctuation marks with multiple semantic meanings, such as "·", ".", and "-", as well as other text content (e.g., basketball tournament). After rewriting the punctuation marks with multiple semantic meanings into text with the target semantic meaning, the target semantic text is integrated with the other text content to obtain, as shown below. Figure 8A The first rewritten content 82 shown is the rewritten text. The first rewritten content 82 is specifically: "Sports News 1: In the basketball open tournament from July 6th to July 7th, player Zhang San scored 17.5 points; 2: Last night, Juventus won 1-0, breaking the record set in September 2000."

[0220] Of course, you can also directly rewrite punctuation marks with multiple meanings, such as "·", ".", and "-", into text with the target meaning in the first content to be rewritten 81 to obtain the first rewritten content 82. That is, you can directly rewrite punctuation marks with multiple meanings into text with the target meaning in the original text to obtain the rewritten text.

[0221] For example, see also Figure 8B The illustration shows another diagram of punctuation rewriting. The second content to be rewritten, 83, is specifically "The highway is now open! Shenzhen → Zhongshan only costs 5 yuan! Tests show that you can purchase tickets via a mini-program: Buy Tickets → Bus → Select Time." Here, the punctuation mark "→" is a punctuation mark with multiple semantic meanings. The second content to be rewritten, 83, is text containing punctuation marks with multiple semantic meanings.

[0222] Using a semantic model, we can identify the semantic usage of "→" in "Shenzhen → Zhongshan" as a change; based on this semantic usage change, from... Figure 7 The corresponding rewriting rule was found in the database, and the rewriting rule was used to rewrite the "→" in "Shenzhen→Zhongshan" as "to".

[0223] Using a semantic model, we identify the semantic usage of "→" in the sequence "purchase ticket → bus → select time" as an operation click path; based on the semantic usage "operation click path", from... Figure 7 The corresponding rewriting rule found is: delete symbols and add "click" and "open" before the operation. Using this rewriting rule, "buy tickets → bus → select time" is rewritten as "open ticket purchase, click bus and select time".

[0224] When rewriting the punctuation mark "→" according to the rewriting rules, since "select time" already contains the verb "select," it is only necessary to delete the "→" before "select time," without adding Chinese verbs such as "open" or "click" before "select time." However, since "purchase ticket → bus" does not contain a verb, not only should the "→" be deleted, but Chinese verbs such as "open" and "click" should also be added before "purchase ticket" and "bus."

[0225] Finally, after rewriting the punctuation marks with multiple semantics into the text of the target semantics, the text of the target semantics is integrated with other content to obtain, for example, Figure 8B the second rewritten content 84 as shown. The second rewritten content 84 is the rewritten text. Specifically, the second rewritten content 84 is "The expressway is open, and it only costs 5 yuan to travel from Shenzhen to Zhongshan! Actual measurement shows that you can open the ticket purchase through the mini-program, select the time by clicking on the bus, and complete the ticket purchase". Certainly, the punctuation marks can also be rewritten directly in the original text (i.e., the second to-be-rewritten content 83) to obtain the second rewritten content 84.

[0226] The first rewritten content 82 and the second rewritten content 84 can be input into the TTS module, so that the TTS module can perform processing such as speech synthesis to output audio data. In this way, when performing voice broadcast according to the audio data, the punctuation marks with multiple semantics in the first rewritten content 82 and the second rewritten content 84 will not be mispronounced or unpronounced, and the broadcast voice can accurately convey the information of the original broadcast content.

[0227] The text of the target semantics may refer to the text content obtained by rewriting the punctuation marks with multiple semantics into the text of the corresponding semantics using the rewriting rules corresponding to the semantic usage. For example, in Figure 8A the first to-be-rewritten content, the "-" in "1-0" is rewritten into the Chinese character "bi" (meaning "ratio"). At this time, the text of the target semantics is "bi".

[0228] It should be noted that identifying the semantic usage of punctuation marks using a semantic model may refer to: inputting the text containing punctuation marks with multiple semantics into the semantic model, so that the semantic model can identify the semantic usage of the punctuation marks in combination with the context (in other words, identify the semantics of the punctuation marks in the current context).

[0229] The above steps S603 and S604 may be executed by the text rewriting module. In specific applications, the text containing punctuation marks with multiple semantics may be input into the text rewriting module; the text rewriting module rewrites the punctuation marks with multiple semantics in combination with the context according to the processes and principles of steps S603 and S604, and outputs the rewritten text content.

[0230] In one implementation, the text rewriting module may be a functional module that performs data processing based on the processes and principles of the above steps S603 and S604. In another implementation, the text rewriting module may be implemented by a Large Language Model (LLM). At this time, the text containing punctuation marks with multiple semantics may be input into the LLM model to obtain the rewritten text output by the LLM model.

[0231] See Figure 9 The diagram illustrates a method for rewriting punctuation marks based on an LLM model. First, the LLM model is trained using training data to obtain the base LLM model. Then, the base LLM model is fine-tuned to obtain the LLM-SFT. Here, Supervised Fine-Tuning (SFT) of the LLM model refers to further training the model using labeled datasets on top of pre-training, enabling the model to better perform specific tasks or adapt to specific application scenarios.

[0232] like Figure 9 As shown, the text to be rewritten is "In the match between April 5th and 7th, 2024, Brazil easily defeated Argentina 4-1, advancing to the Copa America final." The prompt word (i.e., a text correction expert) is: As a text correction expert, you can identify sentences containing punctuation marks "-", "——", "~", ".", ":", and " / " in the input text, and based on the semantic understanding of the context, convert and output the text. The output text format is consistent with the input text, with no new content except for punctuation conversion. Given the input text "In the match between April 5th and 7th, 2024, Brazil easily defeated Argentina 4-1, advancing to the Copa America final," please output the corrected text.

[0233] It should be noted that, Figure 9 The prompts shown below are merely examples. In specific applications, different forms of prompts can be constructed as needed, and no limitations are imposed here.

[0234] Input the constructed Prompt into LLM-SFT to obtain the rewritten text output by LLM-SFT. The rewritten text is: "In the matches from April 5th to 7th, 2024, the Brazilian team easily defeated the Argentine team 4-1 and advanced to the Copa America final."

[0235] Of course, the text rewriting module can also be implemented in other ways, not limited to the LLM model, and will not be restricted here.

[0236] In other embodiments, for text containing punctuation marks with multiple semantic meanings, the electronic device 400 can rewrite the punctuation marks with multiple semantic meanings into texts with different semantic meanings according to various rewriting rules, and then obtain multiple comparison texts; then, based on the similarity between each comparison text and the original text (i.e., the text containing punctuation marks with multiple semantic meanings), the optimal rewriting result is determined, and the rewritten text is obtained.

[0237] For example, see Figure 10 The illustration shows a schematic diagram of a punctuation rewriting process provided in an embodiment of this application. The method may include the following steps:

[0238] Step S1001: Electronic device 400 acquires the content to be broadcast.

[0239] Step S1002: Electronic device 400 determines whether the content to be broadcast contains text with punctuation marks that have multiple semantic meanings. If so, proceed to step S1003.

[0240] It should be noted that text containing punctuation marks with multiple semantic meanings can be the original text in the content to be broadcast, or it can be obtained by converting non-text content (such as images and tables) in the content to be broadcast into text content.

[0241] Step S1003: For text containing punctuation marks with multiple semantic meanings, the electronic device 400 rewrites the punctuation marks with multiple semantic meanings according to the rewriting rules of each punctuation mark with multiple semantic meanings, and obtains multiple comparison texts.

[0242] If the original text (i.e., text containing punctuation marks with multiple semantic meanings) contains only one punctuation mark with multiple semantic meanings, then obtain the rewriting rules corresponding to all semantic uses of that punctuation mark; using all rewriting rules, rewrite the punctuation marks with multiple semantic meanings in the original text into texts with corresponding semantic meanings, obtaining multiple rewritten texts. Each rewritten text is used as a comparison text with the original text.

[0243] For example, suppose the original text contains only one punctuation mark "+" with multiple semantic meanings; according to Figure 7 The mapping table shown indicates that there are three semantic uses for the "+" symbol, each corresponding to a rewriting rule. By applying these three rewriting rules to rewrite the "+" symbol in the original text into text with the corresponding semantic meaning, three contrasting texts can be obtained.

[0244] If the original text contains at least two punctuation marks with multiple semantic meanings, the rewriting rules for each punctuation mark with multiple semantic meanings can be obtained. These rules can then be combined to create multiple combination rules. Using these combination rules, the punctuation marks with multiple semantic meanings in the original text can be rewritten into texts with corresponding semantic meanings, resulting in multiple rewritten texts. Each rewritten text serves as a comparison text with the original text.

[0245] For example, suppose the original text contains the punctuation marks "+" and ".", which have multiple semantic meanings. From Figure 7It can be seen that there are three rewriting rules for "+" and three rewriting rules for ".". At this point, the three rewriting rules for "+" and the three rewriting rules for "." are combined to obtain nine possible combinations. Each combination includes one rewriting rule for "+" and one rewriting rule for ".". When rewriting punctuation marks using one combination rule, the "+" rule within that combination is used to rewrite the "+" in the original text into text with the corresponding meaning, and the "." rule within that combination is used to rewrite the "." in the original text into text with the corresponding meaning. This process continues, using the nine combination rules to rewrite the "+" and "." in the original text to obtain nine contrasting texts. In step S1004, the electronic device 400 determines the optimal rewritten text based on the similarity between each contrasting text and the original text.

[0246] In practical applications, the electronic device 400 can calculate the semantic vectors of each comparison text and the semantic vector of the original text (i.e., text containing punctuation marks with multiple semantic meanings); calculate the similarity between the semantic vectors of each comparison text and the semantic vector of the original text; and determine the comparison text with the highest similarity as the optimal rewritten text. That is, when the semantic vector of a comparison text has the highest similarity to the semantic vector of the original text, the comparison text is determined as the optimal rewritten text.

[0247] Of course, if the original text contains at least two punctuation marks with multiple semantic meanings, multiple rewriting rules can be applied separately for each punctuation mark with multiple semantic meanings to rewrite the punctuation mark in the original text into text with the corresponding semantic meaning. Then, the optimal rewritten text is determined based on the similarity between the original text and the comparison text. Determining the optimal rewritten text for each punctuation mark allows us to determine the optimal rewriting rule for that punctuation mark. By applying the optimal rewriting rule for each punctuation mark to rewrite the corresponding punctuation mark in the original text, we can obtain the final optimal rewritten text.

[0248] Understandably, the content to be broadcast can consist only of text containing punctuation marks with multiple semantic meanings, or it can include other content and text containing punctuation marks with multiple semantic meanings. After rewriting the punctuation marks with multiple semantic meanings to obtain the optimal rewritten text, the rewritten content to be broadcast includes the optimal rewritten text. The rewritten content to be broadcast can be input into the TTS module to perform speech synthesis and obtain audio data.

[0249] Steps S1003 and S1004 described above can be executed by the text rewriting module. In specific applications, text containing punctuation marks with multiple semantic meanings can be input into the text rewriting module; the text rewriting module rewrites the punctuation marks with multiple semantic meanings according to the process and principle of steps S1003 and S1004, combined with the context, and outputs the rewritten text.

[0250] In one implementation, the text rewriting module can be a functional module that processes data based on the flow and principles of steps S1003 and S1004 described above. In another implementation, the text rewriting module can be implemented using a semantic coding model. For example, the text rewriting module can be implemented using a semantic coding model. See also... Figure 11 The diagram illustrates a method for rewriting punctuation marks based on a semantic encoding model. Input text is fed into a text embedding model, and the resulting semantic vector is output by the model. The input text contains punctuation marks with multiple semantic meanings. Various rewriting rules are used to rewrite the punctuation marks with different semantic meanings in the input text into texts with corresponding semantic meanings, resulting in comparison texts 1 through n. n is a positive integer greater than or equal to 2.

[0251] If the input text contains only one punctuation mark with multiple semantic meanings, then multiple rewriting rules can refer to all rewriting rules for that punctuation mark. If the input text contains at least two punctuation marks with multiple semantic meanings, then multiple rewriting rules can refer to multiple combination rules. Combination rules are obtained by arranging and combining the rewriting rules for various punctuation marks.

[0252] The comparison texts 1 through n are input into the text embedding model to obtain semantic vectors (i.e., semantic vectors) output by the model. Each comparison text corresponds to a semantic vector. Based on the semantic vectors of the comparison texts and the semantic vectors of the input texts, semantic space similarity is calculated to obtain the output text. That is, the similarity between the semantic vectors of each comparison text and the semantic vector of the input text is calculated, and the output text (i.e., the optimal rewritten text) is obtained based on the similarity score.

[0253] In this context, the output text is the target comparison text, which refers to the comparison text with the highest similarity semantic vector among comparison texts 1 to n. For example, assuming that the semantic vector of comparison file m has the highest similarity to the semantic vector of the input text, then comparison text m is the target comparison text (i.e., the optimal rewritten text mentioned above), and comparison file m is the output text. Comparison text m consists of texts from comparison text 1 to n.

[0254] see Figure 12 the schematic diagram of the punctuation rewriting process based on the semantic coding model, where the input text is "2024-5-12, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semi-final was in 2022-5, and we look forward to the Chinese team achieving new excellent results". The input text is input into the Text embedding model to obtain the semantic vector of the input text (i.e. 0.004, -0.001, 0.020...-0.023).

[0255] In the input text, the punctuation mark with multiple semantics is "-". According to Figure 7 it can be known that the punctuation mark "-" has multiple rewriting rules, Figure 12 five rewriting rules for the punctuation mark "-" (i.e. rewriting rule 1 to rewriting rule 5) are adopted to rewrite the "-" in the input text respectively, so as to obtain comparison text 1 to comparison text 5. The comparison text 1 to comparison text 5 are respectively input into the Text embedding model to obtain the semantic vector corresponding to each comparison text.

[0256] Wherein, the rewriting rule 1 is to rewrite into Chinese dates such as year, month, day or date, and the comparison text 1 is "May 12, 2024, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semi-final was in May 2022, and we look forward to the Chinese team achieving new excellent results". The semantic vector of comparison text 1 is 0.004, -0.001, 0.020...-0.025.

[0257] The rewriting rule 2 is to rewrite into the pronunciation "gang" of the symbol itself, and the comparison text 2 is "2024gang5gang12, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semi-final was in 2022gang5, and we look forward to the Chinese team achieving new excellent results". The semantic vector of comparison text 2 is 0.025, -0.011, 0.022...-0.014.

[0258] The rewriting rule 3 is to rewrite into "jian" (meaning minus), and the comparison text 3 is "2024jian5jian12, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semi-final was in 2022jian5, and we look forward to the Chinese team achieving new excellent results". The semantic vector of comparison text 3 is 0.050, -0.027, 0.020...-0.023.

[0259] The rewriting rule 4 is to rewrite into "dao" (meaning to), and the comparison text 4 is "2024dao5dao12, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semi-final was in 2022dao5, and we look forward to the Chinese team achieving new excellent results". The semantic vector of comparison text 4 is 0.005, 0.032, -0.020...0.029.

[0260] Rewriting rule 5 is to rewrite it as "negative". Comparative text 5 reads, "2024 -5 -12, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semifinals was in 2022 -5. We look forward to the Chinese team achieving even greater success." The semantic vectors of comparative text 5 are 0.004, 0.001, 0.007…-0.045.

[0261] Calculate the similarity between the semantic vectors of comparison texts 1 through 5 and the semantic vector of the input text, and obtain the output text based on the similarity score. Figure 12 The output text is: "On May 12, 2024, the Chinese badminton team will face the Indonesian team in the Thomas Cup. The last time the two teams met in the semifinals was in May 2022. We look forward to the Chinese team achieving great results again." That is, among comparison files 1 to 5, comparison file 1 has the highest similarity to the semantic vector of the input text, and comparison file 1 is determined as the output text (i.e., the optimal rewritten text).

[0262] Of course, the text rewriting module can also be implemented in other ways, not limited to the semantic coding model, and will not be restricted here.

[0263] As can be seen from the above, this embodiment of the application, by rewriting punctuation marks with multiple semantic meanings in the content to be broadcast into text with corresponding semantic meanings before inputting it into the TTS module, enables the audio data output by the TTS module to more accurately convey the information of the original broadcast content. That is, it reduces the probability of misreading or omitting the semantic meaning of punctuation marks during voice broadcasting, making the voice broadcasting more accurate and able to convey the information of the original broadcast content more precisely.

[0264] For example, in Figure 12 In Chinese, if you directly input text into a TTS module, the module will rewrite the punctuation mark "-" according to its built-in rules, which may result in errors. Figure 5A and Figure 5B The semantics of the punctuation mark "-" are not read or are misread. This application's embodiment rewrites the punctuation mark "-" in the input text into text with the corresponding semantics, and then displays the rewritten text (i.e., Figure 12 The output text is input into the TTS module. Since the punctuation mark "-" has been rewritten as Chinese dates such as year, month, and day (i.e., the semantics of the punctuation mark are already determined), the TTS module does not need to rewrite the punctuation mark "-" according to its built-in rules, reducing the probability of rewriting errors. This makes the audio data output by the TTS module more accurate.

[0265] Rewrite phrases with multiple meanings (composed of numbers and letters):

[0266] Phrases consisting of numbers and letters can have multiple meanings. That is, the meaning of phrases consisting of numbers and letters may differ in different contexts.

[0267] When the content to be played includes phrases composed of numbers and letters with multiple semantic meanings, the relevant technology directly inputs the content to be played into the TTS module. The TTS module then determines the semantic meaning of the phrases according to its built-in rules and outputs audio data based on the determined semantic meaning. However, the built-in rules of the TTS module often cannot accurately understand the semantic meaning of phrases composed of numbers and letters in different contexts, and may misinterpret the semantic meaning of these phrases.

[0268] For example, the content to be broadcast includes the text "The average annual salary of a senior designer is 500,000." Here, "500,000" is a phrase composed of numbers and letters, and has multiple meanings. "500,000" can be understood as "50 watts" or "500,000."

[0269] In related technologies, TTS modules, according to built-in rules, may interpret the semantics of "50w" in "the average annual salary of a senior designer is 50w" as "50 watts", thus leading to the phenomenon of misreading the semantics of the phrase.

[0270] When using words composed of numbers and letters that have multiple semantic meanings, the technology may misread the semantic meaning of the words during voice broadcasting. This can cause the broadcasted voice to fail to accurately convey the information of the original broadcast content, leading to misunderstandings or comprehension difficulties for users and affecting their human-computer interaction experience.

[0271] In this embodiment, phrases consisting of numbers and letters with multiple semantic meanings are rewritten into text content with corresponding semantic meanings before being input into the TTS module. That is, the semantic meaning of phrases consisting of numbers and letters is already determined, and these phrases are rewritten into text content with corresponding semantic meanings, without the TTS module rewriting phrases with multiple semantic meanings according to its built-in rules. This reduces the probability of misreading phrase meanings due to incorrect rewriting of phrases with multiple semantic meanings by the TTS module, allowing the audio data output by the TTS module to more accurately convey the information of the original broadcast content.

[0272] In some embodiments, the electronic device 400 can first identify the semantic usage of phrases with multiple semantic meanings based on the context, and then rewrite the phrases with multiple semantic meanings into text with the target semantic meaning according to the rewriting rules corresponding to the semantic usage. The target semantic meaning is one of the multiple semantic meanings. That is, the electronic device 400 can determine the semantic usage of phrases composed of numbers and letters in the current context based on the context.

[0273] For example, see Figure 13The illustration shows a schematic diagram of a word rewriting process consisting of numbers and letters provided in an embodiment of this application. The method may include the following process steps:

[0274] Step S1301: Electronic device 400 acquires the content to be broadcast.

[0275] In step S1302, the electronic device 400 determines whether the content to be broadcast contains text consisting of phrases with multiple semantic meanings and composed of numbers and letters. If so, proceed to step S1303.

[0276] If a segment of text (such as a word or sentence) in the content to be broadcast contains a phrase consisting of numbers and letters, and that phrase has multiple semantic meanings, then the content to be broadcast is considered to contain text consisting of a phrase consisting of numbers and letters with multiple semantic meanings; otherwise, the content to be broadcast is considered not to contain text consisting of a phrase consisting of numbers and letters with multiple semantic meanings.

[0277] If the content to be broadcast does not contain text consisting of phrases with multiple semantic meanings and composed of numbers and letters, and no audio optimization or other text optimization operations (such as rewriting punctuation marks with multiple semantic meanings) are required, the electronic device 400 can input the content to be broadcast into the TTS module for speech synthesis and output audio data. However, if audio optimization and other text optimization operations are required, these operations will be performed accordingly before speech synthesis via the TTS module. It should be noted that text containing phrases with multiple semantic meanings can be the original text in the content to be broadcast, or it can be obtained by converting non-text content (such as images and tables) in the content to be broadcast into text content.

[0278] In some embodiments, a phrase having multiple semantic meanings and consisting of numbers and letters typically has a character count less than or equal to a first character count threshold. For example, the phrase "50w" has 3 characters, and the first character count threshold can be 4 or 5, which is not limited here.

[0279] Step S1303: Electronic device 400 determines the semantic usage of phrases with multiple semantic meanings.

[0280] Phrases with multiple meanings have different semantic uses in different contexts. Electronic device 400 can determine the semantic use of a phrase with multiple meanings in the current context by considering its surrounding context. For example, electronic device 400 can understand the semantic use of a phrase with multiple meanings in the current context through a semantic model.

[0281] Step S1304: For a text containing a phrase with multiple semantics, the electronic device 400 uses a rewriting rule corresponding to the semantic usage of the phrase to rewrite the phrase with multiple semantics into a text of the target semantics.

[0282] For each phrase with multiple semantics, each semantic usage corresponds to a rewriting rule, which is used to rewrite the phrase composed of numbers and letters into a text of the corresponding semantics. In practical applications, a mapping relationship table between semantic usages and rewriting rules may be preset. After the semantic usage is determined, the corresponding rewriting rule is searched from the mapping relationship table between semantic usages and rewriting rules. Exemplarily, the following Table 1 shows the mapping relationship between each semantic usage of the phrase composed of numbers and letters and the rewriting rule.

[0283] Table 1

[0284]

[0285] In the above Table 1, the phrases composed of numbers and letters with multiple semantics may include: number + w, number + M, number + T, and number + F.

[0286] For the phrase of number + w, there are two semantic usages (that is, two semantics). One semantic usage is capital or quantity, and the other semantic usage is power. The above Table 1 gives examples for these two semantic usages respectively. When the text containing the phrase with multiple semantics is "The average annual salary of a senior designer is 50w", the semantic model is used to understand that the semantic usage of "50w" is "capital". According to the mapping relationship between the semantic usages and the rewriting rules shown in the above Table 1, it is determined that the rewriting rule corresponding to the semantic usage "capital" is: rewrite into Chinese numeral + ten thousand. Using this rewriting rule, "50w" in "The average annual salary of a senior designer is 50w" is rewritten into the Chinese "50万", so as to obtain the rewritten text "The average annual salary of a senior designer is 50万".

[0287] When the text containing the phrase with multiple semantics is "The power of a notebook is generally 50w", the semantic model is used to understand that the semantic usage of "50w" is "power". According to the mapping relationship between the semantic usages and the rewriting rules shown in the above Table 1, it is determined that the rewriting rule corresponding to the semantic usage "power" is: rewrite into Chinese numeral + watt. Using this rewriting rule, "50w" in "The power of a notebook is generally 50w" is rewritten into the Chinese "50瓦", so as to obtain the rewritten text "The power of a notebook is generally 50瓦".

[0288] For number + M, number + T, and number + F, the above Table 1 also gives examples corresponding to various semantic usages respectively. For details, please refer to the above Table 1, which will not be repeated here.

[0289] The target semantic text refers to the text content obtained by rewriting the phrase into text with the corresponding semantic meaning using the rewriting rules corresponding to the semantic usage. For example, in the two examples of the number +w in Table 1 above, the target semantic text is "50 watts" and "500,000".

[0290] In text containing phrases with multiple semantic meanings, electronic device 400 uses rewriting rules corresponding to the semantic usage to rewrite the phrases with multiple semantic meanings into text with the target semantic meaning, thereby obtaining rewritten text. The rewritten text includes text with the target semantic meaning.

[0291] The content to be broadcast can consist only of text containing phrases with multiple semantic meanings, or it can include other content and text containing phrases with multiple semantic meanings. After rewriting the phrases with multiple semantic meanings, the rewritten content to be broadcast includes the rewritten text. The rewritten content to be broadcast can be input into the TTS module to perform speech synthesis and obtain audio data.

[0292] For example, in the content to be broadcast, the text containing phrases with multiple semantic meanings is "The average annual salary of a senior designer is 500,000," while the text with the target semantic meaning is "500,000." Rewriting "500,000" as "500,000" in the text containing phrases with multiple semantic meanings yields the rewritten text "The average annual salary of a senior designer is 500,000."

[0293] The above steps S1303 and S1304 can be executed by the text rewriting module. In specific applications, text containing phrases with multiple semantic meanings can be input into the text rewriting module; the text rewriting module rewrites the phrases with multiple semantic meanings according to the process and principle of steps S1303 and S1304, combined with the context, and outputs the rewritten text.

[0294] In one implementation, the text rewriting module can be a functional module that processes data based on the flow and principles of steps S1303 and S1304 described above. In another implementation, the text rewriting module can be implemented using an LLM model.

[0295] For example, see Figure 14 The diagram illustrates a method for rewriting phrases with multiple semantic meanings based on an LLM model. First, the LLM model is trained using training data to obtain the basic LLM model; then, the basic LLM model is fine-tuned to obtain the LLM-SFT.

[0296] like Figure 14As shown, the text to be rewritten is "How fast is 100M broadband in theory?", with the phrase "100M" having multiple semantic meanings. The prompt word constructed based on this text is: As a text correction expert, you can identify sentences in the input text that contain letters representing units of measurement, and based on the semantic understanding of the context, convert these letters representing units of measurement into appropriate Chinese characters, outputting the converted text with the same format as the input text, with no new content added except for letter conversion. Given the following input text: "How fast is 100M broadband in theory?", please output the corrected text.

[0297] The constructed Prompt is input into LLM-SFT; the rewritten text output by LLM-SFT is obtained. The rewritten text (i.e., the modified text) is specifically "How fast is 100 Mbps broadband in theory?". That is, through the LLM model, the phrase "100M" with multiple semantic meanings is rewritten as "100 Mbps".

[0298] Of course, the text rewriting module can also be implemented in other ways, not limited to the LLM model, and will not be restricted here.

[0299] In other embodiments, for text containing phrases with multiple semantic meanings, the electronic device 400 can rewrite the phrases with multiple semantic meanings into texts with each semantic meaning according to the respective rewriting rules of the phrases, obtaining multiple comparison texts; then, based on the similarity between each comparison text and the original text (i.e., the text containing phrases with multiple semantic meanings), the optimal rewriting result is determined, and thus the rewritten text is obtained. In this case, the specific rewriting process steps can be similar to... Figures 10-12 .Right now, Figures 10-12 Punctuation marks with multiple meanings can be replaced with phrases with multiple meanings to rewrite phrases with multiple meanings.

[0300] For example, regarding the phrase "number + w", the original text (i.e., the text containing phrases with multiple semantic meanings) is "The average annual salary of a senior designer is 500,000". According to Table 1 above, the phrase "500,000" has two semantic uses, resulting in two rewriting rules: rewriting it as "number + ten thousand" and rewriting it as "number + watt". Using these two rewriting rules to rewrite "500,000" in the original text yields two comparison texts: "The average annual salary of a senior designer is 500,000" and "The average annual salary of a senior designer is 50 watts".

[0301] Based on the similarity between the two comparison texts and the original text, the text "The average annual salary of a senior designer is 500,000" has the highest similarity to the original text. Therefore, "The average annual salary of a senior designer is 500,000" is determined as the optimal rewriting result (i.e., the optimal rewritten text).

[0302] Wherein, the similarity between the comparison text and the original text may be calculated through semantic vectors. Specifically, a text embedding model is used to calculate the semantic vector of the comparison text and the semantic vector of the original text; the similarity between the semantic vector of each comparison text and the semantic vector of the original text is calculated; when the similarity between the semantic vector of a certain comparison text and the semantic vector of the original text is the highest, the comparison text is determined as the optimal rewritten text.

[0303] In specific applications, the electronic device 400 rewrites phrases with multiple semantics based on a semantic encoding model. For details, please refer to Figure 11 and Figure 12 the relevant content, which will not be repeated here.

[0304] It can be seen from the above that, in the embodiments of the present application, after rewriting the phrases (composed of numbers and letters) with multiple semantics in the content to be broadcast into texts of corresponding semantics, the processed content is input to the TTS module, which enables the audio data output by the TTS module to more accurately convey the information of the original broadcast content.

[0305] Correcting and replacing the relevant text according to the pinyin annotation or the text annotation:

[0306] The pinyin annotation refers to annotating characters using pinyin. In general, pinyin may be annotated for rare characters or polyphonic characters in the text to assist users in reading.

[0307] The text annotation refers to annotating with characters. In general, when there are some incorrect characters in the text, the correct characters may be annotated at the position of the original text to ensure authenticity and correctness. The text annotation is usually located in parentheses and is used to correct the preceding text.

[0308] For example, refer to Figure 15 which shows a schematic diagram of pinyin annotation and text annotation, the pinyin annotation example 1501 is an introduction to the figure Li Shiqi, and the pinyin "lìyìjī" is used to annotate the pronunciation of "Li Shiqi".

[0309] In Figure 15 the illustrated text annotation example 1502, the original text is "wú kuā zhī jǔ", and to ensure the correctness of the original text, the character "nài" is annotated at the position of the original text, that is, "wú kuā (nài) zhī jǔ".

[0310] When the content to be broadcast includes a pinyin annotation or a text annotation, in the related art, the content to be broadcast is directly input to the TTS module, and the TTS module will not correct and replace the relevant text according to the pinyin annotation and the text annotation, but directly read out the pinyin of the pinyin annotation and the text annotation.

[0311] For example, for Figure 15In the Pinyin annotation example 1501, when the related art performs voice broadcast according to the audio output by the TTS module, the character string "Li Shiqi" is still pronounced as "lìshíqí", and "lìshíqí" will not be corrected to "lìyìjī" according to the Pinyin annotation "lìyìjī". In addition, "郦食其(lìshíqí)" in the Pinyin annotation example 1501 will be pronounced as "lìshíqílìyìjī", that is, the Pinyin annotation "lìyìjī" and the character string "郦食其" annotated by this Pinyin are directly read, resulting in repeated reading. Furthermore, not only "郦食其" in the phrase where the Pinyin annotation "lìyìjī" is located will be mispronounced, but the remaining characters such as "郦食其" and "食其" in the Pinyin annotation example 1501 will also be incorrectly pronounced as "lìshíqí" and "shíqí".

[0312] For Figure 15 The text annotation example 1502 in the related art, when performing voice broadcast according to the audio output by the TTS module, reads "无夸(奈)之举" as "无夸奈之举". That is, the text annotation "奈" and the character "夸" annotated by this text are directly read, and the related text "夸" will not be corrected and replaced according to the text annotation "奈".

[0313] In the related art, the related text is not corrected and replaced according to the Pinyin annotation and the text annotation, but the content to be broadcast is directly input to the TTS module, which will cause problems such as mispronunciation and repeated reading, so that the broadcast voice cannot accurately convey the information of the original broadcast content, further leading to misunderstanding or comprehension obstacles for the user, and affecting the user's human-computer interaction experience.

[0314] However, in the embodiments of the present application, after the related text is corrected and replaced according to the Pinyin annotation or the text annotation, the corrected text is then input to the TTS module, which will not cause problems such as mispronunciation and repeated reading, so that the audio data output by the TTS module can more accurately convey the information of the original broadcast content, and improve the intelligibility of the broadcast content.

[0315] For example, refer to Figure 16 which shows a schematic flowchart of correcting and replacing related text according to a Pinyin annotation, and the process may include the following steps:

[0316] Step S1601: The electronic device 400 acquires the content to be broadcast.

[0317] Step S1602: The electronic device 400 determines whether the content to be broadcast contains a Pinyin annotation. If yes, proceed to step S1603.

[0318] If there is no pinyin annotation in the content to be broadcast, and there is no need to perform sound optimization processing and other text optimization processing operations on the content to be broadcast (for example, rewriting punctuation marks with multiple semantics, etc.), the electronic device 400 may input the content to be broadcast into the TTS module, so as to perform speech synthesis through the TTS module and output audio data. If it is still necessary to perform sound optimization processing and other text optimization operations on the content to be broadcast, the sound optimization processing and other text optimization operations are performed accordingly, and then speech synthesis is performed through the TTS module.

[0319] Step S1603: The electronic device 400 acquires the text containing pinyin annotations from the content to be broadcast.

[0320] Step S1604: The electronic device 400 determines the word group where the pinyin annotation is located from the text containing the pinyin annotation.

[0321] In a specific application, the electronic device 400 may segment the word group where the pinyin annotation is located from the text containing the pinyin annotation (for example, words and sentences containing pinyin annotations). For example, the electronic device 400 may use an LLM model to segment the word group where the pinyin annotation is located.

[0322] For example, the text containing the pinyin annotation is "他是不卑(bùbēi)不亢的人", and the word group segmented by the electronic device 400 using the LLM model is "不卑(bùbēi)不亢", which is the word group where the pinyin annotation is located.

[0323] Step S1605: For the word group where the pinyin annotation is located, the electronic device 400 uses the pinyin to replace the annotated characters in the word group.

[0324] In a specific application, the electronic device 400 may match the pinyin with the characters in the word group according to the number of pinyin and the number of characters in the word group, so as to determine the annotated characters in the word group, and use the pinyin to replace the annotated characters in the word group.

[0325] For example, the word group where the pinyin annotation is located is "不卑(bùbēi)不亢". By matching the pinyin with the characters in the word group, it is determined that "不卑" in the word group is the annotated character. In addition, since the character "不" in "不亢" in the word group is the same as the character "不" in the annotated text "不卑", the character "不" in "不亢" can also be regarded as a character annotated by pinyin. At this time, the characters annotated by pinyin in "不卑(bùbēi)不亢" are "不卑不". The corresponding pinyin is used to replace the annotated characters in the word group, and the rewritten word group "bùbēi bù亢" is obtained.

[0326] Optionally, the method may further include step S1606: the electronic device 400 determines a word group in the content to be broadcast that is completely identical or partially identical to the word group where the phonetic annotation is located, and uses the phonetic alphabet to replace the corresponding characters in the completely identical or partially identical word group.

[0327] If there is no word group in the content to be broadcast that is completely identical or partially identical to the word group where the phonetic annotation is located, step S1606 may not be performed; if there is a word group in the content to be broadcast that is completely identical or partially identical to the word group where the phonetic annotation is located, step S1606 may be performed. Certainly, step S1606 may not be performed regardless of whether there is a word group in the content to be broadcast that is completely identical or partially identical to the word group where the phonetic annotation is located.

[0328] Herein, the word group that is completely identical to the word group where the phonetic annotation is located refers to a word group that does not include the phonetic annotation, but has characters completely identical to the characters of the word group where the phonetic annotation is located.

[0329] The word group that is partially identical to the word group where the phonetic annotation is located refers to a word group that does not include the phonetic annotation, but has characters partially identical to the characters of the word group where the phonetic annotation is located. Herein, the identical characters of the two word groups are the annotated characters.

[0330] In specific applications, the electronic device 400 may match, based on dynamic dictionary matching, a word group that is completely identical or partially identical to the word group where the phonetic annotation is located from the content to be broadcast.

[0331] For example, the word group where the phonetic annotation is located is "bu bei (bùbēi) bu kang", and the word group "bu bei bu kang" is stored in the dynamic dictionary. Based on "bu bei bu kang" stored in the dynamic dictionary, matching is performed within the full text range of the content to be broadcast, to determine whether there is a word group matching "bu bei bu kang".

[0332] At this time, it is determined through dynamic dictionary matching that the content to be broadcast further includes the word group "bu bei bu kang" and the word group "bu bei". The word group "bu bei bu kang" does not include the phonetic annotation, but its characters are completely identical to the characters in the word group "bu bei (bùbēi) bu kang" where the phonetic annotation is located. The word group "bu bei" does not include the phonetic annotation, but its characters are partially identical to part of the characters in the word group "bu bei (bùbēi) bu kang" where the phonetic annotation is located. The identical characters in the two word groups "bu bei" and "bu bei (bùbēi) bu kang" are "bu bei", and the identical characters are the characters annotated by the phonetic alphabet.

[0333] In this case, not only pinyin is used to replace the corresponding characters in the phrase "不卑(bùbēi)不亢" where the pinyin annotation is located; but also pinyin is used to replace the corresponding characters in "不卑不亢" and "不卑" throughout the full text. Using pinyin to replace the characters in "不卑不亢" results in the rewritten phrase "bùbēi bù亢"; using pinyin to replace the characters in "不卑" results in the rewritten phrase "bùbēi".

[0334] To better introduce the process of correcting and replacing relevant text based on pinyin annotations, the following takes Figure 15 the pinyin annotation example 1501 in as an example for description.

[0335] Referring to Figure 17 the shown schematic comparison diagram before and after correcting and replacing relevant text based on pinyin annotations, the content to be broadcast is the pinyin annotation example 1501 in Figure 15 . Figure 15 The left side of is a schematic diagram of the reading process of the related art during voice broadcasting; Figure 15 the right side of is a schematic diagram of the reading process of the embodiment of the present application during voice broadcasting.

[0336] In the pinyin annotation example 1501, there are the characters "郦食其" at the first position 1710, the third position 1730 and the fourth position 1740, and there is a pinyin annotation "lìyìjī" at the second position.

[0337] As shown in the left side of Figure 15 , the related art does not replace "郦食其" in the first position 1710, the third position 1730 and the fourth position 1740 with the pinyin annotation "lìyìjī" according to the pinyin annotation at the second position 1720. Therefore, during the voice broadcasting process, "郦食其" at the first position 1710, the third position 1730 and the fourth position 1740 are all incorrectly pronounced as "lìshíqí". Moreover, after pronouncing "lìshíqí" at the first position 1710, "lìyìjī" at the second position 1720 is then pronounced, followed by "lìshíqí" at the third position 1730, which results in repeated pronunciation and inconsistent pronunciations before and after, causing interference to the user and being unfavorable for the user to correctly understand the broadcast content.

[0338] As shown in Figure 15As shown on the right, in the embodiment of the present application, the first position 1710 and the second position 1720 are the phrases where the phonetic annotation is located. For the phrase with the phonetic annotation, the annotated characters in the phrase are replaced with the corresponding pinyin. At this time, "Li Shiqi" at the first position 1710 is the annotated character, so the pinyin "lìyìjī" at the second position 1720 is used to replace "Li Shiqi" at the first position 1710, such that "Li Shiqi" at the first position 1710 is replaced with "lìyìjī" (specifically as shown in Figure 17 ), and the pinyin annotation "lìyìjī" and the parentheses at the second position 1720 are also deleted.

[0339] Based on dynamic dictionary matching, phrases that are completely identical to the phrase where the phonetic annotation is located are matched across the full text, which are "Li Shiqi" at the third position 1730 and the fourth position 1740 respectively. For these phrases, the corresponding characters are also replaced with the phonetic annotation, such that "Li Shiqi" at the third position 1730 and the fourth position 1740 are replaced with "lìyìjī" (specifically as shown in Figure 17 ).

[0340] After the embodiment of the present application corrects and replaces the relevant text according to the pinyin annotation "lìyìjī", during the voice broadcast process, since the pinyin annotation at the second position 1720 is used to replace "Li Shiqi" at the first position 1710, the third position 1730 and the fourth position 1740, "Li Shiqi" at the first position 1710, the third position 1730 and the fourth position 1740 are all pronounced as "lìyìjī". Moreover, "lìyìjī" is only pronounced once at the first position 1710, which avoids the problem in the related art of repeated reading with inconsistent pronunciations before and after, that is, pronouncing "lìshíqí" first and then pronouncing "lìyìjī". In this way, the broadcast voice can convey the information of the original broadcast content more accurately, which is more conducive to the user's correct understanding of the broadcast content.

[0341] For example, referring to Figure 18 which shows a schematic flowchart of correcting and replacing relevant text according to text annotation, the process may include the following steps:

[0342] Step S1801: The electronic device 400 acquires the content to be broadcast.

[0343] Step S1802: The electronic device 400 determines whether the content to be broadcast contains parentheses. If yes, the process proceeds to step S1803.

[0344] If there are no parentheses in the content to be broadcast, and there is no need to perform sound optimization processing and other text optimization processing operations (such as rewriting punctuation marks with multiple semantics, etc.) on the content to be broadcast, the electronic device 400 may input the content to be broadcast into the TTS module, so as to perform speech synthesis through the TTS module and output audio data. If it is still necessary to perform sound optimization processing and other text optimization operations on the content to be broadcast, the sound optimization processing and other text optimization operations are correspondingly performed, and then speech synthesis is performed through the TTS module.

[0345] Step S1803: The electronic device 400 acquires the text containing parentheses from the content to be broadcast.

[0346] The text containing parentheses may refer to a word or a sentence containing parentheses. For example, the content to be broadcast is Figure 15 the text annotation example 1502 in, and the text containing parentheses is "an act of no alternative (nai) whatsoever".

[0347] Step S1804: For the text containing parentheses, the electronic device 400 uses the text inside the parentheses to replace the preceding text of the parentheses, so as to obtain a comparison text.

[0348] For example, the text containing parentheses is "an act of no boast (nai) whatsoever", and the preceding text of the parentheses is "boast". The text "nai" inside the parentheses is used to replace the preceding text "boast" of the parentheses, so as to obtain the comparison text "an act of no alternative whatsoever".

[0349] Step S1805: The electronic device 400 determines the scores of the comparison text and the text obtained after removing the parentheses and the text inside the parentheses, and obtains the text with a higher score.

[0350] The text obtained after removing the parentheses and the text inside the parentheses refers to: for the text containing parentheses, the text obtained after removing the parentheses and removing the text inside the parentheses. For example, the text containing parentheses is "an act of no boast (nai) whatsoever", the text "nai" inside the parentheses and the parentheses are removed, and the obtained text after removing the parentheses and the text inside the parentheses is "an act of no alternative whatsoever".

[0351] In a specific application, the electronic device 400 may determine the scores of the comparison text and the text obtained after removing the parentheses and the text inside the parentheses from dimensions such as text correctness and / or semantic coherence, and may take the text with a higher score as the final rewritten text.

[0352] For example, the text obtained after removing the parentheses and the text inside the parentheses is "an act of no alternative whatsoever", and the comparison text is "an act of no alternative whatsoever". A semantic model is used to compare the text correctness and / or semantic coherence, etc. of "an act of no alternative whatsoever" and "an act of no alternative whatsoever", so as to obtain the text with a higher score. At this time, the text with a higher score is "an act of no alternative whatsoever".

[0353] Normally, a text annotation is located in parentheses and is a text used to correct the preceding content. For example, Figure 15 In the text annotation example 1502 in which, the text annotation "nai" in "shi shu wu kua (nai) zhi ju" is located in parentheses, and is used to correct the typo "kua" in the preceding text. Based on this, when judging whether the content to be broadcast contains a text annotation, it may first be judged whether the content to be broadcast contains parentheses; if the content to be broadcast contains parentheses, it is then judged whether the text in the parentheses is used to correct an erroneous text. If the text in the parentheses is used to correct an erroneous text, it may be determined that the text in the parentheses is a text annotation, and further determined that the content to be broadcast contains a text annotation. If the content to be broadcast does not contain parentheses, or the content to be broadcast contains parentheses but the text in the parentheses is not used to correct an erroneous text, it may be determined that no text annotation exists in the content to be broadcast.

[0354] Herein, the text in the parentheses may be paralinguistic information, a text annotation, remark information, etc. In the embodiments of the present application, it may be determined through the above step S1804 and step S1805 whether the text in the parentheses is used to correct an erroneous text. When the score of the comparison text is higher, it may be determined that the text in the parentheses is used to correct an erroneous text, and further determined that the text in the parentheses is a text annotation, that is, it is determined that the content to be broadcast includes a text annotation.

[0355] It should be noted that in other embodiments, the electronic device 400 may also identify that the content to be broadcast includes a text annotation in other manners. At this time, the electronic device 400 may acquire a text containing a text annotation; and for the text containing the text annotation, use the text annotation to replace the annotated text in the text, so as to obtain a rewritten text. For example, the text containing the text annotation is "shi shu wu kua (nai) zhi ju", and the text annotation "nai" is used to directly replace the annotated text "kua", so as to obtain the rewritten text "shi shu wu nai zhi ju".

[0356] After the related text is corrected and replaced according to the text annotation in the embodiments of the present application, speech synthesis is performed through a TTS module, such that the audio data output by the TTS module can more accurately convey the information of the original broadcast content, and the intelligibility of the broadcast content is improved. For example, the content to be broadcast is Figure 15 In the text annotation example 1502 in which, in the related art, when broadcasting "shi shu wu kua (nai) zhi ju", it will be read as "shi shu wu kua nai zhi ju", which will cause trouble to the user, resulting in the user having a comprehension obstacle or comprehension deviation for the information of the voice broadcast. However, when broadcasting "shi shu wu kua (nai) zhi ju" in the embodiments of the present application, it will be read as "shi shu wu nai zhi ju", such that the broadcast voice can more accurately convey the information of the original broadcast content, and the user will not have a comprehension obstacle or comprehension deviation for the information of the voice broadcast.

[0357] The content to be broadcast can consist only of text with annotations and / or pinyin annotations, or it can include other content and text with annotations and / or pinyin annotations. After correcting and replacing relevant text in the content to be broadcast based on the annotations and / or pinyin annotations, the rewritten content to be broadcast can be obtained. The rewritten content to be broadcast can be input into the TTS module for speech synthesis to obtain audio data.

[0358] For example, when the content to be broadcast is Figure 15 When using the pinyin annotation example 1501, the rewritten content to be broadcast can be as follows: Figure 17 The text content shown on the right. When the content to be broadcast is... Figure 15 In the example of text annotation 1502, the rewritten content to be broadcast is: Jack said, "She is innocent. She did this not because she wanted to, but because she had no other choice."

[0359] It is understandable that the content to be broadcast may include both pinyin annotations and text annotations. At this time, the electronic device 400 can, respectively, determine the content based on the preceding text... Figure 16 and Figure 18 The process involves correcting and replacing relevant text based on pinyin and text annotations to obtain the rewritten content to be broadcast.

[0360] Steps S1603 to S1606 and S1803 to S1805 described above can be executed by the text rewriting module. In specific applications, text containing pinyin annotations and / or text annotations can be input into the text rewriting module; the text rewriting module, based on the process and principles of steps S1603 to S1606 and S1803 to S1805, and in conjunction with the context, corrects and replaces the annotated text.

[0361] In one implementation, the text rewriting module can be a functional module that processes data based on the above process and principles.

[0362] In another implementation, the text rewriting module can be implemented using an LLM model. For example, see [link to example]. Figure 19 The diagram illustrates a method for correcting and replacing text based on annotations using an LLM model. First, the LLM model is trained using training data to obtain the basic LLM model; then, the basic LLM model is fine-tuned to obtain the LLM-SFT.

[0363] like Figure 14As shown, the text to be rewritten is specifically "实属无夸(奈)之举", which is a text containing a text annotation. The prompt (i.e., Prompt) constructed based on the text to be rewritten is: As a text correction expert, 1. you can identify sentences in the input text that contain parentheses with pinyin annotations inside, and based on the semantic understanding of the context, convert the polyphonic characters corresponding to these pinyin annotations into the corresponding pinyin in the parentheses, and delete the parentheses and the pinyin inside them; 2. you can identify sentences in the input text that contain parentheses, where the character immediately before the parenthesis is a wrong character, and the content inside the parenthesis is the correct text corresponding to the wrong character. Based on the semantic understanding of the context, replace the wrong character before the parenthesis with the correct character inside the parenthesis, and delete the parenthesis and the text inside it; output the text after the above two conversions, and the format of the output text is consistent with that of the input text. The following input text is now available: "实属无夸(奈)之举". Please output the modified text.

[0364] Input the constructed Prompt into LLM-SFT, and obtain the rewritten text output by LLM-SFT. The specific rewritten text is "实属无奈之举".

[0365] The specific text to be rewritten is "单(shàn)老师自己不会骑单(dān)车。单老师经常坐我的车", which is a text containing pinyin annotations. The prompt (i.e., Prompt) constructed based on the text to be rewritten is: As a text correction expert, 1. you can identify sentences in the input text that contain parentheses with pinyin annotations inside, and based on the semantic understanding of the context, convert the polyphonic characters corresponding to these pinyin annotations into the corresponding pinyin in the parentheses, and delete the parentheses and the pinyin inside them; 2. you can identify sentences in the input text that contain parentheses, where the character immediately before the parenthesis is a wrong character, and the content inside the parenthesis is the correct text corresponding to the wrong character. Based on the semantic understanding of the context, replace the wrong character before the parenthesis with the correct character inside the parenthesis, and delete the parenthesis and the text inside it; output the text after the above two conversions, and the format of the output text is consistent with that of the input text. The following input text is now available: "单(shàn)老师自己不会骑单(dān)车。单老师经常坐我的车". Please output the modified text.

[0366] Input the constructed Prompt into LLM-SFT, and obtain the rewritten text output by LLM-SFT. The specific rewritten text is "shàn老师自己不会骑dān车。shàn老师经常坐我的车".

[0367] Of course, the text rewriting module can also be implemented in other manners, and is not limited to the LLM model, which is not limited herein.

[0368] As can be seen from the above, in this embodiment of the application, after correcting and replacing the relevant text in the content to be broadcast according to the annotations (pinyin annotations and / or text annotations) and then inputting it into the TTS module, the audio data output by the TTS module can more accurately convey the information of the original broadcast content.

[0369] Rewrite time text using absolute time description:

[0370] Absolute time and relative time are two ways of representing time. Absolute time refers to expressing time using specific units of time (such as year, month, day, hour, minute, etc.). Relative time refers to expressing time using a specific point in time, event, or phenomenon as a reference point.

[0371] For example, the time text described using absolute time is "August 20, 2024". The current time is August 19, 2024, and the time text described using relative time is "tomorrow".

[0372] When the content to be broadcast includes time text described using absolute time, the relevant technology does not optimize the time text using absolute time; instead, it directly inputs the content to be broadcast into the TTS module. As a result, when broadcasting the audio data output by the TTS module, the absolute time is broadcast in its entirety, leading to high complexity in the broadcast information, increasing the user's cognitive load, and hindering the user's ability to quickly and accurately understand the information conveyed by the voice broadcast.

[0373] In this embodiment, the time text described using absolute time is rewritten into a relative time description and / or a time text that conforms to the user's time description habits before being input into the TTS module. Thus, when performing voice broadcast based on the audio data output by the TTS module, the absolute time is not broadcast in its entirety; instead, relative time and / or voice that conforms to the user's time description habits are broadcast. This reduces the complexity of the voice broadcast information, decreases the user's cognitive load, and facilitates the user's quick and accurate understanding of the information conveyed by the voice broadcast.

[0374] For example, see Figure 20 The diagram illustrates a process for converting absolute time into relative time, which may include the following steps:

[0375] Step S2001: Electronic device 400 acquires the content to be broadcast.

[0376] In step S2002, the electronic device 400 determines whether the content to be broadcast includes time text described using absolute time. If so, proceed to step S2003.

[0377] Step S2003: Electronic device 400 determines the optimization strategy based on the time difference between the time described in the time text and the current time.

[0378] Electronic device 400 can first determine the time span range into which the time difference falls, and then, based on the mapping relationship between the time span range and the optimization strategy, obtain the optimization strategy corresponding to the time span range into which the time difference falls, and thus determine the optimization strategy.

[0379] The optimization strategy is used to optimize time text that uses absolute time description to obtain time text that uses relative time description, or to include time text that uses relative time description.

[0380] In practical applications, optimization strategies corresponding to different time spans can be pre-set to obtain a mapping table between time spans and optimization strategies.

[0381] For example, Table 2 below shows multiple time span intervals and examples of time text optimization under different "current time" settings.

[0382] Table 2

[0383]

[0384] In Table 2 above, the original broadcast time text (i.e., the time text described using absolute time) is "August 20, 2024, 14:30". Therefore, the original broadcast time text describes the time as: August 20, 2024, 14:30.

[0385] The "Rules" column in Table 2 above provides examples of various time span intervals. These time span intervals include: greater than one week, less than or equal to one week but greater than one day, less than or equal to one day but greater than two hours, and less than or equal to two hours.

[0386] It should be noted that in the "Rules" column of Table 2 above, each "rule" (i.e., time span interval) consists of a date span and a time span. The date span is in days, for example, a date span of less than or equal to one week, less than or equal to one day, etc.; the time span is in hours, for example, a time span greater than N hours (e.g., 2 hours). N can be set according to actual needs and is not limited here.

[0387] In Table 2 above, when the date span is greater than one day, the time span interval can include only the date span. In this case, the time span interval can be determined solely based on the date span. For example, if the current time is "July 20, 2024, 11:50", and the date span between the current time and "August 20, 2024, 14:30" is greater than one week, then the time span interval can be determined solely based on the date span between the two times, without calculating the time span.

[0388] In other words, the time span can be considered only when the date span is less than or equal to one day; when the date span is greater than one day, the time span is not considered. Of course, both the date span and the time span can be considered simultaneously, and this is not a limitation here.

[0389] When calculating the time difference between the current time and the time described in the original broadcast time text, the date span (i.e., the number of days) and time span (i.e., the number of hours) between these two times can be calculated. Based on the calculated date span and time span, the time span interval that falls into is determined. For example, in Table 2 above, if the current time is 13:50 on August 20, 2024, by calculating the time difference between "13:50 on August 20, 2024" and "14:30 on August 20, 2024", we can obtain that the date span between these two times is 0 days, the time span is 0.67 hours, and the time span interval that falls into is: the date span is less than or equal to one day, and the time span is less than or equal to 2 hours.

[0390] Of course, in other embodiments, the time span may only have a time interval, which can be in hours or minutes. That is, the time span is uniformly represented in hours. Days and weeks are converted into hours. In this case, the electronic device 400 can calculate the time difference (in hours) between the current time and the time described in the original broadcast time text, and determine the time span it falls into based on the time difference (in hours).

[0391] In Table 2 above, the same "rule" corresponds to two different optimized description methods. One optimized description method is "replacement." That is, the date and time in the absolute time are replaced with the date and time described using relative time. For example, the absolute time "August 20, 2024, 14:30" is rewritten as the relative time "tomorrow 14:30," and the date "tomorrow" in the relative time description replaces the date "August 20, 2024" in the absolute time.

[0392] Another optimized description method is "addition". That is, adding the date and time described using relative time to the absolute time. For example, rewriting the absolute time "August 20, 2024, 14:30" as the relative time "August 20, 2024 (tomorrow) 14:30" adds the date "tomorrow" described using relative time to the absolute time.

[0393] In practical applications, you can determine whether to use the optimized description method of "replacement" or "addition" as needed.

[0394] Different "rules" correspond to different optimization strategies. Table 2 does not show specific optimization strategies, but only the optimized time text corresponding to different "rules". The optimized time text is obtained by optimizing time text described using absolute time using optimization strategies.

[0395] Step S2004: Electronic device 400 uses an optimization strategy to optimize the time text described by absolute time to obtain optimized time text, which includes time text described by relative time.

[0396] For example, referring to Table 2 above, the original broadcast time text description is "August 20, 2024, 14:30".

[0397] If the current time is "July 20, 2024, 11:50", the electronic device 400 calculates the time difference between "August 20, 2024, 14:30" and "July 20, 2024, 11:50" as follows: the date span (i.e., the number of days between the two times) is 10 days, and the time span is 2 hours and 20 minutes. Since the date span of "10 days" is greater than one week, it can be determined that the time span interval (i.e., the "rule") is "time span > one week"; the optimization strategy corresponding to this "rule" is "no optimization". That is, the original broadcast time text "August 20, 2024, 14:30" will not be optimized.

[0398] If the current time is "August 13, 2024, 11:50", the electronic device 400 calculates the time difference between "August 20, 2024, 14:30" and "August 13, 2024, 11:50" as follows: the date span is 7 days, and the time span is 2 hours and 20 minutes. Since the date span of "7 days" is less than or equal to one week, it can be determined that the time span range (i.e., the "rule") is "time span ≤ one week".

[0399] When using the "replacement" optimization method, the optimization strategy for "time span ≤ one week" can be: replace the date in the absolute time with a date described by relative time, and omit the time. Using this optimization strategy to optimize "August 20, 2024, 14:30", that is, replace the absolute date "August 20, 2024" with the relative date "one week later", omitting the time "14:30". The optimized time text is "one week later". Of course, in some other embodiments, the time "14:30" can also be described, in which case the optimized time text is "one week later, 14:30".

[0400] When using the "addition" optimization method, the optimization strategy for "time span ≤ one week" can be: add a date described using a relative time to the absolute time, without describing the exact moment. Using this optimization strategy to optimize "August 20, 2024, 14:30", that is, add the date "one week later" with a relative time description to the absolute time. The optimized time text is "August 20, 2024 (one week later)". Of course, in some other embodiments, the moment "14:30" can also be described, in which case the optimized time text is "August 20, 2024 (one week later) 14:30".

[0401] If the current time is "August 19, 2024, 11:50", the electronic device 400 calculates the time difference between "August 20, 2024, 14:30" and "August 19, 2024, 11:50" as follows: the date span is 1 day, and the time span is 2 hours and 20 minutes. Since the date span "1 day" is less than or equal to one week, it can be determined that the time span interval (i.e., the "rule") is "time span ≤ one week".

[0402] Since the "rules" for "August 19, 2024, 11:50" and "August 13, 2024, 11:50" are both "time span ≤ one week", the corresponding optimization strategies are similar. Please refer to the above for details, and they will not be repeated here.

[0403] At this point, by using optimization strategies to optimize "August 20, 2024, 14:30", the optimized time text can be "Tomorrow 14:30", "Tomorrow", "August 20, 2024 (Tomorrow)", or "August 20, 2024 (Tomorrow) 14:30".

[0404] If the current time is "August 20, 2024, 10:50", the electronic device 400 calculates the time difference between "August 20, 2024, 14:30" and "August 20, 2024, 10:50" as follows: the date span is 0 days, and the time span is 3 hours and 20 minutes. Since the date span "0 days" is less than or equal to one day, and the time span "3 hours and 20 minutes" is greater than 2 hours, it can be determined that the time span interval (i.e., the "rule") is "date span ≤ one day, time span > 2 hours".

[0405] When using the "replacement" optimization method, the optimization strategy for "date span ≤ one day, time span > 2 hours" can be: replace the date in the absolute time with the date described by relative time. Using this optimization strategy to optimize "August 20, 2024, 14:30", that is, replace the date "August 20, 2024" in the absolute time with the date "today" described by relative time. The optimized time text is "Today 14:30".

[0406] When using the "add" optimization method, the optimization strategy for "date span ≤ one day, time span > 2 hours" can be: add a date described using relative time to the absolute time. Using this optimization strategy to optimize "August 20, 2024, 14:30", that is, add the date "today" with relative time description to the absolute time. The optimized time text is "August 20, 2024 (today) 14:30".

[0407] If the current time is "August 20, 2024, 13:50", the electronic device 400 calculates the time difference between "August 20, 2024, 14:30" and "August 20, 2024, 13:50" as follows: the date span is 0 days, and the time span is 1 hour and 20 minutes. Since the date span "0 days" is less than or equal to one day, and the time span "1 hour and 20 minutes" is less than or equal to 2 hours, it can be determined that the time span interval (i.e., the "rule") is "date span ≤ one day, time span ≤ 2 hours".

[0408] Both the "replacement" and "addition" optimization methods, for the condition "date span ≤ one day, time span ≤ 2 hours," employ the following optimization strategy: omit the date description and replace the absolute time with a relative time description. Applying this strategy to "August 20, 2024, 14:30," the absolute time "14:30" is replaced with the relative time description "40 minutes later." The optimized time text is "40 minutes later."

[0409] It should be noted that if the original broadcast time description is a past time, optimization strategies can still be used to optimize the time text. For example, the content to be broadcast includes "The match took place at 14:30 on August 20, 2024". The current time is: 10:30 on August 21, 2024. In this case, "14:30 on August 20, 2024" can be optimized to "14:30 yesterday" or "14:30 on August 20, 2024 (yesterday)". In this case, the content to be broadcast is optimized to "The match took place at 14:30 yesterday".

[0410] The optimized time text can include only time text described using relative time, such as "40 minutes later"; or it can include both time text described using absolute time and time text described using relative time, such as "August 13, 2024 (one week later)".

[0411] As can be seen from the above, by rewriting absolute time into relative time, the embodiments of this application reduce the complexity of the time information broadcast in voice, lower the cognitive load on users, and facilitate users to quickly and accurately understand the information conveyed by the voice broadcast. For example, if the time text described using absolute time is "August 20, 2024, 14:30", related technologies would broadcast this complete time text in voice, resulting in high information complexity. However, the embodiments of this application rewrite "August 20, 2024, 14:30" as "tomorrow 14:30" before broadcasting it in voice. This way, only "tomorrow 14:30" needs to be broadcast in voice, reducing complexity and allowing users to quickly and accurately understand the information conveyed by the voice broadcast.

[0412] In addition to converting absolute time to relative time, it is also possible to convert absolute time into time text that conforms to the user's time description habits, thereby simplifying the complexity of voice broadcast information, reducing the user's cognitive load, and allowing the user to quickly and accurately understand the information conveyed by the voice broadcast.

[0413] For example, see Figure 21 The diagram illustrates a process for rewriting absolute time based on user time description habits. This process may include the following steps:

[0414] Step S2101: Electronic device 400 acquires the content to be broadcast.

[0415] Step S2102: Electronic device 400 determines whether the content to be broadcast includes time text described using absolute time. If so, proceed to step S2103.

[0416] Step S2103: The electronic device 400 optimizes the time text that uses absolute time description according to the user's time description habits to obtain optimized time text that conforms to the user's time description habits.

[0417] In some embodiments, the electronic device 400 can obtain a user's time description habits through user behavior data. User behavior data may include, but is not limited to, voice data input by the user to the electronic device 400 and system time setting data.

[0418] If the voice input by the user to the electronic device 400 includes time information, the user's time description habits can be obtained based on the time information.

[0419] For example, when a user inputs a voice command to the smart assistant of electronic device 400, or engages in a voice conversation with the smart assistant of electronic device 400, the user inputs the voice message "Set an alarm for 2:30 PM." This voice message includes the time information "2:30 PM," which indicates that the user's time description habit is to use a 12-hour clock.

[0420] For example, a user inputs a system time setting operation into electronic device 400 to set the system time of electronic device 400 to 12-hour format. Based on the system time setting data of electronic device 400, it can be determined that the user's time description habit is to use 12-hour format.

[0421] Electronic device 400 can rewrite absolute time into text that conforms to the user's time description habits. For example, the text describing time using absolute time is: August 20, 2024, 14:30. The user's time description habit is to use 12-hour format. Based on the user's time description habit, "August 20, 2024, 14:30" is optimized to obtain the optimized text "August 20, 2024, 2:30 PM".

[0422] As can be seen from the above, by rewriting the absolute time into time text that conforms to the user's time description habits, the complexity of the information broadcast by voice is reduced, the user's cognitive load is reduced, and the user can quickly and accurately understand the information conveyed by the voice broadcast.

[0423] In the embodiments shown above, the electronic device 400 can rewrite absolute time into relative time, or rewrite absolute time into time text that conforms to the user's time description habits. In other embodiments, the electronic device 400 can not only rewrite absolute time into relative time, but also make the optimized time text conform to the user's time description habits.

[0424] At this point, after determining that the content to be broadcast includes time text described using absolute time, the electronic device 400 can determine an optimization strategy based on the time difference between the time described in the time text and the current time. Based on the user's time description habits and the optimization strategy, the time text described using absolute time is optimized to obtain optimized time text. The optimized time text conforms to the user's time description habits and includes time text described using relative time.

[0425] For example, based on users' time description habits, the time text in Table 2 above is optimized to obtain Table 3 below.

[0426] Table 3

[0427]

[0428] Comparing Tables 2 and 3 reveals that Table 3 describes "14:30" in Table 2 as "2:30 PM". This indicates that users typically use a 12-hour time format for time description. When optimizing "August 20, 2024, 14:30" based on user time description habits and optimization strategies, the optimization not only considers the optimization strategy but also the user's time description habits.

[0429] For example, in Table 3 above, if the current time is "August 20, 2024, 10:50", the electronic device 400 calculates the time difference between "August 20, 2024, 14:30" and "August 20, 2024, 10:50" as follows: the date span is 0 days, and the time span is 3 hours and 20 minutes. Since the date span "0 days" is less than or equal to one day, and the time span "3 hours and 20 minutes" is greater than 2 hours, it can be determined that the time span interval (i.e., the "rule") is "date span ≤ one day, time span > 2 hours".

[0430] When using the "replacement" optimization method, the optimization strategy for "date span ≤ one day, time span > 2 hours" can be: replace the date in the absolute time with the date described by relative time. Using this optimization strategy to optimize "August 20, 2024, 14:30", that is, replace the date "August 20, 2024" in the absolute time with the date described by relative time, "today". The optimized time text is "Today 14:30". Additionally, based on the user's time description habit of "using 12-hour format", "14:30" is rewritten as "2:30 PM". The final optimized time text is "Today 2 PM".

[0431] When using the "add" optimization method, the optimization strategy for "date span ≤ one day, time span > 2 hours" can be: add a date described using relative time to the absolute time. Using this optimization strategy to optimize "August 20, 2024, 14:30", that is, add the date "today" with relative time description to the absolute time. The optimized time text is "August 20, 2024 (today) 14:30". Additionally, based on the user's time description habit of "using 12-hour format", "14:30" is rewritten as "2:30 PM". The final optimized time text is "August 20, 2024 (today) 2:30 PM".

[0432] Simplify long texts or long strings of numbers consisting of numbers and letters:

[0433] A long text refers to a text composed of numbers and letters, with the number of characters greater than or equal to a second character number threshold. For example, the second character number threshold is 5, and a certain notification message includes the license plate number Hu A**8907. "A**8907" is composed of numbers and letters and has a total of 7 characters, and 7 is greater than 5. Therefore, "A**8907" is a long text composed of numbers and letters.

[0434] A long number string refers to a number string composed of numbers, with the number of characters greater than or equal to a third character number threshold. For example, the third character number threshold is 6, and a certain notification message includes the mobile phone number 188****8027. "188****8027" is composed of numbers and has a total of 11 characters, and 11 is greater than 6. Therefore, "188****8027" is a long number string.

[0435] Herein, the second character number threshold and the third character number threshold may be set according to actual needs, which are not limited herein.

[0436] When the content to be broadcast includes a long text or a long number string, in the related art, the long text and the long number string are not simplified, but the content to be broadcast is directly input to the TTS module. In this way, when voice broadcast is performed according to the audio data output by the TTS module, the long text or the long number string will be broadcast completely, resulting in high complexity of the voice broadcast information, increasing the user's cognitive load, and being unfavorable for the user to quickly and accurately understand the information conveyed by the voice broadcast.

[0437] However, in the embodiments of the present application, the long text or the long number string is simplified and then input to the TTS module. In this way, when voice broadcast is performed according to the audio data output by the TTS module, the long text or the long number string will not be broadcast completely, but the simplified text is broadcast, which reduces the complexity of the voice broadcast information, reduces the user's cognitive load, and facilitates the user to quickly and accurately understand the information conveyed by the voice broadcast.

[0438] For example, refer to Figure 22 which shows a schematic flowchart of simplifying a long text or a long character string, and the process may include the following steps:

[0439] Step S2201: The electronic device 400 acquires the content to be broadcast.

[0440] Step S2202: The electronic device 400 determines whether the content to be broadcast includes a long text composed of numbers and letters or a long number string. If yes, the process proceeds to step S2203.

[0441] Step S2203: The electronic device 400 extracts N characters from the long text or the long number string, and obtains a simplified text according to the N characters.

[0442] Suppose a long text or long string of numbers has a total of M characters, where N is less than M. That is, extract N characters from the M characters.

[0443] It should be noted that the specific values ​​of M and N may differ for long text and long numeric strings. For example, for the long text "A**8907", M is 7 and N is 4; for the long numeric string "188****8027", M is 11 and N is 4 or 6.

[0444] The extracted N characters can be any characters from long text or long string of numbers. In some embodiments, to make the simplified text convey more accurate information, the last N characters of the long text or long string of numbers can be extracted.

[0445] For example, see Figure 23 The illustrated diagram shows the license plate number description. The notification message 2310 from the ride-hailing application includes the text "Driver has arrived, license plate number A**8907". N is 4. The last four characters "8907" are extracted from the long text "A**8907". Based on these last four characters "8907", the simplified text 2320 "Driver has arrived, license plate number ending in 8907" is generated. Thus, the electronic device 400 only needs to announce "Driver has arrived, license plate number ending in 8907" via voice. Compared to the related technology's voice announcement "Driver has arrived, license plate number A**8907", the voice announcement information in this embodiment is less complex, reducing the user's cognitive load and facilitating quick and accurate understanding of the information conveyed by the voice announcement. Furthermore, the simplified voice announcement information can accurately convey the information of the original announcement.

[0446] For example, for the mobile phone number "188****8027", the last 4 characters can be extracted to simplify the mobile phone number into a description of the last four digits.

[0447] It should be noted that when the content to be broadcast includes both long text and long numeric strings, the electronic device 400 can simplify the long text and long numeric strings separately. For example, both the long text and long numeric strings can be simplified to descriptions with their last two digits.

[0448] Additional key information:

[0449] In some scenarios, the content to be broadcast may be missing crucial information. For example, in a notification broadcast scenario, a ride-hailing app might send a message saying, "Safe Travel - Vehicle Arrival Reminder: Please proceed to the agreed location, wear a mask, and pay attention to travel safety." This notification message lacks the crucial information of the vehicle's license plate number.

[0450] In related technologies, directly inputting the content to be broadcast into the TTS module and then broadcasting it based on the audio data output by the TTS module can lead to unclear and ambiguous information conveyed through the voice broadcast. For example, if the related technology directly broadcasts the notification message "Safe Travel - Vehicle Arrival Reminder, please go to the agreed location to board the vehicle, wear a mask, and pay attention to travel safety," the user cannot obtain the license plate number from the broadcast voice, and therefore cannot find the relevant vehicle based on the license plate number.

[0451] In this embodiment, the missing key information of the content to be broadcast is supplemented before being input into the TTS module, thereby avoiding the ambiguity of the information conveyed by the voice broadcast.

[0452] For example, see Figure 24 The diagram shown illustrates a process for supplementing key information, which may include the following steps:

[0453] Step S2401: Electronic device 400 acquires the content to be broadcast.

[0454] For example, in a notification broadcast scenario, when the electronic device 400 receives a notification message that needs to be broadcast, it uses the content of that notification message as the content to be broadcast.

[0455] Step S2402: Electronic device 400 determines whether the content to be broadcast is missing key information. If so, proceed to step S2403.

[0456] The electronic device 400 can determine whether key information is missing by judging whether the content to be broadcast is missing information in a preset slot. If the content to be broadcast is missing information in a preset slot, it is determined that the content to be broadcast is missing key information; if the content to be broadcast is not missing information in a preset slot, it is determined that the content to be broadcast is not missing key information.

[0457] The preset slots may differ depending on the scenario. For example, in a ride-hailing app's notification message broadcast scenario, the preset slot could include the license plate number. In this case, if the content to be broadcast does not include the license plate number, it is considered that the content to be broadcast is missing key information. As another example, in a train ticket purchasing app's notification message broadcast scenario, the preset slot could be the train's departure time.

[0458] If the content to be broadcast does not lack key information, it can be input into the TTS module; alternatively, a summary of the content can be generated and then input into the TTS module. Since summaries have lower information complexity and typically include key information from the content to be broadcast, the complexity of the voice broadcast can be reduced, and the voice can accurately convey the information of the original broadcast content.

[0459] In step S2403, the electronic device 400 obtains relevant information based on the source of the content to be broadcast.

[0460] Relevant information may include, but is not limited to: historical information from the same source, and information within the relevant application.

[0461] For example, when the content to be broadcast is a notification message, historical information from the same source can include: historical notification messages from the same application, historical notification messages from the same application and the same sender, and historical notification messages from different applications and the same sender. Here, "same source" can refer to: the same application, the same application and the same sender, or different applications and different senders. Specifically, if the notification message is sent by merchant A of application A, then historical notification messages from the same application refer to historical notification messages sent by application A. For instance, taking the received notification message as the current time, we can retrieve notification messages sent by application A within one hour prior to the current time as historical notification messages sent by application A.

[0462] Historical notification messages from the same application and the same sender refer to historical notification messages sent by merchant A through application A.

[0463] Historical notification messages from the same sender across different applications refer to notification messages sent by merchant A through application B. The sender is merchant A. For example, application A is an online shopping application, and application B is a messaging application. A merchant A in the online shopping application sends notification messages to users in application A, and also sends notification messages to users through the messaging application.

[0464] Information within a relevant application can refer to information accessed through the notification message and obtained within that application. The relevant application can include both the application that sent the notification message and other applications associated with it.

[0465] For example, see Figure 25A The diagram shows supplementary information for the notification message. The original notification message was Vehicle Arrival Notification 2510. Electronic device 400 needs to broadcast Vehicle Arrival Notification 2510 via voice. The broadcast content could be: "Safe Travel Vehicle Arrival Reminder: Please board at the agreed location, wear a mask, and pay attention to travel safety."

[0466] exist Figure 25A In the scenario shown, the preset slot is the vehicle's license plate number. At this time, the electronic device 400 determines that the message "Safe Travel Vehicle Arrival Reminder: Please proceed to the agreed location to board the vehicle, wear a mask, and pay attention to travel safety" is missing information from the preset slot, that is, the crucial information of the license plate number.

[0467] After determining that the content to be broadcast lacks key information, the electronic device 400 may obtain the license plate number from the historical notification messages of the same application program; or may enter the related application program to obtain the license plate number from the information in the related application program. As Figure 25A shown in the figure. The electronic device 400 may obtain the historical notification message 2520 of the same application program, so as to obtain the license plate number "Yue B***5562" from the historical notification message 2520. Certainly, the electronic device 400 may also enter the application program that sends the vehicle arrival notification 2510, so as to obtain the license plate number "Yue B***5562" from the related interface 2540 of the application program.

[0468] In Figure 25A , the related information may include at least one of the information displayed by the historical notification message 2520 and the related interface 2540.

[0469] For another example, referring to Figure 25B which is a schematic diagram of supplementing key information of a notification message, the original notification message is a ticket purchase success notification 2550. The electronic device 400 needs to perform voice broadcast on the ticket purchase success notification 2550, and the content to be broadcast may be "Railway ticket purchase waiting list order fulfillment success, notify that you have successfully purchased the ticket".

[0470] In the scenario of the ticket purchase success notification 2550, the preset slot is the vehicle departure time. At this time, the electronic device 400 determines that "Railway ticket purchase waiting list order fulfillment success, notify that you have successfully purchased the ticket" lacks the information of the preset slot, that is, lacks the key information of the vehicle departure time.

[0471] After determining that the content to be broadcast lacks key information, the electronic device 400 may enter the application program that sends the ticket purchase success notification 2550, and obtain the vehicle departure time from the information of the application program. As Figure 25B shown in the figure, enter the application program that sends the ticket purchase success notification 2550, and the waiting list order fulfillment success notification interface 2560 of the application program displays the departure time "18:00, September 22, 2024". At this time, the related information may refer to the information displayed by the waiting list order fulfillment success notification interface 2560.

[0472] When the original notification message is a payment success notification 2580. The electronic device 400 needs to perform voice broadcast on the payment success notification 2580, and the content to be broadcast may be "Safe Travel, the passenger has arrived, thank you for using Safe Travel, the trip has ended, and the password-free payment has been completed".

[0473] In the scenario of the payment success notification 2580, the preset slot is the payment amount. At this time, the electronic device 400 determines that "Assured Travel, the passenger has arrived. Thank you for using Assured Travel. The trip has ended, and the password-free payment has been completed" lacks the information of the preset slot, that is, the key information of the payment amount is missing.

[0474] After determining that the content to be broadcast lacks key information, the electronic device 400 may enter the application program that sends the payment success notification 2580, and obtain the payment amount from the information of the application program. As Figure 25B shown in the figure, the application program that sends the payment success notification 2580 is entered, and the payment amount "36.17" is displayed on the payment success interface 2590 of the application program. At this time, the relevant information may refer to the information displayed on the payment success interface 2590.

[0475] Step S2404: The electronic device 400 supplements the key information of the content to be broadcast according to the relevant information, and obtains the supplemented content to be broadcast.

[0476] After obtaining the relevant information, the key information can be obtained from the relevant information. For example, in Figure 25A , the relevant information may include the historical notification message 2520, and the license plate number "Yue B***5562" can be obtained from the historical notification message 2520.

[0477] In some embodiments, after the key information is obtained from the relevant information, the key information and the content to be broadcast may be integrated to obtain a semantically coherent text, and further obtain the supplemented broadcast content. For example, in Figure 25A , the content to be broadcast is "Assured Travel vehicle arrival reminder, please go to the agreed location to get on the vehicle, wear a mask, and pay attention to travel safety", and the key information is "Yue B***5562". By integrating the content to be broadcast and the key information, the supplemented content to be broadcast is obtained as "Assured Travel vehicle arrival reminder, please go to the agreed location to get on the vehicle, wear a mask, and pay attention to travel safety, the license plate number is Yue B***5562".

[0478] In other embodiments, an abstract may be generated according to the relevant information and the content to be broadcast. The abstract includes the key information. At this time, the supplemented content to be broadcast is the abstract. In specific applications, the electronic device 400 may first generate an abstract of the content to be broadcast, then obtain the key information from the relevant information, and finally supplement the key information into the abstract to obtain the final abstract, which includes the key information. Certainly, the electronic device 400 may also first obtain the relevant information, and then directly generate the final abstract according to the relevant information and the content to be broadcast.

[0479] For example, in Figure 25AIn the context of Vehicle Arrival Notification 2510, the final summary is Vehicle Arrival Notification Summary 2530. Specifically, Vehicle Arrival Notification Summary 2530 reads, "Safe Travel Vehicle has arrived, license plate ending in 5562."

[0480] It should be noted that, in Figure 25A During the process of generating the final summary, electronic device 400 can, according to Figure 22 The process shown simplifies the license plate number into a description of the last two digits.

[0481] exist Figure 25B In the context of the successful ticket purchase notification 2550, the final summary is ticket purchase notification summary 2570. Ticket purchase notification summary 2570 specifically states, "Ticket purchase successful, departing at 6 PM tonight."

[0482] It should be noted that when generating the ticket purchase notification summary 2570, the electronic device 400 can rely on... Figure 21 The process shown rewrites absolute time into relative time descriptions and time expressions that conform to users' time description habits.

[0483] For payment success notification 2580, the final summary is payment notification summary 25100. Payment notification summary 25100 specifically states "Payment successful, 36.17 yuan".

[0484] In comparison, summaries generated from relevant information and the content to be broadcast have lower information complexity and typically include key information about the content to be broadcast. Compared to directly broadcasting the content to be broadcast after supplementing key information, broadcasting summaries not only reduces the information complexity of voice broadcasting but also avoids ambiguity in the information conveyed by voice broadcasting.

[0485] The following is combined Figure 26 and Figure 27 The process of generating a summary based on relevant information and content to be broadcast is introduced.

[0486] For example, see Figure 26 The diagram shown illustrates a process for supplementing key information, which may include the following steps:

[0487] Step S2601: Electronic device 400 acquires the content to be broadcast.

[0488] Step S2602: Electronic device 400 generates a summary of the content to be broadcast.

[0489] Step S2603: Electronic device 400 determines whether the summary is missing key information. If so, proceed to step S2604.

[0490] In practical applications, the electronic device 400 can determine whether information in a preset slot is missing from the summary to ascertain whether the summary is missing key information. If the information in the preset slot is missing, it can be determined that the summary is missing key information; otherwise, it can be determined that the summary is not missing key information. The preset slots differ in different scenarios.

[0491] In step S2604, the electronic device 400 obtains relevant information based on the source of the content to be broadcast.

[0492] In step S2605, the electronic device 400 supplements the missing key information in the abstract based on the relevant information to obtain the final abstract.

[0493] For example, in Figure 25A In the context of Vehicle Arrival Notification 2510, the final summary is Vehicle Arrival Notification Summary 2530. Specifically, Vehicle Arrival Notification Summary 2530 reads, "Safe Travel Vehicle has arrived, license plate ending in 5562."

[0494] For example, see Figure 27 The diagram shown illustrates a process for supplementing key information, which may include the following steps:

[0495] Step S2701: Electronic device 400 acquires the content to be broadcast.

[0496] In step S2702, the electronic device 400 obtains relevant information based on the source of the content to be broadcast.

[0497] In step S2703, the electronic device 400 generates a final summary based on the relevant information and the content to be broadcast.

[0498] In this embodiment, the electronic device 400 does not need to determine whether the content to be broadcast is missing key information, but directly obtains the relevant information. Finally, based on the relevant information and the content to be broadcast, a final summary is generated.

[0499] Rewrite the source description information:

[0500] In notification broadcast scenarios, the source of a notification can include primary and secondary sources. The primary source typically refers to the application from which the notification originates; that is, which application the notification comes from. The secondary source typically refers to the user who sent the notification message.

[0501] For example, when a mobile phone's SMS application receives a text message sent by Zhang San, it displays a text message notification. In this case, the primary source of the notification is the SMS application, and the secondary source is Zhang San.

[0502] When delivering notification messages via voice, the secondary source of the notification can be announced to help users better distinguish the source of the notification message. In this case, the content to be announced can include a description of the message source and the content of the notification message. The message source description is information used to describe the secondary source.

[0503] It should be noted that, typically, to better distinguish different users, notes are added to users' information in electronic devices. For example, Li Si creates a contact named "Zhang San" in his phone's address book, recording Zhang San's phone number and name; in addition, he adds notes such as company, position, and birthday to the "Zhang San" contact. In some cases, to make the "Zhang San" contact appear higher in the address book for easier searching, Li Si also adds sorting information to the "Zhang San" contact. For example, the sorting information might be "AAA." Since the address book displays contacts in alphabetical order, adding "AAA" will make the "Zhang San" contact appear higher in the address book. In this case, the message source description information includes not only the user's name and phone number, but also the sorting information, birthday, company, and position.

[0504] For example, see Figure 28A The diagram illustrates the message source description information. After Li Si's mobile phone 100 receives a text message sent by Zhang San, the SMS application displays SMS notification 2820 on the lock screen. When mobile phone 100 reads SMS notification 2820 aloud, it generates the readout text 2830.

[0505] In the broadcast text 2830, “AAAA–Zhang San–Huawei Company–Sales–188******78–94.10.05” is the message source description information of SMS notification 2820, and “Are you free to have dinner together tonight?” is the content of the notification.

[0506] In the phone's 100 settings, Li Si added notes to the contact "Zhang San" in the address book, including "AAAA," "Zhang San," "Huawei Company," "Sales," and "94.10.05." "AAAA" was used to prioritize "Zhang San" in the address book for easier searching. "94.10.05" is Zhang San's date of birth.

[0507] In related technologies, when notification messages are read aloud via voice, the text is mechanically read word by word. Thus, when the text includes, for example... Figure 28AWhen delivering source descriptions in a message, reading them word for word increases the complexity of the audio presentation and adds to the user's cognitive load. Furthermore, redundant information in the source description is poorly audibly discernible, failing to help users distinguish the source of the notification message.

[0508] In this embodiment, the message source description information is simplified before being broadcast via voice. This way, the voice broadcast does not broadcast the complete message source description information, but rather the simplified version, reducing the complexity of the voice broadcast and lessening the user's cognitive load. Furthermore, in simplifying the message source description information, redundant information is removed, retaining only key and highly recognizable information, making it easier for users to distinguish the source of the notification message through voice broadcast.

[0509] For example, see Figure 28B The diagram illustrates a simplified flow of information describing the source of a message, which may include the following steps:

[0510] Step S2811: Electronic device 400 acquires the content to be broadcast.

[0511] Step S2812: Electronic device 400 determines whether the content to be broadcast includes message source description information. If so, proceed to step S2813.

[0512] For example, in a notification broadcast scenario, it can be determined whether the content to be broadcast includes text content containing a person's name or nickname, in order to determine whether the content to be broadcast includes a message source description. If the content to be broadcast includes text content containing a person's name or nickname, then it is determined that the content to be broadcast includes a message source description; otherwise, it is determined that the content to be broadcast does not include a message source description.

[0513] When it is determined that the content to be broadcast includes a message source description, this message source description information can be obtained. For example, the message source description information can be obtained by retrieving text content containing a person's name or nickname from the content to be broadcast.

[0514] Message source description information typically describes the secondary source of a communication message. This message can be an SMS message or a message from an instant messaging application. For example, in an instant messaging application, a user's notes might include their birthday, phone number, company, and job title. When broadcasting a chat message from that user, this relevant information can be broadcast. In this case, the content to be broadcast can include the message source description information, which includes relevant information about the user who sent the chat message (e.g., nickname, phone number, company, and job title), so that the user can distinguish which user sent the chat message through voice broadcast.

[0515] Step S2813: Electronic device 400 removes the preset type information from the message source description information to obtain simplified message source description information.

[0516] The preset information types are pre-defined and are not limited here. Typically, information with low auditory discrimination, which cannot help users distinguish the source, is identified as a preset information type. For example, preset information types may include, but are not limited to: phone numbers, dates, and sorting information.

[0517] For example, the message source description information is "AAAA – Zhang San – Huawei Company – Sales – 188******78 – 94.10.05". Electronic device 400 removes the information used for sorting, such as "AAAA", the phone number "188******78", and the date "94.10.05", and also removes the connectors before and after these information to obtain the simplified message source description information "Zhang San – Huawei Company – Sales".

[0518] When removing information used for sorting, the electronic device 400 can determine whether the first character of the message source description information is a single letter, or whether the initial string contains multiple consecutive identical letters. If so, the letter or string is considered to be information used for sorting, and this sorting information is removed. For example, in “AAAA–Zhang San–Huawei Company–Sales–188******78–94.10.05”, the first character is A, or the initial string contains multiple A's. As can be seen from the above, this embodiment of the application, after removing redundant information from the message source description information and obtaining simplified message source description information, performs voice broadcast based on the simplified message source description information. This reduces the complexity of the voice broadcast information and makes it easier for users to distinguish the source of notification messages through voice broadcast.

[0519] For example, if the message "AAAA–Zhang San–Huawei Company–Sales–188******78–94.10.05" is directly read aloud, it would require reading 31 characters, making the information read aloud highly complex. Furthermore, the auditory recognition of information such as "AAAA" and "188******78–94.10.05" is low, making them redundant information.

[0520] In this embodiment, the simplified message source description information "Zhang San – Huawei Company – Sales" is broadcast with fewer characters and less redundant information, making it more concise.

[0521] Steps S2812 to S2813 described above can be implemented by the text rewriting module. The text rewriting module can be implemented by an LLM model, and this is not limited to it here.

[0522] Integrate notes and / or paralinguistic information into the text content in a semantically coherent manner:

[0523] Paralinguistic information is related information produced alongside the source language; it refers to non-linguistic components other than spoken language in language communication.

[0524] For example, paralinguistic information may include, but is not limited to: volume, facial expressions, inner thoughts, emotional tone, pitch, speech rate, pauses, silence, coughing, and body language, and may also include vocal or non-vocal characteristics. Emotional tone may include joy, sorrow, and grief (e.g., laughter, wry smile, sighing, sadness, etc.). Pitch may include soft and loud voices. Speech rate may include rapid, slow, and drawn-out sounds. Body language may include waving and clapping.

[0525] The remarks information is supplementary information to the preceding content.

[0526] For example, see Figure 28C The illustrated broadcast text shows that in the text 2810 to be broadcast, "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow," there is a note that "first year of junior high school" is supplementary information to the preceding content "Ning Le," used to further describe that "Ning Le" is a first-year junior high school student.

[0527] The text to be broadcast, 2810, contains the subtext "laughs" in the sentence, "Ning Le: I can control my mindset, but I can't control my stress (laughs), sometimes the stress is a bit too much." This subtext describes Ning Le's emotional tone when she says this.

[0528] Typically, secondary language information and notes are enclosed in parentheses and inserted into the text content. For example, the secondary language information "laugh" in the text to be broadcast, 2810.

[0529] When the content to be broadcast includes notes or paralinguistic information, the relevant technologies do not integrate the notes and paralinguistic information into the text content in a semantically coherent manner. Instead, the content to be broadcast is directly input into the TTS module. As a result, when the audio data output by the TTS module is used for voice broadcast, the notes and paralinguistic information inserted into the text will affect the coherence of the voice broadcast. The broadcast voice will also not reflect the paralinguistic information, which will lead to the voice broadcast failing to accurately convey the information of the original broadcast content.

[0530] For example, the technology reads "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow" in the text to be broadcast 2810 as "Ning Le from Sunshine Middle School will participate in the competition tomorrow". It also reads "Ning Le: I can control my mindset well, but I can't control the pressure (laughs), sometimes the pressure is a bit too much" in the text to be broadcast 2810 as "Ning Le can control her mindset well, but she can't control the pressure well (laughs), sometimes the pressure is a bit too much".

[0531] It can be seen that the inserted notes "first year of junior high school" and the paralinguistic information "laugh" affect the continuity of information during audio playback, thus preventing the broadcast from accurately conveying the original content. Furthermore, the paralinguistic information "laugh" is read as the written word "laugh" during audio playback, failing to be represented as paralinguistic information, resulting in the broadcast not reflecting the paralinguistic information "laugh." This also leads to the broadcast not accurately conveying the original content.

[0532] In other words, the mechanical, word-by-word speech delivery of the technology will affect the coherence of the speech due to the annotations and paralinguistic information inserted into the text content; in addition, the delivered speech does not reflect paralinguistic information.

[0533] In this embodiment, the annotations and paralinguistic information in the content to be broadcast are semantically and coherently integrated into the text content to obtain semantically coherent text content that includes both annotations and paralinguistic information. Then, a TTS module is used for speech synthesis. Thus, when broadcasting audio data output from the TTS module, the broadcasted speech not only reflects the paralinguistic and annotation information but also improves the coherence of the broadcast information, thereby enabling the broadcasted speech to accurately convey the information of the original broadcast content.

[0534] For example, see Figure 29 The diagram illustrates a process for integrating notes and paralanguage information into text content, which may include the following steps:

[0535] Step S2901: Electronic device 400 acquires the content to be broadcast.

[0536] In step S2902, the electronic device 400 determines whether the content to be broadcast includes secondary language information or remarks. If so, proceed to step S2903.

[0537] In some embodiments, secondary language information and remarks are typically enclosed in parentheses, so the electronic device 400 can first determine whether the content to be broadcast includes parentheses. If the content to be broadcast includes parentheses, it further determines whether the content within the parentheses is secondary language information or remarks. If the content within the parentheses is secondary language information or remarks, it is determined that the content to be broadcast includes secondary language information or remarks.

[0538] Conversely, if the content to be broadcast does not include parentheses, or if the content within the parentheses is not secondary language information or remarks, then it is determined that the content to be broadcast does not include secondary language information or remarks.

[0539] Electronic device 400 can use an LLM model to determine whether the content within parentheses is paralinguistic information or annotation information. When the LLM model determines that the content within parentheses describes the speaker's emotional or behavioral paralinguistic information, the content within parentheses is determined to be paralinguistic information; when the LLM model determines that the content within parentheses is supplementary information to the preceding content, the content within parentheses is determined to be annotation information.

[0540] Step S2904: Electronic device 400 obtains text containing secondary language information or notes from the content to be broadcast.

[0541] For example, when the content to be broadcast is text 2810, the text containing sub-language information in text 2810 is "Ning Le: I can control my mindset well, but I can't control my pressure well (laughs), sometimes the pressure is a bit too much"; the text containing remarks is "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow".

[0542] Step S2905: For text containing secondary language information or notes, the electronic device 400 integrates the secondary language information or notes into the text in a semantically coherent manner.

[0543] In practical applications, electronic device 400 can first remove the parentheses, and then use a semantic model to integrate the content within the parentheses into the text in a semantically coherent manner.

[0544] For example, the text containing paralinguistic information is "Ning Le: I can control my mindset well, but I can't control my stress (laughs), sometimes the stress is a bit too much." Removing the parentheses and seamlessly integrating the meaning of "laughs" into the text containing paralinguistic information yields the integrated text. The integrated text could be either "Ning Le said with a smile, 'I can control my mindset well, but I can't control my stress well, sometimes the stress is a bit too much,'" or "Ning Le: I can control my mindset well, but I can't control my stress well. She smiled and said, 'Sometimes the stress is a bit too much.'"

[0545] The text containing the note is "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow." Remove the parentheses and logically integrate "first year of junior high school" into the text containing the note to obtain the text with integrated note information. The text with integrated note information can be: "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow."

[0546] By seamlessly integrating the paralinguistic information "laugh" and the note "first year of junior high school" into the text, and then using a TTS module to convert the text to speech, the audio data is obtained. When broadcasting the audio data, this embodiment reads the text "Ning Le: I can control my mindset well, but I can't control my pressure (laughs), sometimes the pressure is a bit too much" as "Ning Le says with a smile that she can control her mindset well, but she can't control her pressure well, sometimes the pressure is a bit too much," and the text "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow" as "Ning Le from the first year of junior high school at Sunshine Middle School will participate in the competition tomorrow." The audio broadcast has better information coherence and also reflects the paralinguistic information, thus accurately conveying the information of the original broadcast content.

[0547] When the content to be broadcast includes notes and supplementary language information, the electronic device 400 can integrate the notes and supplementary language information into the text in a semantically coherent manner.

[0548] Steps S2902 to S2904 described above can be executed by the text rewriting module. In specific applications, text containing notes and / or paralinguistic information can be input into the text rewriting module; the text rewriting module, based on the process and principles of steps S2902 to S2904, and in conjunction with the context, rewrites the paralinguistic information and / or notes to semantically integrate the paralinguistic information and / or notes into the original text, thereby obtaining the rewritten text.

[0549] In one implementation, the text rewriting module can be a functional module that processes data based on the steps and principles described above. In another implementation, the text rewriting module can be implemented using a large language model. For example, see [link to example]. Figure 30 The diagram illustrates a method for rewriting annotation information and / or paralinguistic information based on an LLM model. First, the LLM model is trained using training data to obtain the basic LLM model; then, the basic LLM model is fine-tuned to obtain the LLM-SFT.

[0550] like Figure 30As shown, the text to be rewritten is "Ning Le: I can control my mindset well, but I can't control my stress (laughs), sometimes the stress is a bit too much." The prompt word (i.e., a text correction expert) constructed based on this text is: As a text correction expert, you can identify sentences with parentheses in the input text. The content within the parentheses is supplementary information to the preceding text or a description of the speaker's emotional actions or other paralinguistic information. Based on the semantic understanding of the context, you integrate the annotations or paralinguistic descriptions within the parentheses into the main text to ensure semantic fluency, delete the parentheses and their contents, and output the integrated text. Given the following input text: "Ning Le: I can control my mindset well, but I can't control my stress (laughs), sometimes the stress is a bit too much.", please output the corrected text.

[0551] Input the constructed Prompt into LLM-SFT; obtain the rewritten text output by LLM-SFT. The rewritten text is: "Ning Le said with a smile, 'I can control my mindset, but I can't control my pressure. Sometimes the pressure is a bit too much.'"

[0552] Of course, the text rewriting module can also be implemented in other ways, not limited to the LLM model, and will not be restricted here.

[0553] Integrate image content with text content:

[0554] The content to be broadcast may include images and text. For example, in news or article broadcasting scenarios, relevant images are often inserted into the news or article to help users better understand the main body of the article or news and improve the user's reading experience. See, for example... Figure 31 The illustration shows a news report to be broadcast, where the first news report 3110 includes text and images.

[0555] Images in the content to be broadcast can be further divided into complementary images and supplementary images. Complementary images are those whose content is already expressed in the text content. Supplementary images are those whose content is partially or entirely not expressed in the text content, and whose content complements the text content to form a complete body of text.

[0556] For example, in Figure 31 In the first news item to be broadcast, 3110, the image depicts two people conversing. The man asks, "How are you going to get there?" and the woman replies, "I have a dual identity." Furthermore, this image content is not included in the text content; the text does not include the image. Moreover, the image and text complement each other; the image supplements the text. Therefore, the image in the first news item to be broadcast, 3110, is a supplementary image.

[0557] When the content to be broadcast includes images, the relevant technology will omit all images during voice broadcast. That is, regardless of whether the image in the content to be broadcast is a complementary or supplementary image, only the text in the content to be broadcast will be broadcast. For example, for... Figure 31 The first news item to be broadcast, 3110, only broadcasts the text: "In some spy movies, we may see the following scenes: For some listed companies, there may be dual listings, but this is completely different from the dual identities in movies," without broadcasting any images.

[0558] If the images in the content to be broadcast are supplementary, since the image content is not expressed in the text and the image content complements the text content, all images will be omitted during voice broadcasting. This will lead to incomplete semantics and disrupt the logical structure of the original broadcast content. Consequently, the broadcast voice will not accurately convey the information of the original broadcast content.

[0559] In this embodiment, the image content and text content of the supplementary image are integrated into a semantically coherent text expression, and then the TTS module is used for speech synthesis. This allows for accurate transmission of the original broadcast content when the audio data output by the TTS module is used for voice broadcast.

[0560] For example, see Figure 32 The diagram illustrates a process for integrating image and text content, which may include the following steps:

[0561] Step S3201: Electronic device 400 acquires the content to be broadcast, which includes images and text.

[0562] Step S3202: Electronic device 400 obtains the text content and image location from the content to be broadcast.

[0563] In step S3203, the electronic device 400 parses the image according to its position to obtain the image content.

[0564] Image location refers to the position of an image within the content to be broadcast. Electronic device 400 can locate the image within the content to be broadcast using the image location, and then parse the image to obtain its content. Image content may include, but is not limited to, image descriptions and text information within the image. For example, in... Figure 31 In the first news item to be broadcast, 3110, the image content may include a picture description of "a man and a woman talking" and text information in the picture such as "What are you going to do about it?" and "I have a dual identity".

[0565] In practical applications, electronic devices 400 can use multimodal LLM models and optical character recognition (OCR) to parse images and obtain their content. For example, OCR can be used to recognize text in an image, while multimodal models can be used to identify the core information or overall descriptive information of the image.

[0566] In step S3204, the electronic device 400 determines whether the image is a supplementary image based on the similarity between the image content and the text content. If so, proceed to step S3205.

[0567] Specifically, the electronic device 400 calculates the similarity between the image content and the text content surrounding the image. If the similarity is less than a preset threshold, it indicates that the image content is not expressed in the text content, and the image is determined to be a supplementary image. If the similarity is greater than or equal to the preset threshold, it indicates that the image content is expressed in the text content, and the image is determined to be a complementary image. When the image is a complementary image, the electronic device 400 does not need to integrate the image content and text content into semantically coherent text.

[0568] In step S3205, the electronic device 400 integrates the image content and text content into semantically coherent text.

[0569] In some embodiments, different types of supplementary images contain different information used to integrate with the text content.

[0570] Supplementary images can be categorized into visual supplementary images, text supplementary images, and table supplementary images.

[0571] A supplementary image is primarily composed of visual elements. For example, the image in the first news item to be broadcast, 3110, is a supplementary image; its main content is a scene of a man and a woman conversing. The image also includes textual information (i.e., the content of their conversation).

[0572] Text-supplemented images consist primarily of text, while table-supplemented images consist primarily of tables. For example, see... Figure 33 The illustrations shown illustrate text-supplementary images and table-supplementary images. The second news item to be broadcast, 3310, includes two images: a supplementary image 3330 and a text-supplementary image 3340. The content of the supplementary image 3330, "I counted, and I still worked 21 days," is already expressed in the text, "Some netizens calculated the number of days and said they actually worked 21 days." The third news item to be broadcast, 3320, includes a table-supplementary image 3350; that is, the table is inserted into the text as an image.

[0573] For example, for a supplementary image, the information in the image content used for integration with the text content may include at least one of the following: overall image description information and text information within the image content. That is, at least one of the overall image description information and text information within the image content can be integrated with the text content to form semantically coherent text.

[0574] For example, see Figure 34A The illustrated diagram shows a supplementary image integration. In related technologies, for the first news item to be broadcast 3110, the first broadcast content 3120 is obtained by acquiring the text content of the first news item to be broadcast 3110. The first broadcast content 3120 does not include image content and is semantically incomplete and incoherent. When the first broadcast content 3120 is broadcast aloud, the broadcasted audio does not include the image content of the supplementary image, and the semantics are incoherent and disjointed, failing to accurately convey the information of the first news item to be broadcast 3110.

[0575] In this embodiment of the application, after determining that the image in the first news item to be broadcast 3110 is a supplementary image, the image is parsed to obtain the following image content: two people are talking; the man asks, "How are you going to get there?", and the woman replies, "I have a dual identity." The image content includes the overall descriptive information "A man and a woman are talking," and the text information within the image is "How are you going to get there?" and "I have a dual identity."

[0576] The image in the first news item to be broadcast, 3110, is a supplementary image. The overall descriptive information of the image and the text information within the image are integrated with the text content to form a semantically coherent text, resulting in the first integrated text, 3130. When the first integrated text, 3130, is broadcast aloud, the broadcasted audio includes the image content of the supplementary image, and is semantically coherent and fluent, accurately conveying the information of the first news item to be broadcast, 3110.

[0577] For text-supplementary images, the information in the image content used to integrate with the text content can include at least one of the following: the core information of the image and all the text information within the image.

[0578] For example, see Figure 34BThe illustrated diagram shows the integration of text-supplementary images. In related technologies, for the second news item to be broadcast 3310, the second broadcast content 3360 is obtained by acquiring the text content of the context of the text-supplementary image 3340. The second broadcast content 3360 does not include the image content of the text-supplementary image 3340. When the second broadcast content 3360 is broadcast aloud, the broadcasted audio does not include the image content of the text-supplementary image 3340, and the semantics are not fluent or coherent, failing to accurately convey the information of the second news item to be broadcast 3310.

[0579] In this embodiment of the application, the image content extracted from the text-supplementary image 3340 may include the word "Mark" and the phrase "What's so complicated? It's just changing from a two-week weekend to a one-week weekend." The phrase "What's so complicated? It's just changing from a two-week weekend to a one-week weekend" is the core information of the image.

[0580] For the text-supplementary image 3340, the core information of the image and the text content are integrated into semantically coherent text to obtain the second integrated text 3370. When the second integrated text 3370 is used for voice broadcasting, the broadcast voice contains the image content of the text-supplementary image 3340, and is semantically coherent and fluent, accurately conveying the information of the second news item 3310 to be broadcast.

[0581] For supplementary images of tables, when the table is a row-listed table, the information in the image content used to integrate with the text content may include at least one of the following: summary information of the table title and header, header and body information, summary information of the entire table content, and the entire table content.

[0582] The summary information of the table titles and headers refers to the summary generated based on the table titles and headers. This summary information is integrated with the text content to form a semantically coherent text, enabling the voice playback of the summary generated from the table titles and headers.

[0583] By integrating the header and body information with the text content into semantically coherent text, the header and body information can be read aloud via voice.

[0584] For example, see Figure 34C The illustrated diagram shows the integration of supplementary table images. In related technologies, for the third news item to be broadcast 3320, the third broadcast content 3380 is obtained from the text content of the third news item to be broadcast 3320. The third broadcast content 3380 does not include the image content of the supplementary table image 3350. When the third broadcast content 3380 is broadcast aloud, the broadcasted audio does not include the image content of the supplementary table image 3350, and the semantics are not smooth or coherent, failing to accurately convey the information of the third news item to be broadcast 3320.

[0585] In this embodiment of the application, the image content extracted from the supplementary table image 3350 may include summary information of the table title and header, as well as header and body information. Specifically, the summary information of the table title and header is "A list of duty-free shops selling foreign exchange goods, including multiple duty-free shops in 19 cities such as Shanghai, Beijing, Guangzhou, and Shenzhen," and the header and body information is "A list of duty-free shops selling foreign exchange goods, with the duty-free shop name being Shanghai Inner Mongolia Duty-Free Shop, and the address being in Pudong New Area, Shanghai..."

[0586] In some embodiments, the summary information of the table title and header is integrated with the text content into semantically coherent text to obtain a third integrated text 3390.

[0587] In other embodiments, the header and body information are integrated with the text content into semantically coherent text to obtain a fourth integrated text 33100.

[0588] When broadcasting a message based on the third integrated text 3390 or the fourth integrated text 33100, the broadcast message includes the image content of the table supplementary image 3350, and the meaning is fluent and coherent, accurately conveying the information of the third news to be broadcast 3320.

[0589] The summary information of the entire table content refers to a summary generated based on the entire table content. This summary information is integrated with the text content to create a semantically coherent text summary for broadcasting the entire table content.

[0590] By integrating all the table content with text content into semantically coherent text, the entire table content can be read aloud.

[0591] When the table is a bar chart, line chart, or pie chart, the information in the image content used to integrate with the text content may include at least one of the following: a summary of the key information of the chart (such as title, table header, table legend, etc.); a summary of key trends; a summary of the entire table content; or the entire table content.

[0592] The summary information of the key information of the chart refers to the summary information generated based on the key information such as the chart title, header, and legend.

[0593] The summary information of key trends refers to the key trends identified through analysis of chart content, and the summary generated based on these key trends.

[0594] In other embodiments, all the parsed image content and text content can be integrated into semantically coherent text, without having to select different image information and text content for integration based on the type of supplementary image.

[0595] In practical applications, the electronic device 400 can employ a semantic model to integrate image and text content into semantically coherent text. The electronic device 400 can then input this semantically coherent text into a TTS module to obtain the audio data output by the TTS module. Thus, when performing voice broadcast based on the audio data output by the TTS module, the broadcast voice includes supplementary image content, and is semantically coherent and fluent, accurately conveying the information of the original broadcast content.

[0596] In some embodiments, the electronic device 400 may adopt a complementary image expression structure, integrating the image content and text content of the complementary image into semantically coherent text.

[0597] The content to be broadcast includes complementary images and supplementary images. Within the context of the complementary image, if a pre-defined expression structure exists, that structure will be used to integrate the image content and text content of the supplementary image into semantically coherent text.

[0598] Preset expression structures may include, but are not limited to: indicating, mentioning, saying, and publishing.

[0599] For example, see Figure 35 The diagram illustrates another process for integrating image and text content, which may include the following steps:

[0600] Step S3501: Electronic device 400 acquires the content to be broadcast, which includes images and text.

[0601] Step S3502: Electronic device 400 obtains the text content and image location from the content to be broadcast.

[0602] In step S3503, the electronic device 400 parses the image according to its location to obtain the image content.

[0603] In step S3504, the electronic device 400 determines supplementary images and complementary images based on the similarity between the image content and the text content.

[0604] When the content to be broadcast includes multiple images, the position of each image is obtained. Based on the position of each image, each image is parsed to obtain its content.

[0605] For each image, the similarity between the image content and the text content is calculated. If the similarity between the image content and the text content is lower than a preset threshold, the image is determined to be a supplementary image. Conversely, if the similarity between the image content and the text content is higher than or equal to the preset threshold, the image is determined to be a complementary image.

[0606] The text content to be broadcast may consist of multiple paragraphs. When determining the image type based on the similarity between the image content and the text content, the text content can be divided into multiple parts, each part comprising one or more text paragraphs. The similarity between each part of the text and the image content is then calculated. If the similarity between each part of the text and the image content is below a preset threshold, the image is determined to be a supplementary image. Conversely, if at least one part of the text and the image content has a similarity greater than or equal to the preset threshold, the image is determined to be a supplementary image.

[0607] In practical applications, electronic device 400 can use semantic models to determine the similarity between image content and text content.

[0608] In this embodiment, the content to be broadcast is determined to include supplementary images and complementary images.

[0609] Step S3505: Electronic device 400 determines that texts with a similarity to the content of the matching image that is higher than the second threshold contain a preset expression structure.

[0610] For a matching image, the electronic device 400 finds text whose content similarity to the matching image is higher than a second threshold, and uses a semantic model to identify whether the text contains a preset expression structure. In this embodiment, if the text whose similarity to the matching image is higher than the second threshold contains words such as "represent," "mention," "say," or "publish," then it is determined that the text contains a preset expression structure; otherwise, it is determined that it does not contain a preset expression structure.

[0611] In step S3506, the electronic device 400 uses a preset expression structure to integrate the image content and text content into semantically coherent text.

[0612] For example, see Figure 36 The diagram illustrates a text integration method using the expressive structure of a reference-matching image. In the second news item to be broadcast 3310, there are matching images 3330 and text-supplementary images 3340. The context of the matching image 3330, "Some netizens calculated the number of days and said they actually worked 21 days," contains the pre-defined expressive structure "says." At this point, when integrating the image content and text content of the text-supplementary image 3340 into semantically coherent text, the expressive structure "says" from the context of the adjacent image (i.e., the matching image 3330) is used for text integration, resulting in the fifth integrated text 33110.

[0613] The second integrated text 3370 was obtained by integrating text without referencing the contextual structure of the accompanying image 3330. By comparing the second integrated text 3370 and the fifth integrated text 33110, it can be seen that the second integrated text 3370 uses "said" to connect the text content and the image content, while the fifth integrated text 33110 uses "represented" to connect the text content and the image content.

[0614] This application embodiment integrates the image content and text content of supplementary images into semantically coherent text by referencing the expressive structure of complementary images, which can further improve the semantic coherence of the preceding and following text, and thus further improve the semantic coherence of the broadcast speech.

[0615] Integrate table content with text content:

[0616] The content to be broadcast may include tables, and the tables are text, not images.

[0617] When the content to be broadcast includes tables in text form, the technology will read the table content aloud word by word. This results in inconsistent semantics in the audio broadcast and fails to accurately convey the information of the original content.

[0618] In this embodiment, the table content and other text content are integrated into semantically coherent text, and then the TTS module is used for text-to-speech conversion. This results in better semantic coherence of the speech delivery, and the delivered speech accurately conveys the information of the original content.

[0619] For example, see Figure 37 The diagram illustrates a process for integrating table content and text content, which may include the following steps:

[0620] Step S3701: Electronic device 400 acquires the content to be broadcast, which includes a table in text format.

[0621] Step S3702: Electronic device 400 integrates the table content and other text content into semantically coherent text.

[0622] In some embodiments, the electronic device 400 can obtain table text content from the content to be broadcast, and analyze information such as table title, table header and table body based on the table text content to determine table information for integration with other text content, and integrate the table information with other text content into semantically coherent text.

[0623] For example, when the table is a row-listed table, the table information used for integration with other text content may include at least one of the following: summary information of the table title and header, header and body information, summary information of the entire table content, and the entire table content.

[0624] The summary information of the table titles and headers refers to the summary generated based on the table titles and headers. This summary information is integrated with the text content to form a semantically coherent text, enabling the voice playback of the summary generated from the table titles and headers.

[0625] By integrating the header and body information with the text content into semantically coherent text, the header and body information can be read aloud via voice.

[0626] For example, see Figure 38 The diagram showing the integrated content of the tables indicates that the fourth news item to be broadcast 3810 includes a text-based table 3820. In contrast, the table in the third news item to be broadcast 3320 is an image, while the fourth news item to be broadcast 3810 is text.

[0627] In related technologies, the text content of the fourth news item to be broadcast 3810 is obtained to obtain the fourth broadcast content 3830. The fourth broadcast content 3830 includes the content of table 3820, such as "serial number" and "duty-free shop name". For the sake of brevity, the fourth broadcast content 3830 does not fully show the content of table 3820.

[0628] When broadcasting the fourth news item 3830, the content of Table 3820 is read aloud word by word. For example, a certain audio segment might be: "Serial Number, Duty-Free Shop Name, Address 1, Shanghai Indoor Duty-Free Shop." It is easy to see that the meaning of the broadcast audio is incoherent and cannot accurately convey the information of the fourth news item 3810 to be broadcast.

[0629] In some embodiments of this application, the summary information of the table title and header is integrated with other text content into semantically coherent text to obtain the sixth integrated text 3840. The summary information of the table title and header is "a list of duty-free shops selling foreign exchange goods, including multiple duty-free shops in 19 cities such as Shanghai, Beijing, Guangzhou, and Shenzhen", and the other text content can be the main text in the fourth news to be broadcast 3810, such as "Let's take a look...".

[0630] In some other embodiments of this application, the header and body information are integrated with other text content into semantically coherent text to obtain the seventh integrated text 3850. The header and body information is "a list of duty-free shops selling foreign exchange goods, the duty-free shop name is Shanghai Inner Mongolia Duty-Free Shop, the address is in Pudong New Area, Shanghai...".

[0631] When the audio is broadcast based on the sixth integrated text 3840 or the seventh integrated text 3850, the broadcast audio contains the contents of table 3820 and is semantically coherent, accurately conveying the information of the fourth news item 381 to be broadcast.

[0632] The summary information of the entire table content refers to a summary generated based on the entire table content. This summary information is integrated with the text content to create a semantically coherent text summary for broadcasting the entire table content.

[0633] By integrating all the table content with text content into semantically coherent text, the entire table content can be read aloud.

[0634] When the table is a bar chart, line chart, or pie chart, the table information used for integration with other text content may include at least one of the following: summary information of the table's key information (such as title, header, table legend, etc.); summary information of key trends; summary information of the entire table content; or the entire table content.

[0635] The summary information of the key information in the table refers to the summary information generated based on the extracted key information such as the table title, header, and table legend.

[0636] The summary information of key trends refers to the key trends obtained by analyzing the table content and the summary summary generated based on the key trends.

[0637] In some embodiments, steps S3202-S3205, S3502-S3506, and S3701-S3702 described above can be executed by the text rewriting module. In this case, the text rewriting module can be a functional module that processes data based on the above steps and principles.

[0638] In another implementation, the text rewriting module can be implemented using a multimodal LLM. For example, see [link to example]. Figure 39 The diagram illustrates a text rewriting method based on a multimodal LLM model. First, the multimodal LLM model is trained using training data to obtain a basic multimodal LLM model. Then, the basic multimodal LLM model is fine-tuned to obtain the multimodal LLM-SFT.

[0639] like Figure 39 As shown, the input to multimodal LLM-SFT can be one or more types of page structured information, such as text, images, and tables, or a page screenshot. For example, the first news item to be broadcast, 3110, can be input into multimodal LLM-SFT, or a screenshot of the page displaying the first news item to be broadcast, 3110, can be input into multimodal LLM-SFT.

[0640] The prompts constructed based on the input are: You can recognize the text, images, and tables on the input page, extract and summarize the information from the images and tables, and if the information in the images or tables is not mentioned in the main text, then integrate the summarized information into the text to maintain contextual coherence and output the integrated text; if the information in the images or tables is mentioned in the main text, then output the original text of the page.

[0641] Input the constructed Prompt into LLM-SFT; obtain the fused text output by LLM-SFT. The fused text is: "In some spy movies, we might see the following scene: two people are talking, one man asks, 'How are you going to get there?' and the other woman replies, 'I have a dual identity.' This is also true for some listed companies, which may have dual listings, but it's completely different from the dual identity depicted in movies."

[0642] Of course, the text rewriting module can also be implemented in other ways, not limited to the multimodal LLM model, and will not be restricted here.

[0643] Remove superscripts and captions:

[0644] When the content to be broadcast includes subtitles and / or captions, the relevant technology directly inputs the content to be broadcast into the TTS module and performs voice broadcast based on the audio data output by the TTS module. In this way, irrelevant information such as subtitles and captions can interfere with the voice broadcast, resulting in semantic incoherence in the voice broadcast, causing comprehension difficulties or misunderstandings for the user, and ultimately making the broadcast voice unable to accurately convey the information of the original broadcast content.

[0645] For example, the text to be broadcast includes the following: "He mentioned that the company's net profit margin is as high as 18% [3]". In this text, "[3]" is a superscript. Related technologies will read this text as "He mentioned that the company's net profit margin is as high as 18.3%". That is, "[3]" affects the accuracy of the voice broadcast.

[0646] In this embodiment, after removing the superscripts and captions from the content to be broadcast, the text-to-speech (TTS) module is used for text-to-speech conversion. This avoids interference from captions and superscripts when broadcasting audio data from the TTS module, ensuring that the broadcasted audio accurately conveys the information of the original content.

[0647] For example, regarding the text "He mentioned that the company's net profit margin is as high as 18% [3]", this application embodiment removes the superscript "[3]" to obtain the superscript-removed text "He mentioned that the company's net profit margin is as high as 18%", and then inputs the superscript-removed text into the TTS module to convert the text into speech. In this way, this application embodiment will read the text "He mentioned that the company's net profit margin is as high as 18% [3]" as "He mentioned that the company's net profit margin is as high as 18%".

[0648] For example, see Figure 40 The diagram shown illustrates a process for deleting superscripts and captions, which may include the following steps:

[0649] Step S4001: Electronic device 400 acquires the content to be broadcast.

[0650] Step S4002: Electronic device 400 determines whether the content to be broadcast contains a superscript or a caption. If so, proceed to step S4003.

[0651] In some embodiments, the electronic device 400 may first obtain the font size information of each character in the content to be broadcast, and determine the mode of the font size based on the font size information of each character; then, determine the single number or single letter whose font size is smaller than the mode of the font size as a subscript. For example, if the mode of the font size is size 4, and the font size of a certain number or letter (e.g., small 4, 5, etc.) is smaller than size 4, then the number or letter is determined to be a subscript.

[0652] In other embodiments, the electronic device 400 may also identify a single number or letter in "[]" as a subscript.

[0653] Captions are annotations and explanations for illustrations. Captions are generally placed below the illustration, but occasionally above it. The line length of a caption should generally not exceed the length of the illustration.

[0654] Electronic device 400 can combine version features and / or text features to determine whether the content to be broadcast contains captions.

[0655] Specifically, for text paragraphs located before or after images, if the font size of the text paragraph is smaller than the mode of font sizes, then the text paragraph is determined to be a caption. The mode of font sizes can be obtained by counting the font sizes of each character in the content to be broadcast. Typically, the mode of font sizes is determined as the size of the body text.

[0656] A picture is identified as a caption when the text preceding or following it contains special caption symbols. Special caption symbols can include, for example, parentheses, upward arrows, and triangles.

[0657] If the text preceding or following an image contains clear textual features, it is identified as an image caption. Clear textual features can include, for example: "Image:", "Image / ", "Image source:", "Image source:", "Photo source:", "Photographed / ", "Photographed by person", etc.

[0658] If the text preceding or following an image contains ambiguous textual features and is not a complete sentence, it is identified as an image caption. Ambiguous textual features can include, for example: image, image source, provided image, photograph, screenshot, screenshot, Visual China, cartoon author.

[0659] If the text paragraph before or after an image contains ambiguous textual features, is a complete sentence, but is centered on the page, it is determined to be a caption.

[0660] For example, see Figure 41 The diagram illustrates the caption determination process. Electronic device 400 first determines the body text size. Specifically, the body text size can be obtained by calculating the mode of the font size in the layout text. That is, the mode of the font size is determined as the body text size.

[0661] Then, the electronic device 400 locates the text paragraphs located before and after the image. It determines whether the font size of the text paragraph is smaller than the font size of the main text; if so, the text paragraph is determined to be a caption; if not, it further determines whether the font size of the text paragraph is larger than the font size of the main text.

[0662] If the font size of the text paragraph is not larger than the font size of the body text, then it is further determined whether the text paragraph contains special caption feature symbols. If the text paragraph contains special caption feature symbols, then the text paragraph is determined to be a caption; if the text paragraph does not contain special caption feature symbols, then it is further determined whether the text paragraph contains explicit text features.

[0663] If a text paragraph contains clear textual features, it is determined to be a caption; if a text paragraph does not contain clear textual features, it is further determined whether the text paragraph contains ambiguous textual features.

[0664] If a text paragraph contains ambiguous textual features, further determine whether the text paragraph is a complete sentence. If the text paragraph is not a complete sentence, then determine whether the text paragraph is a caption. If the text paragraph is a complete sentence, then further determine whether the text paragraph is centered on the page. If the text paragraph is centered on the page, then determine whether the text paragraph is a caption.

[0665] Step S4003: Delete the superscript or caption from the electronic device 400.

[0666] Adjust the audio delivery of different broadcast content based on textual and semantic features:

[0667] In related technologies, directly inputting the content to be broadcast into the TTS module results in the output audio data containing only one voice representation. This means the entire content is mechanically broadcast using a single voice representation. Consequently, the broadcast speech fails to accurately convey the information of the original content.

[0668] For example, see Figure 42 The illustrated diagram of the broadcast text shows that the text to be broadcast, 2810, includes the quoted words "The Shanghai athletes performed very well" in the main text, the subtext "laugh" in the main text, and the voice-over "No" in the main text.

[0669] In related technologies, the text to be broadcast 2810 is directly input into the TTS module, and voice broadcast is performed based on the audio data output by the TTS module. During the voice broadcast, when broadcasting "The Shanghai athletes performed very well," the text content is mechanically read aloud, and the voice expression is consistent with other content, thus the broadcast voice does not reflect the quoted person's words, leading to misunderstandings for the user; when broadcasting "Ning Le: I can control my mentality well, but I can't control the pressure well (laughs), sometimes the pressure is a bit too much," the text content is mechanically read aloud, and the broadcast voice does not reflect "laughs" as paralinguistic information, resulting in poor information coherence in the voice broadcast; when broadcasting the voice-over "No," the text content is mechanically read aloud, and the broadcast voice fails to convey the original meaning of the article.

[0670] In this embodiment, the audio representation of different content is adjusted based on the textual and semantic features of the content to be broadcast, resulting in different audio parameters for each content. This ensures that the audio representation differs for each content during speech broadcast. Adjusting the audio representation can refer to adjusting the audio parameters and converting the audio into speech. Typically, different audio representations have different audio parameters. Audio parameters may include, but are not limited to, volume, speech rate, timbre, tone, style, pauses, and stress.

[0671] Furthermore, embodiments of this application can also convert some content in the content to be broadcast into speech. For example, for the paralinguistic information "laugh", one way is to convert it into the text content "speaking with a smile", and another way is to convert "laugh" into a laugh sound, and when the word "laugh" is read aloud, the laugh sound is broadcast, so that "laugh" is represented as paralinguistic information.

[0672] For example, see Figure 43 The diagram illustrates a process for adjusting the audio representation of different broadcast content based on text and semantic features. This process may include the following steps:

[0673] Step S4301: Electronic device 400 acquires the content to be broadcast.

[0674] In step S4302, the electronic device 400 determines whether the content to be broadcast includes dialogue, paralinguistic information, or statements expressing inner thoughts. If so, proceed to step S4303.

[0675] For example, the electronic device 400 can use a semantic model to identify whether the content following punctuation marks such as colons and quotation marks is spoken by a person; and use a semantic model to identify whether there is semantic text such as "said" or "said" in the context of a certain content. If so, the content is determined to be spoken by a person; if not, the content is determined not to be spoken by a person.

[0676] After determining that the content to be broadcast contains parentheses, the electronic device 400 can further determine whether the content within the parentheses is paralinguistic information through a semantic model. When it is determined that the content within the parentheses contains information such as the speaker's emotions or actions, then the content within the parentheses is determined to be paralinguistic information.

[0677] Electronic device 400 can use a semantic model to determine whether the content within punctuation marks such as parentheses and half-brackets is a statement expressing inner thoughts. For example, the voice-over "(No)" in the text to be broadcast 2810.

[0678] It is understandable that the content to be broadcast may include at least one of the following: a person's speech, paralinguistic information, and speech expressing psychological activities.

[0679] Step S4303: Electronic device 400 outputs tagged text.

[0680] Once it is determined that the content to be broadcast includes dialogue, paralinguistic information, or statements expressing inner thoughts, the text containing dialogue is retrieved and tagged; the text containing paralinguistic information is retrieved and tagged; and the text containing statements expressing inner thoughts is retrieved and tagged, so as to output the tagged text.

[0681] In step S4304, the electronic device 400 performs TTS parameter matching based on the tags of each text to determine the audio parameters of each text.

[0682] It should be noted that a mapping relationship between tags and audio parameters is pre-defined; different tags may correspond to different or the same audio parameters. After obtaining the tagged text, TTS parameter matching is performed. Based on the mapping relationship between tags and audio parameters, the audio parameters corresponding to each tag are determined, and thus the audio parameters for each piece of text are determined. These audio parameters are used in TTS speech synthesis and are therefore also called TTS parameters.

[0683] For example, see Figure 44The illustrated diagram of labels and parameters shows that the first original text content 4410 is: The Shanghai music competition officially kicks off today. The organizers say, "The Shanghai contestants are performing very well." Ning Le (a first-year junior high student) from Sunshine Middle School will participate in the competition tomorrow.

[0684] The first original text content 4410 is a part of the text to be broadcast 2810.

[0685] In this embodiment of the application, after processing the first original text content 4410, a first output text 4420 is obtained. The first output text 4420 is tagged text. Among them, the text "Music Shanghai Station Competition officially kicks off today, organizer:" is tagged "body text", indicating that the text is body text.

[0686] The text “Shanghai athletes performed very well” is tagged as “dialogue, positive and affirmative tone”, indicating that the text is spoken by a person.

[0687] The text “Ning Le from Sunshine Middle School” is labeled as body text, indicating that this text is body text.

[0688] TTS parameter matching is performed on the first output text 4420 to obtain the first TTS parameter matching example 4430. Among them, for the text with the "body text" tag "Music Shanghai Station Competition officially kicks off today, organizer:", the matched TTS parameter (i.e., audio parameter) is "broadcast timbre, speech rate 1".

[0689] For the sentence "Shanghai contestant performed very well" with the phrase "dialogue, positive and affirmative tone", the matched TTS parameter is "human-like tone 1, positive and affirmative tone".

[0690] For the text tag “Ning Le from Sunshine Middle School”, the matched TTS parameters are “broadcast timbre, speech rate 1”.

[0691] After determining the audio parameters of each text, the electronic device 400 can input both the audio parameters and the text into the TTS module. The TTS module then performs speech synthesis based on the audio parameters and outputs audio data. Thus, when broadcasting audio data, the audio expression for different content will differ. For example, taking the first TTS parameter matching example 4430 as an example, using a broadcast tone and a speech rate of 1, the message "The Shanghai Music Competition officially begins today, organized by:" can be broadcast; using a human-like tone of 1 with a positive and affirmative tone, the message "The Shanghai contestants performed very well" can be broadcast. In this way, the user can accurately understand the information in the original broadcast content simply by listening to the audio.

[0692] As can be seen from the above, the embodiments of this application adjust the sound expression of different broadcast content according to the textual and semantic features of the content to be broadcast, so that the sound parameters of different broadcast content are different, thereby enabling the broadcast speech to accurately convey the information of the original broadcast content.

[0693] Adjusting the sound expression of different content structures according to the layout characteristics

[0694] In related technologies, directly inputting the content to be broadcast into the TTS module results in the output audio data having only one sound representation; that is, mechanically broadcasting the entire content using a single sound representation. This prevents the broadcasted speech from accurately conveying the information of the original content. In this embodiment, the sound representation of different content structures is adjusted according to the layout characteristics of the content to be broadcast, so that the sound parameters of different content structures are different, thereby allowing the broadcasted speech to accurately convey the information of the original content.

[0695] The layout features can include the font attributes, alignment attributes (such as center alignment), and paragraph division information of the content to be broadcast. Font attributes can include, but are not limited to: font size, whether it is bold, font color, whether it is highlighted, whether it is underlined, and whether it is italic.

[0696] Based on the layout characteristics of the content to be broadcast, the title, body text, subheadings, and multi-person dialogues can be distinguished.

[0697] For example, see Figure 45 The diagram shown illustrates the layout features of the text to be broadcast. The text to be broadcast, 2810, may include a title, body text, dialogue, subheadings, and key information. Please refer to the diagram for details on each part. Figure 45 This will not be elaborated upon here.

[0698] For example, see Figure 46 The diagram illustrates a process for adjusting the sound expression of different content structures based on page layout characteristics. This process may include the following steps:

[0699] Step S4601: Electronic device 400 acquires the content to be broadcast.

[0700] Step S4602: Electronic device 400 acquires the layout information of the content to be broadcast, including font attributes and alignment attributes.

[0701] In step S4603, the electronic device 400 determines the text size of the content to be broadcast based on the layout information.

[0702] For example, the electronic device 400 calculates the mode of font size by statistically analyzing the font size of each character in the content to be broadcast based on the font attributes in the layout information, and determines the mode of font size as the body font size, while determining the non-mode of font size as a non-body font size.

[0703] In step S4604, the electronic device 400 determines the title, subtitles, key information, and multi-person dialogue in the content to be broadcast based on the version information and body text number.

[0704] For example, if the font size of the first line of text to be broadcast is larger than that of the body text, or if the font and bolding attributes of the first line of text differ from those of the body text, then the first line of text is determined to be a title.

[0705] The font and bolding attributes of the first line of text differ from those of the body text. For example, the first line of text is bolded, but the body text is not. Figure 45 In the text to be broadcast, 2810, the title "Music Competition Shanghai Station Opens" is bolded, but the text itself is not.

[0706] Electronic device 400 can use a semantic model to determine whether a certain part of the content to be broadcast is a multi-person conversation.

[0707] For incomplete sentence text that forms a separate paragraph, electronic device 400 determines whether the font size of the text is larger than the font size of the body text; if so, it determines that the text is a subheading. Furthermore, electronic device 400 can also determine that the text is a subheading when it determines that the text has a center-aligned attribute. Figure 45 In the text to be broadcast, 2810, the subheading "Historical Review" has the attributes of bolding and center alignment.

[0708] After determining the main text of the content to be broadcast based on the font size mode, the electronic device 400 can identify text with attributes such as bold, highlight, underline, and italics as key information within the main text. For example, in Figure 45 In the text to be broadcast, 2810, "Li Yuehong" and "twentieth gold medal" are bolded.

[0709] It is understandable that the content to be broadcast may include at least one of the following: title, subheading, key information, and multi-person dialogue.

[0710] Step S4605: Electronic device 400 outputs tagged text.

[0711] Once the title, subheadings, key information, and multi-person dialogue of the content to be broadcast are determined, the text corresponding to the title is retrieved and tagged; the text corresponding to the subheadings is retrieved and tagged; the text corresponding to the key information is retrieved and tagged; and the text corresponding to the multi-person dialogue is retrieved and tagged. The tagged text is then output.

[0712] In step S4606, the electronic device 400 performs TTS parameter matching based on the tags of each text to determine the audio parameters of each text.

[0713] It should be noted that a mapping relationship between tags and audio parameters is pre-set. The audio parameters corresponding to different tags can be different or the same. After obtaining the tagged text, TTS parameter matching is performed to determine the audio parameters corresponding to each tag based on the mapping relationship between tags and audio parameters, thereby determining the audio parameters of each piece of text.

[0714] For example, see Figure 47 The illustrated diagram of labels and parameters shows that the second original text content 4710 is: "The Shanghai leg of the music competition begins. The Shanghai leg of the music competition officially kicks off today. The organizers say: 'The Shanghai contestants performed very well.' Ning Le (a first-year junior high student) from Sunshine Middle School will participate tomorrow."

[0715] The second original text content 4710 is a part of the text to be broadcast 2810.

[0716] In this embodiment of the application, after processing the second original text content 4710, a second output text 4720 is obtained, which is text with tags.

[0717] The text “Music Competition Shanghai Station Begins” is tagged as “Title”, indicating that the text is a title.

[0718] The text “Music Shanghai Station Competition Officially Kicks Off Today” is tagged with “Main Text”, indicating that this text is the main text.

[0719] The second output text 4720 is processed by TTS parameter matching to obtain the second TTS parameter matching example 4730. Among them, for "Music Competition Shanghai Station Opening" with the "Title" tag, the matched TTS parameter (i.e., audio parameter) is "broadcast timbre, speech rate 1".

[0720] For the text tag "Music Shanghai Station Competition Officially Kicks Off Today", the matched TTS parameters (i.e., audio parameters) are "broadcast timbre, speech rate 1.5".

[0721] After determining the audio parameters of each text, the electronic device 400 can input both the audio parameters and the text into the TTS module. The TTS module then performs speech synthesis based on the audio parameters and outputs audio data. Thus, when broadcasting audio data, the audio expression differs depending on the content structure. For example, taking the second TTS parameter matching example 4730 as an example, using a broadcast voice with a speech rate of 1, it can broadcast "The Shanghai leg of the music competition has begun"; using a broadcast voice with a speech rate of 1.5, it can broadcast "The Shanghai leg of the music competition officially begins today." In this way, the user can accurately understand the information in the original broadcast content simply by listening to the broadcast voice.

[0722] As can be seen from the above, the embodiments of this application adjust the sound expression of different content structures according to the layout characteristics of the content to be broadcast, so that the sound parameters of different content structures are different, thereby enabling the broadcast voice to accurately convey the information of the original broadcast content.

[0723] In some embodiments, steps S4602 to S4604 and step S4302 described above can be implemented by multimodal LLM-SFT. In this case, the content to be broadcast can be input into multimodal LLM-SFT to obtain tagged text output by multimodal LLM-SFT.

[0724] In the various embodiments described above, the processing steps for audio optimization and text optimization are explained respectively. When the electronic device 400 needs to optimize the audio and text of the content to be broadcast, it first performs audio optimization processing on the content to be broadcast to obtain tagged text, then performs text optimization on the tagged text, and finally performs TTS parameter matching on the optimized text.

[0725] For example, see Figure 48 The illustration shown in this application provides a schematic diagram of obtaining audio data based on the content to be broadcast. If only text optimization of the content to be broadcast is required, after text optimization, the optimized text is input into the TTS module to perform speech synthesis and obtain the audio data output by the TTS module.

[0726] If only audio optimization of the content to be broadcast is required, after obtaining the tagged text, TTS parameter matching is performed on the tagged text to obtain the audio parameters of each text; finally, the text and audio parameters are input into the TTS module to perform speech synthesis and obtain the audio data output by the TTS module.

[0727] If audio and text optimization are required for the content to be broadcast, the audio optimization process begins by obtaining tagged text. This tagged text is then combined with the content to be broadcast for further text optimization, integrating the tagged text into the optimized text. TTS parameter matching is then performed on the optimized text to obtain the corresponding audio parameters for the tagged text. Finally, the optimized text and audio parameters are input into the TTS module for speech synthesis, yielding the audio data output by the TTS module. The content to be broadcast can include structured page information such as text, images, and tables. Alternatively, the content can be obtained by processing screenshots (e.g., long screenshots, page screenshots, or screen screenshots).

[0728] All the processes of text optimization can be implemented through the text rewriting module, which can be a functional module or an implementation based on the LLM model.

[0729] See Figure 49 This is a flowchart illustrating a broadcasting method provided in an embodiment of this application. The method may include the following steps:

[0730] Step S4901: Electronic device 400 acquires content to be broadcast, which includes first text content and second text content; the first text content is annotation information, the second text content is annotated text, and the annotated text is the text annotated by the annotation information; or, the first text content is a portion of the second text content.

[0731] In some embodiments, the electronic device 400 displays a first interface; in response to a user's operation on the first interface, it obtains content to be played based on the content displayed on the first interface. The user's operation on the first interface is used to instruct the voice to play the content of the first interface.

[0732] For example, in an article or news broadcasting scenario, the first interface is used to display the article or news. The content displayed on this interface may include, but is not limited to, text, images, and tables. The user's operation on the first interface is a click operation on the reading control. In response to the click operation on the reading control, the electronic device 400 obtains the link to the article or news displayed on the first interface, and retrieves the content of the article or news based on the link, thus obtaining the content to be broadcast.

[0733] For example, in a screen reading scenario, the first interface displays text, images, and tables. User actions on the first interface can trigger the reading of the entire screen content, or they can select content to be read and trigger the reading of the selected content. The electronic device 400 responds to the user's actions on the first interface by using either the entire screen content or the user-selected content as the content to be read.

[0734] In other embodiments, after receiving a notification that requires voice broadcast, the electronic device 400 obtains the content to be broadcast according to the content of the notification. At this time, the content of the notification may be taken as the content to be broadcast. Certainly, the electronic device 400 may also obtain the content to be broadcast according to the content of the notification and the source of the notification. For example, the content of the notification and the description information of the message source may be taken as the content to be broadcast.

[0735] The annotation information may include pinyin annotations and / or text annotations. A text annotation is text that is located in parentheses and used to correct the preceding text; a pinyin annotation is pinyin used to annotate the preceding text.

[0736] It should be noted that text annotations are usually enclosed in parentheses, that is, the text annotation is located within the parentheses. Certainly, the text annotation may not be enclosed in parentheses, but may be enclosed by other punctuation marks, which is not limited herein.

[0737] In some embodiments, the second text content may be a long number string or a long text, and the first text content is N characters in the long number string or the long text. That is, the first text content is part of the text in the second text content.

[0738] Step S4902: The electronic device 400 processes the content to be broadcast to obtain a to-be-broadcast text, where the to-be-broadcast text includes a third text content, and the third text content is text content obtained by rewriting the second text content into the first text content.

[0739] When the first text content is annotation information and the second text content is the annotated text, that is, when the content to be broadcast includes annotation information, the electronic device 400 may correct and replace the annotated text in the content to be broadcast according to the annotation information.

[0740] In an implementation, the annotated text may include the annotated text in the fourth text content. The fourth text content refers to text content that contains annotation information and the annotated text. At this time, the electronic device 400 may acquire the fourth text content in the content to be broadcast; remove the annotation information in the fourth text content, use the annotation information to replace the annotated text in the fourth text content to obtain a first replacement text, and the third text content includes the first replacement text.

[0741] For example, in Figure 15 the pinyin annotation example 1501, the fourth text content is "郦食其(lìyìjī)", which is the phrase where the pinyin annotation is located. In the fourth text content, "郦食其" is the annotated text. The pinyin annotation "lìyìjī" and the parentheses are removed, and the annotated text "郦食其" is replaced with the pinyin annotation "lìyìjī", so that the obtained first replacement text is "lìyìjī".

[0742] For another example, in Figure 15 in the text annotation example 1502, the fourth text content is "this is really a helpless move", which is the text where the text annotation is located. "kua" is the annotated text. After removing the text annotation "nai" and the brackets, and using the text annotation "nai" to replace the annotated text "kua", the first replaced text obtained is "this is really a helpless move".

[0743] In another implementation, the annotated text in the fourth text content and the annotated text in the fifth text content may also be included. The fifth text content refers to text content that matches the fourth text content, contains part or all of the annotated text, but does not include annotation information. That is, the annotated text in the content to be broadcast may refer to the annotated character in the phrase where the annotation information is located, or may refer to the annotated character in the phrase where the annotation information is located and the annotated character at other positions in the content to be broadcast.

[0744] At this time, the electronic device 400 may acquire the fourth text content in the content to be broadcast, and the fourth text content is text content containing annotation information and annotated text; remove the annotation information in the fourth text content, and replace the annotated text in the fourth text content with the annotation information to obtain a first replaced text. In addition, the electronic device 400 further acquires a fifth text content in the content to be broadcast, the fifth text content matches the fourth text content, includes the annotated text, and does not include annotation information; the annotated text in the fifth text content is replaced with the annotation information to obtain a second replaced text. The third text content includes the second replaced text and the first replaced text.

[0745] In a specific application, the electronic device 400 may determine the fifth text content according to the fourth text content through dynamic dictionary matching. For example, for a pinyin annotation, the fifth text content may refer to Figure 16 the phrase that is completely the same as or partially the same as the phrase where the pinyin annotation is located in step S1606.

[0746] For example, referring to Figure 17 the left figure, in the pinyin annotation example 1501, the fourth text content includes "Li Yiji (lìyìjī)" at the first position 1710 and the second position 1720. According to dynamic dictionary matching, it is determined that the fifth text content of the pinyin annotation example 1501 is "Li Yiji" at the third position 1730 and the fourth position 1740.

[0747] As Figure 17As shown in the right figure, replace "lìyìjī" at the first position 1710 with "lìyìjī" at the second position 1720, and remove "lìyìjī" at the second position 1720, so that the obtained first replacement text is "lìyìjī" at the first position 1710. For "lì yì jī" at the third position 1730 and the fourth position 1740, replace "郦食其" with "lìyìjī", and the obtained second replacement text is "lìyìjī" at the third position 1730 and the fourth position 1740. At this time, the text to be broadcast can be as Figure 17 As shown in the right figure, the third text content may include Figure 17 "lìyìjī" at the first position 1710, the third position 1730 and the fourth position 1740 in the right figure.

[0748] In some embodiments, when the annotation information includes a text annotation, the text annotation is usually in parentheses. However, the text in the parentheses may be an annotation, or may be remark information or paralinguistic information, etc. The electronic device 400 may identify whether the text in the parentheses is a text annotation according to the semantic coherence and / or text correctness of the text before and after replacement, and realize replacing and correcting the related text according to the text annotation.

[0749] Specifically, the electronic device 400 acquires the sixth text content in the content to be broadcast, and the sixth text content is text containing parentheses. For example, the sixth text content is "实属无夸(奈)之举".

[0750] The electronic device 400 removes the parentheses and the text in the parentheses from the sixth text content to obtain a first comparison text; removes the parentheses and the text in the parentheses from the sixth text content, and uses the text in the parentheses to replace the preceding text of the parentheses in the sixth text content to obtain a second comparison text. For example, the first comparison text is "实属无夸之举", and the second comparison text is "实属无奈之举".

[0751] The electronic device 400 scores the first comparison text and the second comparison text according to semantic coherence and / or text correctness, and obtains the score of the first comparison text and the score of the second comparison text. A target comparison text is obtained from the first comparison text and the second comparison text, where the target comparison text is the text with a higher score among the first comparison text and the second comparison text; the third text content includes the target comparison text. For example, the score of "实属无夸之举" is lower than that of "实属无奈之举", and "实属无奈之举" is the target comparison text. For example, this embodiment is related to or similar to the content Figure 18 For the related or similar content, reference may be made to the above description, and details are not described herein again.

[0752] In specific applications, the electronic device 400 may use a semantic model to compare the semantic coherence and / or text correctness of the first comparison text and the second comparison text, and output the text with better semantic coherence and / or text correctness as the target comparison text.

[0753] In general, if the first comparison text has better semantic coherence and / or text correctness, it may be considered that the text in the parentheses is not used to correct the preceding text, that is, the text in the parentheses is not a text annotation. If the second comparison text has better semantic coherence and / or text correctness, it may be considered that the text in the parentheses is used to correct the preceding text, that is, the text in the parentheses is a text annotation.

[0754] In specific applications, the electronic device 400 may implement the processing procedure of correcting and replacing the relevant text according to the annotation information based on the text rewriting module. In some embodiments, the text rewriting module may be implemented by an LLM model. At this time, the electronic device 400 may construct a prompt, input the prompt into the LLM-SFT, and obtain the rewritten text output by the LLM-SFT. For example Figure 19 the process shown in. Certainly, in another embodiment, the text rewriting module may not be implemented by an LLM model. For example, in Figure 18 and Figure 16 , the text rewriting module may be a functional module, and the functional module may implement the corresponding step flow.

[0755] When the first text content is part of the text in the second text content, the third text content may include the rewritten first text content.

[0756] For example, the second text content is a long text or a long number string, the long text is composed of numbers and letters, the long number string is composed of numbers, the number of characters of the long text or the long number string is M, and M is greater than or equal to the first threshold. The first text content is N characters in the long text or the long character string. At this time, the electronic device 400 may extract N characters from the long text or the long number string, where N is less than M; by replacing the long text or the long number string in the content to be broadcast with the N characters, the text to be broadcast is obtained. For example, the content to be broadcast includes the text content "The driver has arrived, and the license plate number is Shanghai A**8907", the last four characters "8907" in the long text "A**8907" are extracted, "A**8907" is replaced with "8907", and adaptive adjustment is performed on other text content, so as to obtain the semantically smooth third text content "The driver has arrived, and the license plate tail number is 8907".

[0757] Step S4903: The electronic device 400 converts the text to be broadcast into audio data, where the audio data includes a first audio corresponding to the third text content, and the first audio broadcasts the second text content as the first text content.

[0758] For example, the third text content includes "this is really a last resort". By broadcasting the first audio, the character "kua" in "shi shu wu kua (nai) zhi ju" is pronounced as "nai", and there is no repeated reading that pronounces "kua" followed by "nai".

[0759] For another example, the text to be broadcast is Figure 17 The right figure shown in the figure, by broadcasting the first audio, the name "Li Shiqi" originally at the first position 1710, the third position 170 and the fourth position 1740 is pronounced as "lìyìjī", and there is no repeated reading phenomenon that pronounces "lìshíqí" and then pronounces "lìyìjī".

[0760] For another example, the third text content is "The driver has arrived, and the last digits of the license plate are 8907". By broadcasting the first audio, "A**8907" can be pronounced as "8907".

[0761] Step S4904: The electronic device 400 broadcasts the audio data.

[0762] In the embodiments of the present application, when the first text content is annotation information and the second text content is the annotated information, after correcting and replacing the annotated text in the content to be broadcast according to the annotation information, text-to-speech conversion is performed, which will not mispronounce the annotated text, nor cause the repeated reading phenomenon of pronouncing both the annotated text and the annotation information, so that the broadcast speech can convey the information of the original broadcast content more accurately. It should be noted that for related introductions of correcting and replacing relevant texts according to pinyin annotations and text annotations, reference may also be made to the above Figures 15-19 The relevant content will not be repeated here.

[0763] When the first text content is a part of the text of the second text content, since the number of characters of the first text content is less than the number of characters of the second text content, reading the second text content as the first text content through the first audio can not only make the broadcast speech convey the information of the original broadcast content more accurately, but also reduce the complexity of the speech broadcast information.

[0764] In some embodiments, when the first text content is a partial text of the second text content, the second text content may be a long text or a long number string. The long text is composed of numbers and letters, the long number string is composed of numbers, the number of characters of the long text or the long number string is M, and M is greater than or equal to a first threshold; the first text content may be N characters in the long text or the long number string, where N is less than M.

[0765] At this time, in the process of processing the content to be broadcast to obtain the text to be broadcast, the electronic device 400 may extract N characters from the long text or the long number string; and obtain the text to be broadcast by replacing the long text or the long number string in the content to be broadcast with the N characters.

[0766] For example, the N characters can be the last N characters of a long text or a long string of numbers. The long text could be a license plate number, and the long string of numbers could be a mobile phone number or a telephone number.

[0767] The first threshold can be set according to actual needs and is not limited here. For example, the first threshold can be the second character count threshold or the third character count threshold mentioned above.

[0768] For more information on simplifying long texts or strings in the content to be broadcast, please refer to [link / reference needed]. Figure 22 and Figure 23 The relevant content will not be repeated here.

[0769] In some embodiments, when the content to be broadcast includes elements with at least two semantics, the electronic device 400 can rewrite the elements with at least two semantics into text with corresponding semantics, and then use the TTS module to perform text-to-speech conversion.

[0770] At this time, during the process of processing the content to be broadcast and obtaining the text to be broadcast, the electronic device 400 can obtain the seventh text content in the content to be broadcast. The seventh text content is text content containing elements with at least two semantics. By converting the elements in the seventh text content into text with the target semantics, the eighth text content is obtained. The target semantics is one of at least two semantics. The text to be broadcast includes the eighth text content.

[0771] Elements with at least two semantic meanings include punctuation marks and / or phrases consisting of numbers and letters. The number of characters in the phrase is less than or equal to a first character count threshold. That is, when the content to be broadcast includes punctuation marks or phrases with multiple semantic meanings, these punctuation marks or phrases can be rewritten into text with the target semantic meaning.

[0772] For example, the seventh text content is "The average annual salary of a senior designer is 500,000", and the eighth text content is "The average annual salary of a senior designer is 500,000". The target semantic text is "500,000".

[0773] For example, the seventh text reads, "In the match between April 5th and 7th, 2024, Brazil easily defeated Argentina 4-1 and advanced to the Copa America final," while the eighth text reads, "In the match between April 5th and 7th, 2024, Brazil easily defeated Argentina 4-1 and advanced to the Copa America final."

[0774] In one implementation, the electronic device 400 can determine the semantic usage of elements in the seventh text content; determine the target rewriting rule corresponding to the semantic usage of the elements based on the mapping relationship between the semantic usage and the rewriting rule; and use the target rewriting rule to convert the elements in the seventh text content into text with the target semantic meaning to obtain the eighth text content.

[0775] For example, in Figure 7 In the text, the punctuation mark "-" is used. If the semantic usage of "-" in the seventh text is determined to be temperature, the rewriting rule corresponding to the semantic usage "temperature" is (i.e.) Figure 7 The transcription rule in the code is the target rewriting rule, where the target is rewritten as "below zero". For details on the implementation process, please refer to [link to implementation details]. Figures 6-9 and Figures 13-14 The relevant content will not be repeated here.

[0776] In another implementation, the electronic device 400 can convert the elements in the seventh text content into text with corresponding semantics according to the rewriting rules corresponding to each semantic usage of the elements, and obtain candidate text content corresponding to each semantic usage; convert the seventh text content into the original semantic vector; convert each candidate text content into a candidate semantic vector; and obtain the eighth text content from each candidate text content based on the similarity between the original semantic vector and each candidate semantic vector. The eighth text content is the text content with the highest similarity among the candidate text contents.

[0777] Electronic device 400 can use a text embedding model to convert the seventh text content and candidate text content into semantic vectors. The candidate text content can serve as comparison text for the seventh text content. For details on the implementation process, please refer to [link to relevant documentation]. Figures 10-12 The relevant content will not be repeated here.

[0778] In some embodiments, when the content to be broadcast includes first time text described using absolute time, the electronic device 400 can rewrite the absolute time into relative time or a time that conforms to the user's time description habits.

[0779] At this time, during the process of processing the content to be broadcast and obtaining the text to be broadcast, the electronic device 400 can convert the first time text into a second time text based on the time difference between the time described in the first time text and the current time, and / or the user's time description habits; wherein, the second time text adopts a relative time description, or conforms to the user's time description habits, or includes text described by absolute time and text described by relative time, or adopts a relative time description and conforms to the user's time description habits, or includes text described by absolute time and text described by relative time and conforms to the user's time description habits; the text to be broadcast includes the second time text.

[0780] For example, the first time text is "August 20, 2024, 14:30". The second time text can be "August 20, 2024 (tomorrow) 14:30" or "Tomorrow 14:30", or "August 20, 2024 (tomorrow) 2:30 PM" or "Tomorrow 2:30 PM". The user's time description habits are determined based on the target voice input by the user to the electronic device and / or the system time setting of the electronic device; the target voice is voice that includes time information. For a detailed implementation of this embodiment, please refer to... Figures 20-21 The relevant content will not be repeated here.

[0781] In some embodiments, when the content to be broadcast includes supplementary images, the electronic device 400 can integrate part or all of the content of the supplementary images with the text content to form semantically coherent text, and then use a TTS module to convert the text to speech in order to broadcast the image content of the supplementary images.

[0782] At this time, the content to be broadcast includes the first image. During the processing of the content to be broadcast to obtain the text to be broadcast, the electronic device 400 can obtain the image position of the first image within the broadcast content; parse the first image according to its position to obtain its image content; if the image content of the first image is not expressed in the text content of the content to be broadcast, then some or all of the image content of the first image and the text content of the content to be broadcast are merged into the ninth text content; the text to be broadcast includes the ninth text content; the audio data includes the second audio corresponding to the ninth text content, the second audio being used to broadcast some or all of the image content of the first image.

[0783] In practical applications, the electronic device 400 calculates the similarity between the image content and the text content of the image context; if the similarity is less than a preset threshold, it indicates that the image content is not expressed in the text content, and the image is determined to be a supplementary image; if the similarity is greater than or equal to the preset threshold, it indicates that the image content has been expressed in the text content, and the image is determined to be a complementary image.

[0784] The electronic device 400 can integrate all or part of the image content of the first image into the text content. In some embodiments, the appropriate image content can be selected and integrated with the text content based on the type of the first image.

[0785] For example, when the first image is a supplementary image, the overall description information of the image or the text information within the image can be integrated with the text content. Alternatively, both the overall description information and the text information within the image can be integrated with the text content simultaneously. For a related description of this embodiment, please refer to... Figures 31-34CThe relevant content will not be repeated here.

[0786] Furthermore, when integrating the image content and text content of the first image, the electronic device 400 can refer to the expressive structure of the complementary image.

[0787] At this point, the content to be broadcast also includes a second image. The content of the second image is already expressed in the text content of the content to be broadcast; that is, the second image is a complementary image. During the process of merging part or all of the image content of the first image and the text content of the content to be broadcast into the ninth text content, if the electronic device 400 determines that a text segment with a similarity to the image content of the second image exceeding a second threshold has a preset expression structure, then according to the preset expression structure, the text content of the content to be broadcast and part or all of the image content of the first image are merged into the ninth text content; the ninth text content has a preset expression structure. For example, the preset expression structure is "represent" or "said".

[0788] For a related description of this embodiment, please refer to [link / reference]. Figures 35-36 The relevant content will not be repeated here.

[0789] In some embodiments, when the content to be broadcast includes notes and / or secondary language information, the electronic device 400 can semantically integrate the notes and / or secondary language information into the text content.

[0790] At this time, during the process of processing the content to be broadcast and obtaining the text to be broadcast, the electronic device 400 can acquire the tenth text content in the content to be broadcast. The tenth text content is text content containing notes and / or paralinguistic information. The notes and / or paralinguistic information in the tenth text content are integrated into the text content to obtain the semantically coherent eleventh text content. The text to be broadcast includes the eleventh text content. The notes are information located in parentheses to supplement the preceding content. The paralinguistic information is non-verbal components other than spoken language located in parentheses and used to describe the character's speaking state.

[0791] For example, the tenth text includes "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow" or "Ning Le: I can control my mindset well, but I can't control the pressure well (laughs), sometimes the pressure is a bit too much." The eleventh text includes "Ning Le (first year of junior high school) from Sunshine Middle School will participate in the competition tomorrow" or "Ning Le said with a smile that she can control her mindset well, but she can't control the pressure well, sometimes the pressure is a bit too much."

[0792] For a related description of this embodiment, please refer to [link / reference]. Figures 28C to 31 The relevant content will not be repeated here.

[0793] In some embodiments, when the content to be broadcast includes a table in text form, the electronic device 400 can integrate all or part of the table content into the text.

[0794] At this time, during the process of processing the content to be broadcast and obtaining the text to be broadcast, the electronic device 400 can extract the content of the table; based on the content of the table and the text content of the content to be broadcast, the text to be broadcast is obtained, which includes part or all of the content of the table.

[0795] For a related description of this embodiment, please refer to [link / reference]. Figures 37-38 The relevant content will not be repeated here.

[0796] In some embodiments, when the content to be broadcast includes captions and / or superscripts, the electronic device 400 can remove the captions and / or superscripts before using the TTS module to convert text to speech.

[0797] At this time, during the process of processing the content to be broadcast and obtaining the text to be broadcast, the electronic device 400 removes the captions and / or superscripts from the content to be broadcast and obtains the text to be broadcast, which does not include captions and / or superscripts.

[0798] In some embodiments, the content to be broadcast includes message source description information, which describes the source of the notification message. In this case, during the processing of the content to be broadcast to obtain the text to be broadcast, the electronic device 400 removes the preset type information from the message source description information to obtain simplified message source description information; the text to be broadcast includes the simplified message source description information.

[0799] In some embodiments, the content to be broadcast does not include target information. In this case, during the process of processing the content to be broadcast to obtain the text to be broadcast, the electronic device 400 obtains supplementary information based on the source of the content to be broadcast. The supplementary information includes historical broadcast content and / or information related to the content to be broadcast within the application. Based on the supplementary information and the content to be broadcast, a content summary text is generated, which includes the target information. The text to be broadcast is the content summary text.

[0800] The target information can be the key information mentioned above, and the supplementary information can be... Figures 24-27 The relevant information. Historical broadcast content can be, for example, historical notification messages. For a related description of this embodiment, please refer to... Figures 24-27 The relevant content will not be repeated here.

[0801] In some embodiments, during the process of processing the content to be played and obtaining the text to be played, if the text content of the text to be played includes at least one feature text, the electronic device 400 tags each feature text to obtain tagged text content; the text to be played includes tagged text content; wherein, different feature texts have different tags; the tags are used to determine the sound parameters for playing the feature text.

[0802] At this time, the electronic device 400 can determine the sound parameters corresponding to each tagged text content according to the mapping relationship between the tags and the sound parameters; input the text to be played and the sound parameters into the text-to-speech module to obtain the audio data output by the text-to-speech module; wherein, the text to be played includes text content with the first tag and text content with the second tag; the audio data includes the third audio corresponding to the text content with the first tag and the fourth audio corresponding to the text content with the second tag; the sound parameters of the third audio and the fourth audio are different.

[0803] The characteristic text includes at least one of the following: first-level heading text, body text, dialogue text, subheading text, body text with a first font attribute, text containing sub-language, and voice-over text; the plural font attribute of the body text is the second font attribute, and the first font attribute is different from the second font attribute.

[0804] For example, the first font attribute is bold, highlighted, or underlined. The first font attribute is neither bold nor highlighted.

[0805] Dialogue text can be text within quotation marks, text after a colon, or text from segmented question-and-answer dialogues. Subheading text can refer to subheading text.

[0806] For a related description of this embodiment, please refer to [link / reference]. Figures 42-47 The relevant content will not be repeated here.

[0807] It should be noted that each item in the text optimization process can be implemented by the text rewriting module. The text rewriting module can be implemented using an LLM model or as a functional module.

[0808] See Figure 50 This is a flowchart illustrating a broadcasting method provided in an embodiment of this application. The method may include the following steps:

[0809] Step S5001: Electronic device 400 acquires the content to be broadcast, which includes annotation information.

[0810] Step S5002: The electronic device 400 processes the content to be broadcast to obtain a to-be-broadcast text, where the to-be-broadcast text includes a third text content, and the third text content is a text content obtained by replacing the annotated text in the content to be broadcast with annotation information.

[0811] Step S5003: The electronic device 400 converts the to-be-broadcast text into audio data, where the audio data includes a first audio corresponding to the third text content, and the first audio broadcasts the annotated text as the annotation information.

[0812] For example, the third text content includes "it is truly a last resort", and by playing the first audio, the character "kua" in "truly no kua(nai) move" is pronounced as "nai", without the repeated reading of pronouncing "kua" first and then pronouncing "nai".

[0813] For another example, the to-be-broadcast text is Figure 17 the right diagram shown, and by playing the first audio, "Li Shiqi" originally at the first position 1710, the third position 170 and the fourth posit...

Claims

1. A broadcasting method, characterized in that, Applied to electronic devices, the method includes: Obtain the content to be broadcast, which includes first text content and second text content; The content to be broadcast is processed to obtain the text to be broadcast, which includes a third text content, which is obtained by rewriting the second text content into the first text content; The text to be broadcast is converted into audio data, and the audio data includes a first audio corresponding to the third text content. The first audio broadcasts the second text content as the first text content. The audio data will be broadcast.

2. The method according to claim 1, characterized in that, The first text content is annotation information, and the second text content is the annotated text, which is the text annotated by the annotation information; The content to be broadcast is processed to obtain the text to be broadcast, including: Obtain the fourth text content from the content to be broadcast, wherein the fourth text content is text content containing the annotation information and the annotated text; Remove the annotation information from the fourth text content, and replace the annotated text in the fourth text content with the annotation information to obtain a first replacement text. The third text content includes the first replacement text.

3. The method according to claim 2, characterized in that, Processing the content to be broadcast to obtain the text to be broadcast also includes: Obtain the fifth text content from the content to be broadcast, wherein the fifth text content includes part or all of the annotated text, but does not include the annotation information; The annotation information is used to replace the annotated text in the fifth text content to obtain the second replacement text, and the third text content also includes the second replacement text.

4. The method according to claim 1, characterized in that, The first text content is annotation information, and the second text content is the annotated text, wherein the annotation information includes text annotations; The content to be broadcast is processed to obtain the text to be broadcast, including: Obtain the sixth text content from the content to be broadcast, wherein the sixth text content is text containing parentheses; Remove the parentheses and the text within the parentheses from the sixth text content to obtain the first comparison text; Remove the parentheses and the text within the parentheses from the sixth text content, and replace the text preceding the parentheses in the sixth text content with the text within the parentheses to obtain the second comparison text; Based on semantic coherence and / or textual correctness, the first comparison text and the second comparison text are scored to obtain the scores of the first comparison text and the second comparison text. Obtain the target comparison text from the first comparison text and the second comparison text, wherein the target comparison text is the text with the higher score between the first comparison text and the second comparison text; The third text content includes the target comparison text.

5. The method according to claim 1, characterized in that, The second text content is a long text or a long number string, wherein the long text consists of numbers and letters, the long number string consists of numbers, and the number of characters in the long text or the long number string is M, where M is greater than or equal to a first threshold. The first text content is N characters from the long text or the long number string, where N is less than M; The content to be broadcast is processed to obtain the text to be broadcast, including: Extract the N characters from the long text or the long string of numbers; The text to be broadcast is obtained by replacing the long text or the long number string in the content to be broadcast with the N characters.

6. The method according to claim 5, characterized in that, The N characters are the last N characters of the long text or the long number string; the long text is a license plate number, and the long number string is a mobile phone number or telephone number.

7. The method according to any one of claims 1 to 6, characterized in that, The content to be broadcast includes elements with at least two semantic meanings; The content to be broadcast is processed to obtain the text to be broadcast, including: Obtain the seventh text content from the content to be broadcast, wherein the seventh text content is text content containing elements having at least two semantic meanings; The eighth text content is obtained by converting the elements in the seventh text content into text with the target semantics, wherein the target semantics is one of the at least two semantics, and the text to be broadcast includes the eighth text content.

8. The method according to claim 7, characterized in that, The eighth text content is obtained by converting the elements in the seventh text content into text with the target semantics, including: Determine the semantic usage of the elements in the seventh text content; Based on the mapping relationship between semantic usage and rewriting rules, determine the target rewriting rule corresponding to the semantic usage of the element; Using the target rewriting rule, the elements in the seventh text content are transformed into text with the target semantics to obtain the eighth text content.

9. The method according to claim 7, characterized in that, The eighth text content is obtained by converting the elements in the seventh text content into text with the target semantics, including: Based on the rewriting rules corresponding to each semantic usage of the element, the elements in the seventh text content are converted into text with corresponding semantics to obtain candidate text content corresponding to each semantic usage. The seventh text content is converted into an original semantic vector; Each of the candidate text contents is converted into a candidate semantic vector; Based on the similarity between the original semantic vector and each of the candidate semantic vectors, the eighth text content is obtained from each of the candidate text contents, and the eighth text content is the text content with the highest similarity among the candidate text contents.

10. The method according to claim 7, characterized in that, The elements having at least two semantic meanings include punctuation marks, and / or phrases consisting of numbers and letters.

11. The method according to any one of claims 1 to 10, characterized in that, The content to be broadcast includes first-time text described using absolute time; The content to be broadcast is processed to obtain the text to be broadcast, including: Based on the time difference between the time described in the first time text and the current time, and / or the user's time description habits, the first time text is converted into a second time text; The second time text is described using relative time, or conforms to the user's time description habits, or includes text described using absolute time and text described using relative time, or uses relative time description and conforms to the user's time description habits, or includes text described using absolute time and text described using relative time and conforms to the user's time description habits; The text to be broadcast includes the second time text.

12. The method according to claim 11, characterized in that, The user time description habit is determined based on the target voice input by the user to the electronic device, and / or the system time setting of the electronic device; The target speech is speech that includes time information.

13. The method according to any one of claims 1 to 12, characterized in that, The content to be broadcast includes the first image; The content to be broadcast is processed to obtain the text to be broadcast, including: Obtain the image position of the first image within the broadcast content; The image content of the first image is obtained by parsing the first image based on its position. If the image content of the first image is not expressed in the text content of the content to be broadcast, then part or all of the image content of the first image and the text content of the content to be broadcast will be merged into the ninth text content. The text to be broadcast includes the content of the ninth text; The audio data includes a second audio corresponding to the ninth text content, which is an audio recording of part or all of the content of the first image.

14. The method according to claim 13, characterized in that, The content to be broadcast also includes a second image, the content of which is already expressed in the text content of the content to be broadcast; The ninth text content is formed by merging part or all of the image content of the first image with the text content of the content to be broadcast, including: If a text segment is determined to have a preset expression structure that is more similar to the image content of the second image than the second threshold, the text content of the content to be broadcast and part or all of the image content of the first image are merged into the ninth text content according to the preset expression structure. The ninth text content has the preset expression structure.

15. The method according to any one of claims 1 to 14, characterized in that, The content to be broadcast includes notes and / or secondary language information; The content to be broadcast is processed to obtain the text to be broadcast, including: Obtain the tenth text content from the content to be broadcast, wherein the tenth text content is text content containing the remarks information and / or the secondary language information; The annotation information and / or the secondary language information in the tenth text content are integrated into the text content to obtain the eleventh text content; The text to be broadcast includes the content of the eleventh text; The remarks information is located in parentheses and is used to supplement the description of the preceding content; The paralinguistic information refers to non-linguistic components other than spoken language.

16. The method according to any one of claims 1 to 15, characterized in that, The content to be broadcast includes tables in text format; The content to be broadcast is processed to obtain the text to be broadcast, including: Extract the contents of the table; Based on the content of the table and the text content of the content to be broadcast, the text to be broadcast is obtained, and the text to be broadcast includes part or all of the content of the table.

17. The method according to any one of claims 1 to 16, characterized in that, The content to be broadcast includes captions and / or subscripts; The content to be broadcast is processed to obtain the text to be broadcast, including: Remove the captions and / or superscripts from the content to be broadcast to obtain the text to be broadcast, wherein the text to be broadcast does not include the captions and / or superscripts.

18. The method according to any one of claims 1 to 17, characterized in that, The content to be broadcast includes message source description information, which describes the source of the notification message; The content to be broadcast is processed to obtain the text to be broadcast, including: Remove the preset type information from the message source description information to obtain simplified message source description information; The text to be broadcast includes the simplified message source description information.

19. The method according to any one of claims 1 to 18, characterized in that, The content to be broadcast does not include target information; The content to be broadcast is processed to obtain the text to be broadcast, including: Based on the source of the content to be broadcast, supplementary information is obtained, including historical broadcast content and / or information within the application related to the content to be broadcast; Based on the supplementary information and the content to be broadcast, a content summary text is generated, which includes the target information; The text to be broadcast is the content summary text.

20. The method according to any one of claims 1 to 19, characterized in that, The content to be broadcast is processed to obtain the text to be broadcast, including: If the text content of the text to be broadcast includes at least one feature text, each feature text is tagged to obtain the text content with the tags. The text to be broadcast includes text content with the aforementioned tags; The labels are different for different feature texts; the labels are used to determine the sound parameters for playing the feature text.

21. The method according to claim 20, characterized in that, Converting the text to be broadcast into audio data includes: Based on the mapping relationship between tags and sound parameters, determine the sound parameters corresponding to each text content with the aforementioned tags; The text to be played and the audio parameters are input into the text-to-speech module to obtain the audio data output by the text-to-speech module; The text to be broadcast includes text content with a first tag and text content with a second tag; The audio data includes a third audio corresponding to the text content with a first tag, and a fourth audio corresponding to the text content with a second tag. The third audio and the fourth audio have different sound parameters.

22. The method according to claim 20, characterized in that, The feature text includes at least one of the following: first-level heading text, body text, dialogue text, second-level heading text, body text with a first font attribute, text containing sub-language, and voice-over text; The plural font attribute of the main text is the second font attribute, and the first font attribute is different from the second font attribute.

23. A broadcasting method, characterized in that, Applied to electronic devices, the method includes: Obtain the content to be broadcast, which includes a first time text, which is text described using absolute time. The content to be broadcast is processed to obtain the text to be broadcast, which includes a second time text, which is converted from the first time text. The text to be broadcast is converted into audio data, the audio data including the audio corresponding to the second time text, and the audio corresponding to the second time text broadcasts the first time text as the second time text. The audio data will be broadcast.

24. The method according to claim 23, characterized in that, The content to be broadcast is processed to obtain the text to be broadcast, including: converting the first time text into the second time text based on the time difference between the time described in the first time text and the current time, and / or the user's time description habits.

25. The method according to claim 24, characterized in that, The user time description habit is determined based on the target voice input by the user to the electronic device, and / or the system time setting of the electronic device; The target speech is speech that includes time information.

26. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 22 or 23 to 25.

27. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 22 or 23 to 25.

28. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1 to 22 or 23 to 25.

Images