Streaming voice broadcasting method and system based on large model

Abstract

The invention provides a streaming voice broadcast method and system based on a large model, and belongs to the technical field of voice processing, and the method comprises the steps: carrying out the anomaly detection and correction of a first text, obtaining a target text, and dividing the target text into a plurality of text blocks based on a first time interval; the first text is an original text input to the first equipment; matching a voice processing library for each text block based on the content type of each text block, and performing text-voice conversion on each text block based on the voice processing library to obtain a plurality of voice blocks; and splicing the plurality of voice blocks based on the time point corresponding to each voice block to obtain a target voice, and broadcasting the target voice. According to the streaming voice broadcasting method and system based on the large model, the text processing efficiency and the broadcasting quality can be improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of speech processing technology, and more specifically, to a streaming speech broadcasting method and system based on a large model. Background Art

[0002] In the field of speech processing technology, with the continuous development of artificial intelligence and big data technology, streaming speech broadcast methods and systems have gradually become a hot topic of research. Traditional speech broadcast methods often rely on small-scale models or fixed algorithms, making it difficult to achieve efficient and accurate text-to-speech conversion. When processing long texts or continuous text streams, these methods may encounter problems such as processing delays and incoherent broadcasts, which affects the user experience. Summary of the invention

[0003] The purpose of the present disclosure is to provide a streaming voice broadcast method and system based on a large model to improve text processing efficiency and broadcast quality.

[0004] According to a first aspect of an embodiment of the present disclosure, a streaming voice broadcasting method based on a large model is provided, which is applied to a first device and includes: Performing anomaly detection and correction on a first text to obtain a target text, and dividing the target text into a plurality of text blocks based on a first time interval; the first text is an original text input into the first device; Matching a speech processing library to each text block based on the content type of each text block, and performing text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks; Based on the time point corresponding to each speech block, multiple speech blocks are spliced ​​to obtain a target speech, and the target speech is broadcasted.

[0005] A second aspect of the embodiments of the present disclosure provides a streaming voice broadcasting system based on a large model, which is applied to a first device and includes: a segmentation module, configured to perform anomaly detection and correction on a first text to obtain a target text, and to divide the target text into a plurality of text blocks based on a first time interval; the first text is an original text input into the first device; A processing module, configured to match a speech processing library to each text block based on the content type of each text block, and perform text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks; The broadcasting module is used to splice multiple voice blocks based on the time point corresponding to each voice block to obtain a target voice, and broadcast the target voice.

[0006] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned large model-based streaming voice broadcast method when executing the computer program.

[0007] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned large-model-based streaming voice broadcasting method are implemented.

[0008] The beneficial effects of a large model-based streaming voice broadcast method and system provided by the embodiments of the present disclosure are: The present disclosure can significantly improve the processing efficiency and broadcast quality of text, which is specifically embodied in: first, the target text is obtained by performing anomaly detection and correction on the first input text, thereby ensuring the accuracy and readability of the target text. Secondly, the target text is divided into multiple text blocks using the first time interval, and the most suitable speech processing library is matched for each text block according to the content type. This personalized matching strategy makes the text-to-speech conversion process more accurate and efficient, and can generate more natural and smooth speech blocks. After these speech blocks are spliced ​​based on their respective corresponding time points, the target speech formed is not only highly coherent, but also can accurately convey the information and emotions of the original text. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flowchart of a large model-based streaming voice broadcasting method provided in one embodiment of the present disclosure; Figure 2 A structural block diagram of a streaming voice broadcasting system based on a large model provided in an embodiment of the present disclosure; Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0011] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0012] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0013] Please refer to Figure 1 , Figure 1 A flow chart of a large model-based streaming voice broadcasting method provided in an embodiment of the present disclosure is applied to a first device. The method includes: S101: Perform anomaly detection and correction on a first text to obtain a target text, and divide the target text into a plurality of text blocks based on a first time interval; the first text is an original text input into a first device.

[0014] In this embodiment, the first device is a device that executes the large model-based streaming voice broadcast method. The first device can be any device that can receive text input and has a voice broadcast function, such as a smart phone, a tablet computer, or a car system.

[0015] The first text is the original text input into the first device. The target text is the text obtained by performing anomaly detection and correction on the original text. The first time interval represents the time length or unit for dividing the target text into multiple text blocks, and the time interval can be set according to the broadcast speed, the device performance of the first device, and the user's needs. The first time interval can be 10s, 5s, 2s, etc.

[0016] The text blocks are smaller text segments obtained by segmenting the target text based on the first time interval, and each text block contains a portion of the original text.

[0017] Performing anomaly detection and correction on the first text to obtain a target text includes: Determine the language background domain of the first text according to the content similarity between the first text and the input text within the first historical time period, and select a data set with the same language background domain as the first text from the text database; Anomaly detection and correction are performed on the first text based on the data set to obtain the target text.

[0018] In this embodiment, the first device has a recording and storage mechanism that can store text data within a first historical time period. The first time period can be a fixed value or can be adjusted according to the amount of text input to the first device. The first device can calculate the content similarity between the first text (i.e., the text currently to be processed) and these historical texts.

[0019] If the content similarity between the first text and the input text within the first historical time period is greater than a first threshold, the language background field of the input text within the first historical time period is used as the language background field of the first text; if the content similarity between the first text and the input text within the first historical time period is less than or equal to the first threshold, the language background field of the first text is determined according to the content of the first text.

[0020] For example, assuming that the content of the input text in the first historical duration is the working principle of the engine, and the first text is the components of the engine, then the language background field of the components of the engine and the working principle of the engine are the same, indicating that the first text may be a continuation of the content of the text input in the first historical duration. Assuming that the content of the input text in the first historical duration is the working principle of the engine, and the first text is the principle of rectilinear propagation of light, then the language background field of the first text and the content of the input text in the first historical duration are different, indicating that the first text has reopened a new topic compared with the content of the input text in the first historical duration. At this time, the language background field of the first text can only be determined based on the content of the first text. Therefore, the language background field of the first text can be determined based on the content similarity between the first text and the input text in the first historical duration.

[0021] When the language background domain of the first text is determined, the device can search for a data set with the same language background domain from the text database, and perform anomaly detection and correction on the first text based on the data set to obtain the target text. Anomaly detection includes grammar detection, semantic detection, and spelling detection. When such errors occur in the first text, the device can correct these error messages in time to obtain the correct target text.

[0022] Assuming that the first time interval is a fixed number of lines (n lines), the target text is divided into multiple text blocks based on the first time interval, including: Counting starts from the first character of the target text. When the text reaches n lines, the text between the first character and the last character of the nth line is taken as the first text block, and the text blocks are divided in sequence until the target text is read. n is a positive integer.

[0023] S102: Matching a speech processing library for each text block based on the content type of each text block, and performing text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks.

[0024] In this embodiment, the content type refers to the type of information or style expressed by the text block. For example, the text block can be a textbook, news, novel, poem, explanatory text, dialogue, etc. The content type determines the language characteristics, structural features and emotional color of the text block, thereby affecting the selection of the speech processing library and the effect of text-to-speech conversion.

[0025] A speech processing library is a software tool or library that can convert text into speech. Speech processing libraries usually contain pre-trained speech models, sound samples, and algorithms that can generate natural and fluent speech output. Text-to-Speech (TTS) is a technology that can convert text into speech output.

[0026] In this embodiment, according to the content type of the text block, a suitable speech processing library is selected for TTS conversion, and different speech processing libraries can process different types of content. Exemplarily, Microsoft Azure Text-to-Speech supports multiple languages ​​and voices, and is suitable for multiple content types, including news, novels, poetry, etc. It provides a rich selection of voices, and the most suitable voice can be selected according to the content type of the text block. Google Text-to-Speech: Based on Google's online TTS service, it has high voice quality and is suitable for scenarios that require high-quality voice output, such as poetry recitation, explanatory text explanation, etc. Amazon Polly provides high-quality speech synthesis services that support multiple languages ​​and voices. It is suitable for scenarios that require highly natural and fluent voice output, such as news broadcasts, business presentations, etc.

[0027] S103: splicing multiple voice blocks based on the time point corresponding to each voice block to obtain a target voice, and broadcasting the target voice.

[0028] In this embodiment, a speech block refers to a piece of generated audio data, which contains a continuous sound information. These speech blocks can be words, phrases, sentences or longer speech paragraphs. In speech processing or synthesis technology, speech blocks can be processed, edited or spliced ​​as basic units.

[0029] A time point refers to a moment or position that can be clearly identified on the timeline. In this embodiment, a time point is generally used to identify the moment when a voice block starts or ends. Through the time point, the playback order and duration of each voice block can be accurately controlled.

[0030] The target voice refers to the final audio obtained by splicing multiple voice blocks. It contains the content of all the spliced ​​voice blocks and is played in a predetermined order and time point. The target voice can be any form of audio output, such as a complete voice message, a voice story or a song.

[0031] In this embodiment, each speech block has one or more time points associated with it, and these time points are used to determine the position of each speech block in the final splicing result. Because each text block uses different speech processing libraries during the processing, the speech blocks obtained are not necessarily continuous in time, so it is necessary to splice the speech blocks according to the time points corresponding to each speech block, so as to obtain a complete and smooth audio. Time points play a vital role in the splicing process. They not only determine the position of each speech block in the final audio, but also affect the coherence and naturalness of the spliced ​​audio. If the time point setting is inaccurate, it may cause obvious pauses, overlaps or breaks in the spliced ​​audio. In the splicing process, the transition between the speech blocks also needs to be considered to ensure that the spliced ​​audio sounds natural and coherent. By splicing multiple speech blocks, we can get a target speech.

[0032] It can be concluded from the above that the present disclosure can significantly improve the processing efficiency and broadcast quality of text, which is specifically embodied in: first, the target text is obtained by performing anomaly detection and correction on the first input text, thereby ensuring the accuracy and readability of the target text. Secondly, the target text is divided into multiple text blocks using the first time interval, and the most suitable speech processing library is matched for each text block according to the content type. This personalized matching strategy makes the text-to-speech conversion process more accurate and efficient, and can generate more natural and smooth speech blocks. After these speech blocks are spliced ​​based on their respective corresponding time points, the target speech formed is not only highly coherent, but also can accurately convey the information and emotions of the original text.

[0033] In one embodiment of the present disclosure, performing anomaly detection and correction on a first text to obtain a target text includes: If the first text does not belong to the specific field, a first type of anomaly detection is performed on the first text to obtain a first anomaly detection result, and the first text is corrected based on the first anomaly detection result to obtain a target text; If the first text belongs to a specific field, performing a first type of anomaly detection and a second type of anomaly detection on the first text in sequence to obtain a second anomaly detection result, and correcting the first text based on the second anomaly detection result to obtain a target text; The number of detection types of the first anomaly detection is greater than the number of detection types of the second anomaly detection.

[0034] In this embodiment, a specific field refers to a field with specific expertise, specific industry specifications, or a specific subject matter scope. For example, the medical field, the legal field, the computer science field, etc., the texts in these fields often follow the terminology usage rules, logical structure, and expression methods that are unique to the field, which are different from ordinary general texts. The first type of anomaly detection is an anomaly detection method for the first text, and the content of the detection is relatively general. When the first text does not belong to a specific field, sufficient anomaly information can be obtained through this detection method alone for subsequent correction; when the first text belongs to a specific field, it is also necessary to perform the first type of anomaly detection first, and conduct a preliminary screening of various types of anomalies in the text, such as grammatical errors, spelling errors, basic semantic rationality, and other detection types.

[0035] The second type of anomaly detection is an anomaly detection method for texts in specific fields. After the first type of anomaly detection is performed on the first text, the second type of anomaly detection is required to further check whether the text meets the professional requirements of the specific field (including but not limited to the accurate use of professional terms, compliance with logical rules in the field, and compliance with industry standards and specifications). Compared with the first type of anomaly detection, the second type of anomaly detection focuses more on the professional aspects of a specific field. The number of detection types is relatively small, but it plays a key role in ensuring the accuracy and professionalism of texts in specific fields.

[0036] In this embodiment, when the input first text does not belong to a specific field, the first text is comprehensively screened mainly by relying on the first type of anomaly detection. The first type of anomaly detection has multiple detection types, and can check possible problems in the text from multiple common angles such as grammar, spelling, and semantics, such as checking whether the sentence is complete, whether the wording is accurate, and whether the semantics is reasonable. The first anomaly detection result is obtained through the first type of anomaly detection; according to the first anomaly detection result, the abnormal part of the first text is corrected in a targeted manner, so as to convert the original first text into a target text that has been optimized and common anomalies have been removed, so that it can better meet the requirements of processing steps such as dividing text blocks and speech synthesis.

[0037] When the first text input belongs to a specific field, the first type of anomaly detection will be performed first, using its multiple detection types to conduct a preliminary and comprehensive investigation of the text, covering grammar, spelling, basic semantics and other aspects, and obtain the corresponding detection results. However, since it is a text in a specific field, the first type of anomaly detection alone is not enough to ensure that the text fully meets the professional requirements of the field, so the second type of anomaly detection will continue on this basis. The second type of anomaly detection focuses on the accurate use of professional terms, the logical rationality within the field, and whether it complies with industry standards and specifications. By performing these two types of detections in succession, a comprehensive second anomaly detection result is obtained, which fully reflects all the anomalies of the first text in a specific field. The first text is corrected based on the second anomaly detection result to obtain the target text that meets the subsequent processing requirements.

[0038] It can be concluded from the above that, in this embodiment, for non-specific field texts, the first type of anomaly detection and correction can quickly and accurately handle common problems, improve text quality, and ensure the fluency and accuracy of subsequent voice broadcasts. For specific field texts, two types of detection are performed in sequence, which can comprehensively and deeply troubleshoot anomalies, ensure the correctness of professional terms, logic and specifications, and enhance the reliability and professionalism of voice broadcast content in professional fields. Overall, the quality of the target text is improved and the voice broadcast effect is optimized.

[0039] In an embodiment of the present disclosure, the first type of anomaly detection includes grammatical error detection, spelling error detection and / or semantic error detection.

[0040] In this embodiment, performing a first type of anomaly detection on the first text to obtain a first anomaly detection result includes: Grammatical error detection is performed on the first text to obtain abnormal grammatical information, and / or spelling error detection is performed on the first text to obtain abnormal text information, and / or semantic error detection is performed on the first text to obtain abnormal semantic information.

[0041] In this embodiment, grammar refers to the structural rules of a language, which are manifested differently in different languages. For example, in English, subject-verb agreement in sentences, the correct use of tenses, the structure of clauses, etc. all fall within the scope of grammar. A sentence like "He go to school" has a grammar error, and the correct form should be "He goes to school". Through grammar error detection, similar parts that do not conform to grammar rules in the text can be identified. In Chinese, the collocation of sentence components and word order are the key points of grammar detection. For example, the word order of "我饭吃在食堂" does not conform to Chinese grammar habits, and the correct form is "我在食堂吃饭". Such grammar problems can be discovered and marked through grammar error detection. Spelling error detection mainly targets problems in word spelling. In English, some words that are prone to spelling mistakes can be detected. For example, a spelling mistake like "accomodation" (the correct spelling is "accommodation") can be discovered. Spelling error detection will compare the words in the input text with a standard vocabulary list or spelling rules to find misspelled words. In Chinese, typos can also be detected. For example, a typo like "即然" (the correct form is "既然") can be identified to ensure the accuracy of the words used in the text.

[0042] Semantic error detection focuses on whether the meaning of a sentence is reasonable. In natural language, the combination of words should conform to logic. For example, the sentence "The cat is driving a car" is semantically unreasonable. Through semantic error detection, such sentences that do not conform to normal semantic understanding can be discovered. In Chinese, a sentence like "石头开心地笑了" will be judged as a semantic error from a semantic perspective if it is in a non-personified context, because stones generally do not have emotions and behaviors like "smiling happily".

[0043] It can be concluded from the above that grammar error detection can ensure that the text follows the language structure rules, making the sentences smooth and easy to understand, and improving the normativity of the text. Spelling error detection can guarantee the accuracy of words and avoid ambiguity in understanding caused by spelling problems. Semantic error detection focuses on the logical rationality of sentences, enhancing the accuracy and coherence of text expression. In summary, the first type of anomaly detection can effectively improve the text quality, enhance the quality of voice broadcasting, and enhance the user experience.

[0044] In one embodiment of the present disclosure, the voice processing library includes a sound effect library and a language model processing library; Matching a voice processing library for each text block based on the content type of each text block, including: If the content type of the text block is the sound effect type, it is determined that the voice processing library corresponding to the text block is the sound effect processing library; If the content type of the text block is a non-sound effect type, it is determined that the speech processing library corresponding to the text block is a language model processing library.

[0045] In this embodiment, the sound effect library is one of the components of the speech processing library, and mainly stores various sound effect resources, such as natural sounds (such as wind, rain, birdsong, etc.), common sound effects in life (such as doorbells, telephone rings, car horns, etc.), and some sounds with special effects (such as science fiction sound effects, horror sound effects, etc.). When it is determined based on the content of the text block that the text block needs to be added with sound effects, the device will call this sound effect library to process the speech content corresponding to the text block and add corresponding sound effects to the speech.

[0046] The language model processing library is also part of the speech processing library. It mainly implements related functions based on language model technology. The language model is a tool for modeling natural language. It can generate natural and fluent speech based on given text, and can also perform processing on the speech content such as speech recognition and semantic understanding. When the content type of the text block is non-sound effect type, the device can select this language model processing library to process the speech content corresponding to the text block, convert the text into natural and fluent speech, or perform other related processing on the speech.

[0047] When the content type of the text block is a sound effect type, such as "The rain is pouring down, and the ducks are happily playing in the water", the sound of rain and ducks can be added to the text block. When the content type of the text block is a non-sound effect type, such as "The school held a sports meeting yesterday", the text block can be input into the language model and processed by the language model.

[0048] From the above, it can be concluded that this embodiment can match the sound effect library for the sound effect type text block, accurately add appropriate sound effects to the relevant content, and enhance the richness and interest of the voice. For the non-sound effect type matching language model processing library, this embodiment can ensure that the text is accurately converted into natural speech, and improve the clarity and coherence of the speech. In summary, this embodiment optimizes the voice processing process as a whole, improves the quality and effect of voice broadcasting, meets the voice processing requirements of different content types, and brings a better auditory experience to users.

[0049] In one embodiment of the present disclosure, text-to-speech conversion is performed on each text block based on a speech processing library to obtain multiple speech blocks, including: If the content type of the text block is a sound effect type, then performing text-to-speech conversion on the text block based on the sound effect processing library to obtain a first type of speech block; If the content type of the text block is non-sound effect type, the loss function corresponding to the language model is determined according to the amount of information corresponding to the text block; based on the language model and the loss function corresponding to the language model, the text block is converted from text to speech to obtain a second type of speech block.

[0050] In this embodiment, text-to-speech conversion refers to processing each divided text block using a specific speech processing library to convert the text content into a corresponding speech form, thereby obtaining a plurality of speech blocks.

[0051] The loss function measures the gap between the result of speech generated by the language model and the expected target. By continuously adjusting the parameters of the language model to reduce this gap, the final generated speech is more consistent with the content and requirements of the text block, thereby obtaining the second type of speech block.

[0052] When the content type of the text block is a sound effect type, for example, the text block content is a description related to a specific sound effect such as "clear bird chirping". At this time, the device will call the sound effect processing library in the speech processing library to perform text-to-speech conversion operations. Through the speech resources corresponding to various sound effects and related processing algorithms pre-stored in the sound effect processing library, the text block is accurately converted into a first-class speech block that can express the sound effect of "clear bird chirping".

[0053] When the content type of the text block is non-sound effect type, for example, the content of the text block is "The weather is very good today". First, the loss function corresponding to the language model should be determined based on the amount of information contained in this text block. The amount of information can be considered from multiple aspects, such as the complexity of the sentence (including factors such as vocabulary and sentence structure), semantic richness, etc. Different information amounts require different loss functions to optimize the performance of the language model in the process of converting text to speech, so as to better adapt to the specific characteristics of the text block and make the converted speech more in line with expectations.

[0054] After determining the appropriate loss function, the device will use the language model in the language model processing library and the determined loss function to perform text-to-speech conversion on the text block. Through the language model's understanding of the text, its ability to generate speech, and the optimization of the conversion process by the loss function, this non-sound effect type text block is eventually converted into a second-class speech block. The second-class speech block obtained in this way can express the content of the text block in a more natural and accurate speech form.

[0055] It can be concluded from the above that in this embodiment, for sound effect type text blocks, the sound effect processing library can accurately generate realistic sound effect voice blocks, enhancing the richness and attractiveness of the voice. For non-sound effect types, the loss function is determined according to the amount of information for conversion, which can make the voice more in line with the text content and natural and smooth. The overall accuracy and quality of voice conversion are improved, meeting the conversion requirements of different types of text blocks, and improving the effects and user experience of applications such as voice broadcasting.

[0056] In one embodiment of the present disclosure, determining a loss function corresponding to a language model according to the amount of information corresponding to a text block includes: If the amount of information corresponding to the text block is greater than the first value, the loss function corresponding to the language model is the first loss function; If the amount of information corresponding to the text block is less than or equal to the first value and greater than the second value, the loss function corresponding to the language model is the second loss function; If the amount of information corresponding to the text block is less than or equal to the second value, the loss function corresponding to the language model is the third loss function; The text-to-speech conversion quality adjustment accuracy of the first loss function is greater than the text-to-speech conversion quality adjustment accuracy of the second loss function; the text-to-speech conversion quality adjustment accuracy of the second loss function is greater than the text-to-speech conversion quality adjustment accuracy of the third loss function.

[0057] In this embodiment, the first loss function is a loss function used when the amount of information in the text block is greater than the first value. Due to the large amount of information, the text-to-speech conversion quality needs to be adjusted more finely, so the text-to-speech conversion quality adjustment accuracy of the first loss function is the highest. This high-precision adjustment can better handle complex, information-rich text blocks and ensure that the speech can accurately express the rich content of the text.

[0058] For example, if the amount of information in the text block is greater than the first value, the loss function corresponding to the language model is the first loss function. The first loss function has a higher text-to-speech conversion quality adjustment accuracy, and it can pay more attention to the details of the speech, such as making more detailed adjustments in the intonation, speed, pauses, etc. of the speech, so as to better convey the rich emotions and scene atmosphere in the text. For example, for "ancient and magnificent castle", it will be read in a relatively low and solemn tone, and for "the vines seem to have life, swaying gently with the breeze", there will be appropriate soft and soothing speed and pauses to enhance the expressiveness of the speech, so that the audience can better feel the unique charm of the castle in the mysterious forest.

[0059] The second loss function is a loss function used when the amount of information corresponding to the text block is less than or equal to the first value and greater than the second value. Its text-to-speech conversion quality adjustment accuracy is lower than that of the first loss function, and is suitable for text blocks with medium information content. While ensuring a certain conversion quality, the adjustment accuracy is slightly lower than that of the first loss function, so as to achieve a balance between accuracy and computing resource consumption.

[0060] For example, when the amount of information corresponding to the text block is less than or equal to the first value and greater than the second value, the loss function corresponding to the language model is the second loss function. The second loss function has a moderate text-to-speech conversion quality adjustment accuracy, and will appropriately emphasize some key contents while ensuring the basic natural fluency of the speech. For example, when reading aloud "a book about history", the pronunciation of the word "history" can be slightly emphasized to highlight the type of book. For the "sound of turning pages", some slight sound effects of turning pages can be added to make the speech more vivid.

[0061] The third loss function is used when the amount of information corresponding to the text block is less than or equal to the second value. The third loss function has the lowest text-to-speech conversion quality adjustment accuracy among the three, and is suitable for simple text blocks with small amounts of information. For such simple texts, overly fine adjustments are not required, and using a loss function with lower accuracy can more efficiently complete text-to-speech conversion.

[0062] For example, when the amount of information corresponding to the text block is less than or equal to the second value, the loss function corresponding to the language model is the third loss function. The third loss function has a relatively low text-to-speech conversion quality adjustment accuracy, and it mainly focuses on converting text into speech quickly and concisely to ensure the basic fluency and comprehensibility of the speech. For the text block "She smiled", it can be converted into speech with a relatively ordinary tone and a steady speaking speed, without too much complicated processing, because the text itself is very simple and does not require overly fine adjustments to convey rich information or emotions.

[0063] It can be concluded from the above that in this embodiment, for high-information text blocks, the high-precision first loss function is used, which can finely adjust the voice conversion, accurately convey rich content and emotions, and improve the voice quality. The second loss function is used for medium-information text blocks to improve processing efficiency while ensuring a certain quality. The third loss function corresponds to low-information text blocks, which is quickly processed and ensures basic comprehensibility, and optimizes resource utilization and voice conversion effects as a whole.

[0064] In one embodiment of the present disclosure, multiple speech blocks are concatenated based on the time point corresponding to each speech block to obtain a target speech, including: The emotional features between speech blocks at every two adjacent time points are determined, and multiple speech blocks are concatenated based on the emotional features to obtain the target speech.

[0065] In this embodiment, the emotional features are emotion-related attributes contained in the speech block, such as joy, sadness, anger, calmness, etc. These emotional features can be reflected by factors such as voice intonation, speech speed, volume, etc., and are used to make the transition between adjacent speech blocks more natural when splicing speech blocks, which conforms to human hearing and emotional cognition habits.

[0066] In this embodiment, the speech blocks of every two adjacent time points are determined. For example, the time points corresponding to speech block A are from the 1st second to the 3rd second, and the time points corresponding to speech block B are from the 3rd second to the 6th second, and these two speech blocks are adjacent. Then, the emotional characteristics between adjacent speech blocks are analyzed: assuming that the emotional characteristics of speech block A are pleasant, and the emotional characteristics of speech block B are sad. During the splicing process, the device will adjust the transition method according to these emotional characteristics. In order to make the transition from pleasant to sad natural, the process of human emotional conversion can be simulated by gradually lowering the tone and slowing down the speed at the splicing point, so as to make the audience more auditory and receptive to this emotional change. In this way, all speech blocks are spliced ​​together in order and emotional characteristics, and the target speech is finally obtained, making it sound more coherent and natural, and conforming to the logic of emotional expression.

[0067] It can be concluded from the above that this embodiment can make the target speech after splicing more coherent and natural in emotional expression, avoid abrupt transitions, and enhance the expressiveness and appeal of the speech. It makes it easier for the audience to understand and feel the emotional information conveyed by the speech, and improves the quality and audibility of the speech. At the same time, it also helps to improve the user experience of applications such as voice broadcasting, making the voice content more attractive and friendly.

[0068] Corresponding to a streaming voice broadcasting method based on a large model in the above embodiment, Figure 2 This is a structural block diagram of a streaming voice broadcast system based on a large model provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 2 The large model-based streaming voice broadcasting system 20 is applied to the first device, and includes: a segmentation module 21, a processing module 22 and a broadcasting module 23.

[0069] The segmentation module 21 is used to perform anomaly detection and correction on the first text to obtain a target text, and divide the target text into a plurality of text blocks based on a first time interval; the first text is the original text input into the first device; A processing module 22, configured to match a speech processing library for each text block based on the content type of each text block, and perform text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks; The broadcasting module 23 is used to splice multiple voice blocks based on the time point corresponding to each voice block to obtain the target voice, and broadcast the target voice.

[0070] In one embodiment of the present disclosure, the segmentation module 21 is specifically used for: If the first text does not belong to the specific field, a first type of anomaly detection is performed on the first text to obtain a first anomaly detection result, and the first text is corrected based on the first anomaly detection result to obtain a target text; If the first text belongs to a specific field, performing a first type of anomaly detection and a second type of anomaly detection on the first text in sequence to obtain a second anomaly detection result, and correcting the first text based on the second anomaly detection result to obtain a target text; The number of detection types of the first anomaly detection is greater than the number of detection types of the second anomaly detection.

[0071] In an embodiment of the present disclosure, the first type of anomaly detection includes grammatical error detection, spelling error detection and / or semantic error detection.

[0072] In one embodiment of the present disclosure, the speech processing library includes a sound effect library and a language model processing library; The processing module 22 is specifically used for: If the content type of the text block is a sound effect type, determining that the speech processing library corresponding to the text block is a sound effect processing library; If the content type of the text block is a non-sound effect type, it is determined that the speech processing library corresponding to the text block is a language model processing library.

[0073] In one embodiment of the present disclosure, the processing module 22 is specifically used for: If the content type of the text block is a sound effect type, then performing text-to-speech conversion on the text block based on the sound effect processing library to obtain a first type of speech block; If the content type of the text block is non-sound effect type, the loss function corresponding to the language model is determined according to the amount of information corresponding to the text block; based on the language model and the loss function corresponding to the language model, the text block is converted from text to speech to obtain a second type of speech block.

[0074] In one embodiment of the present disclosure, the processing module 22 is specifically used for: If the amount of information corresponding to the text block is greater than the first value, the loss function corresponding to the language model is the first loss function; If the amount of information corresponding to the text block is less than or equal to the first value and greater than the second value, the loss function corresponding to the language model is the second loss function; If the amount of information corresponding to the text block is less than or equal to the second value, the loss function corresponding to the language model is the third loss function; The text-to-speech conversion quality adjustment accuracy of the first loss function is greater than the text-to-speech conversion quality adjustment accuracy of the second loss function; the text-to-speech conversion quality adjustment accuracy of the second loss function is greater than the text-to-speech conversion quality adjustment accuracy of the third loss function.

[0075] In one embodiment of the present disclosure, the broadcast module 23 is specifically used for: The emotional features between speech blocks at every two adjacent time points are determined, and multiple speech blocks are concatenated based on the emotional features to obtain the target speech.

[0076] See also Figure 3 , Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 3 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules in the above-mentioned system embodiments, such as Figure 2 The functions of modules 21 to 23 are shown.

[0077] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0078] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0079] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0080] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of a large model-based streaming voice broadcasting method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device described in the embodiments of the present disclosure, which will not be repeated here.

[0081] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0082] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0083] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0085] In the several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0086] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0087] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0088] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A streaming voice broadcasting method based on a large model, applied to a first device, characterized in that: include: Performing anomaly detection and correction on a first text to obtain a target text, and dividing the target text into a plurality of text blocks based on a first time interval; the first text is an original text input into the first device; Matching a speech processing library to each text block based on the content type of each text block, and performing text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks; Based on the time point corresponding to each speech block, multiple speech blocks are spliced ​​to obtain a target speech, and the target speech is broadcasted.

2. A streaming voice broadcasting method based on a large model as claimed in claim 1, characterized in that: The step of performing anomaly detection and correction on the first text to obtain the target text includes: If the first text does not belong to a specific field, performing a first type of anomaly detection on the first text to obtain a first anomaly detection result, and correcting the first text based on the first anomaly detection result to obtain a target text; If the first text belongs to a specific field, performing a first type of anomaly detection and a second type of anomaly detection on the first text in sequence to obtain a second anomaly detection result, and correcting the first text based on the second anomaly detection result to obtain a target text; The number of detection types of the first anomaly detection is greater than the number of detection types of the second anomaly detection.

3. A streaming voice broadcasting method based on a large model as claimed in claim 2, characterized in that: The first type of anomaly detection includes grammatical error detection, spelling error detection and / or semantic error detection.

4. A streaming voice broadcasting method based on a large model as claimed in claim 1, characterized in that: The speech processing library includes a sound effect library and a language model processing library; The matching of each text block with a speech processing library based on the content type of each text block includes: If the content type of the text block is a sound effect type, determining that the speech processing library corresponding to the text block is a sound effect processing library; If the content type of the text block is a non-sound effect type, it is determined that the speech processing library corresponding to the text block is a language model processing library.

5. A streaming voice broadcasting method based on a large model as claimed in claim 4, characterized in that: The step of performing text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks includes: If the content type of the text block is a sound effect type, performing text-to-speech conversion on the text block based on the sound effect processing library to obtain a first type of speech block; If the content type of the text block is a non-sound effect type, the loss function corresponding to the language model is determined according to the amount of information corresponding to the text block; and the text block is converted from text to speech based on the language model and the loss function corresponding to the language model to obtain a second type of speech block.

6. A streaming voice broadcasting method based on a large model as claimed in claim 5, characterized in that: The step of determining a loss function corresponding to the language model according to the amount of information corresponding to the text block includes: If the amount of information corresponding to the text block is greater than the first value, the loss function corresponding to the language model is the first loss function; If the amount of information corresponding to the text block is less than or equal to the first value and greater than the second value, the loss function corresponding to the language model is the second loss function; If the amount of information corresponding to the text block is less than or equal to the second value, the loss function corresponding to the language model is the third loss function; The text-to-speech conversion quality adjustment accuracy of the first loss function is greater than the text-to-speech conversion quality adjustment accuracy of the second loss function; the text-to-speech conversion quality adjustment accuracy of the second loss function is greater than the text-to-speech conversion quality adjustment accuracy of the third loss function.

7. A streaming voice broadcasting method based on a large model as claimed in claim 1, characterized in that: The step of splicing the multiple speech blocks based on the time point corresponding to each speech block to obtain the target speech includes: The emotional features between speech blocks at every two adjacent time points are determined, and multiple speech blocks are concatenated based on the emotional features to obtain the target speech.

8. A streaming voice broadcasting system based on a large model, applied to a first device, characterized in that: include: a segmentation module, configured to perform anomaly detection and correction on a first text to obtain a target text, and to divide the target text into a plurality of text blocks based on a first time interval; the first text is an original text input into the first device; A processing module, configured to match a speech processing library to each text block based on the content type of each text block, and perform text-to-speech conversion on each text block based on the speech processing library to obtain a plurality of speech blocks; The broadcasting module is used to splice multiple voice blocks based on the time point corresponding to each voice block to obtain a target voice, and broadcast the target voice.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images