Dubbing polyphone correction method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a dubbing polyphone correction method, which comprises the following steps of: obtaining a dubbing text corresponding to a dubbing audio, performing word segmentation processing, and splitting the dubbing text into single vocabularies; recognizing and marking polyphones in the word sequence after word segmentation, extracting a preset number of words in front of and behind the marked polyphones, constructing a context window, and combining the words to summarize to form sentences; performing part-of-speech tagging and syntactic analysis on sentences in the context window to obtain semantic information of polyphones; different pronunciation modes of the polyphone are matched with the semantic information, the matching degree of each pronunciation is determined, the pronunciation with the highest matching degree is selected for correction, and target dubbing is generated. According to the method and the device, the correct pronunciation of the polyphone is automatically selected by combining the context and the semantic information, the pronunciation accuracy in dubbing is improved, manual proofreading caused by wrong pronunciation of the polyphone is reduced, the dubbing making efficiency is improved, the making cost is reduced, and the dubbing quality is optimized.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a method, device, equipment and storage medium for correcting dubbed polyphones. Background Art

[0002] In the financial sector, the application of audio and voice technologies is becoming more and more widespread, especially in intelligent customer service, automatic voice response (IVR) systems, speech recognition and speech synthesis, etc. Speech processing technology has become an important tool for financial service automation. However, the existing audio processing technology still has significant deficiencies when processing the Chinese language, especially in the recognition and pronunciation correction of polyphones, which brings many challenges to the customer service and data processing of financial institutions.

[0003] Polyphones are widely present in the Chinese language. In particular, in the speech recognition and speech generation process in the financial field, traditional audio processing technology often has difficulty in correctly processing these polyphones. Polyphones have different pronunciations in different contexts. Existing speech recognition systems and text-to-speech (TTS) systems often fail to accurately transmit information in financial conversations due to incorrect pronunciation of polyphones, affecting service quality.

[0004] In the field of finance, especially when automatically reporting financial products, market dynamics, and transaction confirmations, existing TTS systems often cannot adapt to the changes in polyphones in different scenarios, resulting in stiff or inaccurate pronunciations, which seriously affects the naturalness and fluency of speech synthesis. In addition, this unnatural pronunciation may affect customers' trust in financial services, thereby reducing the service efficiency and customer satisfaction of financial institutions.

[0005] Since existing technologies cannot automatically and accurately handle the pronunciation of polyphones, financial institutions often need to invest a lot of manpower in manual proofreading and post-processing to ensure the accuracy and professionalism of voice output. However, this manual intervention not only increases the operating costs of financial services, but also prolongs the service delivery cycle, affecting the service efficiency and customer experience of financial institutions.

[0006] Although some speech processing systems have introduced rule-based polyphone processing methods, these methods often rely on static lexicons and simple rule matching, and are unable to flexibly cope with the complex language environment in financial services. It is also difficult to accurately select pronunciation methods based on the actual context, further restricting their application in financial scenarios. Summary of the invention

[0007] The main purpose of the present invention is to provide a method, device, equipment and storage medium for correcting dubbing polyphones, aiming to solve the technical problem that the prior art lacks contextual understanding when processing polyphones, resulting in frequent pronunciation errors and difficulty in automatic correction.

[0008] To achieve the above object, the present invention provides a method for correcting a dubbing polyphonetic character, comprising:

[0009] Obtaining a dubbing text corresponding to the dubbing audio, performing word segmentation processing on the dubbing text, and splitting the dubbing text into individual words;

[0010] Identify the position of polyphones in the word sequence after word segmentation, and mark the identified polyphones;

[0011] Extracting a preset number of words before and after each marked polyphone, and constructing a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words;

[0012] Performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain semantic information of the corresponding polyphonetic characters in the current context;

[0013] Matching different pronunciations of the corresponding polyphonetic character with the semantic information to determine the matching degree corresponding to each pronunciation;

[0014] Select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character;

[0015] Based on the correct pronunciation of the polyphonetic character, the dubbing audio is corrected to generate a target dubbing.

[0016] Further, to achieve the above object, the present invention provides a device for correcting polyphonetic characters in dubbing, comprising:

[0017] A text processing module is used to obtain a dubbing text corresponding to the dubbing audio, perform word segmentation processing on the dubbing text, and split the dubbing text into individual words;

[0018] A polyphone recognition and marking module is used to identify the position of polyphones in the word sequence after word segmentation and mark the recognized polyphones;

[0019] A context construction module, for extracting a preset number of words before and after each marked polyphone, and constructing a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words;

[0020] A semantic analysis module, used to perform part-of-speech tagging and syntactic analysis on the sentences in the context window, and obtain semantic information of the corresponding polyphonetic characters in the current context;

[0021] A pronunciation matching module, used to match different pronunciations of the corresponding polyphonetic character with the semantic information, and determine the matching degree corresponding to each pronunciation;

[0022] A correct pronunciation determination module is used to select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character;

[0023] The audio correction module is used to correct the dubbing audio based on the correct pronunciation of the polyphonetic character to generate a target dubbing.

[0024] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a dubbing polyphonetic character correction program stored in the memory and executable on the processor, wherein the dubbing polyphonetic character correction program implements the steps of the dubbing polyphonetic character correction method as described above when executed by the processor.

[0025] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a dubbing polyphonetic character correction program is stored, and when the dubbing polyphonetic character correction program is executed by a processor, the steps of the dubbing polyphonetic character correction method as described above are implemented.

[0026] Beneficial effects: The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a method for correcting polyphones in dubbing, which includes obtaining the dubbing text corresponding to the dubbing audio and performing word segmentation processing to split it into individual words; identifying polyphones in the word sequence after word segmentation and marking them, extracting a preset number of words before and after the marked polyphones, constructing a context window, and combining the words to form sentences; performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain the semantic information of the polyphones; matching the different pronunciations of the polyphones with the semantic information, determining the matching degree of each pronunciation, selecting the pronunciation with the highest matching degree for correction, and generating a target dubbing. The present invention automatically selects the correct pronunciation of polyphones by combining context and semantic information, thereby improving the pronunciation accuracy in dubbing, reducing manual proofreading due to polyphone pronunciation errors, improving dubbing production efficiency, reducing production costs, and optimizing dubbing quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0028] Figure 1 A schematic diagram of an application environment of a method for correcting polyphonetic characters in dubbing according to an embodiment of the present invention;

[0029] Figure 2 A schematic flow chart of an embodiment of a method for correcting polyphonetic characters in dubbing according to the present invention;

[0030] Figure 3 It is a functional module schematic diagram of a preferred embodiment of the device for correcting polyphonetic characters in dubbing of the present invention;

[0031] Figure 4A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0032] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0033] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0034] The method for correcting polyphonetic characters in dubbing provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment, the user terminal communicates with the server terminal through the network. The server terminal can obtain the dubbing text corresponding to the dubbing audio through the user terminal and perform word segmentation processing to split it into individual words; identify polyphones in the word sequence after word segmentation and mark them, extract a preset number of words before and after the marked polyphones, build a context window, and combine the words to form sentences; perform part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain the semantic information of the polyphones; match the different pronunciation modes of the polyphones with the semantic information, determine the matching degree of each pronunciation, select the pronunciation with the highest matching degree for correction, and generate the target dubbing. The present invention automatically selects the correct pronunciation of the polyphones by combining the context and semantic information, improves the pronunciation accuracy in dubbing, reduces the manual proofreading caused by the pronunciation errors of the polyphones, improves the dubbing production efficiency, reduces the production cost, and optimizes the dubbing quality. Among them, the user terminal can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server terminal can be implemented by an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0035] See also Figure 2 , Figure 2 The flowchart of the embodiment of the method for correcting polyphonetic characters in dubbing provided by the present invention is as follows: It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that here.

[0036] like Figure 2 As shown, the dubbing polyphone correction method proposed by the present invention comprises the following steps:

[0037] S10, obtaining a dubbing text corresponding to the dubbing audio, performing word segmentation processing on the dubbing text, and splitting the dubbing text into individual words;

[0038] In this embodiment, text data corresponding to the voiceover is extracted from the audio file. Usually, the voiceover text may be embedded in the metadata of the audio file, or may exist separately as a subtitle file or a manually input text file. The synchronization of the text and the audio is crucial, so it is necessary to ensure that each audio segment corresponds one-to-one with the text segment based on the timeline.

[0039] The voiceover text can be directly obtained through subtitle files (such as SRT, ASS formats). If there is no clear association between the text and the audio, the corresponding voiceover text can be generated through speech-to-text (such as ASR technology).

[0040] Word segmentation is to split the text at the level of words or characters. Since there are no obvious word boundaries in Chinese text, word segmentation needs to rely on word segmentation algorithms in natural language processing technology. The accuracy of word segmentation will affect subsequent polyphone recognition and semantic analysis.

[0041] Use existing Chinese word segmentation tools (such as Jieba, HanLP, etc.) to perform word segmentation, and disassemble the entire text into words or single characters. Professional terms or custom dictionaries can be added during the word segmentation process to improve the accuracy of word segmentation.

[0042] Process the segmented text into a sequence of single words, which is convenient for subsequent polyphone recognition and context analysis. The semantic and pronunciation features of each word can be presented more clearly in this form, supporting further semantic processing. Store the segmented text as a serialized data structure, such as a list, an array, or a tree structure, for subsequent analysis. Each word contains additional information such as its position and part of speech.

[0043] Example illustration: In the financial field, financial terms with polyphones are often involved in the voiceover text. For example, the character '重' in the word '存折重发' has different pronunciations in different contexts. It is pronounced 'chóng' when it means 'again', and pronounced 'zhòng' when it means 'weight'. For the financial service of'reissuing a savings passbook', the correct pronunciation should be 'chóng', indicating reissuing the savings passbook. Through the polyphone correction technology, the system can recognize the polyphonic nature of the character '重' and, combined with context analysis, identify '重' as pronounced 'chóng'. This technology ensures the pronunciation accuracy in financial service announcements, avoids semantic misunderstandings caused by incorrect polyphone pronunciations, and thus improves the professionalism of financial services and customer satisfaction.

[0044] By accurately acquiring and segmenting the dubbing text, the consistency of the audio and text content is ensured, and a basis is provided for the subsequent polyphone recognition and pronunciation correction. The word segmentation process can refine the semantic structure of the text, which helps to more accurately identify polyphone pronunciation errors in complex contexts. The word segmentation process of the dubbing text greatly improves the efficiency of automated processing, reduces manual intervention, and optimizes the pronunciation quality of the dubbing.

[0045] S20, identifying the position of the polyphone in the word sequence after word segmentation, and marking the identified polyphone;

[0046] In this embodiment, in the text sequence after word segmentation, polyphonic characters are identified based on the position of the vocabulary. This process usually relies on a predefined polyphonic word library, or uses a context-aware algorithm to identify the pronunciation of polyphonic characters in different contexts. The goal is to find characters that may have polyphonic characteristics in each vocabulary.

[0047] Compare with polyphone word library: quickly identify which words contain polyphones by comparing the segmented words with a word library containing all common polyphones. Context awareness: dynamically determine whether the characters in a specific word are polyphones and determine their specific pronunciation by analyzing the context before and after the words and combining semantic analysis technology.

[0048] After identifying polyphones, they need to be marked so that they can be pronounced correctly in subsequent processing steps. The marking should not only indicate the location of the polyphone, but also include its context information, glyph information, and pronunciation-related features (such as the exact position of the polyphone in the vocabulary).

[0049] Add markers to the vocabulary sequence for each polyphone. The markers can be special characters, comments, or metadata to ensure that the polyphone can be quickly identified and processed in subsequent steps. The markers can include: the start and end positions of the polyphone, context information, possible pronunciations, etc.

[0050] Example illustration: In the financial field, dubbing texts often contain some polysemous words. For example, the word "bank passbook". In this scenario, the "zhe" in "passbook" is a polysemous word, which can be pronounced as "zhé" (such as "fold") or "shé" (such as "lose money"). In financial dubbing, the "zhe" in "passbook" should be pronounced as "zhé", indicating the name of the bank passbook item. If the system misidentifies this word as "shé", it may lead to pronunciation errors, thus causing misunderstandings for customers. First, the system performs word segmentation on "bank passbook" and splits it into two words: "bank" and "passbook". Next, the system identifies the word "zhe" as a polysemous word in the segmented word sequence and marks it. In the subsequent processing, the system combines context analysis and pronunciation databases to select the correct pronunciation method to ensure that the "zhe" in "passbook" is pronounced correctly in the dubbing. This can effectively reduce pronunciation errors in dubbing and improve the professionalism of financial information broadcasting and the customer experience.

[0051] By identifying and marking polysemous words in the segmented word sequence, the system can accurately locate the polysemous words in the text, ensuring that the polysemous word problems can be targeted in the subsequent pronunciation matching and correction processes. It can effectively reduce the inaccurate information transmission caused by incorrect pronunciation of polysemous words and improve the accuracy and quality of dubbing.

[0052] S30, Extract a preset number of words before and after each marked polysemous word to construct a context window for each marked polysemous word, and the context window contains sentences formed by summarizing the words;

[0053] In this embodiment, starting from the already marked polysemous word, a fixed number of words before and after the polysemous word are extracted. These words help to understand the context environment of the polysemous word and provide information support for subsequent semantic analysis and pronunciation matching. This preset number is usually set according to the requirements of the application scenario and can flexibly adapt to different types of contexts.

[0054] According to the position of the polysemous word in the word sequence, a preset number of words are extracted before and after the polysemous word to construct a window containing context information. For example, if 3 words are extracted before and after each, a context window with a range of 7 words can be constructed.

[0055] The context window is a set of words extracted around the marked polysemous word. By constructing the context window, sufficient context information can be provided for the polysemous word, helping the system to understand the specific meaning and pronunciation requirements of the word in the current environment. The context window can help the system to perform word sense disambiguation and pronunciation matching more accurately.

[0056] Combine the extracted front and back words into a window as an independent context unit. The context window not only contains the words themselves but may also include additional information for each word, such as part of speech, position, etc., providing input for subsequent semantic processing.

[0057] The context window is not just a simple collection of words but needs to be recombined into the sentence structure of natural language for easier semantic understanding. By integrating the words in the context into sentences, the system can analyze the grammar and semantics of the sentences to determine the semantic status and pronunciation choice of the polyphone in the whole sentence.

[0058] Rearrange the extracted words to form complete sentences or fragments. Through sentence-level analysis, the system can gain a deeper understanding of the context window, laying a foundation for subsequent pronunciation matching and correction.

[0059] Example illustration: In the dubbing in the financial field, for example, in the sentence "The bank president chairs the meeting", the system recognizes that the "行" in "银行行长 (bank president)" is a polyphone and constructs a context window with complete semantics by extracting the 2 words before and after the sentence "The bank president chairs the meeting". Through context analysis, the system can determine that the "行" here should be pronounced as "háng" because the "chairs the meeting" in the context implies a position rather than "行动 (action)". This method of constructing the context window helps to avoid pronunciation errors in dubbing and ensures the accurate transmission of financial information.

[0060] By extracting a preset number of words before and after the polyphone to construct the context window, the system can provide sufficient context information for the pronunciation judgment of the polyphone. The sentence structure in the context window provides more comprehensive support for subsequent word sense disambiguation and semantic matching, improving the accuracy of the polyphone pronunciation. It effectively reduces pronunciation errors caused by insufficient context information and improves the overall dubbing quality.

[0061] S40, perform part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain the semantic information of the corresponding polyphone in the current context;

[0062] In this embodiment, part-of-speech tagging classifies each word in the context window to determine its grammatical role in the sentence (such as noun, verb, adjective, etc.). Through part-of-speech tagging, the system can understand the grammatical functions of each word, thus providing a basis for subsequent semantic analysis.

[0063] Use pre-trained part-of-speech tagging models (such as HMM, CRF, BERT, etc.) to perform word-by-word analysis on the sentences in the context window, and automatically assign corresponding part-of-speech tags to each word. For example, in the sentence "The bank president chairs the meeting", "bank president" will be tagged as a noun, "chairs" as a verb, and "meeting" as a noun. The system can use a custom dictionary or rules to improve the accuracy of part-of-speech tagging in a specific domain (such as the financial domain).

[0064] Syntactic analysis is to further analyze the sentence structure and determine the grammatical relationships between words. Through syntactic analysis, the system can construct the syntactic structure tree of the sentence, understand the dependency relationships between words (such as subject, predicate, object, etc.), and provide a reference for the pronunciation selection of polyphonic characters.

[0065] Use methods such as Dependency Parsing or Constituency Parsing to perform structured analysis on the sentences in the context window and construct the syntactic structure tree. For example, in the sentence "The bank president chairs the meeting", the system will identify that "bank president" is the subject of "chairs" and "meeting" is the object of "chairs".

[0066] After completing part-of-speech tagging and syntactic analysis, the system can further obtain the semantic information of polyphonic characters in the context. By analyzing the part of speech, grammatical relationships, and context of the sentence where the polyphonic character is located, the system can infer the specific semantics of the polyphonic character to determine its correct pronunciation in the current environment.

[0067] Combining the results of part-of-speech tagging and syntactic analysis, the system makes semantic inferences about polyphonic characters. For example, in the sentence "The bank president chairs the meeting", by analyzing the subject-predicate relationship between "bank president" and "chairs", it can be inferred that "行" should be pronounced as "háng" because in this context, "bank president" represents a position rather than an action. Use Word Sense Disambiguation (WSD) technology combined with the context to eliminate the ambiguity of polyphonic characters and determine their pronunciation in the current sentence.

[0068] Example illustration: In the sentence "The存折 is reissued", the system identifies that "重" is a polyphonic character. Through part-of-speech tagging of the sentence, the system identifies that "存折" is a noun and "重发" is a verb. Then through syntactic analysis, the system constructs the structure tree of the sentence and determines that "重" grammatically modifies "发". Combining the context, the system infers that "重发" means "reissue", so "重" should be pronounced as "chóng" rather than "zhòng". Through this combination of part-of-speech tagging and syntactic analysis, the system effectively avoids pronunciation errors of polyphonic characters and ensures the accurate transmission of financial information.

[0069] By performing part-of-speech tagging and syntactic analysis on the sentences in the context window, the system can accurately understand the semantic information of polyphonic characters in the current context. This not only provides strong semantic support for the pronunciation selection of polyphonic characters but also effectively reduces pronunciation errors caused by incorrect syntactic relations. The system can find the correct semantics of polyphonic characters in complex sentence structures and automatically select the correct pronunciation according to the semantics, significantly improving the accuracy and naturalness of dubbing.

[0070] S50, match the different pronunciation methods of the corresponding polyphonic characters with the semantic information, and determine the matching degree corresponding to each pronunciation method;

[0071] In this embodiment, after obtaining the different pronunciation methods of polyphonic characters, they are matched with the semantic information extracted from the current context. By analyzing the pronunciation differences of polyphonic characters in different contexts and combining the semantic clues provided by the context, the system can match the appropriate pronunciation method according to the context.

[0072] The system accesses the pronunciation database to obtain all possible pronunciation methods of the polyphonic character (for example, for "行", obtain two pronunciations "háng" and "xíng"). Then, the system uses the semantic information (such as part of speech, syntactic relation, word sense disambiguation result, etc.) extracted from the context window, combines it with the pronunciation method of the polyphonic character, and performs matching through an algorithm. The matching method can be based on a semantic similarity model (such as using a word vector model, such as Word2Vec, BERT, etc.) to calculate the matching degree between the pronunciation method and the context.

[0073] After matching the pronunciation method with the semantic information, the system generates a matching degree score for each pronunciation method. This matching degree reflects the suitability of the pronunciation method in the current context. The higher the score, the more suitable the pronunciation method is for the current context.

[0074] The system calculates the matching degree of each pronunciation method based on semantic similarity or the syntactic and word sense information of the context. For example, the system can use cosine similarity or other similarity measurement methods to calculate the matching degree between the pronunciation of the polyphonic character and the current context. The matching degree calculation result generates a value for each pronunciation method, and the system selects the best pronunciation through this value.

[0075] Example illustration: In the financial field, assume that the word to be pronounced is the character "chong" in "debt restructuring". First, the system retrieves two possible pronunciations of "chong", namely "zhòng" and "chóng", from the pronunciation database. Then, through context analysis, the system determines that the "chong" in "debt restructuring" should be pronounced as "chóng", meaning to reorganize or arrange again. Subsequently, the system assigns a higher matching degree to the pronunciation "chóng", while the pronunciation "zhòng" receives a lower matching degree due to semantic inconsistency. Finally, the system selects the pronunciation "chóng" with the highest matching degree, ensuring that the pronunciation of "debt restructuring" correctly conveys its semantics.

[0076] By matching different pronunciation modes of polyphonic characters with semantic information, the system can accurately select the pronunciation of polyphonic characters according to the specific context. The matching degree scoring mechanism provides a quantitative basis for the selection of polyphonic character pronunciations, thus avoiding misunderstandings or inaccurate information transmission caused by inappropriate pronunciation selections. It improves the accuracy of polyphonic character pronunciations, enabling the polyphonic characters in the voiceover to blend naturally and precisely into the context, and enhancing the overall voiceover quality.

[0077] S60, select the pronunciation mode with the highest matching degree as the correct pronunciation mode of the polyphonic character;

[0078] In this embodiment, after the system calculates the matching degrees of different pronunciation modes of the polyphonic character with the context semantics, it selects the pronunciation mode with the highest matching degree as the correct pronunciation of the polyphonic character. This step ensures that the most appropriate pronunciation mode is used for the polyphonic character in a specific context.

[0079] After the system obtains the matching degrees of each pronunciation mode, it sorts these matching degrees and selects the pronunciation mode with the highest score. Usually, the system automatically selects the item with the highest score according to the calculation results of the matching degree algorithm without manual intervention. The pronunciation mode with the highest matching degree will be marked as the correct pronunciation of the polyphonic character in the current context and passed to the subsequent audio processing module for pronunciation correction.

[0080] Based on the context information and pronunciation matching degree, the system finally determines which pronunciation mode better conforms to the current semantics and takes this pronunciation as the corrected standard pronunciation. Through the previous matching degree calculation and sorting, the system automatically selects the pronunciation mode with the highest matching degree and takes it as the standard pronunciation of the polyphonic character in the current context. This result will then be passed to the audio editing module for replacing the original incorrect pronunciation.

[0081] In one specific implementation, the closest pronunciation mode can be selected by comparing with the pronunciation modes in the pronunciation dictionary. The system can customize a pronunciation dictionary for the financial industry, which contains the pronunciation modes of industry-specific terms to ensure accurate matching.

[0082] It is also possible to use a word vector model to calculate the similarity between the pronunciation of polyphonic characters and the semantic meaning of the context, and select the pronunciation with the highest semantic matching degree. In a complex financial context, BERT or other language models can be used to calculate the semantic similarity between the pronunciation and the context. For example, the pronunciation of the character "重" in "重组" (reorganization) and "重写" (rewrite) is different, and the system can determine the most appropriate pronunciation based on the context.

[0083] It is also possible to perform pronunciation matching by combining a rule engine and a deep learning model. Use rule matching in simple contexts and deep learning models in complex contexts. For example, in financial news broadcasts, the system can analyze complex financial terms through deep learning, while in common terms, it directly calls the rule engine to judge the pronunciation. For example, for the character "重" in "重组", the pronunciation "chóng" is automatically selected.

[0084] Example: In a financial broadcast, assume the dubbed content is "银行行长主持会议" (The bank president presides over the meeting). The system recognizes that "行" is a polyphonic character, and through a matching algorithm, determines that the matching degree of the pronunciation "háng" is 0.9, while the matching degree of "xíng" is 0.1. The system automatically selects the pronunciation "háng" with a matching degree of 0.9 as the correct pronunciation, ensuring the accurate pronunciation of the term "银行行长". This selection mechanism effectively avoids misunderstandings caused by incorrect pronunciation of polyphonic characters in financial broadcasts, ensuring professionalism and the accuracy of information transmission.

[0085] By selecting the pronunciation with the highest matching degree as the correct pronunciation of the polyphonic character, the system can automatically correct incorrect pronunciations of polyphonic characters based on the semantic information of the context. It improves the accuracy of the pronunciation of polyphonic characters in complex contexts and reduces the need for manual intervention. The automated matching and selection mechanism improves the overall efficiency of dubbing, ensures accurate and natural pronunciation, and enhances the fluency and professionalism of the auditory experience.

[0086] S70, based on the correct pronunciation of the polyphonic character, correct the dubbed audio to generate the target dubbing.

[0087] In this embodiment, the system has selected the correct pronunciation of the polyphonic character through semantic analysis and matching calculations in the previous steps. The purpose of this step is to use the correct pronunciation to correct the incorrect pronunciation in the dubbed audio and ensure that the pronunciation of the audio is consistent with the text.

[0088] The system extracts the corresponding audio segment from the pronunciation library according to the correct pronunciation of the polyphonic character, or uses speech synthesis technology to generate the audio of the correct pronunciation. The audio segment of the correct pronunciation will be used to replace the incorrect pronunciation in the original audio.

[0089] After identifying the incorrect pronunciation of polyphones in the dubbing audio, the system needs to correct it by replacing the original incorrect pronunciation with the correct pronunciation. This process includes locating the polyphones in the audio, extracting and replacing audio clips, and adjusting the continuity and sound quality of the audio.

[0090] The system first locates the pronunciation segment of the polyphone through the timeline or audio frame information of the dubbing audio. Next, the original audio segment is extracted and replaced with the correct pronunciation segment. After the replacement is completed, the system will smooth the replaced audio to ensure a seamless connection between the new and old audio, while ensuring the continuity of tone, timbre and volume.

[0091] The final target dubbing audio is generated by correcting the pronunciation of polyphones in the dubbing audio. In the dubbing audio after this step, all polyphones have been corrected to the correct pronunciation.

[0092] After the audio correction and processing is completed, the system outputs the target dubbing audio as the final file to ensure smooth audio and accurate pronunciation. It can support audio file output in different formats (such as MP3, WAV, etc.) to adapt to different application scenarios.

[0093] By applying the correct pronunciation of polyphones to dubbing audio correction, the system can automatically and efficiently correct polyphone pronunciation errors in dubbing. This reduces the workload of manual proofreading while ensuring that the final generated target dubbing audio meets high standards in pronunciation accuracy and coherence. It improves the efficiency and quality of dubbing production, especially when batch processing is required, the system's automated audio correction technology significantly shortens working time.

[0094] The present invention relates to the fields of artificial intelligence technology and financial technology, and discloses a method for correcting polyphonic characters in dubbing, which comprises obtaining the dubbing text corresponding to the dubbing audio and performing word segmentation processing to split it into individual words; identifying polyphonic characters in the word sequence after word segmentation and marking them, extracting a preset number of words before and after the marked polyphonic characters, constructing a context window, and combining the words to form sentences; performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain the semantic information of the polyphonic characters; matching different pronunciation modes of the polyphonic characters with the semantic information, determining the matching degree of each pronunciation, selecting the pronunciation with the highest matching degree for correction, and generating a target dubbing. The present invention automatically selects the correct pronunciation of polyphonic characters by combining context and semantic information, improves the pronunciation accuracy in dubbing, reduces manual proofreading caused by polyphonic character pronunciation errors, improves dubbing production efficiency, reduces production costs, and optimizes dubbing quality.

[0095] In one embodiment, the above S40 includes:

[0096] S401, using a pre-trained part-of-speech tagging model to perform part-of-speech tagging on the words in the context window to determine the part-of-speech category of each word;

[0097] S402, performing dependency syntactic analysis on the sentences in the context window, constructing a syntactic structure tree of the sentences, and determining dependency grammatical relations between words;

[0098] S403, applying word meaning disambiguation technology, combining the part of speech category of each word and the grammatical relationship between words, to determine the semantic information of the corresponding polyphone in the current context;

[0099] S404: Use a context word vector model to semantically embed the semantic information to optimize the semantic information.

[0100] In this embodiment, a pre-trained part-of-speech tagging model (such as HMM, CRF, BERT, etc.) is used to analyze the words in the context window one by one, and each word is assigned its corresponding part-of-speech category (such as noun, verb, adjective, etc.), thereby providing a basis for subsequent grammatical and semantic analysis.

[0101] By loading the pre-trained part-of-speech tagging model and inputting the vocabulary sequence in the context window, the model automatically generates part-of-speech tags. For example, in "the bank president chaired the meeting", "the president" is tagged as a noun, "chaired" is tagged as a verb, and "meeting" is tagged as a noun. This automated processing method can improve the efficiency of text analysis while ensuring the accuracy of annotation.

[0102] Dependency parsing is a technique used to analyze sentence structure by constructing a syntactic structure tree to show the dependency relationships between each word in a sentence. This step helps the system understand which words are the subject or object of a polyphone, and how to correctly parse the semantic structure of the entire sentence.

[0103] Use dependency parsing tools (such as spaCy, Stanford Parser, etc.) to parse the sentence. The system constructs a syntax structure tree based on the parsing results to display the grammatical relationship between words. For example, in "The president of the bank presided over the meeting", the syntax structure tree shows that "President" is the subject of "presided over" and "meeting" is the object of "presided over".

[0104] Word Sense Disambiguation (WSD) technology is used to find the correct meaning or pronunciation in polysemous words or polyphones. By combining the results of part-of-speech tagging and the information of the syntactic structure tree, the system can disambiguate the polyphones and determine their semantics in the current context.

[0105] The system analyzes the context information of polyphonic characters, including the part-of-speech categories and dependency relationships of surrounding words, and then determines the exact pronunciation of the polyphonic character through a word sense disambiguation model (such as rule-based WSD or context-based deep learning model). For example, in the phrase "bank president", it can be judged from the context that the character "行" should be pronounced as "háng".

[0106] Semantic embedding is a process of converting semantic information into vector form through a context word vector model (such as Word2Vec, BERT, etc.). The goal of this step is to further optimize the semantic expression of polyphonic characters through context information, making the pronunciation selection more accurate.

[0107] By converting the polyphonic character and its context into vector form through the context word vector model, the system can judge the best pronunciation method of the polyphonic character through the similarity of semantic vectors. The advantage of the word vector model is that it can capture the subtle semantic differences of words in different contexts, ensuring that the pronunciation selection is more reasonable.

[0108] Example: In financial broadcasts, in a sentence like "debt restructuring", the system marks "重" as an adjective through part-of-speech tagging. The dependency syntactic analysis shows that "重" modifies "组". The system then uses word sense disambiguation technology to judge that "重" should be pronounced as "chóng" in this context. Finally, the system uses the context word vector model to embed this semantic information to ensure that this pronunciation method best fits the current context, thereby generating a pronunciation-accurate voiceover file. This process effectively improves the accuracy and professionalism of financial information transmission.

[0109] In this embodiment, by performing part-of-speech tagging, dependency syntactic analysis, and semantic embedding on the sentences in the context window, the system can accurately capture the semantic information of polyphonic characters in the current context. The word sense disambiguation technology combines with grammatical relationships to effectively eliminate the ambiguity of polyphonic character pronunciations, ensuring the accuracy of pronunciation selection. Semantic embedding further improves the rationality of polyphonic character pronunciations, making the entire voiceover process more natural and coherent. The entire process has a high degree of automation, reducing the need for manual intervention while improving the voiceover quality.

[0110] In one embodiment, the above S20 includes:

[0111] S201, in the segmented lexical sequence, use a predefined polyphonic word library or context-aware method to identify the positions of polyphonic characters;

[0112] S202, record the specific positions of the identified polyphonic characters in the lexical sequence, where the specific positions include the start index and the end index;

[0113] S203, mark the polyphonic characters in the lexical sequence using a preset marking format.

[0114] In this embodiment, by using a preset multi - tone word library or a context - based perception algorithm, it is automatically recognized which words contain multi - tone characters. The multi - tone characters in the word library record various possible pronunciation methods of the same character in different contexts. The context - aware method determines whether a multi - tone character appears in the sequence by analyzing the context.

[0115] The system can look up the segmented words in a dictionary to identify whether they are multi - tone characters. For example, the character "行" may appear in "行长" (bank president) and "银行" (bank) with different pronunciations. If the context - aware method is used, the system will analyze whether the multi - tone character appears in a specific word or phrase based on the context environment to determine whether the character is a multi - tone character.

[0116] When the system identifies a multi - tone character, it records its specific position in the word sequence, including the start and end indices in the text sequence. This provides precise position information for subsequent processing (such as pronunciation selection and audio correction).

[0117] The system stores the start and end indices of each identified multi - tone character in a data structure. For example, if the character "行" appears at the 5th to 6th character positions in the sequence, the system will record its start index as 5 and end index as 6.

[0118] For the convenience of subsequent processing, the system needs to clearly mark the multi - tone characters. These marks can use preset symbols or annotations to ensure that subsequent operations such as pronunciation correction can quickly locate and identify these multi - tone characters.

[0119] The system can use specific marking symbols (such as XML tags or JSON marks) to annotate the multi - tone characters. For example, mark the character "行" as <polyphone> OK< / polyphone> , or add additional metadata such as pronunciation methods to it.

[0120] Example illustration: In the text processing in the financial field, for example, when processing the dubbing of "银行行长" (bank president), the system recognizes through the multi - tone word library that the character "行" is a multi - tone character. The system records the specific position of the character "行" in the sequence, with the start index as 5 and the end index as 6. Next, the system marks the character "行" as <polyphone> OK< / polyphone> , so that the pronunciation problem of this multi - tone character can be quickly located and processed in subsequent pronunciation selection. This marking method ensures the pronunciation accuracy of complex terms in financial broadcasts and avoids ambiguity in information transmission.

[0121] In this embodiment, by identifying and marking polyphonic characters in the lexical sequence after word segmentation, the system can accurately locate the specific positions of polyphonic characters, providing precise data support for subsequent pronunciation selection and correction. This improves the automation level of polyphonic character processing, reduces the need for manual intervention, and enhances the text processing efficiency. The standardization of marking also ensures that polyphonic characters can be quickly recognized and processed in subsequent steps.

[0122] In one embodiment, the above S50 includes:

[0123] S501, constructing a pronunciation database and obtaining pronunciation samples corresponding to different pronunciation methods of the polyphonic character from the pronunciation database;

[0124] S502, matching the semantic information with different pronunciation samples of the corresponding polyphonic character;

[0125] S503, analyzing the matching degree of each pronunciation method of the polyphonic character with the current context based on the matching result.

[0126] In this embodiment, a database containing all common polyphonic characters and their corresponding different pronunciation samples is created. The pronunciation samples are pre-recorded audio segments that can represent the pronunciation methods of polyphonic characters in different contexts. This database can include standard pronunciations or multi-dialect pronunciations according to different fields.

[0127] The pronunciation database can be constructed by collecting different pronunciation samples of polyphonic characters (such as manual recording or speech synthesis). For example, the pronunciations of the character "重" may include "zhòng" and "chóng", and the system enters them into the database respectively. Each pronunciation sample in the database should be accompanied by metadata related to its pronunciation (such as usage scenarios, contexts, etc.).

[0128] Match the context semantic information with the pronunciation samples in the pronunciation database to judge the applicability of each pronunciation method in the current context. The system will use the grammatical and semantic information of the context (such as part of speech, grammatical structure, word sense disambiguation, etc.) to evaluate whether different pronunciation methods are suitable for this situation.

[0129] The system generates a semantic vector through context information, uses natural language processing techniques (such as word vector models) to compare the context information with the pronunciation samples in the database, and calculates the matching degree of different pronunciation methods in the current context. For example, for the character "行" in "行长", the system matches the pronunciation "háng" as more suitable for the context through semantic analysis.

[0130] After the system matches the semantic information with the pronunciation samples, it is necessary to calculate a matching degree score for each pronunciation method. The matching degree score reflects the semantic consistency of each pronunciation method with the current context. The higher the score, the more suitable the pronunciation method is for this situation.

[0131] The system analyzes the matching degree of each pronunciation method with the context by calculating the similarity of semantic vectors (such as cosine similarity) or through other statistical models. For example, the system calculates the matching degrees for the two pronunciation methods "háng" and "xíng" of the character "行" respectively, and assigns a higher matching degree to "háng" because it fits the current context better.

[0132] Example illustration: In the broadcast in the financial field, for example, for the character "重" in "债务重组", the system first obtains the pronunciation samples of "zhòng" and "chóng" from the pronunciation database. Through semantic analysis, the system matches the semantic information of the character "重" with the two pronunciation methods, and finally analyzes that the pronunciation "chóng" has a higher matching degree in the current context, so this pronunciation method is selected for the broadcast. This pronunciation matching technology ensures the pronunciation accuracy of financial information in the broadcast and improves the professionalism and reliability of the broadcast.

[0133] In this embodiment, by constructing a pronunciation database and matching it with the context semantic information, the system can intelligently select the correct pronunciation method of polyphonic characters according to the actual context. The matching degree calculation provides a quantitative basis for pronunciation selection, which helps to avoid inaccurate pronunciation caused by incorrect context understanding. It effectively improves the correctness and naturalness of the pronunciation of polyphonic characters, especially suitable for dubbing scenarios that require high accuracy.

[0134] In one embodiment, in the above S501, constructing a pronunciation database includes:

[0135] S5011, collecting multiple polyphonic characters and their corresponding different pronunciation methods, and collecting usage examples of polyphonic characters in different contexts;

[0136] S5012, based on the different pronunciation methods and usage examples of the polyphonic characters, recording different pronunciation samples of the polyphonic characters according to preset standards;

[0137] S5013, based on the different pronunciation methods and corresponding pronunciation samples of each polyphonic character, constructing a pronunciation database.

[0138] In this embodiment, common polyphonic characters and their different pronunciation methods are collected. These polyphonic characters may have different pronunciations in different contexts. The usage examples include specific examples of how polyphonic characters are pronounced in specific contexts, which provide a reference for subsequent pronunciation selection.

[0139] The system collects a list of common polyphonic characters, including characters such as "行 (xíng / háng)", "重 (zhòng / chóng)", "折 (zhé / shé)", etc., through manual or automated tools, and separately records their different pronunciation methods. For example, the character "行" can be pronounced as "háng" in "银行 (yín háng)" and "xíng" in "行动 (xíng dòng)". At the same time, the system also needs to record examples of polyphonic characters in various actual uses. For example, the "行" in "银行行长 (yín háng xíng zhǎng)" is pronounced as "háng", which can be stored as a context example.

[0140] To ensure that the sample quality of the pronunciation database meets the standards, the system records pronunciation samples for each polyphonic character according to preset standards such as the standard of Mandarin Chinese. The samples must cover all pronunciation methods of the polyphonic character in different contexts, so that the system can select the appropriate pronunciation when making a match in the future.

[0141] The system can record standard pronunciation samples of polyphonic characters in different contexts through professional voice actors or TTS (Text-to-Speech) technology. For example, a voice actor records the two pronunciations "zhòng" and "chóng" for the character "重" according to the standard of Mandarin Chinese and assigns them to the corresponding usage scenarios. Each recorded sample should carry metadata indicating the context and pronunciation method of the pronunciation, so as to facilitate quick search during pronunciation correction.

[0142] Organize all the recorded pronunciation samples into a pronunciation database. This database includes all pronunciation methods of each polyphonic character and their corresponding audio samples, providing data support for subsequent automated pronunciation correction.

[0143] The system constructs a structured pronunciation database based on the polyphonic characters and their pronunciation samples. Each polyphonic character in the database will contain all its pronunciation samples and their associated context examples. The structured data includes the character, pronunciation, context, and audio file of the pronunciation sample of the polyphonic character.

[0144] Example illustration: In financial broadcasts, the system needs to process characters such as "重" in "重组 (chóng zǔ)". By constructing a pronunciation database, the system has collected two pronunciation methods of the character "重": pronounced as "chóng" in "重组" and "zhòng" in "严重 (yán zhòng)". The system records standard audio samples for each pronunciation method and stores them in the database in association with the corresponding usage examples. When broadcasting financial news, the system can quickly extract the correct pronunciation sample from the pronunciation database according to the context, ensuring that the "重" in "重组" is accurately pronounced as "chóng", thus improving the accuracy and professionalism of information transmission.

[0145] In this embodiment, by collecting different pronunciation methods of polyphonic characters and their examples in actual use, the system can provide comprehensive data support for the pronunciation of polyphonic characters in different contexts. The recorded standard pronunciation samples ensure the accuracy of pronunciation, while the pronunciation database provides a reliable basis for subsequent pronunciation matching and correction. It not only improves the accuracy of the dubbing and speech recognition systems, but also significantly reduces pronunciation errors of polyphonic characters and optimizes the overall dubbing quality.

[0146] In one embodiment, the above S70 includes:

[0147] S701, determining the position of each polyphonic character in the dubbing audio through the timeline information of the dubbing text and the dubbing audio;

[0148] S702, using speech recognition technology to analyze the current pronunciation method of the polyphonic character in the dubbing audio and determining whether the current pronunciation method of the polyphonic character matches the corresponding correct pronunciation method;

[0149] S703, if the current pronunciation method does not match the correct pronunciation method, replacing the current pronunciation method with the correct pronunciation method to generate a corrected target dubbing.

[0150] In this embodiment, by synchronizing the dubbing text and the dubbing audio, the timeline information is used to locate the specific position of the polyphonic character in the audio. Each polyphonic character in the text is associated with the corresponding segment in the audio through a timestamp, providing precise positioning for subsequent pronunciation analysis and correction.

[0151] The system first aligns the dubbing text and the dubbing audio on the timeline, and uses the timestamp information of the audio to determine the specific position of each polyphonic character. Through the synchronous processing of the text and the audio, the system can accurately find the audio segment corresponding to the polyphonic character in the audio. For example, the system determines through text synchronization technology that the time range of the character "行" in "行长" in the audio is from 5 seconds to 6 seconds.

[0152] Through speech recognition technology, the system can extract the actual pronunciation method of the polyphonic character in the audio and compare it with the pre-determined correct pronunciation method to determine whether the current pronunciation is correct. This step can automatically identify and correct pronunciation errors in the audio.

[0153] The system analyzes the pronunciation of the polyphonic character in the audio through a speech recognition model (such as an ASR model), extracts the pronunciation features. Then these features are compared with the correct pronunciation in the pronunciation database to determine whether the pronunciation matches. For example, the system determines that the pronunciation of "行" in the audio is "xíng", while the correct pronunciation should be "háng".

[0154] If the system detects a mispronunciation of a polyphonetic character, it will automatically replace the incorrect pronunciation in the audio with the correct pronunciation. This replacement operation requires that the correct pronunciation be seamlessly embedded into the original audio to generate a corrected dubbing while ensuring audio continuity and sound quality.

[0155] The system locates the audio clip that needs to be replaced based on the timeline information and replaces it with an audio clip with the correct pronunciation. The replaced audio clip should smoothly connect with the surrounding audio to avoid abrupt sound quality. After the replacement is completed, the system generates a corrected target dubbing.

[0156] This embodiment applies the correct pronunciation of polyphones to the correction of dubbing audio, so that the system can realize automatic detection and correction of pronunciation errors. It reduces the workload of manual review and editing, and ensures the accuracy and naturalness of the pronunciation of the dubbing audio finally generated. For scenes that require high-quality pronunciation, such as news broadcasts or commercial dubbing, this technology can significantly improve efficiency and accuracy.

[0157] In one embodiment, the step of replacing the current pronunciation mode with the correct pronunciation mode in S703 includes:

[0158] S7031, based on the position of the polyphonetic character in the dubbing audio, determining the original audio segment corresponding to the polyphonetic character in the dubbing audio;

[0159] S7032, extracting pronunciation features of the current pronunciation mode from the original audio segment, wherein the pronunciation features include tone, volume and timbre;

[0160] S7033, adjusting the correct pronunciation using the pronunciation feature, and generating a modified audio segment of the polyphonetic character based on the adjusted correct pronunciation;

[0161] S7034: Replace the original audio segment of the polyphonetic character with the modified audio segment of the polyphonetic character in the dubbing audio.

[0162] In this embodiment, the system first accurately locates the polyphone segment in the audio by the position of the polyphone in the dubbing text and the timeline information. This step is used to identify the original audio segment that needs to be replaced, providing a basis for subsequent pronunciation correction.

[0163] The system uses audio and text synchronization technology to locate the position of polyphones in the dubbing audio according to the position of the polyphones in the dubbing text and its corresponding timestamp. The system extracts the audio clip at that position as the basis for subsequent processing.

[0164] Use audio analysis technology to extract the main features of the current pronunciation (such as tone, volume, timbre) from the original audio clip. These features are used to ensure that the replaced audio clip can seamlessly integrate into the entire audio, maintaining consistency in sound quality, pitch, and timbre.

[0165] The system extracts acoustic features from the original audio through audio analysis tools. Tone is extracted through the fundamental frequency information of the audio, volume is determined through amplitude analysis, and timbre is calculated through spectral analysis. For example, the system will analyze the tone and timbre information in the original pronunciation of the character "行" and use these features for subsequent pronunciation adjustment.

[0166] To make the replaced pronunciation consistent with the original audio, the system adjusts the correct pronunciation method based on the extracted pronunciation features. The system adjusts the tone, volume, and timbre of the correct pronunciation to match the original audio clip.

[0167] After recording or generating the audio clip of the correct pronunciation, the system adjusts the characteristic parameters of these audio, such as tone, timbre, volume, so that the audio clip is consistent with the pronunciation features of the original clip. The adjusted audio clip will be used to replace the incorrect pronunciation.

[0168] The system inserts the corrected audio clip into the original dubbed audio to replace the incorrect pronunciation of the polyphonic character. This step requires the replaced audio clip to seamlessly connect with the original audio to ensure the continuity and naturalness of the sound quality.

[0169] Based on the timeline information, the system replaces the incorrect pronunciation in the original audio with the adjusted audio clip of the correct pronunciation. During the replacement process, the system smooths the audio to ensure smooth sound quality and avoid abruptness or noise at the connection.

[0170] Example illustration: In the dubbing process of financial broadcasts, such as for the phrase "债务重组" (debt restructuring), the system first determines the position of the character "重" in the audio through the timeline information. Then, the system extracts the pitch, timbre, and volume features of the original pronunciation of the character "重". Next, the system adjusts the correct pronunciation clip of "chóng" to make its features consistent with the original audio. Finally, the system seamlessly replaces the adjusted pronunciation clip of "chóng" into the original audio to generate the corrected target dubbing.

[0171] In this embodiment, by extracting the pronunciation features of the original audio clip, the system can maintain the sound quality consistency of the entire audio when replacing incorrect pronunciations, avoiding problems such as abrupt or discontinuous sound quality during the pronunciation replacement process. It ensures the accuracy of pronunciation correction, enabling the corrected dubbed audio to not only maintain the correctness of pronunciation but also smoothly integrate into the original audio, improving the overall audio quality and user experience.

[0172] In one embodiment, a device for correcting polyphonetic characters in dubbing is provided, and the device for correcting polyphonetic characters in dubbing corresponds to the method for correcting polyphonetic characters in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of the dubbing polyphone correction device of the present invention. A text processing module 10, a polyphone recognition and marking module 20, a context construction module 30, a semantic analysis module 40, a pronunciation matching module 50, a correct pronunciation determination module 60 and an audio correction module 70. The functional modules are described in detail as follows:

[0173] A text processing module 10 is used to obtain a dubbing text corresponding to the dubbing audio, perform word segmentation processing on the dubbing text, and split the dubbing text into individual words;

[0174] The polyphone recognition and marking module 20 is used to recognize the position of the polyphone in the word sequence after word segmentation and mark the recognized polyphone;

[0175] A context construction module 30 is used to extract a preset number of words before and after each marked polyphone, and construct a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words;

[0176] A semantic analysis module 40, for performing part-of-speech tagging and syntactic analysis on the sentences in the context window, and obtaining semantic information of the corresponding polyphonetic characters in the current context;

[0177] A pronunciation matching module 50 is used to match different pronunciations of the corresponding polyphonetic character with the semantic information and determine the matching degree corresponding to each pronunciation;

[0178] A correct pronunciation determination module 60 is used to select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character;

[0179] The audio correction module 70 is used to correct the dubbing audio based on the correct pronunciation of the polyphonetic character to generate a target dubbing.

[0180] In one embodiment, the semantic analysis module 40 is specifically used for:

[0181] Using a pre-trained part-of-speech tagging model to perform part-of-speech tagging on the words in the context window to determine the part-of-speech category of each word;

[0182] Performing dependency syntactic analysis on the sentences in the context window, constructing a syntactic structure tree of the sentences, and determining dependency grammatical relations between words;

[0183] Apply word meaning disambiguation technology, combine the part of speech category of each word and the grammatical relationship between words, and determine the semantic information of the corresponding polyphone in the current context;

[0184] The semantic information is semantically embedded using a contextual word vector model to optimize the semantic information.

[0185] In one embodiment, the polyphone recognition and marking module 20 is specifically used for:

[0186] In the word segmentation sequence, the position of the polyphone is identified using a predefined polyphone word library or a context-aware method;

[0187] Recording the specific position of the recognized polyphone in the vocabulary sequence, wherein the specific position includes a start index and an end index;

[0188] The polyphonetic characters in the vocabulary sequence are marked using a preset marking format.

[0189] In one embodiment, the pronunciation matching module 50 is specifically used for:

[0190] Constructing a pronunciation database, and obtaining pronunciation samples corresponding to different pronunciation modes of the polyphonetic character from the pronunciation database;

[0191] Matching the semantic information with different pronunciation samples of the corresponding polyphonetic characters;

[0192] Based on the matching results, the matching degree between each pronunciation mode of the polyphonetic character and the current context is analyzed.

[0193] In one embodiment, the pronunciation matching module 50 is specifically used for:

[0194] Collect multiple polyphonetic characters and their corresponding different pronunciations, and collect examples of the use of polyphonetic characters in different contexts;

[0195] Based on the different pronunciations and usage examples of the polyphonetic characters, different pronunciation samples of the polyphonetic characters are recorded according to preset standards;

[0196] A pronunciation database is constructed based on the different pronunciation methods of each polyphone and the corresponding pronunciation samples.

[0197] In one embodiment, the audio correction module 70 is specifically configured to:

[0198] Determine the position of each polyphone in the dubbing audio by using the timeline information of the dubbing text and the dubbing audio;

[0199] Using speech recognition technology to analyze the current pronunciation of polyphonetic characters in the dubbing audio, and determine whether the current pronunciation of the polyphonetic characters matches the corresponding correct pronunciation;

[0200] If the current pronunciation mode does not match the correct pronunciation mode, the correct pronunciation mode is used to replace the current pronunciation mode to generate a corrected target dubbing.

[0201] In one embodiment, the audio correction module 70 is specifically configured to:

[0202] Based on the position of the polyphonetic character in the dubbing audio, determining the original audio segment corresponding to the polyphonetic character in the dubbing audio;

[0203] Extracting pronunciation features of the current pronunciation mode from the original audio clip, wherein the pronunciation features include tone, volume and timbre;

[0204] Using the pronunciation feature to adjust the correct pronunciation, and generating a modified audio segment of the polyphonetic character based on the adjusted correct pronunciation;

[0205] The modified audio segment of the polyphonetic character is used to replace the original audio segment of the polyphonetic character in the dubbing audio.

[0206] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal through a network connection. When the computer program is executed by the processor, the functions or steps on the service side of a method for correcting polyphonetic characters in dubbing are realized.

[0207] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a method for correcting polyphonetic characters in dubbing on the user side.

[0208] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0209] Obtaining a dubbing text corresponding to the dubbing audio, performing word segmentation processing on the dubbing text, and splitting the dubbing text into individual words;

[0210] Identify the position of polyphones in the word sequence after word segmentation, and mark the identified polyphones;

[0211] Extracting a preset number of words before and after each marked polyphone, and constructing a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words;

[0212] Performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain semantic information of the corresponding polyphonetic characters in the current context;

[0213] Matching different pronunciations of the corresponding polyphonetic character with the semantic information to determine the matching degree corresponding to each pronunciation;

[0214] Select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character;

[0215] Based on the correct pronunciation of the polyphonetic character, the dubbing audio is corrected to generate a target dubbing.

[0216] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0217] Obtaining a dubbing text corresponding to the dubbing audio, performing word segmentation processing on the dubbing text, and splitting the dubbing text into individual words;

[0218] Identify the position of polyphones in the word sequence after word segmentation, and mark the identified polyphones;

[0219] Extracting a preset number of words before and after each marked polyphone, and constructing a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words;

[0220] Performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain semantic information of the corresponding polyphonetic characters in the current context;

[0221] Matching different pronunciations of the corresponding polyphonetic character with the semantic information to determine the matching degree corresponding to each pronunciation;

[0222] Select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character;

[0223] Based on the correct pronunciation of the polyphonetic character, the dubbing audio is corrected to generate a target dubbing.

[0224] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0225] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0226] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0227] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for correcting polyphonetic characters in dubbing, characterized in that: The following steps are involved: Obtaining a dubbing text corresponding to the dubbing audio, performing word segmentation processing on the dubbing text, and splitting the dubbing text into individual words; Identify the position of polyphones in the word sequence after word segmentation, and mark the identified polyphones; Extracting a preset number of words before and after each marked polyphone, and constructing a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words; Performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain semantic information of the corresponding polyphonetic characters in the current context; Matching different pronunciations of the corresponding polyphonetic character with the semantic information to determine the matching degree corresponding to each pronunciation; Select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character; Based on the correct pronunciation of the polyphonetic character, the dubbing audio is corrected to generate a target dubbing.

2. The method for correcting a polyphonetic character in dubbing as claimed in claim 1, wherein: Performing part-of-speech tagging and syntactic analysis on the sentences in the context window to obtain semantic information of the corresponding polyphonetic characters in the current context includes: Using a pre-trained part-of-speech tagging model to perform part-of-speech tagging on the words in the context window to determine the part-of-speech category of each word; Performing dependency syntactic analysis on the sentences in the context window, constructing a syntactic structure tree of the sentences, and determining dependency grammatical relations between words; Apply word meaning disambiguation technology, combine the part of speech category of each word and the grammatical relationship between words, and determine the semantic information of the corresponding polyphone in the current context; The semantic information is semantically embedded using a contextual word vector model to optimize the semantic information.

3. The method for correcting a dubbing polyphonetic character as claimed in claim 1, wherein: Identify the position of polyphones in the word sequence after word segmentation and mark the identified polyphones, including: In the word segmentation sequence, the position of the polyphone is identified using a predefined polyphone word library or a context-aware method; Recording the specific position of the recognized polyphone in the vocabulary sequence, wherein the specific position includes a start index and an end index; The polyphonetic characters in the vocabulary sequence are marked using a preset marking format.

4. The method for correcting polyphonetic characters in dubbing as claimed in claim 1, wherein: Matching different pronunciations of the corresponding polyphonetic character with the semantic information and determining the matching degree corresponding to each pronunciation, including: Constructing a pronunciation database, and obtaining pronunciation samples corresponding to different pronunciation modes of the polyphonetic character from the pronunciation database; Matching the semantic information with different pronunciation samples of the corresponding polyphonetic characters; Based on the matching results, the matching degree between each pronunciation mode of the polyphonetic character and the current context is analyzed.

5. The method for correcting a dubbing polyphonetic character as claimed in claim 4, wherein: Build a pronunciation database, including: Collect multiple polyphonetic characters and their corresponding different pronunciations, and collect examples of the use of polyphonetic characters in different contexts; Based on the different pronunciations and usage examples of the polyphonetic characters, different pronunciation samples of the polyphonetic characters are recorded according to preset standards; A pronunciation database is constructed based on the different pronunciation methods of each polyphone and the corresponding pronunciation samples.

6. The method for correcting polyphonetic characters in dubbing as claimed in claim 1, wherein: Based on the correct pronunciation of the polyphonetic character, the dubbing audio is corrected to generate a target dubbing, including: Determine the position of each polyphone in the dubbing audio by using the timeline information of the dubbing text and the dubbing audio; Using speech recognition technology to analyze the current pronunciation of polyphonetic characters in the dubbing audio, and determine whether the current pronunciation of the polyphonetic characters matches the corresponding correct pronunciation; If the current pronunciation mode does not match the correct pronunciation mode, the correct pronunciation mode is used to replace the current pronunciation mode to generate a corrected target dubbing.

7. The method for correcting a dubbing polyphonetic character as claimed in claim 6, wherein: Replacing the current pronunciation mode with the correct pronunciation mode includes: Based on the position of the polyphonetic character in the dubbing audio, determining the original audio segment corresponding to the polyphonetic character in the dubbing audio; Extracting pronunciation features of the current pronunciation mode from the original audio clip, wherein the pronunciation features include tone, volume and timbre; Using the pronunciation feature to adjust the correct pronunciation, and generating a modified audio segment of the polyphonetic character based on the adjusted correct pronunciation; The modified audio segment of the polyphonetic character is used to replace the original audio segment of the polyphonetic character in the dubbing audio.

8. A device for correcting polyphonetic characters in dubbing, characterized in that: The dubbing polyphonetic character correction device comprises: A text processing module is used to obtain a dubbing text corresponding to the dubbing audio, perform word segmentation processing on the dubbing text, and split the dubbing text into individual words; A polyphone recognition and marking module is used to identify the position of polyphones in the word sequence after word segmentation and mark the recognized polyphones; A context construction module is used to extract a preset number of words before and after each marked polyphone, and construct a context window for each marked polyphone, wherein the context window contains a sentence formed by summarizing the words; A semantic analysis module, used to perform part-of-speech tagging and syntactic analysis on the sentences in the context window, and obtain semantic information of the corresponding polyphonetic characters in the current context; A pronunciation matching module, used to match different pronunciations of the corresponding polyphonetic character with the semantic information, and determine the matching degree corresponding to each pronunciation; A correct pronunciation determination module is used to select the pronunciation with the highest matching degree as the correct pronunciation of the polyphonetic character; The audio correction module is used to correct the dubbing audio based on the correct pronunciation of the polyphonetic character to generate a target dubbing.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a dubbing polyphonetic character correction program stored in the memory and executable on the processor. When the dubbing polyphonetic character correction program is executed by the processor, the steps of the dubbing polyphonetic character correction method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a dubbing polyphonetic character correction program, and when the dubbing polyphonetic character correction program is executed by the processor, the steps of the dubbing polyphonetic character correction method according to any one of claims 1 to 7 are implemented.

Images