Methods, apparatus, equipment, media, and products for processing annotated text.

Abstract

This application proposes a method, apparatus, device, medium, and product for processing annotated text. The method includes: acquiring original annotated text and corresponding audio data, wherein the original annotated text does not include target symbols; performing end-to-end speech recognition on the audio data to obtain a first audio recognition result, wherein the first audio recognition result includes target symbols; and backfilling the target symbols into the original annotated text based on the first audio recognition result to obtain target annotated text, which is used for end-to-end training of a speech recognition model. This approach can improve the accuracy and recall of constructing end-to-end annotated text.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium and product for processing annotated text. Background Technology

[0002] Speech recognition is the process of converting speech signals into text or commands using computer technology. It involves analyzing and understanding human speech input through speech recognition models and converting it into text that can be used in various applications.

[0003] Currently, the text output by speech recognition models usually does not contain target symbols, such as punctuation marks and Arabic numerals. The target symbols need to be added to the text through a post-processing module to obtain the final recognized text, which results in low speech recognition efficiency. Summary of the Invention

[0004] Based on the above-mentioned technological status, this application proposes a method, apparatus, device, medium, and product for processing annotated text, which can improve speech recognition efficiency.

[0005] To achieve the above-mentioned technical objectives, this application proposes the following technical solution:

[0006] The first aspect of this application proposes a model training method, comprising: acquiring original labeled text and audio data corresponding to the original labeled text, wherein the original labeled text does not include target symbols; performing end-to-end speech recognition on the audio data to obtain a first audio recognition result corresponding to the audio data, wherein the first audio recognition result includes target symbols; and backfilling the target symbols into the original labeled text based on the first audio recognition result to obtain target labeled text, wherein the target labeled text is used for end-to-end training of a speech recognition model.

[0007] In some embodiments, the step of backfilling the target symbol into the original labeled text based on the first audio recognition result to obtain the target labeled text includes: backfilling the target symbol into the original labeled text based on the first audio recognition result to obtain the first candidate labeled text; and obtaining the target labeled text based on the first candidate labeled text.

[0008] In some embodiments, the step of backfilling the target symbol into the original labeled text based on the first audio recognition result to obtain the first candidate labeled text includes: aligning the first audio recognition result and the original labeled text to obtain the first aligned text; and backfilling the target symbol into the original labeled text based on the first aligned text to obtain the first candidate labeled text.

[0009] In some embodiments, the first aligned text includes alignment results corresponding to multiple statistical units. The alignment result of each statistical unit includes the original annotation data, alignment type, and recognition result data corresponding to that statistical unit. The alignment type characterizes the degree of consistency between the original annotation data and the recognition result data. The step of backfilling target symbols into the original annotation text based on the first aligned text to obtain a first candidate annotation text includes: for each alignment result among the multiple alignment results, if the recognition result data in the alignment result is an Arabic numeral or punctuation mark, determining the statistical period from the statistical unit corresponding to the previous first target alignment type to the statistical unit corresponding to the next first target alignment type; if the statistical period is consistent with the... If the statistical results adjacent to the alignment result are non-Arabic numerals or non-mathematical symbols, and the alignment type of the remaining alignment results (excluding the current alignment result) within the statistical period is the second target alignment type, then the recognition result data in the alignment result is backfilled to the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit; if the statistical results adjacent to the alignment result within the statistical period are Arabic numerals or mathematical symbols, and the alignment type of the remaining alignment results (excluding the current alignment result) within the statistical period is the third target alignment type, then the recognition result data in the alignment result is backfilled to the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit; based on the backfill results corresponding to multiple statistical units, the first candidate labeled text is obtained.

[0010] In some embodiments, obtaining the target annotation text based on the first candidate annotation text includes: obtaining the second candidate annotation text corresponding to the original annotation text; and determining the target annotation text based on the first candidate annotation text and the second candidate annotation text.

[0011] In some embodiments, the first audio recognition result is obtained using a first model, which is a model capable of end-to-end speech recognition; wherein, obtaining the second candidate labeled text corresponding to the original labeled text includes: performing end-to-end speech recognition on the audio data using a second model to obtain a second audio recognition result corresponding to the audio data, wherein the second audio recognition result includes target symbols, the second model is a model capable of end-to-end speech recognition, and the first model is different from the second model; based on the second audio recognition result, backfilling punctuation marks and Arabic numerals into the original labeled text to obtain the second candidate labeled text; or, using a dictionary to fill the original labeled text with target symbols to obtain the second candidate labeled text.

[0012] In some embodiments, determining the target annotation text based on the first candidate annotation text and the second candidate annotation text includes: aligning the first candidate annotation text and the second candidate annotation text to obtain a third aligned text between the first candidate annotation text and the second candidate annotation text; and backfilling the original annotation text with punctuation marks and Arabic numerals based on the third aligned text to obtain the target annotation text.

[0013] In some embodiments, obtaining the target annotation text based on the first candidate annotation text includes: determining the first candidate annotation text as the target annotation text.

[0014] In some embodiments, the target symbol includes Arabic numerals and / or punctuation marks.

[0015] A second aspect of this application provides an apparatus for processing labeled text, comprising: an acquisition unit for acquiring original labeled text and audio data corresponding to the original labeled text, wherein the original labeled text does not include target symbols; a speech recognition unit for performing end-to-end speech recognition on the audio data to obtain a first audio recognition result corresponding to the audio data, wherein the first audio recognition result includes target symbols; and a backfilling unit for backfilling target symbols into the original labeled text based on the first audio recognition result to obtain target labeled text, wherein the target labeled text is used for end-to-end training of a speech recognition model.

[0016] A third aspect of this application provides an electronic device, including a memory and a processor; the memory is connected to the processor and is used to store a program; the processor is used to implement the text processing method described in the first aspect and any of its implementations by running the program in the memory.

[0017] The sixth aspect of this application proposes a storage medium storing a computer program, which, when executed by a processor, implements the text processing method described in the first aspect and any of its implementations.

[0018] The seventh aspect of this application provides a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to implement the text processing method described in the first aspect and any of its implementations.

[0019] The method, apparatus, device, medium, and product for processing annotated text proposed in this application, through end-to-end speech recognition of the audio data corresponding to the original annotated text, can obtain a first audio recognition result containing target symbols. Then, based on the first audio recognition result, the target symbols are backfilled into the original annotated text to obtain the target annotated text. The target annotated text can be used as end-to-end training corpus to train a speech recognition model for end-to-end speech recognition tasks, thereby obtaining a speech recognition model with end-to-end speech recognition capabilities. Furthermore, when applying this speech recognition model for speech recognition, end-to-end speech recognition results can be directly obtained, saving the steps of punctuation and digit normalization in the post-processing module and improving speech recognition efficiency. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for processing labeled text provided in an embodiment of this application;

[0022] Figure 2 An example diagram of an alignment type provided in an embodiment of this application;

[0023] Figure 3 A schematic diagram of the backfilling process provided in an embodiment of this application;

[0024] Figure 4 A schematic diagram illustrating another backfilling process provided in an embodiment of this application;

[0025] Figure 5 A schematic diagram illustrating another backfilling process provided in an embodiment of this application;

[0026] Figure 6 A schematic diagram illustrating another backfilling process provided in an embodiment of this application;

[0027] Figure 7 A schematic diagram illustrating another backfilling process provided in an embodiment of this application;

[0028] Figure 8 A flowchart illustrating the process of determining target annotation text provided in this application embodiment;

[0029] Figure 9 A schematic diagram of the structure of the annotation text processing device provided in the embodiments of this application;

[0030] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions proposed in this application are applicable to speech recognition application scenarios. By adopting the technical solutions in this application, end-to-end corpora can be constructed and applied to the training of end-to-end speech recognition models, thereby obtaining end-to-end speech recognition results.

[0032] Traditional multilingual speech recognition systems use labeled text that does not contain punctuation marks or Arabic numerals during the training phase, and then use text annotation techniques to fill in punctuation marks and normalize numbers in the output recognized text. This approach suffers from low speech recognition efficiency.

[0033] In addition, text annotation technology mainly includes punctuation tasks and number tasks. For punctuation tasks, text with punctuation marks can be crawled from the web, and then the punctuation marks can be removed according to certain rules to obtain text without punctuation marks. Then, parallel data with and without punctuation marks can be used to train a punctuation model to achieve punctuation mark filling in of the recognized text.

[0034] For numerical tasks, it is difficult to obtain the text before and after numerical normalization, so processing based on predefined rules is usually required. This results in a low recall rate for this method.

[0035] In addition, due to the limited training data for rare languages, the post-processing module has lower accuracy when handling punctuation and number annotation tasks in these languages.

[0036] In view of the above-mentioned technical status, this application proposes a method for processing labeled text. By adding target symbols to labeled text, the standard accuracy and recall rate of labeled text can be improved.

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] This application first proposes a method for processing annotated text, see [link to relevant documentation]. Figure 1 As shown, the method includes steps S101-S103:

[0039] S101. Obtain the original annotation text and the corresponding audio data.

[0040] The original annotation text refers to the annotation text that does not include the target symbols. Target symbols include Arabic numerals and / or punctuation marks. The original annotation text may not include the target symbols, which can include the following situations: for example, the original annotation text may not include Arabic numerals, may not include punctuation marks, or may not include both Arabic numerals and punctuation marks.

[0041] The audio data corresponds to the original annotated text, and contains richer annotation information compared to the original annotated text. By performing end-to-end speech recognition on the audio data, the target symbols missing in the original annotated text can be obtained.

[0042] There are several ways to acquire audio data, including the following: In some embodiments, audio data can be obtained through open datasets. For example, publicly available audio datasets can be obtained through open dataset platforms such as Kaggle and UCI. These datasets typically contain audio files in different language types.

[0043] In addition to obtaining audio data through open datasets, some embodiments can also obtain audio data by building their own datasets. For example, recording a large amount of audio data using recording equipment.

[0044] After obtaining the audio data, a speech recognition model can be used to obtain the original labeled text. For example, by inputting the audio data into a speech recognition model, the audio data can be automatically converted into the original labeled text.

[0045] S102. Perform end-to-end speech recognition on the audio data to obtain the first audio recognition result corresponding to the audio data.

[0046] In this embodiment, end-to-end speech recognition means that the speech recognition result of the audio data includes the target symbol. That is, by performing end-to-end speech recognition on the audio data, not only can the recognized text be obtained, but also the target symbol within the recognized text. The recognized text containing the target symbol can be referred to as the first audio recognition result.

[0047] End-to-end speech recognition can be implemented using existing large-scale open-source models. These models offer high speech recognition accuracy, and using them to perform speech recognition on audio data yields highly accurate first audio recognition results. Furthermore, when backfilling target symbols into the original annotated text based on these first audio recognition results, the accuracy of the backfilling results can be improved. Moreover, when training end-to-end speech recognition tasks using the backfilling results, the recognition accuracy of the end-to-end speech recognition model can be improved, thereby enhancing the overall accuracy of the end-to-end speech recognition task.

[0048] In some embodiments, a first model can be used to perform end-to-end speech recognition on the audio data to obtain a first audio recognition result corresponding to the audio data. The first model is a model capable of performing end-to-end speech recognition, such as an open-source multilingual recognition model, an open-source language model, etc.

[0049] When the first model is an open-source multilingual recognition model, such as the Whisper model, by inputting audio data into this model, end-to-end first audio recognition results can be obtained. The Whisper model is trained on large datasets of various audio data and performs excellently in speech recognition, tone recognition, and automatic sentence segmentation. In multilingual speech recognition tasks, it exhibits near-human robustness and accuracy. Therefore, using the first model can yield highly accurate first audio recognition results.

[0050] When the first model is an open-source large-scale language model, such as a Generative Pre-Trained (GPT) model, a first audio recognition result can be obtained by inputting a prompt word corresponding to the language category of the audio data along with the input audio data. This prompt word can provide a task description and examples, and specify the format and requirements of the returned result to ensure that the pronunciation of the returned text after numerical normalization is the same as the original annotated text. For example, the prompt word could be "perform speech recognition on the audio data, and ensure that the obtained speech recognition result includes the target symbol." The GPT model learns the syntax and semantics of natural language through unsupervised pre-training on a large-scale text corpus, and it performs excellently in natural language processing tasks such as text generation. Using the GPT model for end-to-end speech recognition of audio data can yield a first audio recognition result with high accuracy.

[0051] S103. Based on the first audio recognition result, the target symbol is backfilled into the original labeled text to obtain the target labeled text, which is used for end-to-end training of the speech recognition model.

[0052] The first audio recognition result is the recognized text corresponding to the original labeled text, but with the target symbol added. Compared to the original labeled text, the first audio recognition result includes the target symbol. Therefore, based on the first audio recognition result, missing target symbols in the original labeled text can be supplemented, thus obtaining the target labeled text. The target labeled text is the labeled text with the target symbol. Using it for training an end-to-end speech recognition task yields an end-to-end speech recognition model. Applying this end-to-end speech recognition model enables end-to-end speech recognition, saving post-processing steps and improving speech recognition efficiency.

[0053] Specifically, step S103 includes: based on the first audio recognition result, backfilling the target symbol into the original labeled text to obtain the first candidate labeled text; and based on the first candidate labeled text, obtaining the target labeled text.

[0054] In some embodiments, based on the first audio recognition result, the target symbol is backfilled into the original labeled text to obtain the first candidate labeled text, including: aligning the first audio recognition result and the original labeled text to obtain the first aligned text; and based on the first aligned text, the target symbol is backfilled into the original labeled text to obtain the first candidate labeled text.

[0055] The first aligned text represents the correspondence between the content in the original annotated text and the content in the first audio recognition result. This content can be words, punctuation marks, Arabic numerals, etc. For example, the corresponding positions of target symbols in the first audio recognition result and words in the original annotated text. The purpose of the alignment process is to find the correspondence between each recognition position in the first audio recognition result and each annotation position in the original annotated text.

[0056] In some embodiments, text alignment can be used to align the first audio recognition result and the original labeled text. Text alignment is primarily achieved by measuring the difference between the content at the same location in the first audio recognition result and the original labeled text. Simply put, it's the minimum number of single-character edits (such as insertion, deletion, or replacement) required to change the content in the first audio recognition result to the content in the original labeled text. For a detailed explanation of the specific implementation of text alignment, please refer to the alignment rules of the edit distance algorithm; further details are omitted here.

[0057] The first aligned text includes alignment results corresponding to multiple statistical units. These statistical units can be understood as the smallest processing unit during text alignment, encompassing the correspondence between the content at the recognition and annotation positions. Specifically, the alignment result for each statistical unit includes the original annotation data, alignment type, and recognition result data corresponding to that unit. Note that the original annotation data and recognition result data may be empty. The alignment type characterizes the degree of consistency between the original annotation data and the recognition result data. The alignment types are illustrated below with examples:

[0058] Each statistical unit in a set of multiple statistical units can be represented in L / T / R format. Each statistical unit can include the annotation position in the original annotation text and the recognition position in the first audio recognition result. Here, L represents the content of the original annotation text at the statistical unit position, R represents the content of the first audio recognition result at the current statistical unit position, and T represents the alignment type of L and R. The alignment type can include four types: H, S, I, and D. H indicates that both L and R are not empty and are consistent; S indicates that both L and R are not empty and are inconsistent; I indicates that L is empty and R is not empty; D indicates that L is not empty and R is empty. The following is a detailed description of the above alignment types with reference to the accompanying figures and examples:

[0059] Figure 2 An example diagram illustrating an alignment type provided in an embodiment of this application. For example... Figure 2 As shown, for "citai / H / citai", "citai" at position L represents the content at the annotation position corresponding to the current statistical unit position in the original annotation text; "citai" at position R represents the content at the recognition position corresponding to the current statistical unit position in the first audio recognition result. Since the content at position L and the content at position R are both non-empty and identical, their alignment type is "H".

[0060] Continue reading Figure 2 For "tuvojas / S / ojas", "tuvojas" at position L represents the content at the annotation position in the original annotation text corresponding to the current statistical unit position; "ojas" at position R represents the content at the recognition position in the first audio recognition result corresponding to the current statistical unit position. Since the content at position L is different from the content at position R, and neither of them is empty, their alignment type is "S".

[0061] Continue reading Figure 2 Regarding " / I / 100", it can be seen that the L position is empty; the "100" at the R position represents the content at the recognition position corresponding to the current statistical unit position in the first audio recognition result. Since the content at the L position is empty, while the content at the R position is not empty, the alignment type of the two is "I".

[0062] Continue reading Figure 2 Regarding ", / I / ", it can be seen that the R position is empty; the "100" at the L position represents the content at the recognition position corresponding to the current statistical unit position in the first audio recognition result. Since the content at the R position is empty, while the content at the L position is not empty, the alignment type of the two is "D".

[0063] Based on the above embodiments, the correspondence between each annotation position in the original annotated text and the content at each recognition position in the first audio recognition result can be obtained, thus obtaining the first aligned text. After obtaining the first aligned text, the target symbols can be backfilled into the original annotated text based on the first aligned text to obtain the first candidate annotated text. Specifically, this includes: for each alignment result among multiple alignment results, if the recognition result data in the alignment result is Arabic numerals or punctuation marks, determining the statistical period from the statistical unit corresponding to the previous first target alignment type to the statistical unit corresponding to the next first target alignment type; if the statistical result adjacent to the alignment result within the statistical period is a non-Arabic numeral or non-mathematical symbol, and the remaining alignment results within the statistical period other than the current alignment result... If the alignment type is the second target alignment type, the recognition result data in the alignment result is backfilled to the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit; if the statistical result adjacent to the alignment result in the statistical period is an Arabic numeral or a mathematical symbol, and the alignment type of the other alignment results in the statistical period other than the current alignment result is the third target alignment type, the recognition result data in the alignment result is backfilled to the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit; based on the backfill results corresponding to multiple statistical units, the first candidate labeled text is obtained.

[0064] The first target alignment type refers to the alignment type where T = H. The second target alignment type refers to the alignment type where T = H or T = D. The third target alignment type refers to the alignment type where T = H, T = I, or T = D.

[0065] For example, each of the multiple alignment results is taken as the current alignment result, and the process is traversed backward from the current alignment result position to find the previous T=H. Assuming that the previous alignment result at the current alignment result position is T=H, the alignment result from the previous T=H to the current alignment result position is determined as the forward traversal result, and the forward traversal ends. Then, the process is traversed backward from the current alignment result position to find the next T=H. Assuming that the next alignment result at the current alignment result position is T=H, the alignment result from the next T=H to the current alignment result position is determined as the backward traversal result, and the backward traversal ends.

[0066] During the above forward traversal, if T=H is not found even after traversing to the beginning of the sentence, the current alignment result is moved to the beginning of the sentence as the forward traversal result, and the forward traversal ends.

[0067] Similarly, during the backward traversal, if T=H is not found even after traversing to the end of the sentence, the current alignment result is moved to the end of the sentence as the result of the backward traversal, and the backward traversal ends.

[0068] Next, the results of the forward and backward traversals are merged to form a statistical period. It is then determined whether the alignment result adjacent to the current alignment result is an Arabic numeral or a mathematical symbol. If not, it is determined whether the alignment type of the remaining alignment results within this statistical period (excluding the current alignment result) is the second target alignment type. If so, the recognition result data from that alignment result is backfilled into the original labeled text position in the statistical unit, resulting in the backfilled result for that statistical unit.

[0069] Furthermore, if the alignment result adjacent to the current alignment result is an Arabic numeral or a mathematical symbol, then it is determined whether the alignment type of the other alignment results in the statistical period other than the current alignment result is a third target alignment type. If so, the alignment result and the adjacent and continuous at least one Arabic numeral or mathematical symbol are backfilled to the position of the original annotation text in each statistical unit to obtain the backfill result of each statistical unit.

[0070] Once all the above steps have been performed in multiple statistical units, the first candidate labeled text can be obtained.

[0071] The first target alignment type includes T=H; the second target alignment type includes T=H and T=D; the third target alignment type includes T=I, T=D, and T=H, where T is not S. Here, the mathematical symbols refer to symbols that can represent a number at the current position or adjacent positions, such as the percent sign.

[0072] The traversal process described above will be explained in detail below with reference to the accompanying diagram:

[0073] Figure 3 This is a schematic diagram illustrating the backfilling process provided in an embodiment of this application. Figure 3As shown, the row containing LAB represents the original annotated text, and the row containing REC represents the first audio recognition result. Assuming the current alignment result is the 11th statistical result from left to right, corresponding to the recognition position "133", we iterate forward and backward from position "133". In the forward iteration, we see that the contents of L and R at position "se" are the same, and neither is empty; therefore, its alignment type is "H". Thus, the forward iteration ends. In the backward iteration, the contents of L and R at position "vai" are the same, and neither is empty; therefore, its alignment type is "H". Thus, the backward iteration ends. The content corresponding to the rectangles in the figure represents one statistical period. Since the other statistical results outside the current statistical result within this statistical period are se / H / se, seta / D / , and vai / H / vai, and their alignment type is all non-S, it indicates that the recognition results near the current statistical result are highly accurate. Therefore, it indicates that the recognition results at position 133 are also highly accurate, so 133 can be backfilled into the original annotation text.

[0074] Figure 4 This is a schematic diagram illustrating another backfilling process provided in an embodiment of this application. For example... Figure 4 As shown, the row containing LAB represents the original labeled text, and the row containing REC represents the first audio recognition result. Assuming the current alignment result is the 4th statistical result from left to right, corresponding to the recognition position of "40", we iterate forward and backward from the position "40". In the forward iteration, we see that the L and R values ​​at the position "euroa" are identical, and neither is empty, therefore the alignment type is "H", and the forward iteration ends. In the backward iteration, the L and R values ​​at the position "senttia" are identical, and neither is empty, therefore the alignment type is "H", and the backward iteration ends. The content corresponding to the rectangle in the figure represents a statistical period. Because the statistical result between the two H values ​​within this statistical period, before the number 40, is not an Arabic numeral or mathematical symbol, and there is an insertion error (empty in the original labeled text, "mika" in the first audio recognition result, i.e., T=I), 40 is not backfilled into the original labeled text.

[0075] Figure 5 This is a schematic diagram illustrating another backfilling process provided in an embodiment of this application. For example... Figure 5As shown, the row containing LAB represents the original labeled text, and the row containing REC represents the first audio recognition result. Assuming the current alignment result is the second statistical result from left to right, i.e., the position "2", we traverse forward and backward from the position "2". In the forward traversal, we see that the contents of L and R at the position "maksaa" are the same, and neither is empty, therefore its alignment type is "H", and the forward traversal ends. In the backward traversal, we see that the contents of L and R at the position "euroa" are the same, and neither is empty, therefore its alignment type is "H", and the backward traversal ends. The content corresponding to the rectangle in the figure represents a statistical period. Since the statistical result after the number 2 between the two Hs within this statistical period is not an Arabic numeral or mathematical symbol, and there is an insertion error (empty in the original labeled text, "se" in the first audio recognition result, i.e., T=I), 2 is not backfilled into the original labeled text.

[0076] Figure 6 This is a schematic diagram illustrating another backfilling process provided in an embodiment of this application. For example... Figure 6 As shown, the row containing LAB represents the original labeled text, and the row containing REC represents the first audio recognition result. Assuming the current alignment result is the 4th statistical result from left to right, i.e., position "40", we iterate forward and backward from position "40". In the forward iteration, we see that the contents of L and R at position "euroa" are the same, and neither is empty, therefore its alignment type is "H", and the forward iteration ends. In the backward iteration, we see that the contents of L and R at position "senttia" are the same, and neither is empty, therefore its alignment type is "H", and the backward iteration ends. The content corresponding to the rectangle in the figure represents a statistical period. Because there is a substitution error in the statistical results before "40" between the two H values ​​within this statistical period—that is, "mi" in the original labeled text and "mika" in the first audio recognition result—and these two are inconsistent (T=S), therefore, 40 is not backfilled into the original labeled text.

[0077] Figure 7 This is a schematic diagram illustrating another backfilling process provided in an embodiment of this application. For example... Figure 7As shown, the row containing LAB represents the original labeled text, and the row containing REC represents the first audio recognition result. Assuming the current alignment result is the second statistical result from left to right, i.e., position "2", we iterate forward and backward from position "2". In the forward iteration, we see that the alignment type at position "maksaa" is "H", therefore, the forward iteration ends. In the backward iteration, we see that the alignment type at position "euroa" is "H", therefore, the backward iteration ends. The content corresponding to the rectangle in the figure represents a statistical period. Since the statistical results between the two H's within this statistical period, after 2, are "mika" in the original labeled text and "se" in the first audio recognition result, and these two are inconsistent (T=S), therefore, 2 is not backfilled into the original labeled text.

[0078] The following example illustrates the case where the statistical results adjacent to the current alignment result within the statistical period are Arabic numerals or mathematical symbols, and the alignment type of all other alignment results within the statistical period, excluding the current alignment result, is a third-target alignment type:

[0079] For example, given LAB as "Is this string of numbers 10199?" and REC as "Is this string of numbers 101 99?", assuming the current statistical unit's corresponding REC is "101", we would traverse backwards from the position of 101 to find the previous T=H, i.e., "Yes", and traverse backwards from the position of 101 to find the next T=H, i.e., "99". Therefore, the statistical period obtained from the current traversal is "Yes 101 99". Since T is not S within this statistical period, and there are consecutive numbers at the position of 101, we can backfill 101 and 99 together.

[0080] After obtaining the first candidate annotation text based on the above embodiments, the target annotation text can be determined based on the first candidate annotation text. Specifically, obtaining the target annotation text based on the first candidate annotation text includes: determining the first candidate annotation text as the target annotation text. This embodiment uses a single backfill method, directly using the first candidate annotation text as the target annotation text.

[0081] To further improve the recall rate of the backfill results, another backfill method can be used to backfill the original labeled text, and the two backfill results can be combined to obtain a more comprehensive target labeled text.

[0082] Figure 8 This is a flowchart illustrating the process of determining the target annotation text provided in an embodiment of this application. Figure 8 As shown, the target labeled text is obtained based on the first candidate labeled text, including steps S801 and S802:

[0083] S801. Obtain the second candidate annotation text corresponding to the original annotation text.

[0084] There are several ways to obtain the second candidate annotation text corresponding to the original annotation text, as follows:

[0085] In some embodiments, obtaining the second candidate labeled text corresponding to the original labeled text includes: performing end-to-end speech recognition on the audio data using a second model to obtain a second audio recognition result corresponding to the audio data, wherein the second audio recognition result includes target symbols, the second model is an end-to-end speech recognition model, and the first model is different from the second model; based on the second audio recognition result, backfilling punctuation marks and Arabic numerals into the original labeled text to obtain the second candidate labeled text.

[0086] The second model is a large model with end-to-end speech recognition capabilities. Examples include open-source multilingual recognition models and open-source language models.

[0087] When the second model is an open-source multilingual recognition model, inputting audio data into this multilingual recognition model will yield end-to-end second audio recognition results.

[0088] When the second model is an open-source language model, a prompt word corresponding to the language category of the audio data can be input along with the input audio data to obtain the second audio recognition result. This prompt word can provide a task description and examples, and specify the format and requirements of the returned result to ensure that the pronunciation of the returned, digitized text is the same as the original annotated text. For example, the prompt word could be "perform speech recognition on the audio data, and ensure that the obtained speech recognition result includes the target symbol."

[0089] After obtaining the second audio recognition result, punctuation marks and Arabic numerals can be backfilled into the original labeled text based on the second audio recognition result to obtain the second candidate labeled text. The specific backfilling process can be found in the foregoing embodiments. The implementation method of backfilling punctuation marks and Arabic numerals into the original labeled text based on the first audio recognition result to obtain the first candidate labeled text will not be repeated here.

[0090] Since the aforementioned large-scale models are commercially available and costly, in some embodiments, a dictionary can be used to fill the original annotated text with target symbols to obtain a second candidate annotated text, in order to save costs. In this embodiment, a dictionary for each language can be pre-created, containing all possible representations of numbers in all languages, as well as various punctuation marks. Specifically, this can be obtained through online collection, large-scale language models, or manual generation. Then, based on this dictionary, the original annotated text is traversed to extract the portions that may contain target symbols, and then the subsequent filling task is performed.

[0091] In some embodiments, a number warping tool can be used to fill the original annotation text with target symbols to obtain a second candidate annotation text.

[0092] S802. Determine the target annotation text based on the first candidate annotation text and the second candidate annotation text.

[0093] After obtaining the second candidate annotation text, the target annotation text can be determined based on the first and second candidate annotation texts. Specifically, this involves: aligning the first and second candidate annotation texts to obtain a third aligned text; and then, based on the third aligned text, backfilling the original annotation text with punctuation marks and Arabic numerals to obtain the target annotation text.

[0094] In this embodiment, either the first candidate labeled text or the second candidate labeled text is used as L, and the other is used as R for re-alignment, and the target symbol is backfilled again, thereby improving the recall rate of the target symbol. The specific implementation process of re-alignment and backfilling can be referred to the detailed description in the embodiment above regarding text alignment of the first audio recognition result and the original labeled text, and backfilling of the original labeled text based on the first aligned text, which will not be repeated here.

[0095] The method for processing annotated text proposed in this application involves performing end-to-end speech recognition on the audio data corresponding to the original annotated text to obtain a first audio recognition result containing target symbols. Then, based on the first audio recognition result, the target symbols are backfilled into the original annotated text to obtain the target annotated text. The target annotated text can be used as end-to-end training corpus to train a speech recognition model for end-to-end speech recognition tasks, thereby obtaining a speech recognition model with end-to-end speech recognition capabilities. Furthermore, when applying this speech recognition model for speech recognition, end-to-end speech recognition results can be obtained directly, saving the steps of punctuation and digit normalization in the post-processing module and improving speech recognition efficiency. In addition, a large amount of end-to-end training corpus can be obtained through the speech data and the corresponding original annotated text, allowing for end-to-end training of the speech recognition model for speech recognition tasks, resulting in a speech recognition model with end-to-end speech recognition capabilities. And when applying this speech recognition model for speech recognition, the accuracy of speech recognition can be improved. Furthermore, this solution can also address the difficulty in obtaining training data for rare languages. By employing the above solution, a large amount of end-to-end training data for rare languages ​​can be obtained, which can then be used to train the speech recognition model for end-to-end speech recognition tasks. The resulting speech recognition model can improve the accuracy of speech recognition when dealing with rare languages.

[0096] Corresponding to the above-described method for processing labeled text, this application also proposes a device for processing labeled text, such as... Figure 9 As shown, the device includes:

[0097] Acquisition unit 901 is used to acquire the original annotation text and the audio data corresponding to the original annotation text, wherein the original annotation text does not include the target symbol;

[0098] The speech recognition unit 902 is used to perform end-to-end speech recognition on the audio data to obtain a first audio recognition result corresponding to the audio data, wherein the first audio recognition result includes the target symbol.

[0099] The backfilling unit 903 is used to backfill target symbols into the original labeled text based on the first audio recognition result to obtain target labeled text, which is used for end-to-end training of the speech recognition model.

[0100] In some embodiments, the backfilling unit 903 backfills target symbols into the original labeled text based on the first audio recognition result to obtain target labeled text, including: backfilling target symbols into the original labeled text based on the first audio recognition result to obtain first candidate labeled text; and obtaining the target labeled text based on the first candidate labeled text.

[0101] In some embodiments, the backfilling unit 903 backfills target symbols into the original labeled text based on the first audio recognition result to obtain a first candidate labeled text, including: aligning the first audio recognition result and the original labeled text to obtain a first aligned text; and backfilling target symbols into the original labeled text based on the first aligned text to obtain the first candidate labeled text.

[0102] In some embodiments, the first aligned text includes alignment results corresponding to multiple statistical units. The alignment result of each statistical unit includes the original annotation data, alignment type, and recognition result data corresponding to that statistical unit. The alignment type characterizes the degree of consistency between the original annotation data and the recognition result data. The step of backfilling target symbols into the original annotation text based on the first aligned text to obtain a first candidate annotation text includes: for each alignment result among the multiple alignment results, if the recognition result data in the alignment result is an Arabic numeral or punctuation mark, determining the statistical period from the statistical unit corresponding to the previous first target alignment type to the statistical unit corresponding to the next first target alignment type; if the statistical period is consistent with the... If the statistical results adjacent to the alignment result are non-Arabic numerals or non-mathematical symbols, and the alignment type of the remaining alignment results (excluding the current alignment result) within the statistical period is the second target alignment type, then the recognition result data in the alignment result is backfilled to the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit; if the statistical results adjacent to the alignment result within the statistical period are Arabic numerals or mathematical symbols, and the alignment type of the remaining alignment results (excluding the current alignment result) within the statistical period is the third target alignment type, then the recognition result data in the alignment result is backfilled to the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit; based on the backfill results corresponding to multiple statistical units, the first candidate labeled text is obtained.

[0103] In some embodiments, the backfilling unit 903 obtains the target annotation text based on the first candidate annotation text, including: obtaining the second candidate annotation text corresponding to the original annotation text; and determining the target annotation text based on the first candidate annotation text and the second candidate annotation text.

[0104] In some embodiments, the first audio recognition result is obtained using a first model, which is a model capable of end-to-end speech recognition; wherein, the backfilling unit 903 obtains the second candidate labeled text corresponding to the original labeled text by: performing end-to-end speech recognition on the audio data using a second model to obtain a second audio recognition result corresponding to the audio data, wherein the second audio recognition result includes target symbols, the second model is a model capable of end-to-end speech recognition, and the first model is different from the second model; based on the second audio recognition result, backfilling punctuation marks and Arabic numerals into the original labeled text to obtain the second candidate labeled text; or, using a dictionary to fill the original labeled text with target symbols to obtain the second candidate labeled text.

[0105] In some embodiments, the backfilling unit 903 determines the target annotation text based on the first candidate annotation text and the second candidate annotation text, including: aligning the first candidate annotation text and the second candidate annotation text to obtain a third aligned text between the first candidate annotation text and the second candidate annotation text; and backfilling punctuation marks and Arabic numerals onto the original annotation text based on the third aligned text to obtain the target annotation text.

[0106] In some embodiments, the backfilling unit 903 obtains the target annotation text based on the first candidate annotation text, including: determining the first candidate annotation text as the target annotation text.

[0107] The annotation text processing apparatus provided in this embodiment belongs to the same application concept as the annotation text processing method provided in the above embodiments of this application. It can execute the annotation text processing method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the annotation text processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the annotation text processing method provided in the above embodiments of this application, and will not be repeated here.

[0108] The functions implemented by each unit in the above-mentioned text processing device can be implemented by the same or different processors, and this application embodiment does not limit this.

[0109] It should be understood that the units in the processing device described above can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration text, thereby implementing the functions of some or all of the above units. All units of the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0110] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0111] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0112] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0113] This application also proposes a control device, which includes a processor and an interface circuit. The processor in the control device is connected to a data input component through the interface circuit of the control device.

[0114] The data input component specifically refers to a functional component that can input or collect voice data, such as a microphone, etc.

[0115] The aforementioned interface circuit can be any interface circuit capable of implementing data communication functions, such as a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIe circuit, etc.

[0116] The processor in this control device is also a circuit with signal processing capabilities, which executes the text processing method described in the above embodiments. For specific implementation details of the processor, please refer to the processor implementation methods described above; this application does not impose strict limitations on these implementations.

[0117] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 10 As shown, the device includes:

[0118] Memory 200 and processor 210;

[0119] The memory 200 is connected to the processor 210 and is used to store programs;

[0120] The processor 210 is configured to implement the text processing method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0121] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0122] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0123] A bus can include a pathway for transmitting information between various components of a computer system.

[0124] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0125] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0126] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0127] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0128] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0129] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0130] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of the annotation text processing method provided in the above embodiments of this application.

[0131] This application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the text annotation processing method described in any of the above embodiments. For details of the processing procedure and its beneficial effects, please refer to the embodiments of the text annotation processing method described above.

[0132] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the annotation text processing method described in any of the above embodiments of this specification.

[0133] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0134] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor of the steps in the annotation text processing method described in any of the above embodiments of this specification.

[0135] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0136] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0137] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0138] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0139] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0140] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0141] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0142] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0144] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing annotated text, characterized in that, include: Obtain the original labeled text and the corresponding audio data, wherein the original labeled text does not include the target symbol; The audio data is subjected to end-to-end speech recognition using a large language model to obtain a first audio recognition result corresponding to the audio data, wherein the first audio recognition result includes the target symbol. Based on the first audio recognition result, the target symbols are backfilled into the original labeled text to obtain the target labeled text, which is used for end-to-end training of the speech recognition model.

2. The method according to claim 1, characterized in that, The step of backfilling the target symbols into the original labeled text based on the first audio recognition result to obtain the target labeled text includes: Based on the first audio recognition result, the target symbol is backfilled into the original labeled text to obtain the first candidate labeled text; The target labeled text is obtained based on the first candidate labeled text.

3. The method according to claim 2, characterized in that, The step of backfilling the target symbols into the original labeled text based on the first audio recognition result to obtain the first candidate labeled text includes: The first audio recognition result and the original labeled text are aligned to obtain the first aligned text; Based on the first aligned text, the target symbol is backfilled into the original labeled text to obtain the first candidate labeled text.

4. The method according to claim 3, characterized in that, The first aligned text includes alignment results corresponding to multiple statistical units. The alignment result of each statistical unit includes the original annotation data, alignment type and recognition result data corresponding to the statistical unit. The alignment type represents the degree of consistency between the original annotation data and the recognition result data. The step of backfilling the target symbol into the original annotation text based on the first aligned text to obtain the first candidate annotation text includes: For each alignment result among multiple alignment results, if the recognition result data in the alignment result is Arabic numerals or punctuation marks, the statistical period is determined to be from the statistical unit corresponding to the previous first target alignment type to the statistical unit corresponding to the next first target alignment type. If the statistical result adjacent to the alignment result within the statistical period is a non-Arabic numeral or a non-mathematical symbol, and the alignment type of the remaining alignment results within the statistical period other than the current alignment result is the second target alignment type, then the recognition result data in the alignment result is backfilled into the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit. If the statistical result adjacent to the alignment result within the statistical period is an Arabic numeral or a mathematical symbol, and the alignment type of the remaining alignment results within the statistical period other than the current alignment result is a third target alignment type, then the recognition result data in the alignment result is backfilled into the position of the original labeled text in the statistical unit to obtain the backfill result of the statistical unit. Based on the backfill results corresponding to multiple statistical units, the first candidate labeled text is obtained.

5. The method according to claim 2, characterized in that, The step of obtaining the target labeled text based on the first candidate labeled text includes: Obtain the second candidate annotation text corresponding to the original annotation text; The target annotation text is determined based on the first candidate annotation text and the second candidate annotation text.

6. The method according to claim 5, characterized in that, The first audio recognition result is obtained using a first model, which is a model capable of end-to-end speech recognition. The step of obtaining the second candidate annotation text corresponding to the original annotation text includes: The second model is used to perform end-to-end speech recognition on the audio data to obtain a second audio recognition result corresponding to the audio data. The second audio recognition result includes the target symbol. The second model is a speech recognition model capable of performing end-to-end speech recognition, and the first model is different from the second model. Based on the second audio recognition result, punctuation marks and Arabic numerals are backfilled into the original labeled text to obtain the second candidate labeled text; or, The original annotated text is filled with target symbols using a dictionary database to obtain the second candidate annotated text.

7. The method according to claim 5, characterized in that, The step of determining the target annotation text based on the first candidate annotation text and the second candidate annotation text includes: Align the first candidate annotation text and the second candidate annotation text to obtain the third aligned text between the first candidate annotation text and the second candidate annotation text; Based on the third aligned text, punctuation marks and Arabic numerals are backfilled into the original labeled text to obtain the target labeled text.

8. The method according to claim 2, characterized in that, The step of obtaining the target labeled text based on the first candidate labeled text includes: The first candidate labeled text is determined as the target labeled text.

9. The method according to claim 1, characterized in that, The target symbols include Arabic numerals and / or punctuation marks.

10. A text processing apparatus for annotated text, characterized in that, include: The acquisition unit is used to acquire the original annotation text and the audio data corresponding to the original annotation text, wherein the original annotation text does not include the target symbol; The speech recognition unit is used to perform end-to-end speech recognition on the audio data through a large language model to obtain a first audio recognition result corresponding to the audio data, wherein the first audio recognition result includes the target symbol. The backfilling unit is used to backfill target symbols into the original labeled text based on the first audio recognition result to obtain target labeled text, which is used for end-to-end training of the speech recognition model.

11. An electronic device, characterized in that, Including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method as described in any one of claims 1 to 9 by running a program in the memory.

12. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.

13. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 9.

Images