Vowel restoration method and apparatus, electronic device, and storage medium

Abstract

The application provides a vowel restoration method and device, electronic equipment and storage medium, and belongs to the technical field of natural language processing, and comprises the following steps: obtaining a first to-be-processed text, the first to-be-processed text comprising a first text requiring vowel restoration; inputting the first to-be-processed text into a vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model, the predicted label being three types of vowel symbol labels corresponding to each character in the first text; and determining a target text after vowel restoration based on the first to-be-processed text and the predicted label output by the vowel restoration model. In this process, three types of vowel symbol labels corresponding to each character in the first text can be synchronously predicted, avoiding conflicts between different vowel symbol labels, so that the vowel symbol label of each character in the first to-be-processed text can be accurately predicted, thereby improving the accuracy of vowel restoration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a vowel restoration method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, Hebrew adopts a full letter writing form, however, this form omits the diacritics appearing in full diacritics or dotted variants, which can cause pronunciation errors and semantic ambiguities of words.

[0003] In order to avoid pronunciation errors and semantic ambiguities of words, it is necessary to annotate vowels of Hebrew to ensure accurate pronunciation of words and avoid semantic ambiguities.

[0004] From the perspective of vowel restoration, it is difficult to restore a single word, and usually it is necessary to combine context to add diacritics in a vowelless word, and then restore the actual complete spelling, and the related technical solution adopts a serial prediction mode for prediction, and for the case that a single character has multiple diacritics, rule conflicts are prone to occur, so that the restored vowels are inaccurate. SUMMARY

[0005] The present application provides a vowel restoration method, device, electronic equipment and storage medium to solve the problem that the related technical solution adopts a serial prediction mode for prediction, and for the case that a single character has multiple diacritics, rule conflicts are prone to occur, so that the restored vowels are inaccurate.

[0006] The present application provides a vowel restoration method, comprising the following steps:

[0007] Obtaining a first to-be-processed text, the first to-be-processed text comprising a first text requiring vowel restoration;

[0008] Inputting the first to-be-processed text into a vowel restoration model for multi-diacritic label synchronous prediction to obtain a prediction label output by the vowel restoration model, the prediction label being three types of diacritic labels corresponding to each character in the first text, the three types of diacritic labels comprising a first dot symbol label for consonant fricative, a second dot symbol label for correcting consonant sound value, and a symbol label for annotating vowels, the vowel restoration model being trained based on a character sample and three types of diacritic labels associated with the character sample;

[0009] Determining a target text after vowel restoration based on the first to-be-processed text and the prediction label output by the vowel restoration model.

[0010] The vowel restoration method provided by the present application further comprises a second text not requiring vowel restoration.

[0011] The first to-be-processed text is input into a vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model.

[0012] The first text in the first to-be-processed text is kept unchanged, the second text in the first to-be-processed text is replaced with a third text, and a first to-be-processed text after replacement is obtained, the third text being a text that does not affect vowel restoration.

[0013] The first to-be-processed text after replacement is input into a vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model.

[0014] The target text after vowel restoration is determined based on the first to-be-processed text and the predicted label output by the vowel restoration model.

[0015] The third text in the first to-be-processed text after replacement is kept unchanged, the first text in the first to-be-processed text after replacement is replaced with the first text after vowel annotation, a first to-be-processed text after annotation is obtained, and the first text after vowel annotation is obtained by annotating the first text in the first to-be-processed text after replacement with the predicted label output by the vowel restoration model.

[0016] The first text after vowel annotation in the first to-be-processed text after annotation is kept unchanged, the third text in the first to-be-processed text after annotation is replaced with the second text to obtain the target text after vowel restoration.

[0017] In the vowel restoration method provided by the application, the second text includes one or more of the following:

[0018] Text in a language other than the first language, space, number, and punctuation mark.

[0019] The first language is a language corresponding to the first text.

[0020] In the vowel restoration method provided by the application, the first to-be-processed text is input into a vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model.

[0021] Each character in the first to-be-processed text is represented by a basic symbol to obtain a basic symbol string, and the basic symbol is a symbol constituting each character.

[0022] The basic symbol string is split with a delimiter between different characters in the first to-be-processed text as a boundary to obtain a first sequence composed of a plurality of basic symbol subsequences.

[0023] inputting the first sequence into a vowel restoration model to perform multi-vowel symbol label synchronous prediction, to obtain a predicted label output by the vowel restoration model.

[0024] The vowel restoration method provided by the application comprises an embedding layer, a first extraction layer, a second extraction layer and an output layer.

[0025] The embedding layer is configured to convert each of the basic symbol subsequences into a corresponding text distribution feature, and the text distribution feature is fused with a position code corresponding to each of the text distribution features to obtain a first feature.

[0026] The first extraction layer is configured to perform feature extraction on the first feature to obtain a first extraction result.

[0027] The second extraction layer is configured to perform feature extraction on the second feature to obtain a second extraction result.

[0028] The output layer is configured to output a predicted label based on the second extraction result.

[0029] The element repetition enhancement processing comprises the following steps:

[0030] Based on the first extraction result, each target basic symbol in each basic symbol subsequence at the character level is replaced by an enhanced target basic symbol to obtain an enhanced first extraction result.

[0031] The target basic symbol is a basic symbol in the basic symbol subsequence other than the first basic symbol and the last basic symbol, and the enhanced target basic symbol is obtained by repeating embedding each basic symbol in the target basic symbol N times and splicing.

[0032] The vowel restoration model is trained based on the following manner:

[0033] A training sample set is constructed, and the training sample set comprises a plurality of character samples, and each of the character samples is associated with three types of vowel symbol labels.

[0034] The character sample is input into the vowel restoration model to obtain a probability value of the three types of vowel symbol labels output by the vowel restoration model.

[0035] determine a total training loss based on the probability value and the three types of diacritic labels associated with the character sample;

[0036] update the trainable parameters of the vowel restoration model based on the total training loss.

[0037] The present application provides a vowel restoration device, comprising the following modules:

[0038] The acquisition module is configured to acquire a first to-be-processed text, wherein the first to-be-processed text comprises a first text requiring vowel restoration.

[0039] The processing module is configured to input the first to-be-processed text into a vowel restoration model for multi-diacritic label synchronous prediction, to obtain a predicted label output by the vowel restoration model, wherein the predicted label is a three-type diacritic label corresponding to each character in the first text, the three-type diacritic label comprises a first dot symbol label for consonant fricative, a second dot symbol label for correcting consonant phonetic value, and a symbol label for labeling vowel, and the vowel restoration model is trained based on a character sample and three-type diacritic labels associated with the character sample.

[0040] The restoration module is configured to determine a target text after vowel restoration based on the first to-be-processed text and the predicted label output by the vowel restoration model.

[0041] The present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the vowel restoration method according to any one of the above when executing the program.

[0042] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the vowel restoration method according to any one of the above.

[0043] The present application also provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the vowel restoration method according to any one of the above.

[0044] The vowel restoration method, device, electronic device and storage medium provided by the present application can input a first to-be-processed text comprising a first text requiring vowel restoration into a vowel restoration model for multi-diacritic label synchronous prediction, can obtain a predicted label output by the vowel restoration model, can synchronously predict three-type diacritic labels corresponding to each character in the first text in the process, avoid conflicts between different diacritic labels, and thus accurately predict diacritic labels for each character in the first to-be-processed text, thereby improving the accuracy of vowel restoration. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the present application or the prior art, the drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0046] Figure 1 is one of the flowcharts of the vowel restoration method provided by the present application;

[0047] Figure 2 is the second flowchart of the vowel restoration method provided by the present application;

[0048] Figure 3 is the structure diagram of the vowel restoration model provided by the present application;

[0049] Figure 4 is the flowchart of the first to-be-processed text processing process provided by the present application;

[0050] Figure 5 is the schematic block diagram of the vowel restoration device provided by the present application;

[0051] Figure 6 is the structural schematic diagram of the electronic device provided by the present application.

[0052] Reference signs:

[0053] 501, acquisition module; 502, processing module; 503, restoration module; 610, processor; 620, communication interface; 630, memory; 640, communication bus. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0055] It should be noted that in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, method, article or device including the element. The terms "upper", "lower" and the like indicate the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise specified and limited, the terms "mounting", "connecting", "connecting" should be broadly understood, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0056] The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" and the like are generally a class, and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" means at least one of the connected objects, and the character " / ", generally means that the front and rear associated objects are in an "or" relationship.

[0057] The following will be described in conjunction with Figures 1-6 The vowel restoration method, device, electronic equipment and storage medium provided by the present application are described, which aims to solve the problem that the related technical solutions use a serial prediction method for prediction, and for the case that a single character has multiple diacritics, rule conflicts are prone to occur, making the restored vowels inaccurate.

[0058] Figure 1 is one of the flowcharts of the vowel restoration method provided by the present application, as Figure 1 shown, including but not limited to the following steps:

[0059] Step 101, obtaining a first to-be-processed text, the first to-be-processed text including a first text requiring vowel restoration.

[0060] The first text can be a Hebrew text.

[0061] In some embodiments, the first text is a part or all of the first to-be-processed text.

[0062] In step 102, the first to-be-processed text is input into the vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model. The predicted label is a three-type vowel symbol label corresponding to each character in the first text. The three-type vowel symbol label includes a first dot symbol label for a consonant fricative, a second dot symbol label for correcting a consonant phonetic value, and a symbol label for labeling a vowel. The vowel restoration model is trained based on a character sample and three-type vowel symbol labels associated with the character sample.

[0063] In step 103, a target text with restored vowels is determined based on the first to-be-processed text and the predicted label output by the vowel restoration model.

[0064] In this embodiment, for the first to-be-processed text including the first text that needs to restore vowels, the first to-be-processed text is input into the vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model. In this process, a three-type vowel symbol label corresponding to each character in the first text can be synchronously predicted to avoid conflicts between different vowel symbol labels, thereby accurately predicting the vowel symbol label for each character in the first to-be-processed text to improve the accuracy of vowel restoration.

[0065] The predicted label is a three-type vowel symbol label corresponding to each character in the first text. The three-type vowel symbol label includes a first dot symbol label for a consonant fricative, a second dot symbol label for correcting a consonant phonetic value, and a symbol label for labeling a vowel. The three-type vowel symbol label one-to-one corresponds to three types of vowel symbols used in the Hebrew orthography.

[0066] Specifically, in the Hebrew orthography, the Hebrew language uses three types of vowel symbols, namely, a Sin / Shin dot (4 types of representation, used to distinguish consonant fricatives, a dot symbol located at the upper right corner or the upper left corner of a letter), a Dagesh dot (3 types of representation, used to correct the phonetic value of a consonant, a dot symbol located at the center of a consonant letter), and a Niqqud symbol (16 types of representation, used to label a vowel, located above or below a letter).

[0067] The first dot symbol label for a consonant fricative is a Sin / Shin dot, the second dot symbol label for correcting a consonant phonetic value is a Dagesh dot, and the symbol label for labeling a vowel is a Niqqud symbol.

[0068] In the above embodiment, after obtaining the predicted label output by the vowel restoration model, the first text in the first to-be-processed text can be labeled according to the labeling manner on the character based on the three types of diacritics, so as to obtain the target text after the vowel is restored.

[0069] In the above embodiment, the vowel restoration model is trained based on the character sample and the three types of diacritic labels associated with the character sample, so that the process of obtaining the predicted label does not need to rely on the built-in diacritic combination rule of the dictionary, but learns the rule autonomously through the training data. Therefore, vowel restoration can be realized without the dictionary.

[0070] In some embodiments, the first to-be-processed text further includes a second text that does not need to restore the vowel; the first to-be-processed text is input into the vowel restoration model for synchronous prediction of multiple diacritic labels, to obtain a predicted label output by the vowel restoration model, including:

[0071] The first text in the first to-be-processed text is kept unchanged, and the second text in the first to-be-processed text is replaced by a third text, to obtain the first to-be-processed text after the replacement, the third text being a text that does not affect vowel restoration;

[0072] The first to-be-processed text after the replacement is input into the vowel restoration model for synchronous prediction of multiple diacritic labels, to obtain a predicted label output by the vowel restoration model;

[0073] Based on the first to-be-processed text and the predicted label output by the vowel restoration model, a target text after the vowel is restored is determined, including:

[0074] The third text in the first to-be-processed text after the replacement is kept unchanged, the first text in the first to-be-processed text after the replacement is replaced by a first text after the vowel is labeled, to obtain a first to-be-processed text after the labeling, the first text after the vowel is labeled being obtained by labeling the first text in the first to-be-processed text after the replacement with the predicted label output by the vowel restoration model, the first text after the vowel is labeled in the first to-be-processed text after the labeling is kept unchanged, the third text in the first to-be-processed text after the labeling is replaced by the second text, to obtain the target text after the vowel is restored.

[0075] In this embodiment, by replacing the second text that does not need to restore the vowel with the third text that does not affect vowel restoration, the range of the text that needs to restore the vowel in the first to-be-processed text can be determined. In this process, the vowel restoration model can focus on the text that needs to restore the vowel, so as to reduce the redundancy of the prediction task.

[0076] For example, the first to-be-processed text is: (Hebrew "Genesis" + number "1" + English phrase).

[0077] In which "1" and "In the beginning" belong to the second text (no need to restore the Hebrew vowel), can be replaced by the third text "5" (numeric uniform symbol) and "O" (illegal character MASK symbol).

[0078] In this embodiment, compared with shielding or deleting the second text, the embodiment of the application can avoid information loss caused by shielding or deleting the second text, and ensure that the target text after restoring the vowel contains all the contents of the first to-be-processed text.

[0079] In the above embodiment, if the second text is directly input into the model, the model may learn incorrect character-vowel symbol associations, such as incorrect binding of English characters and Hebrew vowel symbols, which reduces the prediction accuracy. By replacing the second text with the third text, the non-target text is converted into a non-interference text known to the model and does not affect the training rules, avoiding noise from damaging the model's learning of the target character-vowel symbol association.

[0080] Specifically, after the second text is replaced by the illegal character MASK O, the word accuracy (WOR) of the vowel restoration model is improved by 20%. Especially when the test set contains a large amount of mixed text, the WOR can be improved from 87.41% to about 89%, close to the performance of the dictionary scheme.

[0081] In addition, for the multi-language mixed special symbol containing scene, by replacing the second text with the third text, the application scenario of the application can be expanded from pure Hebrew text to multi-language mixed text and special symbol containing text, covering more than 80% of real application scenarios such as news, technical documents, social media, and breaking through the scene limitations of traditional solutions.

[0082] For example, a Hebrew news (containing the number 555 and the English Breaking News), the text input into the model after replacement is , and after prediction, it is restored to the original number and English, and only the vowel of the Hebrew part is supplemented.

[0083] For example, a Hebrew technical document (containing English AI), after replacing AI with O O, the prediction is restored to retain AI and supplement the Hebrew vowel.

[0084] Table 1 shows the replacement relationship of replacing the second text with the third text, as shown in Table 1:

[0085] Table 1

[0086]

[0087] In some embodiments, the second text includes one or more of the following:

[0088] non-first language text, space, number, punctuation mark;

[0089] wherein the first language is a language corresponding to the first text.

[0090] In some embodiments, the first to-be-processed text is input into the vowel restoration model for multi-diacritic synchronous prediction, to obtain a predicted label output by the vowel restoration model, comprising:

[0091] Each character in the first to-be-processed text is represented by a basic symbol to obtain a basic symbol string, and the basic symbol is a symbol constituting each character;

[0092] The basic symbol string is split with a delimiter between different characters in the first to-be-processed text as a boundary to obtain a first sequence composed of a plurality of basic symbol subsequences;

[0093] The first sequence is input into the vowel restoration model for multi-diacritic synchronous prediction, to obtain a predicted label output by the vowel restoration model.

[0094] In this embodiment, the basic symbol is an element constituting the first text, which is equivalent to a component in Chinese characters. By representing each character in the first to-be-processed text by a basic symbol, the vowel restoration model can learn the underlying rules of basic symbol combination→diacritic. In this process, the vowel restoration model accurately predicts the label based on the basic symbol, avoiding misjudgment caused by the overall input at the character level.

[0095] In addition, by representing each character in the first to-be-processed text by a basic symbol, the first sequence is finally obtained, which can unify the dimension learned by the vowel restoration model and reduce the cognitive load of the vowel restoration model.

[0096] For example, different forms of (with Dagesh / without Dagesh), the core basic symbol is consistent after disassembly, the vowel restoration model can reuse the same set of component→diacritic rules, and does not need to learn the rules of different forms of characters separately.

[0097] In this process, the learning efficiency of the vowel restoration model for character features is improved by 35%, the training convergence step number is reduced from 140k steps to 100k steps, and the overfitting risk in the low-resource scenario is significantly reduced.

[0098] The delimiters (spaces, punctuation marks) in Hebrew text are like the word boundaries in Chinese character text, naturally dividing semantic units (words, phrases), enabling the vowel restoration model to accurately capture the collaborative rules of basic symbols (radicals) within the same semantic unit (just like the combination and pronunciation association of "艹", "平", "木", "果" in the word "苹果"), avoiding interference from basic symbols across semantic units (such as not wrongly associating the "木" in "苹果" with the "艹" in "香蕉"), making the prediction of vowel points more accurate, and solving the pain point of fuzzy semantic boundaries in long sequences in traditional solutions. 〈0000216〉〈0000217〉The length of the basic symbol substrings (radical groups) after decomposition is more compact (corresponding to the combination of basic symbols in the original word, just like an idiom is still grouped by words after being decomposed into radicals, with a controllable length), avoiding the redundant attention calculation caused by long character sequences + no clear grouping in traditional solutions and the problem of gradient attenuation caused by long sequences. 〈0000218〉〈0000219〉Among them, the basic symbols are the effective inputs of the vowel restoration model, which are represented by Letter - Symbols, and the effective outputs of the vowel restoration model include the first dot symbol labels (Sin - Symbols) for consonant fricatives, the second dot symbol labels (Dagesh - Symbols) for correcting consonant values, and the symbol labels (Niqqud - Symbols ) for annotating vowels. 〈0000220〉〈0000221〉Among them, the first dot symbol labels for consonant fricatives can distinguish 〈0000222〉and 〈0000223〉, the second dot symbol labels for correcting consonant values can affect the central points of the pronunciation of certain consonants, and the symbol labels for annotating vowels are used for all other vowel points. The combination of the above three vowel points determines the pronunciation 〈0000224〉〈0000225〉Table 2 shows the specific relationships between the input data types, output data types and corresponding tags of the vowel restoration model, as shown in Table 2: 〈0000226〉〈0000227〉[[ID=ID=19]]Table 2 〈0000228〉〈0000229〉〈0000230〉〈0000231〉〈0000232〉In some embodiments, the vowel restoration model includes an embedding layer, a first extraction layer, a second extraction layer and an output layer; 〈0000233〉〈0000234〉The embedding layer is used to convert each basic symbol substring into a corresponding text distribution feature, and the first feature is obtained by fusing the text distribution feature and the position encoding corresponding to each text distribution feature; 〈0000235〉〈0000236〉The first extraction layer is configured to perform feature extraction on the first feature to obtain a first extraction result. After performing element repetition enhancement processing on the first extraction result, an enhanced first extraction result is obtained. After performing a masking operation on the padding bits in the enhanced first extraction result, an updated first extraction result is obtained. The second feature is obtained by fusing the updated first extraction result and updated position encoding.

[0108] The second extraction layer is configured to perform feature extraction on the second feature to obtain a second extraction result.

[0109] The output layer is configured to output a prediction label based on the second extraction result.

[0110] In this embodiment, the double-extraction-layer scheme of the first extraction layer and the second extraction layer can focus on the local features of the basic symbol sub-string (radical group) by the first extraction layer, such as understanding the collocation rules of radicals in the same semantic unit (such as the local association of day + month to form Ming), to output the first extraction result. The second extraction layer captures global context association based on the enhanced second feature (such as the semantic association of Ming in tomorrow), solving the cross-sub-string feature dependency problem of long sentences and compound words.

[0111] In this process, the prediction accuracy of diacritics for long words and compound words is improved by 8%-10%, especially for Hebrew compound words The cross-sub-string diacritic collaborative prediction is more accurate, and the WOR is further broken through to about 90%.

[0112] In the above embodiment, by performing element repetition enhancement processing, the model can be forced to focus on the semantic integrity of the basic symbol combination, avoid semantic fragmentation caused by radical splitting, and significantly improve the long word segmentation accuracy.

[0113] In addition, the execution masking operation can remove invalid placeholders, reduce noise interference, reduce prediction errors caused by padding bits, and improve the prediction stability of the vowel restoration model in long sequences and irregular texts to 99%. The model adapts to scenarios where the lengths of basic symbol sub-strings are inconsistent, and the updated position encoding is further fused to ensure the accuracy of the enhanced sequence position information.

[0114] In the above embodiment, the simplified modular design of the embedding layer, the first extraction layer, the second extraction layer, and the output layer can reduce the computing power consumption by 25% during training, and improve the training speed on low-configuration devices by 30%. At the same time, due to the structured process, the model debugging and iteration efficiency is improved by 40%.

[0115] In some embodiments, the structure of the vowel restoration model is a Transformer Encoder structure, and can also be any one of a fully connected layer network, a convolutional neural network, a recurrent neural network, a long short-term memory neural network, a residual neural network, and an attention mechanism deep learning model.

[0116] In some embodiments, the structure of the first extraction layer and the second extraction layer is a Transformer Encoder structure, a 6-head attention 4-layer Encoder structure is used, the optimizer Adam is set to 0.005, dropout=0.1 is performed on a downstream task, the batch-size is set to 256, and 140k steps of training are used as a current state-of-the-art model (SOTA).

[0117] In some embodiments, the first extraction layer and the second extraction layer can be absolute position encoding, or relative position encoding or rotational position encoding.

[0118] In some embodiments, the element repetition enhancement processing includes:

[0119] Based on the first extraction result, replacing each target base symbol in each base symbol substring at a character level with an enhanced target base symbol to obtain an enhanced first extraction result;

[0120] wherein the target base symbol is a base symbol in the base symbol substring other than the first base symbol and the last base symbol, and the enhanced target base symbol is obtained by repeating embedding each base symbol in the target base symbol N times and concatenating.

[0121] For example, the base symbol substring is “APPLE”, and the target base symbol is “PPL” when N is 3, and the enhanced target base symbol is “PPPPPPLLL”.

[0122] In some embodiments, the vowel restoration model is trained based on the following manner:

[0123] A training sample set is constructed, and the training sample set includes a plurality of character samples, each character sample being associated with three types of diacritic labels;

[0124] The character sample is input into the vowel restoration model to obtain a probability value of the three types of diacritic labels associated with the character sample output by the vowel restoration model;

[0125] Based on the probability value and the three types of diacritic labels associated with the character sample, a total training loss is determined;

[0126] The trainable parameters of the vowel restoration model are updated based on the total training loss.

[0127] In this embodiment, the unified target of the three types of label binding is the character sample, and the model is forced to learn the symbol co-occurrence constraint, which can improve the collaborative prediction accuracy of the three types of diacritics by 15%-20%, reduce the symbol combination error rate by 60%, and completely solve the rule conflict problem caused by independent training.

[0128] In addition, by taking characters as the minimum unit instead of words or sentences, limited low-resource labeled data can be decomposed into more independent training samples, avoiding the waste of training data in low-resource scenarios.

[0129] As shown in Figure 2 In the case of the first text to be processed being Hebrew, and the process of replacing the second text with the third text, the basic symbol transformation, and the separator segmentation as preprocessing, the vowel restoration method mainly includes the following steps:

[0130] Step 201, obtaining the original Hebrew text to be processed;

[0131] Step 202, preprocessing the original Hebrew text in a character-level manner, distinguishing between legal characters, illegal characters, numbers, and punctuation, and performing index indexing operations thereon;

[0132] Step 203, using a vowel restoration model to obtain the text distribution features of each character in the original Hebrew text to be processed and the feature distribution of the three types of diacritics labels;

[0133] Step 204, based on the text distribution features of each character in the original Hebrew text to be processed, combining the learning rules of the feature distribution of the three types of diacritics labels, determining the corresponding vowel restoration result of each character in the original Hebrew text to be processed.

[0134] The index indexing operation is a process of representing by basic symbols.

[0135] The preprocessed vowel restoration text and the input form input to the vowel restoration model are shown in Table 3.

[0136] Table 3

[0137]

[0138] In some embodiments, the accuracy is calculated at the word-level granularity with the true labeled data, and two examples of test set input and prediction result extraction are shown in Table 4. For example, the prediction result "O OOOOOO" after the MASK of "I am Rich" in the table; the true vowel restoration and the prediction result of the second sentence have differences, and the word is recorded as a prediction error. ​

[0139] Table 4

[0140]

[0141] In the present application, the prediction result statistics indicators of the vowel restoration model adopt word accuracy (WOR) and vocalization accuracy (VOC), the word accuracy refers to the part of the word without diacritic errors, and the vocalization accuracy refers to the part of the word that will not cause pronunciation errors regardless of any diacritic point errors, wherein the baseline effect is 67.08% (17076 / 25453 words).

[0142] In some embodiments, the training sample set is obtained by processing a 78,000 token-level diacritic corpus and a token-level fine-grained pos corpus, wherein the pos corpus refers to a ready-made text collection in which each word in the text is labeled with a part-of-speech tag.

[0143] In some embodiments, as shown in Figure 3 The initial model structure of the vowel restoration model includes an embedding layer, a first extraction layer, an output layer, and a full connection layer, wherein the first to-be-processed text (x1, x2, … xn) is input into the embedding layer, the output of the embedding layer is input into the first extraction layer after position encoding, the output of the first extraction layer is input into the output layer after performing element repetition enhancement processing, the output of the output layer is input into the full connection layer, and the output prediction label is obtained.

[0144] In the case of using a model setting of 6 heads and 6 layers with 768 hidden layer dimensions, the accuracy on the validation set is more than 90%, but the accuracy on the test set is only 67.08%, but there is no obvious overfitting from the loss performance.

[0145] The best WOR of adjusting the hidden layer dimension to 192, 384, 512, and 1024 in different configurations is 71.15%. At the same time, the number of heads and the number of layers are also modified, and using a 384 hidden layer dimension, the WOR is best at 68.2% after adjusting the model structure to 6 heads and 8 layers, 8 heads and 6 layers, and 8 heads and 8 layers, respectively. In the best structure of 6 heads and 8 layers with 384 nodes, the number of repeated embedding times, i.e., the proportion, is also adjusted, and the input data is expanded at the word level, and the WOR is obviously improved to 73%. After analyzing the model structure, considering the parameter size and performance of the model integration deployment, the finally used structure is 6 heads, 4 layers, and 384 hidden layer dimensions, and the repeated embedding is 3 times. After the input data passes through two layers of Encoder (i.e., the extraction of the first extraction layer and the second extraction layer in the present application), a dropout operation (i.e., the operation performed by the output layer in the present application) is performed to avoid overfitting caused by insufficient data.

[0146] The vowel restoration model proposed in the application has a WOR of 87.41% (22250 / 25453 words) on a 2124-sentence test set, which is 20% more accurate than the baseline effect of 67.08% (17076 / 25453 words). The WOR is close to the current optimal effect and the Morfix scheme using a word-level dictionary (89.43%) and a pre-trained model (90.76%), but the cost and the number of model parameters are much lower than the above schemes.

[0147] In some embodiments, as shown in Figure 4 the first to-be-processed text is input into the vowel restoration model, and finally the target text after the vowels are restored is obtained. If the target text is a dictionary word, the phonemes corresponding to the target text are output according to the dictionary. If the target text is not a dictionary word, the C45 module is called to predict the speech corresponding to the target text, and the prediction result of the C45 module is input into the stress identification module. The stress identification module is used to annotate the stress of the speech corresponding to the predicted target text, so as to obtain the phonemes corresponding to the target text, wherein the C45 module is a speech prediction module.

[0148] It should be noted that the vowel restoration device provided by the application can execute the vowel restoration method of any of the above embodiments when actually running, and therefore the present embodiment will not be described in detail.

[0149] As shown in Figure 5 the vowel restoration device provided by the application comprises:

[0150] The acquisition module 501 is configured to acquire a first to-be-processed text, and the first to-be-processed text comprises a first text in which vowels need to be restored.

[0151] The processing module 502 is configured to input the first to-be-processed text into a vowel restoration model for multi-vowel symbol tag synchronous prediction, to obtain a prediction tag output by the vowel restoration model. The prediction tag is a three-type vowel symbol tag corresponding to each character in the first text. The three-type vowel symbol tag comprises a first point symbol tag for a consonant fricative, a second point symbol tag for correcting a consonant sound value, and a symbol tag for labeling a vowel. The vowel restoration model is trained based on a character sample and three-type vowel symbol tags associated with the character sample.

[0152] The restoration module 503 is configured to determine a target text after the vowels are restored based on the first to-be-processed text and the prediction tag output by the vowel restoration model.

[0153] In the embodiment, for a first to-be-processed text including a first text requiring restored vowels, the first to-be-processed text is input to the vowel restoration model for multi-vowel symbol label synchronous prediction, and a predicted label output by the vowel restoration model can be obtained. In this process, three types of vowel symbol labels corresponding to each character in the first text can be synchronously predicted, conflicts between different vowel symbol labels are avoided, and thus the vowel symbol label of each character in the first to-be-processed text is accurately predicted, thereby improving the accuracy of vowel restoration.

[0154] Figure 6 is a structural schematic diagram of an electronic device provided by the present application, as Figure 6 shown, the electronic device can include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 complete mutual communication through the communication bus 640. The processor 610 can invoke a logical instruction in the memory 630 to execute a vowel restoration method, which includes: obtaining a first to-be-processed text, the first to-be-processed text including a first text requiring restored vowels; inputting the first to-be-processed text to a vowel restoration model for multi-vowel symbol label synchronous prediction to obtain a predicted label output by the vowel restoration model, the predicted label being three types of vowel symbol labels corresponding to each character in the first text, the three types of vowel symbol labels including a first dot symbol label for a consonant fricative, a second dot symbol label for correcting a consonant phonetic value, and a symbol label for labeling a vowel, the vowel restoration model being trained based on a character sample and three types of vowel symbol labels associated with the character sample; and determining a target text after the vowels are restored based on the first to-be-processed text and the predicted label output by the vowel restoration model.

[0155] In addition, the logical instruction in the memory 630 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0156] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer can execute the vowel restoration method provided by the above-mentioned embodiments, and the method comprises: obtaining a first to-be-processed text, the first to-be-processed text comprising a first text in which vowels need to be restored; inputting the first to-be-processed text into a vowel restoration model to perform multi-vowel symbol label synchronous prediction, to obtain a prediction label output by the vowel restoration model, the prediction label being three types of vowel symbol labels corresponding to each character in the first text, the three types of vowel symbol labels comprising a first dot symbol label for consonant fricative, a second dot symbol label for correcting consonant sound value, and a symbol label for labeling vowels, the vowel restoration model being trained based on character samples and the three types of vowel symbol labels associated with the character samples; and determining a target text after the vowels are restored based on the first to-be-processed text and the prediction label output by the vowel restoration model.

[0157] In another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the vowel restoration method provided by the above-mentioned embodiments, and the method comprises: obtaining a first to-be-processed text, the first to-be-processed text comprising a first text in which vowels need to be restored; inputting the first to-be-processed text into a vowel restoration model to perform multi-vowel symbol label synchronous prediction, to obtain a prediction label output by the vowel restoration model, the prediction label being three types of vowel symbol labels corresponding to each character in the first text, the three types of vowel symbol labels comprising a first dot symbol label for consonant fricative, a second dot symbol label for correcting consonant sound value, and a symbol label for labeling vowels, the vowel restoration model being trained based on character samples and the three types of vowel symbol labels associated with the character samples; and determining a target text after the vowels are restored based on the first to-be-processed text and the prediction label output by the vowel restoration model.

[0158] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0159] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the implementation can also be through hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of the various embodiments or some parts of the embodiments.

[0160] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A vowel restoration method, characterized in that, include: Obtain the first text to be processed, which includes the first text for which vowels need to be recovered, and the first text is Hebrew text; The first text to be processed is input into the vowel recovery model for simultaneous prediction of multiple altered note labels, and the predicted labels output by the vowel recovery model are obtained. The predicted labels are three types of altered note labels corresponding to each character in the first text. The three types of altered note labels include a first dot symbol label for consonant fricatives, a second dot symbol label for correcting consonant pitch values, and a symbol label for marking vowels. The vowel recovery model is trained based on character samples and the three types of altered note labels associated with the character samples. The target text after vowel recovery is determined based on the first text to be processed and the predicted labels output by the vowel recovery model.

2. The vowel restoration method according to claim 1, characterized in that, The first text to be processed also includes a second text that does not require the recovery of vowels; The step of inputting the first text to be processed into the vowel recovery model for simultaneous prediction of variable note labels, and obtaining the predicted labels output by the vowel recovery model, includes: Keeping the first text in the first text to be processed unchanged, replacing the second text in the first text to be processed with the third text, and obtaining the replaced first text to be processed, wherein the third text is text that does not affect vowel recovery; The replaced first text to be processed is input into the vowel recovery model for synchronous prediction of variable note labels, and the predicted labels output by the vowel recovery model are obtained. The step of determining the target text after vowel recovery based on the first text to be processed and the predicted labels output by the vowel recovery model includes: Keeping the third text in the replaced first text to be processed unchanged, the first text in the replaced first text to be processed is replaced with the first text after the vowel is annotated, and the first text after the vowel is annotated is obtained by annotating the first text in the replaced first text to be processed with vowels using the predicted labels output by the vowel recovery model. Keeping the first text after the marked vowels in the first text to be processed unchanged, the third text in the first text to be processed after the marked vowels is replaced with the second text to obtain the target text after the vowels are restored.

3. The vowel restoration method according to claim 2, characterized in that, The second text includes one or more of the following: Non-native language text, spaces, numbers, and punctuation marks; Wherein, the first language is the language corresponding to the first text.

4. The vowel restoration method according to claim 1, characterized in that, The step of inputting the first text to be processed into the vowel recovery model for simultaneous prediction of variable note labels, and obtaining the predicted labels output by the vowel recovery model, includes: Each character in the first text to be processed is represented by a basic symbol to obtain a basic symbol string, wherein the basic symbols are the symbols that constitute each character; Using the delimiters between different characters in the first text to be processed as boundaries, the basic symbol string is split to obtain a first sequence composed of multiple basic symbol substrings; The first sequence is input into the vowel restoration model for synchronous prediction of variable note labels, and the predicted labels output by the vowel restoration model are obtained.

5. The vowel restoration method according to claim 4, characterized in that, The vowel restoration model includes an embedding layer, a first extraction layer, a second extraction layer, and an output layer; The embedding layer is used to convert each of the basic symbol substrings into a corresponding text distribution feature, and the first feature is obtained by fusing the text distribution feature and the positional encoding corresponding to each of the text features; The first extraction layer is used to extract features from the first feature to obtain a first extraction result. The first extraction result is enhanced after performing element repetition enhancement processing. The padding bits in the enhanced first extraction result are masked to obtain an updated first extraction result. The updated first extraction result and the updated position code are fused to obtain a second feature. The second extraction layer is used to extract features from the second feature to obtain the second extraction result; The output layer is used to output predicted labels based on the second extraction result.

6. The vowel restoration method according to claim 5, characterized in that, The element repetition enhancement process includes: Based on the first extraction result, the target basic symbol in each basic symbol substring at the character level is replaced with the enhanced target basic symbol to obtain the enhanced first extraction result; The target basic symbol is the basic symbol in the basic symbol substring except for the first and last basic symbols. The enhanced target basic symbol is obtained by repeatedly embedding each basic symbol in the target basic symbol N times and concatenating them.

7. The vowel restoration method according to any one of claims 1 to 6, characterized in that, The vowel restoration model was trained using the following method: Construct a training sample set, which includes multiple character samples, each of which is associated with three types of altered note labels; Input the character sample into the vowel recovery model and obtain the probability values ​​of the three types of altered note labels associated with the character sample output by the vowel recovery model; The total training loss is determined based on the probability value and the three types of altered note labels associated with the character sample; The trainable parameters of the vowel recovery model are updated based on the total training loss.

8. A vowel restoration device, characterized in that, include: The acquisition module is used to acquire the first text to be processed, which includes the first text for which vowels need to be recovered, and the first text is Hebrew text; The processing module is used to input the first text to be processed into the vowel recovery model for synchronous prediction of multiple altered note labels, and obtain the predicted labels output by the vowel recovery model. The predicted labels are three types of altered note labels corresponding to each character in the first text. The three types of altered note labels include a first dot symbol label for consonant fricatives, a second dot symbol label for correcting consonant pitch values, and a symbol label for marking vowels. The vowel recovery model is trained based on character samples and the three types of altered note labels associated with the character samples. The recovery module is used to determine the target text after recovering vowels based on the first text to be processed and the predicted labels output by the vowel recovery model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the vowel restoration method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the vowel restoration method as described in any one of claims 1 to 7.

Images