A spoken language translation method, system and electronic device
By standardizing colloquial language data and incorporating masking and fluency information into the machine translation model, the problems of literal translation, mistranslation, and omission of colloquial expressions in Chinese during machine simultaneous interpretation were solved, achieving more accurate and fluent translation results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH OF CHINA
- Filing Date
- 2022-10-09
- Publication Date
- 2026-08-04
AI Technical Summary
Existing machine simultaneous interpretation technologies suffer from problems such as literal translation, mistranslation, and omission when faced with complex spoken Chinese expressions in real-world conference scenarios. Current labeled data and model training schemes cannot effectively cover all forms of spoken Chinese expressions.
Colloquial language data is regularized through dependency parsing and part-of-speech tagging to generate first-language data. This data is then input into a machine translation model that incorporates an encoder with masking information and a decoder with fluency information for translation. Masking and fluency information are added during model training to improve the accuracy and fluency of the translation.
It improves the accuracy and error tolerance of machine translation models in translating spoken expressions, reduces literal translation and omissions, and ensures that the translation is clearer and easier to understand.
Smart Images
Figure CN115587600B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of natural language processing and speech recognition, and in particular to a spoken language translation method, system, and electronic device. Background Technology
[0002] In machine simultaneous interpretation, facing real-world conference scenarios and complex, colloquial Chinese expressions, existing labeled data and model training methods are often uncontrollable. Specifically, current model training methods still suffer from problems such as literal translation and mistranslation for most colloquial Chinese expressions. Summary of the Invention
[0003] In view of this, this application provides a spoken language translation method, system, and electronic device, the specific solutions of which are as follows:
[0004] A spoken translation method includes:
[0005] Obtain spoken language data;
[0006] The colloquial language data is regularized based on dependency parsing and part-of-speech tagging to obtain the first monolingual data.
[0007] The first monolingual data is input into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0008] Furthermore, it also includes:
[0009] Train the machine translation model;
[0010] The training of the machine translation model includes:
[0011] The training parameters are obtained, and the training parameters include at least: second monolingual data and second translation data corresponding to the second monolingual data;
[0012] The machine translation model is obtained by training the model based on at least the second monolingual data and the second translated data.
[0013] Furthermore, the step of training the model based at least on the second monolingual data and the second translated data to obtain the machine translation model includes:
[0014] The second monolingual data is input into a pre-trained anti-smoothing regularization model to obtain the first monolingual colloquial data output by the anti-smoothing regularization model.
[0015] The machine translation model is trained based on the first monolingual spoken language data and the second translated text data.
[0016] Furthermore, it also includes:
[0017] Training the anti-smooth regularization model;
[0018] The training of the anti-smooth regularization model includes:
[0019] Obtain multiple second monolingual colloquialization data;
[0020] Obtain the third monolingual data after normalizing each second monolingual colloquial data;
[0021] The second monolingual colloquial data and the third monolingual data are used as training data to train the model, resulting in an anti-smooth regularization model.
[0022] Furthermore, the step of training the model based on the first monolingual spoken language data and the second translated text data to obtain the trained machine translation model includes:
[0023] Key information was determined based on the second monolingual data and the first monolingual colloquial data.
[0024] Based on the key information, add masking information to the first monolingual colloquial data;
[0025] The machine translation model is obtained by training the first monolingual colloquial data and the second translated data after adding masking information.
[0026] Furthermore, it also includes:
[0027] Determine the translation loss data of the machine translation model;
[0028] Determine the mask loss data resulting from adding mask information;
[0029] Determine the smoothness decoding loss data resulting from incorporating smoothness information;
[0030] The overall loss data of the machine translation model is determined based on translation loss data, loss data with added masking information, and fluency decoding loss data.
[0031] The machine translation model is optimized based on the overall loss data.
[0032] Furthermore, the colloquial language data is normalized based on dependency parsing and part-of-speech tagging to obtain the first monolingual data, including:
[0033] Based on the aforementioned dependency parsing and part-of-speech tagging methods, the core words and subject-verb structure of the colloquial language data are determined.
[0034] If the colloquial language data has a subject-verb structure, determine the attributive-head structure corresponding to the subject-verb structure;
[0035] When the subject-predicate structure contains a structure that satisfies the colloquial modification condition, the colloquial language data is normalized, and the structure that satisfies the colloquial modification condition is deleted.
[0036] A spoken language translation system, comprising:
[0037] The acquisition unit is used to acquire spoken language data;
[0038] The standardization unit is used to standardize the colloquial language data based on dependency parsing and part-of-speech tagging to obtain the first monolingual data;
[0039] The translation unit is used to input the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0040] An electronic device, comprising:
[0041] A processor is used to acquire spoken language data; to normalize the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data; to input the first monolingual data into a pre-trained machine translation model to obtain first translated data output by the machine translation model, wherein the machine translation model includes at least: an encoder incorporating masking information and a decoder incorporating fluency information.
[0042] The memory is used to store the program used by the processor to execute the above-described processing procedure.
[0043] A readable storage medium for storing at least one set of instructions;
[0044] The instruction set is used to be invoked and to perform at least the oral translation method as described in any of the preceding items.
[0045] As can be seen from the above technical solutions, the spoken language translation method, system, and electronic device disclosed in this application obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. After obtaining the spoken language data, this solution inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text. The masking information in the machine translation model enables the inclusion of masking information during training, thereby enhancing the ability to predict missing words in the monolingual data input to the machine translation model and improving error tolerance. At the same time, the addition of fluency information to the machine translation model optimizes the output of the machine translation model, improving the fluency of the model's output translation and thus mitigating mistranslation, literal translation, and omissions. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 This is a flowchart of a spoken language translation method disclosed in an embodiment of this application;
[0048] Figure 2 This is a flowchart of a spoken language translation method disclosed in an embodiment of this application;
[0049] Figure 3 This is a flowchart of a spoken language translation method disclosed in an embodiment of this application;
[0050] Figure 4 This is a schematic diagram of the structure of an encoder disclosed in an embodiment of this application;
[0051] Figure 5 This is a schematic diagram of the structure of a decoder disclosed in an embodiment of this application;
[0052] Figure 6 This is a schematic diagram of the structure of a machine translation model disclosed in an embodiment of this application;
[0053] Figure 7 This is a flowchart of a spoken language translation method disclosed in an embodiment of this application;
[0054] Figure 8This is a schematic diagram of a dependency parsing tree disclosed in an embodiment of this application;
[0055] Figure 9 This is a schematic diagram of the structure of a spoken language translation system disclosed in an embodiment of this application;
[0056] Figure 10 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation
[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0058] This application discloses a spoken language translation method, the flowchart of which is as follows: Figure 1 As shown, it includes:
[0059] Step S11: Obtain colloquial language data;
[0060] Step S12: Based on dependency parsing and part-of-speech tagging, the colloquial language data is regularized to obtain the first monolingual data;
[0061] Step S13: Input the first monolingual data into the pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0062] In existing machine simultaneous interpretation technologies, when faced with complex spoken expressions in real-world meeting scenarios, such as spoken Chinese, the existing labeled data and model training solutions usually cannot cover all forms of spoken Chinese expressions. When faced with complex spoken Chinese expressions, mistranslation, literal translation, overtranslation, and omission are still prone to occur, which means that the audience cannot understand the information conveyed by the speaker by just reading the translation.
[0063] Based on this, the proposed solution first standardizes the obtained colloquial language data, and then inputs the standardized data into the machine translation model for translation. This ensures that the translated text corresponds to the standardized colloquial language data, thereby improving the accuracy of translating colloquial expressions and reducing mistranslations, literal translations, and omissions in machine simultaneous interpretation.
[0064] Spoken language data refers to the language output by the speaker that needs to be translated. This language is usually spoken language, that is, language with the user's speaking habits. For example, spoken language data includes words such as "this", "that", "um", and "uh".
[0065] Before translating the obtained colloquial language data, it is necessary to first normalize it. Specifically, this can be done by normalizing the colloquial language data based on dependency parsing and part-of-speech tagging to obtain normalized first monolingual data.
[0066] When a speaker uses colloquialisms, it often affects the translation. Phrases like "this" and "our" can lead to literal translation problems. For example, "This increases the difficulty of this and that" could be directly translated as "This adds to the difficulty of this and that," which would be difficult for the user to understand. Furthermore, different speakers have different speaking habits, and their colloquialisms often vary, but these colloquialisms are generally categorized.
[0067] Therefore, the spoken language data output by the speaker is standardized by removing colloquial words, so that the standardized first monolingual data can be expressed more concisely and clearly than the spoken language data, without redundant components, so as to produce a clearer and more accurate translation.
[0068] Specifically, dependency parsing techniques can identify the dependency relationships between words in a sentence. These dependency relationships are directional, with dominant and subordinate elements. Core components typically have multiple sub-components. By combining colloquial words with the results of part-of-speech tagging and the dependency relationships, colloquial expressions in sentences can be safely regularized while preserving the semantics of the original text.
[0069] After standardizing the spoken language data to obtain the first monolingual data, the first monolingual data is used as input to the machine translation model. The machine translation model translates the first monolingual data to obtain the output of the machine translation model, namely the first translated data. The first translated data is the translation corresponding to the spoken language data output by the speaker. The first translated data is the translation based on the standardized sentence obtained after removing the speaker's spoken words, so that the content expressed by the first translated data is consistent with the content expressed by the speaker's spoken language data, which facilitates other users' understanding of the speaker's speech and reduces the occurrence of literal translation, omissions and other errors.
[0070] The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. That is, during the training of the machine translation model, the training parameters are processed using masking information to enable the model to accurately translate sentences with the masking, thereby improving the accuracy of translating sentences with missing words during the application of the machine translation model.
[0071] Integrating fluency information into machine translation models means that during the training process, the machine translation model can be optimized based on the fluency parameter of the translated text, so that the fluency of the translated text can meet the requirements during the application of the machine translation model.
[0072] The spoken language translation method disclosed in this embodiment obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. After obtaining the spoken language data, this scheme inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text, thereby achieving translation based on normalized data and improving the accuracy of spoken expression translation. Furthermore, the masking information in the machine translation model allows for the inclusion of masking information during model training, enhancing the ability to predict missing words in the monolingual data input to the machine translation model, thus improving error tolerance. Simultaneously, the addition of fluency information to the machine translation model optimizes the model's output, improving the fluency of the output translation and thus mitigating mistranslation, literal translation, and omissions.
[0073] This embodiment discloses a spoken language translation method, the flowchart of which is as follows: Figure 2 As shown, it includes:
[0074] Step S21: Obtain colloquial language data;
[0075] Step S22: Based on dependency parsing and part-of-speech tagging, the colloquial language data is regularized to obtain the first monolingual data;
[0076] Step S23: Obtain training parameters, which include at least: the second monolingual data and the corresponding second translation data of the second monolingual data;
[0077] Step S24: Train the model based on at least the second monolingual data and the second translated data to obtain the machine translation model;
[0078] Step S25: Input the first monolingual data into the machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0079] Training a machine translation model can be specifically done by: obtaining training parameters, training based on the training parameters, and thus obtaining a machine translation model for application.
[0080] The training parameters include at least: second monolingual data and corresponding second translation data. The second monolingual data and the second translation data are obtained in advance. The second monolingual data is language data obtained by normalizing the spoken language output by the speaker in a real meeting scenario, and the second translation data is the translated data obtained by translating the normalized language data.
[0081] Furthermore, the training process may also include: inputting the second monolingual data into the pre-trained anti-smoothing regularization model to obtain the first monolingual colloquial data output by the anti-smoothing regularization model; training the model based on the first monolingual colloquial data and the second translation data to obtain the trained machine translation model.
[0082] During the training of a machine translation model, it can be trained based on the second monolingual data and the second translated data; alternatively, it can be trained first on the second monolingual data and the second translated data, and then trained on the first monolingual colloquial data and the second translated data; or, it can be trained simultaneously on the second monolingual data and the second translated data, and on the first monolingual colloquial data and the second translated data. This ensures that the final machine translation model can take normalized monolingual data as input and output the corresponding translation, and can also take monolingual colloquial data obtained from the monolingual data as input and output the corresponding translation.
[0083] The first monolingual colloquial data is obtained by inputting the normalized second monolingual data into the anti-smoothing normalization model, resulting in first monolingual colloquial data with colloquial elements. The anti-smoothing normalization model can adjust the normalized monolingual data to resemble colloquial data, simulating the spoken language used by speakers in real-world meeting scenarios. The first monolingual colloquial data can include sentences with colloquial words or sentences with grammatical errors, i.e., sentences that conform to everyday speaking habits.
[0084] After obtaining the first monolingual colloquial data based on the second monolingual data, since the first monolingual colloquial data is simply the second monolingual data with the addition of colloquial words, the second translation data corresponding to the second monolingual data is also a translation of the first monolingual colloquial data. Therefore, the first monolingual colloquial data and the second translation data can be used as training parameters to optimize the machine translation model. This optimized machine translation model can then receive sentences containing colloquial words and output the translation data corresponding to the monolingual data containing those colloquial words. In other words, the machine translation model takes sentences containing colloquial words as input and outputs the translation data of sentences after removing colloquial words, thus avoiding literal translation and omissions.
[0085] For example, given a second monolingual data X and its corresponding translation Y, if the second monolingual data X is input into an anti-smoothing regularization model, the output will be the first monolingual colloquial data X'. The machine translation model can then be trained using (X,Y) as training parameters. Furthermore, it can be trained using (X',Y) as training parameters to ensure that the trained machine translation model can output translation data after receiving regularized monolingual data, and also output corresponding translation data after receiving unregistered colloquialized monolingual data.
[0086] Machine translation models come in various forms, depending on the language of the input monolingual or spoken monolingual data and the language into which it needs to be translated. For example, if the input is Chinese and the output is English, the machine translation model is a Chinese-English translation model; if the input is Chinese and the output is Korean, the machine translation model is a Chinese-Korean translation model; and if the input is Russian and the output is Chinese, the machine translation model is a Russian-Chinese translation model.
[0087] The anti-smoothing regularization model is obtained through pre-training. The training of the anti-smoothing regularization model includes: obtaining multiple second monolingual colloquial data, obtaining third monolingual data after regularizing each second monolingual colloquial data, and using the second monolingual colloquial data and the third monolingual data as training data to train the model and obtain the anti-smoothing regularization model.
[0088] Anti-smoothing regularization is a sequence generation model similar to machine translation. However, while machine translation translates input (e.g., inputting Chinese and outputting English), anti-smoothing regularization outputs colloquial Chinese from input, that is, it outputs sentences with colloquial vocabulary from regularized input sentences.
[0089] The training data used to train the anti-smooth regularization model includes: real speakers' statements in a conference scenario and sentences with grammatical errors, T. pre and T pre Sentence T after colloquialization post , will T post As input, T pre The output is used to train the model to obtain an anti-smooth regularization model.
[0090] The spoken language translation method disclosed in this embodiment obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. After obtaining the spoken language data, this scheme inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text, thereby achieving translation based on normalized data and improving the accuracy of spoken expression translation. Furthermore, the masking information in the machine translation model allows for the inclusion of masking information during model training, enhancing the ability to predict missing words in the monolingual data input to the machine translation model, thus improving error tolerance. Simultaneously, the addition of fluency information to the machine translation model optimizes the model's output, improving the fluency of the output translation and thus mitigating mistranslation, literal translation, and omissions.
[0091] This embodiment discloses a spoken language translation method, the flowchart of which is as follows: Figure 3 As shown, it includes:
[0092] Step S31: Obtain spoken language data;
[0093] Step S32: Based on dependency parsing and part-of-speech tagging, the colloquial language data is regularized to obtain the first monolingual data;
[0094] Step S33: Obtain training data, which includes at least: second monolingual data and second translation data corresponding to the second monolingual data;
[0095] Step S34: Input the second monolingual data into the pre-trained anti-smoothing regularization model to obtain the first monolingual colloquial data output by the anti-smoothing regularization model;
[0096] Step S35: Determine key information based on the second monolingual data and the first monolingual colloquial data, and add masking information to the first monolingual colloquial data based on the key information;
[0097] Step S36: Train the model based on the first monolingual spoken language data with added masking information and the second translated data to obtain the machine translation model;
[0098] Step S37: Input the first monolingual data into the machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0099] Existing machine translation models typically consist of an encoder and a decoder. Incorporating masking information into the encoder can enhance its ability to predict missing words by adding masks to some words in difficult colloquial expressions. Incorporating fluency information into the decoder can incentivize the model to produce more fluent and coherent translations, thereby significantly improving the problems of mistranslation, literal translation, and omission in existing Chinese simultaneous interpretation models.
[0100] The encoder is used to parse the input sequence, i.e., the monolingual data input to the machine translation model, and its output is the source text vector representation, i.e., the monolingual data vector representation. The encoder model structure is as follows: Figure 4 As shown, it includes: the Add&Normalize module, the Feed Forward module, and the Self-Attention module.
[0101] The decoder obtains the source text vector representation output by the encoder and the history token of the translated text result decoded in the previous time step, and outputs the translated text. The decoder model structure is as follows: Figure 5 As shown, it includes: the Add & Normalize module, the Feed Forward module, the Encoder-decoderAttention module, and the Self-Attention module.
[0102] There can be multiple encoders and decoders in a machine translation model. When there are M encoders in a machine translation model, the monolingual data is input into the machine translation model and will be input into each encoder in turn, and finally calculated M times. Similarly, when there are N decoders in a machine translation model, they will be calculated N times in turn.
[0103] In addition, the machine translation model may also include a language model, which is used to calculate fluency information. Its inputs are the source text vector representation output by the encoder, the current text vector representation output by the decoder, and the translated text result decoded in the previous time step. The output is fluency information.
[0104] Then as Figure 6 The diagram shown is a schematic of a machine translation model, which includes an encoder, a decoder, and a language model.
[0105] In this method, masking information is incorporated into the encoder, which forces the machine translation model to use less phrasing information for translation, thereby improving the machine translation model's tolerance for colloquial expressions.
[0106] Machine translation models need to add masks of an appropriate proportion to the training data, placing the masks at the locations corresponding to key information.
[0107] Key information is determined based on second-language data and first-language spoken language data. Specifically, by comparing the second-language data and the first-language spoken language data, overlapping portions are identified and designated as key information. Words identified as key information in the training data are selected using a Bernoulli distribution, and masks are added to these selected key information words. In other words, one or more words are selected from multiple key information words, and masks are added to these selected words.
[0108] For example: the second monolingual data is X2, and its corresponding translation is Y2. The first monolingual colloquial data X2' generated by the anti-smoothing regularization model from X2 is then the training sentence pair to be input into the model is (X2', Y). Its key information can be represented as KEY, and its corresponding formula is:
[0109] KEY=[X∩X']
[0110] Based on the Bernoulli distribution, a mask is selectively added to the key information (KEY), and the formula is as follows:
[0111] m~Bernoulli(p)
[0112]
[0113]
[0114] Where m ~ Bernoulli(p) represents constructing an m-matrix with a Bernoulli distribution, such as assuming the generating matrix [0,0,0,0,1]; This means adding <MASK> to the keyword candidates according to the Bernoulli distribution m matrix, i.e., a mask; This indicates that the content after generating the mask will be merged into X2'.
[0115] For example, taking "The weather is quite nice today" as X2 for illustration, sending X2 into the anti-smoothing and regularization model, we get X2' as "Well, the weather is quite nice today". Segmenting the overlapping part of X2 and X2' gives KEY = ["today", "weather", "is", "quite", "nice"]. Assuming the matrix generated based on the Bernoulli distribution is [0, 0, 0, 0, 1], after adding the mask, we get KEY = ["today", "weather", "is", "quite", "MASK"]. Then, after merging the keyword with the mask into X2', we can get X2' as "Well, the weather is quite <MASK>".
[0116] Specifically, during the model training process, we can determine the keyword corresponding to the mask by analyzing context information or current sentence patterns, etc.
[0117] After obtaining the first monolingual colloquial data X2' with the mask added, using it as training data for model training, the translation result loss function of this machine translation model is:
[0118]
[0119] Furthermore, if we want to enhance the robustness of the encoder and the ability to predict keywords with mask information added, at the encoder end, the model also needs to determine which words in the current training data are replaced, and predict which words are replaced. Its loss function is as follows:
[0120]
[0121]
[0122] Among them, d(x) represents all words with masks added, G represents the generator, that is, the output of the encoder, and D represents the discriminator, which consists of non-linear hidden units.
[0123] Among them, L D represents predicting which words are replaced and which are not; L P represents predicting what the replaced words are specifically, such as "nice" in "The weather is quite nice today".
[0124] Furthermore, to incorporate fluency information into the decoder, it can be specifically:
[0125] Determine the translation loss data of the machine translation model; determine the mask loss data generated by adding mask information; determine the fluency decoding loss data generated by incorporating fluency information; determine the overall loss data of the machine translation model based on the translation loss data, the loss data of adding mask information, and the fluency decoding loss data; optimize the machine translation model based on the overall loss data.
[0126] By fusing the hidden information units from the encoder and decoder at the current time step and combining them with information from historical time steps, a cross-attention mechanism is used to extract key information, and finally, the final information is used to calculate fluency. The hidden information unit is the hidden state module in the transformer structure, which refers to the vector terminology of the intermediate layers of the neural network, representing the source text vector output by the encoder and the vector representation learned by the decoder at the current time step.
[0127] The smoothness information can be calculated as follows: First, the smoothness of the information fused from the encoder end and the smoothness of the information fused from the decoder end are calculated, and the calculation formula is as follows:
[0128]
[0129]
[0130] Among them, H enc (x) represents the encoder output, F enc L represents the smoothness information of the encoder information. enc Let F be the loss function for the language module, and F be the decoder information fluency information. dec The calculation method is similar to that used for calculating the fluency information of the encoder.
[0131] The smoothness decoding loss data resulting from incorporating smoothness information is:
[0132] L F =F enc +F dec
[0133] By combining historical decoding information and incorporating fluency information calculations into the encoder and decoder information, the machine translation model can learn and decode more fluent translations by adding fluency information constraints. In other words, the translation result at the next moment needs to refer to the translation information at previous moments, such as... Figure 6 As shown, the language module needs to combine the history token, which has been decoded from the translation at a historical moment.
[0134] The total loss data is as follows:
[0135] L = L M +L D +L P +L F
[0136] Based on this overall loss function, the machine translation model can be optimized to make the output of the machine translation model more fluent.
[0137] The spoken language translation method disclosed in this embodiment obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. After obtaining the spoken language data, this scheme inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text, thereby achieving translation based on normalized data and improving the accuracy of spoken expression translation. Furthermore, the masking information in the machine translation model allows for the inclusion of masking information during model training, enhancing the ability to predict missing words in the monolingual data input to the machine translation model, thus improving error tolerance. Simultaneously, the addition of fluency information to the machine translation model optimizes the model's output, improving the fluency of the output translation and thus mitigating mistranslation, literal translation, and omissions.
[0138] This embodiment discloses a spoken language translation method, the flowchart of which is as follows: Figure 7 As shown, it includes:
[0139] Step S71: Obtain colloquial language data;
[0140] Step S72: Determine the core words and subject-predicate structure of colloquial language data based on dependency parsing and part-of-speech tagging.
[0141] Step S73: If the colloquial language data contains a subject-predicate structure, determine the attributive-head structure corresponding to the subject-predicate structure;
[0142] Step S74: When there is a structure that satisfies the colloquial modification condition in the subject-predicate structure corresponding to the attributive-head structure, the colloquial language data is normalized, and the structure that satisfies the colloquial modification condition is deleted to obtain the first monolingual data;
[0143] Step S75: Input the first monolingual data into the pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0144] Dependency parsing techniques can identify the dependency relationships between words in a sentence. These dependencies are directional, categorized as dominant and subordinate, with core components typically having multiple subordinate components. By combining colloquial words with part-of-speech tagging and dependency relationships, colloquial expressions in sentences can be safely regularized while preserving the semantics of the original text.
[0145] Take the sentence "Because I believe many brothers and sisters in our team of this company and their brothers and sisters" as an example. The above sentence is an oral sentence, which is determined as oral language data and analyzed based on the dependency syntax analysis technology. As Figure 8 shown, it is the syntax analysis tree obtained by analyzing the oral language data using the dependency syntax analysis technology.
[0146] Among them, the core word is the last "brothers and sisters" in the sentence. Based on this core word, the subject-predicate structure is determined. Since there is a subject-predicate structure in this sentence, it can be clarified that the subject-predicate structures include "I believe many" and "brothers and sisters and theirs".
[0147] Among them, the attributive-middle structures of the first subject-predicate structure include "many companies", "this company", "company team", "this team", "our team", etc. Select the structures that meet the oral modification conditions from them.
[0148] The conditions for meeting the oral modification can be: when a subject-predicate structure includes no less than two attributive-middle structures, and at least one of the attributive-middle structures includes an oral modification word, and there is a right adjunct word after the attributive-middle structure, then delete the oral modification word in the at least one attributive-middle structure and the right adjunct word after the at least one attributive-middle structure.
[0149] Take the above sentence as an example. Among them, for the first subject-predicate structure, the attributive-middle structures that meet the oral modification conditions, that is, the attributive-middle structures with oral modification words and right adjunct words after them, include "this company". Among them, the right adjunct word of "this company" is the "de" after "this company". Then delete the oral modification word in "this company's", that is, "this" before "company", and the right adjunct word, that is, "de" after "company".
[0150] In addition, when determining the relationship between a certain word and a certain word in a sentence, it is determined through dependency syntax analysis. For oral words such as "this" and "our", they are only filtered when they are attributive-middle structures and there are other attributive-middle structure modifiers in the sentence. That is, when there are only oral words such as "this" and "our" as attributive-middle structure modifiers in the sentence, these oral words cannot be filtered. After filtering the oral words, it must be ensured that there is still at least one attributive-middle structure in the sentence.
[0151] Among them, in the attributive-middle relationship phrases, there are various Chinese oral expressions. In the actual application scenarios of Chinese conferences, there are many oral expressions, which will not be elaborated here one by one.
[0152] The spoken language translation method disclosed in this embodiment obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. After obtaining the spoken language data, this scheme inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text, thereby achieving translation based on normalized data and improving the accuracy of spoken expression translation. Furthermore, the masking information in the machine translation model allows for the inclusion of masking information during model training, enhancing the ability to predict missing words in the monolingual data input to the machine translation model, thus improving error tolerance. Simultaneously, the addition of fluency information to the machine translation model optimizes the model's output, improving the fluency of the output translation and thus mitigating mistranslation, literal translation, and omissions.
[0153] This embodiment discloses a spoken language translation system, the structural diagram of which is shown below. Figure 9 As shown, it includes:
[0154] Obtain unit 91, regularize unit 92 and translate unit 93.
[0155] Among them, the acquisition unit 91 is used to acquire colloquial language data;
[0156] The regularization unit 92 is used to regularize colloquial language data based on dependency parsing and part-of-speech tagging to obtain the first monolingual data.
[0157] The translation unit 93 is used to input the first monolingual data into the pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0158] Furthermore, the spoken language translation system disclosed in this embodiment may also include:
[0159] The first training unit is used to train the machine translation model;
[0160] The first training unit is used for:
[0161] Obtain training parameters, which include at least: second monolingual data and corresponding second translation data; train the model based on at least the second monolingual data and the second translation data to obtain a machine translation model.
[0162] Furthermore, the first training unit trains the model based at least on the second monolingual data and the second translated data to obtain a machine translation model, including:
[0163] The first training unit inputs the second monolingual data into the pre-trained anti-smoothing regularization model to obtain the first monolingual colloquial data output by the anti-smoothing regularization model; the model is trained based on the first monolingual colloquial data and the second translation data to obtain the trained machine translation model.
[0164] Furthermore, the spoken language translation system disclosed in this embodiment may also include:
[0165] The second training unit is used to train the anti-smooth regularization model.
[0166] The second training unit is used for:
[0167] Multiple second monolingual colloquial data are obtained; third monolingual data are obtained after normalizing each second monolingual colloquial data; the second monolingual colloquial data and the third monolingual data are used as training data to train the model and obtain the anti-smoothing normalization model.
[0168] Furthermore, the first training unit is used for:
[0169] Key information is determined based on the second monolingual data and the first monolingual colloquial data; masking information is added to the first monolingual colloquial data based on the key information; the model is trained based on the first monolingual colloquial data with masking information and the second translation data to obtain the machine translation model.
[0170] Furthermore, the spoken language translation system disclosed in this embodiment may also include: a model optimization unit, used to determine the translation loss data of the machine translation model; determine the masking loss data generated by adding masking information; determine the fluency decoding loss data generated by incorporating fluency information; determine the overall loss data of the machine translation model based on the translation loss data, the loss data of adding masking information, and the fluency decoding loss data; and optimize the machine translation model based on the overall loss data.
[0171] Furthermore, the regularization unit is used for:
[0172] Based on dependency parsing and part-of-speech tagging, the core words and subject-predicate structure of the colloquial language data are determined; if the colloquial language data has a subject-predicate structure, the corresponding modifier-head structure is determined; when there is a structure in the modifier-head structure corresponding to the subject-predicate structure that satisfies the colloquial modification conditions, the colloquial language data is normalized and the structure that satisfies the colloquial modification conditions is deleted.
[0173] The spoken language translation system disclosed in this embodiment is based on the spoken language translation method disclosed in the above embodiments, and will not be described again here.
[0174] The spoken language translation system disclosed in this embodiment obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. After obtaining the spoken language data, this solution inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text, thereby achieving translation based on normalized data and improving the accuracy of spoken language translation. Furthermore, the masking information in the machine translation model allows for the inclusion of masking information during model training, enhancing the ability to predict missing words in the monolingual data input to the machine translation model, thus improving error tolerance. Simultaneously, incorporating fluency information into the machine translation model optimizes the output of the model, improving the fluency of the output translation and thus mitigating mistranslation, literal translation, and omissions.
[0175] This embodiment discloses an electronic device, the structural schematic diagram of which is shown below. Figure 10 As shown, it includes:
[0176] Processor 101 and storage 102.
[0177] The processor 101 is used to obtain spoken language data; to normalize the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data; and to input the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
[0178] The memory 102 is used to store the program for the processor to execute the above-mentioned processing.
[0179] The electronic device disclosed in this embodiment is based on the spoken language translation method disclosed in the above embodiments, and will not be described again here.
[0180] The electronic device disclosed in this embodiment obtains spoken language data, normalizes the spoken language data based on dependency parsing and part-of-speech tagging to obtain first monolingual data, and inputs the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. This solution, after obtaining spoken language data, inputs the normalized spoken language data into a machine translation model incorporating masking and fluency information to obtain the translated text, thereby achieving translation based on normalized data and improving the accuracy of translating spoken expressions. Furthermore, the masking information in the machine translation model allows for the inclusion of masking information during training, enhancing the ability to predict missing words in the monolingual data input to the machine translation model, thus improving error tolerance. Simultaneously, incorporating fluency information into the machine translation model optimizes its output, improving the fluency of the translated text and thus mitigating mistranslation, literal translation, and omissions.
[0181] This application embodiment also provides a readable storage medium on which a computer program is stored. The computer program is loaded and executed by a processor to implement the steps of the above-described spoken language translation method. The specific implementation process can be referred to the description of the corresponding part of the above embodiment, and will not be repeated in this embodiment.
[0182] This application also proposes a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above-described spoken translation method or spoken translation system. Specific implementation processes can be referred to the descriptions of the corresponding embodiments above, and will not be repeated here.
[0183] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0184] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0185] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0186] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A spoken language translation method, characterized in that, include: Obtain spoken language data; The colloquial language data is regularized using dependency parsing and part-of-speech tagging to obtain first monolingual data. This includes: determining the core words and subject-predicate structure of the colloquial language data based on dependency parsing and part-of-speech tagging; if the colloquial language data has a subject-predicate structure, determining the corresponding modifier-head structure; when the modifier-head structure corresponding to the subject-predicate structure contains a structure that satisfies the colloquial modification condition, the colloquial language data is regularized, and the structure that satisfies the colloquial modification condition is deleted; wherein, the condition for satisfying the colloquial modification condition is: the subject-predicate structure includes at least two modifier-head structures, and at least one modifier-head structure includes a colloquial modifier, and the modifier-head structure is followed by a right-addressable structure word; deleting the structure that satisfies the colloquial modification condition includes: deleting the colloquial modifier in the at least one modifier-head structure and the right-addressable structure word following the at least one modifier-head structure; The first monolingual data is input into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information.
2. The method according to claim 1, characterized in that, Also includes: Train the machine translation model; The training of the machine translation model includes: The training parameters are obtained, and the training parameters include at least: second monolingual data and second translation data corresponding to the second monolingual data; The machine translation model is obtained by training the model based on at least the second monolingual data and the second translated data.
3. The method according to claim 2, characterized in that, The process of training the model based at least on the second monolingual data and the second translated data to obtain the machine translation model includes: The second monolingual data is input into a pre-trained anti-smoothing regularization model to obtain the first monolingual colloquial data output by the anti-smoothing regularization model. The machine translation model is trained based on the first monolingual spoken language data and the second translated text data.
4. The method according to claim 3, characterized in that, Also includes: Training the anti-smooth regularization model; The training of the anti-smooth regularization model includes: Obtain multiple second monolingual colloquialization data; Obtain the third monolingual data after normalizing each second monolingual colloquial data; The second monolingual colloquial data and the third monolingual data are used as training data to train the model, resulting in an anti-smooth regularization model.
5. The method according to claim 3, characterized in that, The process of training a machine translation model based on the first monolingual spoken language data and the second translated text data to obtain the trained model includes: Key information was determined based on the second monolingual data and the first monolingual colloquial data. Based on the key information, add masking information to the first monolingual colloquial data; The machine translation model is obtained by training the first monolingual colloquial data and the second translated data after adding masking information.
6. The method according to claim 5, characterized in that, Also includes: Determine the translation loss data of the machine translation model; Determine the mask loss data resulting from adding mask information; Determine the smoothness decoding loss data resulting from incorporating smoothness information; The overall loss data of the machine translation model is determined based on translation loss data, loss data with added masking information, and fluency decoding loss data. The machine translation model is optimized based on the overall loss data.
7. A spoken language translation system, characterized in that, include: The acquisition unit is used to acquire spoken language data; The standardization unit is used to standardize the colloquial language data based on dependency parsing and part-of-speech tagging to obtain the first monolingual data; The translation unit is used to input the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model. The machine translation model includes at least an encoder incorporating masking information and a decoder incorporating fluency information. The regularization unit is used for: The core words and subject-predicate structure of colloquial language data are determined based on dependency parsing and part-of-speech tagging. If the colloquial language data contains a subject-predicate structure, the corresponding modifier-head structure is determined. When the modifier-head structure corresponding to the subject-predicate structure contains a structure that satisfies the colloquial modification condition, the colloquial language data is regularized, and the structure that satisfies the colloquial modification condition is deleted. The condition for satisfying the colloquial modification condition is: the subject-predicate structure includes at least two modifier-head structures, and at least one modifier-head structure includes a colloquial modifier, and the modifier-head structure is followed by a right-addressing structure. Deleting the structure that satisfies the colloquial modification condition includes: deleting the colloquial modifier in the at least one modifier-head structure and the right-addressing structure following the at least one modifier-head structure.
8. An electronic device, characterized in that, include: Processor, used to acquire spoken language data; The colloquial language data is regularized using dependency parsing and part-of-speech tagging to obtain first monolingual data, including: determining the core words and subject-predicate structure of the colloquial language data based on dependency parsing and part-of-speech tagging; if the colloquial language data has a subject-predicate structure, determining the corresponding attributive-head structure; when the attributive-head structure corresponding to the subject-predicate structure contains a structure that satisfies the colloquial modification condition, the colloquial language data is regularized, and the structure that satisfies the colloquial modification condition is deleted; wherein, the condition for satisfying the colloquial modification is: The subject-predicate structure includes at least two attributive-head structures, and at least one of the attributive-head structures includes a colloquial modifier, and the attributive-head structure is followed by a right-addressable word; deleting the structure that satisfies the colloquial modifier condition includes: deleting the colloquial modifier in the at least one attributive-head structure and the right-addressable word following the at least one attributive-head structure; inputting the first monolingual data into a pre-trained machine translation model to obtain the first translated data output by the machine translation model, wherein the machine translation model includes at least: an encoder incorporating masking information and a decoder incorporating fluency information; The memory is used to store the program used by the processor to execute the above-described processing procedure.
9. A readable storage medium for storing at least one set of instructions; The instruction set is used to be invoked and to perform at least the oral translation method as described in any one of claims 1-6.