Language file processing method and model training method applied to language file processing

Abstract

The embodiment of the invention provides a language file processing method and a model training method applied to language file processing. The method comprises the steps of obtaining a to-be-processed language file; wherein the to-be-processed language file comprises text information of at least one language type used for being displayed on a preset interface; inputting a to-be-processed language file into the first model for processing to obtain a to-be-translated text; inputting the to-be-translated text into a second model for translation to obtain a translation result of the target language type; the translation result is written into the to-be-processed language file, and a processed language file is obtained; wherein the processed language file is used for displaying text information of different language types on a preset interface. The method is used for automatically translating the language file, so that the processing efficiency of the language file is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and the technical field of text translation, in particular to a language file processing method and a model training method applied to language file processing. BACKGROUND

[0002] With the acceleration of globalization, multilingual text information processing is often required. For example, in the scenario of system development, language files are usually configured during system development. These language files enable the system to display text information of different language types in the language files on a preset interface.

[0003] In some technologies, to realize translation of the language types of language files, the text content in the language files is translated by manual operation, thereby obtaining text content of other language types in the language files. In the above technology, the translation by manual operation is low in efficiency.

[0004] Therefore, there is an urgent need for a solution capable of automatically processing language files. SUMMARY

[0005] The language file processing method and the model training method applied to language file processing provided by the embodiments of the present application are used to realize automatic translation processing of language files, thereby improving the processing efficiency of language files.

[0006] In a first aspect, the embodiments of the present application provide a language file processing method, comprising:

[0007] obtaining a language file to be processed; wherein the language file to be processed includes text information of at least one language type to be displayed on a preset interface;

[0008] inputting the language file to be processed into a first model for processing to obtain text to be translated;

[0009] inputting the text to be translated into a second model for translation to obtain a translation result of a target language type; and writing the translation result into the language file to be processed to obtain a processed language file; wherein the processed language file is used to display text information of different language types on the preset interface.

[0010] In a possible implementation, inputting the language file to be processed into the first model for processing to obtain the text to be translated comprises:

[0011] inputting the language file to be processed into the first model for processing to obtain a feature vector of the text information in the language file to be processed; the feature vector is used to represent the context relationship features of the text information in the language file to be processed;

[0012] Based on the first model, the feature vector is processed to obtain the text to be translated.

[0013] In a possible implementation, based on the first model, the feature vector is processed to obtain the text to be translated, including:

[0014] Based on the first model, the feature vector is processed to determine the to-be-processed information of the text information of the language file to be processed; wherein the to-be-processed information includes one or more of the following: string start position, string end position, special character, professional term;

[0015] The to-be-processed information in the language file to be processed is marked respectively to obtain the text to be translated.

[0016] In a possible implementation, the text to be translated is input into the second model for translation to obtain the translation result of the target language type, including:

[0017] The text to be translated is input into the second model, and based on the second model, the first text in the text to be translated is translated to obtain the second text of the target language type; wherein the first text is the text content between the mark of the string start position and the mark of the string end position in the text to be translated;

[0018] Based on the second model, the second text is translated to obtain the third text; wherein the language type of the third text is the same as the language type of the first text;

[0019] The semantic similarity between the third text and the first text is determined, and if it is determined that the semantic similarity is greater than or equal to a preset threshold, the second text is determined as the translation result of the target language type.

[0020] In a possible implementation, the second model is configured with a knowledge base; wherein the knowledge base includes: a professional term knowledge base, and / or a standard specification knowledge base.

[0021] In a possible implementation, the translation result is written into the language file to be processed to obtain the processed language file, including:

[0022] The text to be translated is input into the third model for position calibration processing to obtain target position information; the target position information is used to indicate the position in the language file to be processed where the translation result needs to be written;

[0023] Based on the target position information, the translation result is written into the language file to be processed to obtain the processed language file.

[0024] In a possible implementation, the language file to be processed is obtained, including:

[0025] In response to receiving the code file, a language file to be processed is obtained from an engineering source code path corresponding to the code file, and a target language type corresponding to the language file to be processed is determined; wherein the code file is used to call the language file to be processed from the engineering source code path corresponding to the code file, so as to display text information in the language file to be processed on a preset interface.

[0026] In a possible implementation, the method further includes:

[0027] The target language type is added in the language support list; wherein the language support list indicates a language type supported by the processed language file.

[0028] In response to a selected language type in the language support list, text information corresponding to the selected language type in the processed language file is called and displayed on the preset interface.

[0029] In a second aspect, an embodiment of the present application provides a model training method applied to language file processing, including:

[0030] A training data set is obtained, the training data set including a plurality of language types of language files to be trained, part of text information in the language files to be trained being subjected to mask processing, and the language files to be trained being marked with a plurality of original labels, the plurality of original labels indicating string start positions, string end positions, special characters and professional terms of text information of the language files;

[0031] The initial model is trained based on the training data set to obtain a first model; wherein the first model is the first model provided in the first aspect.

[0032] In a possible implementation, the initial model is trained based on the training data set to obtain the first model, including:

[0033] The language files to be trained are input into the initial model, and part of the text information in the language files to be trained subjected to mask processing is predicted to obtain a feature vector;

[0034] The feature vector is processed based on the initial model to obtain a predicted label of the language files to be trained;

[0035] The initial model is trained based on the predicted label and the original label to obtain the first model.

[0036] In a third aspect, an embodiment of the present application provides a language file processing apparatus, including:

[0037] An obtaining module is configured to obtain a language file to be processed; wherein the language file to be processed includes text information of at least one language type for display on a preset interface.

[0038] a processing module, configured to input a language file to be processed into the first model for processing to obtain a text to be translated;

[0039] The processing module is further configured to input the text to be translated into the second model for translation to obtain a translation result of a target language type; and write the translation result into the language file to be processed to obtain a processed language file; wherein the processed language file is used to display text information of different language types on a preset interface.

[0040] In a possible implementation, the processing module is configured to:

[0041] input the language file to be processed into the first model for processing to obtain a feature vector of text information in the language file to be processed; and the feature vector is used to represent a context relationship feature of the text information in the language file to be processed.

[0042] The processing module is configured to process the feature vector based on the first model to obtain the text to be translated.

[0043] In a possible implementation, the processing module is configured to:

[0044] The processing module is configured to process the feature vector based on the first model to determine to-be-processed information of the text information in the language file to be processed; wherein the to-be-processed information includes one or more of the following: a string start position, a string end position, a special character, and a professional term.

[0045] The processing module is configured to mark the to-be-processed information in the language file to be processed to obtain the text to be translated.

[0046] In a possible implementation, the processing module is configured to:

[0047] The processing module is configured to input the text to be translated into the second model, and translate a first text in the text to be translated based on the second model to obtain a second text of a target language type; wherein the first text is text content between a mark of a string start position and a mark of a string end position in the text to be translated.

[0048] The processing module is configured to translate the second text based on the second model to obtain a third text; wherein a language type of the third text is the same as a language type of the first text.

[0049] Determine the semantic similarity between the third text and the first text, and if the semantic similarity is greater than or equal to a preset threshold, determine the second text as the translation result of the target language type.

[0050] In a possible implementation, the second model is configured with a knowledge base; wherein the knowledge base comprises: a professional term knowledge base, and / or a standard specification knowledge base.

[0051] In a possible implementation, the translation result is written into the language file to be processed to obtain a processed language file, and the processing module is configured to:

[0052] The text to be translated is input into the third model to perform position calibration processing to obtain target position information; the target position information is used to indicate a position in the language file to be processed where the translation result needs to be written;

[0053] Based on the target position information, the translation result is written into the language file to be processed to obtain a processed language file.

[0054] In a possible implementation, the language file to be processed is obtained, and the obtaining module is configured to:

[0055] In response to receiving the code file, the language file to be processed is obtained from the engineering source code path corresponding to the code file; and a target language type corresponding to the language file to be processed is determined; wherein the code file is used to call the language file to be processed from the engineering source code path corresponding to the code file, so as to display text information in the language file to be processed on a preset interface.

[0056] In a possible implementation, the processing module is further configured to:

[0057] In the language support list, the target language type is added; wherein the language support list indicates a language type supported by the processed language file;

[0058] In response to a selected language type in the language support list, text information corresponding to the selected language type in the processed language file is called to be displayed on a preset interface.

[0059] In a fourth aspect, the embodiments of the present application provide a model training device applied to language file processing, comprising:

[0060] The first training module is configured to obtain a training data set, wherein the training data set comprises a plurality of language types of language files to be trained, part of the text information in the language files to be trained is subjected to mask processing, and the language files to be trained are marked with a plurality of original labels, and the plurality of original labels represent the string start position, the string end position, the special character and the professional term of the text information of the language file.

[0061] The second training module is configured to train the initial model based on the training data set to obtain a first model.

[0062] In a possible implementation, the initial model is trained based on the training data set to obtain the first model, and the second training module is configured to:

[0063] The language file to be trained is input into the initial model, and the part of the text information in the language file to be trained that has been subjected to the mask processing is predicted to obtain a feature vector;

[0064] The feature vector is processed based on the initial model to obtain a predicted label of the language file to be trained;

[0065] The initial model is trained based on the predicted label and the original label to obtain the first model.

[0066] In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor.

[0067] The memory stores computer execution instructions.

[0068] The processor executes the computer execution instructions stored in the memory, so that the processor executes the method provided in the first aspect and / or the second aspect.

[0069] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the method provided in the first aspect and / or the second aspect.

[0070] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the computer program implements the method provided in the first aspect and / or the second aspect.

[0071] The language file processing method and the model training method applied to language file processing provided in the embodiments of the present application obtain a language file to be processed, input the language file to be processed into a first model for processing to obtain a text to be translated, input the text to be translated into a second model for translation to obtain a translation result, and write the translation result into the language file to be processed to obtain a processed language file. The method does not need manual translation, and improves the translation processing efficiency of the language file. BRIEF DESCRIPTION OF DRAWINGS

[0072] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0073] Figure 1 Flowchart of the processing method of the language file provided by the present application Figure 1

[0074] Figure 2 Flowchart of the processing method of the language file provided by the present application Figure 2

[0075] Figure 3 Flowchart of the processing method of the language file provided by the present application Figure 3

[0076] Figure 4 Flowchart of the translation processing of the exemplary language file

[0077] Figure 5 Flowchart of the model training method provided by the present application applied to language file processing

[0078] Figure 6 Structural diagram of the language file processing device provided by the present application

[0079] Figure 7 Structural diagram of the model training device provided by the present application applied to language file processing

[0080] Figure 8 Structural diagram of the electronic device provided by the present application

[0081] The specific embodiments of the present application have been shown in the above-described drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application concept in any way, but to illustrate the present application concept to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0082] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, the same numbers refer to the same or similar elements throughout the drawings, unless otherwise represented. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they only represent examples of apparatus and methods consistent with some aspects of the present application, as detailed in the appended claims.

[0083] First, the terms involved in the present application are explained:

[0084] ​​​Language file: Also known as language resource file; refers to a code file containing text information displayed on a preset interface. For example, taking the Basic Input / Output System (BIOS) as an example, the preset interface in the BIOS can display text information. This process is usually implemented based on calling a language resource file. In the language resource file, text information of multiple language types is stored in a specific format. These text information are used to display on the preset interface. For example, a language file can be a Unicode string definition file, referred to as a UNI file; wherein Unicode refers to a unified character encoding standard for characters of various languages.

[0085] With the acceleration of globalization, support for multiple languages ​​is often necessary during application or system development. Application or system default interfaces typically display text information. Furthermore, the language information on these default interfaces can be switched. This process typically involves calling a language file and displaying text information in different languages ​​on the default interface.

[0086] This requires multilingual text information processing in the language file. For example, the English text information originally contained in the language file must be translated to obtain Chinese text information.

[0087] In some embodiments, text information in a language file is translated by manual translation to obtain text information in other language types.

[0088] In the above embodiment, manual translation has the technical problem of low efficiency in processing the translation of language files.

[0089] The language file processing method provided in the present application obtains a language file to be processed, inputs the language file to be processed into a first model for processing, and obtains a text to be translated; then inputs the text to be translated into a second model for translation to obtain a translation result, and writes the translation result into the language file to be processed to obtain a processed language file. This method eliminates the need for manual translation, thereby improving the translation processing efficiency of language files.

[0090] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0091] Figure 1 Schematic diagram of the process of processing language files provided in this applicationFigure 1 As shown in Figure 1 The method comprises the following steps:

[0092] Step 101. Obtain a language file to be processed.

[0093] The language file to be processed includes text information of at least one language type for display on a preset interface.

[0094] An exemplary language file to be processed is obtained. The language file to be processed includes text information. The text information can be text information of multiple language types, and the text information can be used for display on a preset interface. The preset interface can be an interface of an application program or an interface developed by a system.

[0095] For example, taking a BIOS interface scenario as an example, in the BIOS interface scenario, the language file to be processed can be a UNI file. In the UNI file, there is text information of at least one language type that needs to be displayed on the BIOS interface.

[0096] Specifically, in the UNI file, each option name corresponds to an option identifier, which can be denoted as "#string". The "#string" option identifier is followed by a language identifier, which can be denoted as "#language". The language identifier has a language label, which represents a language type, such as "eng" representing English. After the language label, the option name is recorded, and the string in the language type of the language label, i.e., the text information.

[0097] If the "#string" option identifier is followed by multiple language types, the "#string" option identifier is followed by multiple "#language" language identifiers, and each language identifier corresponds to a language label of a language type.

[0098] Step 102. Input the language file to be processed into a first model for processing to obtain a text to be translated.

[0099] The first model can include a Bidirectional Encoder Representations from Transformers (BERT) and a Bidirectional Long Short-Term Memory-Conditional Random Field (BiLSTM-CRF).

[0100] The language file to be processed is input into the first model for analysis and processing, so as to obtain the text to be translated. Based on the BERT network of the first model, the features of the text information in the language file to be processed are extracted from the language file to be processed; based on the BiLSTM-CRF of the first model, the sequence labeling is performed according to the features of the text information in the language file to be processed, and the boundary information and the like of the text information is accurately divided. Based on the divided boundary information and the like of the text information, the text to be translated which needs to be processed subsequently is determined.

[0101] Step 103. The text to be translated is input into the second model for translation, so as to obtain the translation result of the target language type; and the translation result is written into the language file to be processed, so as to obtain the processed language file.

[0102] The processed language file is used to display the text information of different language types on the preset interface.

[0103] For example, the text to be translated obtained in step 102 is input into the second model for translation, so as to obtain the translation result corresponding to the text to be translated.

[0104] Further, the translation result is written into the language file to be processed again, so as to obtain the processed language file. It can be understood that, in the processed language file, in addition to the text information of at least one language type included in the language file to be processed, the text information of the target language type obtained after translation is also included.

[0105] The processed language file can be used to display the text information of different language types on the preset interface.

[0106] Optionally, the second model can be a trained artificial intelligence (AI) translation intelligent agent. It refers to an intelligent system capable of autonomous learning, understanding and converting different languages based on artificial intelligence technology. The technical core is to realize the automatic translation of cross-language text and the like information through a deep learning algorithm and natural language processing (NLP) technology.

[0107] The language file processing method provided by the embodiment of the application acquires a language file containing text information to be displayed on a preset interface, inputs the language file into a model for processing, obtains text information to be translated, inputs the text information into a model for translating text, performs translation to obtain a translation result, and writes the translation result into the original language file again, so as to obtain a language file containing a target translation language type. The efficiency of the translation processing of the language file is improved.

[0108] Figure 2Flowchart of processing method of language file provided by language file of the present application Figure 2 As shown in Figure 2 the embodiment is based on Figure 1 the embodiment, the processing method of the language file is described in detail, and step 102 can specifically include:

[0109] Step 201. Input the language file to be processed into the first model for processing to obtain the feature vector of the text information in the language file to be processed.

[0110] The feature vector is used to represent the context relationship feature of the text information in the language file to be processed.

[0111] For example, the language file to be processed is input into the first model for processing. Based on the BERT network in the first model, the language file to be processed is analyzed and processed to obtain the context relationship feature of the text information in the language file to be processed, i.e. the feature vector.

[0112] For example, in the UNI file, the BERT network in the first model analyzes the UNI file and identifies the language definition block in the UNI file to obtain the context embedding vector of each token, i.e. the feature vector.

[0113] Step 202. Based on the first model, the feature vector is processed to obtain the text to be translated.

[0114] For example, based on the BiLSTM-CRF network in the first model, the feature vector is sequentially labeled to identify the boundary containing the text information, and then the text to be translated is obtained.

[0115] For example, in the UNI file, there can be multiple segments of text information; by using the BiLSTM-CRF network, the feature vector is sequentially labeled to identify the boundary of the multiple segments of text information.

[0116] Specifically, when identifying the boundary containing the text information, other information to be processed of the language file to be processed can also be identified. For example, in the language file to be processed, there can be many special characters and professional terms in addition to the text information. These information needs to be kept in the original text and not translated during translation.

[0117] In the above embodiment, by processing the language file to be processed by the first model, the context relationship of the text information in the language file can be identified first, and based on the context relationship of the text information, it is determined which text information needs to be translated. Based on the above process, the grammatical structure in the language file to be processed and the context relationship between the text information can be accurately identified, and the text information to be translated can be accurately extracted, which on the one hand improves the efficiency of translation, and on the other hand lays a foundation for the subsequent translation process of text information, and can improve the accuracy of the translation result.

[0118] Specifically, in an example, the process of processing the feature vector by the first model in step 202 to obtain the text to be translated can specifically include:

[0119] Step 2021. Based on the first model, processing the feature vector to determine the to-be-processed information of the text information of the language file to be processed.

[0120] Among them, the to-be-processed information includes one or more of the following: string start position, string end position, special character, professional term.

[0121] For example, based on the BiLSTM-CRF network in the first model, the feature vector output by the BERT network is received, the feature vector is processed, and the to-be-processed information of the text information of the language file to be processed is determined.

[0122] Among them, the to-be-processed information can include: string start position, string end position, special character and professional term.

[0123] For example, the string start position refers to the position where the text information of the language file to be processed starts, and the string end position refers to the position where the text information of the language file to be processed ends. It can be understood that in the subsequent translation process, the text information between the string start position and the string end position should be translated.

[0124] For example, the special character refers to a character with special meaning in the language file to be processed, which can be used to indicate line feed or separation or placeholder. For example: "%d" or "%x".

[0125] For example, the professional term refers to the professional information defined in the language file to be processed. For example, the language label "eng" of the language identifier "#language" in the UNI file can be regarded as a professional term.

[0126] In the actual translation process, these special characters and professional terms do not need to be translated and need to be kept in the original text.

[0127] Therefore, identifying the string start position, the string end position, the special character and the professional term can lay a foundation for subsequent marking.

[0128] Step 2022. The to-be-processed information in the to-be-processed language file is marked respectively to obtain the to-be-translated text.

[0129] For example, the corresponding positions of the to-be-processed information identified above are marked respectively to obtain the to-be-translated text.

[0130] It can be understood that the to-be-translated text includes a plurality of position marks, in which the string start position, the string end position, the special character and the professional term are marked respectively.

[0131] In the above example, by processing the feature vector, the relevant to-be-processed information in the to-be-processed language file can be obtained, which includes the string start position, the string end position, the special character and the professional term. In actual text information translation, only the text information of the string needs to be translated. Therefore, the string start position and the string end position are determined and marked, the structure characteristics of the language file are considered, the text information part that needs to be translated is accurately marked, other contents that do not need to be translated are not translated, and the efficiency of the language file translation processing is improved.

[0132] As can be known from the foregoing example, the to-be-translated text is translated by the second model. Figure 3 The flowchart of the language file processing method provided in the present application is shown in Figure 3 In an example, as shown in Figure 4 The process of translating the to-be-translated text by the second model can specifically include:

[0133] Step 301. The to-be-translated text is input into the second model, and the first text in the to-be-translated text is translated based on the second model to obtain a second text of a target language type.

[0134] The first text is the text content between the mark of the string start position and the mark of the string end position in the to-be-translated text.

[0135] For example, the to-be-translated text is input into the second model, and the second model translates the to-be-translated text.

[0136] Specifically, the first text in the to-be-translated text is extracted based on the second model, and the first text is translated to obtain a second text of a target language type.

[0137] The first text refers to the text content between the mark of the starting position of the string and the mark of the ending position of the string in the text to be translated, that is, the text information that needs to be translated mentioned in the foregoing example.

[0138] In combination with the foregoing embodiments, the second model can be a trained AI translation agent. To better translate language files, in an example, the second model is configured with a knowledge base. The knowledge base includes a professional term knowledge base and / or a standard specification knowledge base.

[0139] For example, the AI translation agent is configured with a knowledge base of a specific field, such as a professional term knowledge base and / or a standard specification knowledge base.

[0140] For example, in the BIOS scenario, the professional term knowledge base can be a BIOS term library. In the BIOS term library, information such as original text terms, standard translations, and technical explanations can be included, which can allow the AI translation agent to call the BIOS term library to translate the first text in the text to be translated.

[0141] For example, in the BIOS scenario, the standard specification knowledge base can be a Unified Extensible Firmware Interface (UEFI) standard specification library. In the UEFI standard specification library, standard translations of official terms, data structures, and protocol processes can be included. The AI translation agent can be allowed to call the UEFI standard specification library to translate the first text in the text to be translated.

[0142] In the foregoing example, by configuring the second model with a knowledge base of a specific field, the text information that needs to be translated in the text to be translated can be more accurately translated, thereby improving the accuracy of the translation result.

[0143] Step 302. Translate the second text based on the second model to obtain a third text.

[0144] The language type of the third text is the same as that of the first text.

[0145] For example, based on the second model, the second text translated into the target language type is translated again to obtain a third text. The language type of the third text is the same as that of the first text.

[0146] It can be understood that based on the second model, after the second text is translated, the original language type is translated again to obtain a third text.

[0147] Step 303. Determine the semantic similarity between the third text and the first text, and if it is determined that the semantic similarity is greater than or equal to a preset threshold, determine that the second text is the translation result of the target language type.

[0148] For example, the semantic similarity between the third text and the first text is calculated.

[0149] Specifically, the third text and the first text are segmented and feature extracted respectively, and the cosine similarity or Euclidean distance is calculated to obtain the semantic similarity.

[0150] Further, if it is determined that the semantic similarity is greater than or equal to the preset threshold, it means that the semantics of the second text obtained by translation does not deviate from the semantics of the first text in the original text to be translated. Therefore, the second text can be determined as the translation result of the target language type.

[0151] In the above embodiment, the first text in the text to be translated is translated by the second model to obtain the second text, and the second text is retranslated into the third text of the same language type as the first text. The semantic similarity between the first text and the third text is determined, and if the semantic similarity is greater than the preset threshold, it means that the semantics of the second text obtained by translation is consistent with the semantics of the original text, so the second text is determined as the translation result. By translating through the second model and rolling back to the original text, the accuracy of the translation result can be improved.

[0152] In combination with the foregoing embodiments, after obtaining the translation result, the translation result needs to be written back to the language file to be processed to obtain the processed language file. In one example, the method can specifically include:

[0153] The text to be translated is input into the third model for position calibration processing to obtain target position information.

[0154] The target position information is used to indicate the position in the language file to be processed where the translation result needs to be written.

[0155] For example, the third model can be a Transformer network with a self-attention mechanism. The text to be translated is input into the Transformer network for position calibration, and the target position information can be obtained. The target position information is used to indicate the position in the language file to be processed where the translation result needs to be written.

[0156] Specifically, the Transformer network can calculate a position matrix P based on the original Query, Key and Value during the calculation of the attention score, so as to represent the relative position importance weight of the token in the string. For example, the position matrix P[i,j] represents the weight matrix of the importance of the token in the string position i and the importance of the position j. It can be understood that if the token in the string position i is more important, the translation result corresponding to the token should be written in the position i.

[0157] Based on the weight of the relative position of the token represented by the position matrix calculated by the Transformer network, the target position information is determined to realize the position calibration process.

[0158] Based on the target position information, the translation result is written into the language file to be processed to obtain the processed language file.

[0159] For example, based on the position matrix in the Transformer network, the accurate position of the original text information (the first text in the text to be translated) in the language file to be processed, i.e. the target position information, can be determined. Based on the target position information, the translation result is written into the corresponding position of the original text information (the first text in the text to be translated) in the language file to be processed, so as to obtain the processed language file.

[0160] Figure 4 For example, the flowchart of the translation process of the language file is shown. As shown in Figure 4 , the language file to be processed is input into the BERT network in the first model Figure 4 ("BERT") to obtain the feature vector; input into the BiLSTM-CRF network in the first model Figure 4 ("BiLSTM-CRF") to mark the information to be processed, i.e. the start position of the string Figure 4 ("TAG_START"), the end position of the string Figure 4 ("TAG_END"), special characters Figure 4 ("ESCAPE"), professional terms Figure 4 ("TERM") to obtain the text to be translated; input into the second model Figure 4 ("AI translation") for translation processing to obtain the translation result; input the text to be translated into the third model Figure 5The target position information is obtained, and the translation result is written into the position indicated by the target position information in the language file to be processed, to obtain a processed language file.

[0161] In the above examples, by adding the position matrix in the self-attention mechanism in the third model, accurate position calibration processing can be realized, and the position where the translation result should be written in the language file to be processed is determined, and the writing is performed. It can be ensured that the translation result is accurately written in the corresponding position, and the accurate rewriting of the text information of the target language type in the processed language file is ensured.

[0162] On the basis of any of the preceding embodiments, in an example, in the process of obtaining the language file to be processed, the following steps can be specifically included:

[0163] In response to receiving the code file, the language file to be processed is obtained from the engineering source code path corresponding to the code file, and the target language type corresponding to the language file to be processed is determined.

[0164] The code file is used to call the language file to be processed from the engineering source code path corresponding to the code file, so as to display the text information in the language file to be processed on the preset interface.

[0165] For example, a user or a developer selects to upload a code file on a system interface. In response to receiving the code file, the language file to be processed is obtained from the engineering source code path corresponding to the code file. The code file includes executable code instructions, which are used to call the language file to be processed from the corresponding engineering source code path, so as to display the text information in the language file to be processed on the preset interface.

[0166] Further, after obtaining the language file to be processed, the user or the developer can also select a target language type to be translated for the language file to be processed. It can be understood that the target language type refers to the language type of the translation result to be translated for the text to be translated in the language file to be processed.

[0167] In the above examples, when obtaining the language file to be processed, in response to the code file uploaded by the user, all the language files under the path of the code file can be traversed to ensure the integrity of the language file translation. It prevents the text information in part of the language file from not being translated, so that the text information of the target language type cannot be displayed on the preset interface.

[0168] On the basis of any of the preceding embodiments, in an example, the method can further include:

[0169] In the language support list, a target language type is added. The language support list indicates a language type supported by the processed language file.

[0170] For example, after obtaining the processed language file, the target language type is added in the language support list. The language support list includes a plurality of language types. It can be understood that the language types included in the language support list are the language types supported by the processed language file.

[0171] For example, the language types supported by the language file to be processed include Chinese and English, and the language support list includes Chinese and English. After translation processing, the language types supported by the obtained processed language file include Chinese, English, and Japanese, and Japanese is added in the language support list.

[0172] In response to the selected language type in the language support list, the text information corresponding to the selected language type in the processed language file is displayed on the preset interface.

[0173] For example, the user selects Chinese in the language support list. The text information of Chinese in the processed language file is displayed on the preset interface.

[0174] For example, the user selects Chinese in the language support list. The text information of Chinese in the processed language file is displayed on the preset interface.

[0175] In the above example, by updating the supported target language types in the language file to be processed in real time, the user can select a language type, and the text information corresponding to the selected language type is displayed on the preset interface in real time, so that one-key switching after compilation is realized.

[0176] The language file processing method provided by the embodiment of the application includes obtaining a language file to be processed, inputting the language file to be processed into a first model for processing to obtain a text to be translated, inputting the text to be translated into a second model for translation to obtain a translation result, and writing the translation result into the language file to be processed to obtain a processed language file. The method does not need manual translation, and improves the translation processing efficiency of the language file.

[0177] The application further provides a model training method applied to language file processing. Figure 5 As shown in the flowchart of the model training method applied to language file processing provided by the application, Examples of original tags the method includes:

[0178] Step 501. Obtain a training data set, the training data set including a plurality of language types of language files to be trained, part of the text information in the language files to be trained being subjected to mask processing, and the language files to be trained being marked with a plurality of original labels, the plurality of original labels representing string start positions, string end positions, special characters and professional terms of the text information of the language files.

[0179] For example, in order to train the model, a training data set needs to be constructed. The training data set is obtained, wherein the training data set includes a plurality of language files to be trained. The language files to be trained have text information of a plurality of language types.

[0180] Optionally, the language files to be trained can be different kinds of language files. For example, the language files to be processed, the processed language files or the language files updated according to user demand mentioned in the foregoing embodiments. In this way, the training data set can be continuously updated and the model can be continuously trained during the translation processing of the language files.

[0181] Further, part of the text information in the language files to be trained is randomly selected for mask processing. The mask processing refers to controlling the range of information processing by "masking" or "marking" specific elements. For example, a binary mask matrix can be used to operate with target data so that the model only pays attention to information at specific positions during training.

[0182] Further, the language files to be trained have a plurality of original labels. The original labels are used to represent the string start positions, string end positions, special characters and professional terms of the text information in the language files to be trained.

[0183] Optionally, the original labels in the language files to be trained can refer to the labels shown in Table 1.

[0184] Table 1: Example of original label table

[0185] Functional description of original tags TAG START Used to mark the start of a string TAG END Used to mark the end of a string ESCAPE Used to mark special characters TERM Used to mark professional terms that need to be preserved Figure 6

[0186] Step 502. Train the initial model based on the training data set to obtain a first model.

[0187] The first model is the first model provided in the foregoing embodiments.

[0188] For example, the initial model is trained based on the language files to be trained in the training data set. The initial model can include a BERT network and a BiLSTM-CRF network.

[0189] The BERT network and the BiLSTM-CRF network in the initial model are jointly iteratively trained based on the training data set. When the number of times of performing the iterative training reaches a preset threshold, a first model is obtained. The first model is used to process a language file to be processed to obtain a text to be translated.

[0190] The model training method applied to language file processing provided in the embodiments of the present application comprises the following steps: constructing a training data set, wherein the training data set comprises a language file to be trained, part of the text information in the language file to be trained is subjected to a mask processing, and the part of the text information is marked with an original label; and training a first model based on the training data set to obtain a trained first model. In the foregoing manner, the training of the first model can be implemented, so that the first model can quickly and efficiently obtain a text to be translated in the application process.

[0191] On the basis of the foregoing embodiments, in an example, the process of step 502 of training can further comprise the following steps.

[0192] Step 5021. The language file to be trained is input into an initial model, and part of the text information in the language file to be trained that is subjected to a mask processing is predicted to obtain a feature vector.

[0193] In an example, the language file to be trained is input into the initial model, and the BERT network of the initial model predicts part of the text information in the language file to be trained that is subjected to a mask processing.

[0194] The text information before the mask processing is taken as a label in the training process of the BERT network, and the text information predicted by the BERT network and the text information before the mask processing are used to train the BERT network of the initial model.

[0195] Further, the BERT network of the initial model predicts the part of the text information subjected to the mask processing to further obtain a feature vector of the language file to be processed.

[0196] Step 5022. The feature vector is processed based on the initial model to obtain a predicted label of the language file to be trained.

[0197] In an example, the BiLSTM-CRF network of the initial model is used to identify and process the feature vector to obtain the predicted label of the language file to be trained. The predicted label is a string start position, a string end position, a special character and a professional term of the text information in the language file to be trained that are predicted by the BiLSTM-CRF network of the initial model.

[0198] Step 5023. The initial model is trained based on the predicted label and the original label to obtain a first model.

[0199] For example, the initial model based BiLSTM-CRF network is trained based on the predicted label of the language file to be trained obtained by the initial model based BiLSTM-CRF network and the original label of the language file to be trained.

[0200] Specifically, the value of the loss function is determined according to the predicted label and the original label, and the parameters in the BiLSTM-CRF network are adjusted in the training process to make the value of the loss function converge, so as to realize the training of the BiLSTM-CRF network in the initial model.

[0201] Based on the training of the BERT network and the BiLSTM-CRF network in the initial model, the first model provided in the foregoing embodiments is obtained.

[0202] In the foregoing example, the initial model is used to predict the masked part in the language file to be trained, so as to obtain the feature vector; the feature vector is processed based on the initial model to obtain the predicted label of the language file to be trained; and the model is trained based on the predicted label and the original label to obtain the first model. The model can learn the grammatical structure of the language file, and can identify the boundary of the text information, special characters and professional terms. Furthermore, in the application process, the boundary of the text information is accurately determined based on the grammatical structure of the language file, and the text to be translated is accurately extracted.

[0203] Figure 6 The structure diagram of the language file processing apparatus provided in the present application is shown in Figure 7 The language file processing apparatus 60 provided in the present embodiment includes:

[0204] The acquisition module 601 is configured to acquire a language file to be processed; wherein the language file to be processed includes text information of at least one language type to be displayed on a preset interface.

[0205] The processing module 602 is configured to input the language file to be processed into a first model for processing to obtain text to be translated.

[0206] The processing module 602 is further configured to input the text to be translated into a second model for translation to obtain a translation result of a target language type; and write the translation result into the language file to be processed to obtain a processed language file; wherein the processed language file is used to display text information of different language types on the preset interface.

[0207] In a possible implementation, the processing module 602 is configured to:

[0208] inputting the language file to be processed into the first model to obtain a feature vector of the text information in the language file to be processed; the feature vector is used to represent a context relationship feature of the text information in the language file to be processed;

[0209] processing the feature vector based on the first model to obtain the text to be translated.

[0210] In a possible implementation, the feature vector is processed based on the first model to obtain the text to be translated, and the processing module 602 is configured to:

[0211] processing the feature vector based on the first model to determine to-be-processed information of the text information in the language file to be processed; the to-be-processed information includes one or more of the following: a string start position, a string end position, a special character, and a professional term;

[0212] labeling the to-be-processed information in the language file to be processed respectively to obtain the text to be translated.

[0213] In a possible implementation, the text to be translated is input into the second model to obtain a translation result of a target language type, and the processing module 602 is configured to:

[0214] inputting the text to be translated into the second model and translating a first text in the text to be translated based on the second model to obtain a second text of the target language type; the first text is text content between a label of a string start position and a label of a string end position in the text to be translated;

[0215] translating the second text based on the second model to obtain a third text; the language type of the third text is the same as that of the first text;

[0216] determining a semantic similarity between the third text and the first text, and determining that the second text is the translation result of the target language type if the semantic similarity is greater than or equal to a preset threshold.

[0217] In a possible implementation, the second model is configured with a knowledge base; the knowledge base includes a professional term knowledge base and / or a standard specification knowledge base.

[0218] In a possible implementation, the translation result is written into the language file to be processed to obtain a processed language file, and the processing module 602 is configured to:

[0219] inputting the text to be translated into a third model to perform position calibration processing to obtain target position information; the target position information is used to indicate a position in the language file to be processed where the translation result needs to be written;

[0220] Based on the target position information, the translation result is written into the language file to be processed to obtain a processed language file.

[0221] In a possible implementation, the language file to be processed is obtained, and the obtaining module 601 is configured to:

[0222] In response to receiving the code file, the language file to be processed is obtained from the project source code path corresponding to the code file, and the target language type corresponding to the language file to be processed is determined; wherein the code file is used to call the language file to be processed from the project source code path corresponding to the code file, so as to display the text information in the language file to be processed on a preset interface.

[0223] In a possible implementation, the processing module 602 is further configured to:

[0224] In the language support list, the target language type is added; wherein the language support list indicates the language types supported by the processed language file.

[0225] In response to the selected language type in the language support list, the text information corresponding to the selected language type in the processed language file is called to be displayed on the preset interface.

[0226] The language file processing apparatus provided in this embodiment can execute the method provided in the method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0227] Figure 7 The structural schematic diagram of the model training apparatus for language file processing provided in this application is shown in FIG. 7, and the model training apparatus 70 provided in this embodiment includes: Figure 8

[0228] The first training module 701 is configured to obtain a training data set, the training data set including a plurality of language files to be trained of multiple language types, part of the text information in the language files to be trained being subjected to mask processing, and the language files to be trained being marked with a plurality of original labels, the plurality of original labels indicating the string start position, the string end position, special characters and professional terms of the text information of the language file.

[0229] The second training module 702 is configured to train an initial model based on the training data set to obtain a first model; wherein the first model is the first model provided in the first aspect.

[0230] In a possible implementation, the initial model is trained based on the training data set to obtain the first model, and the second training module 702 is configured to:

[0231] ​The language file to be trained is input to the initial model, and the part of the text information in the language file to be trained that has been subjected to mask processing is predicted to obtain a feature vector;

[0232] Based on the initial model, the feature vector is processed to obtain a predicted label of the language file to be trained.

[0233] Based on the predicted label and the original label, the initial model is trained to obtain a first model.

[0234] The model training device for language file processing provided in the embodiment can execute the method provided in the method embodiment, and has similar implementation principles and technical effects, which will not be described here.

[0235] Figure 8 The structure of the electronic device provided in the present application is shown in the figure. ​ As shown, the electronic device 80 provided in the embodiment includes at least one processor 801 and a memory 802. Optionally, the electronic device 80 further includes a communication component 803. The processor 801, the memory 802, and the communication component 803 are connected through a bus 804.

[0236] In the specific implementation process, the at least one processor 801 executes the computer execution instructions stored in the memory 802, so that the at least one processor 801 executes the method described above.

[0237] The specific implementation process of the processor 801 can refer to the method embodiment described above, which has similar implementation principles and technical effects, and will not be described here.

[0238] In the above embodiment, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The steps of the method disclosed in the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0239] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0240] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0241] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described above.

[0242] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the method described above is implemented.

[0243] The readable storage medium described above can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0244] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0245] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0246] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0247] In addition, each functional unit in various embodiments of the application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0248] If the function is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiment methods of the application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0249] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction-related hardware. The aforementioned program can be stored in a computer readable storage medium. The program executes the steps including the above-mentioned method embodiments when executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0250] Finally, it should be noted that those skilled in the art, after considering the specification and practicing the application disclosed herein, will easily think of other embodiments of the application. The application is intended to cover any variations, uses, or adaptations of the application that follow the general principles of the application and include common knowledge or conventional techniques in the art that are not disclosed by the application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the application is only limited by the appended claims.

Claims

1. A method for processing a language file, characterized in that: include: Obtaining a language file to be processed; wherein the language file to be processed includes text information of at least one language type for display on a preset interface; Inputting the language file to be processed into the first model for processing to obtain a text to be translated; The text to be translated is input into the second model for translation to obtain a translation result of the target language type; and the translation result is written into the language file to be processed to obtain a processed language file; wherein the processed language file is text information of different language types displayed on a preset interface.

2. The method according to claim 1, characterized in that Inputting the language file to be processed into the first model for processing to obtain the text to be translated, including: Inputting the language file to be processed into a first model for processing to obtain a feature vector of the text information in the language file to be processed; the feature vector is used to represent the contextual relationship characteristics of the text information in the language file to be processed; The feature vector is processed based on the first model to obtain the text to be translated.

3. The method according to claim 2, characterized in that Processing the feature vector based on the first model to obtain the text to be translated includes: Based on the first model, the feature vector is processed to determine information to be processed of the text information in the language file to be processed; wherein the information to be processed includes one or more of the following: a starting position of a string, an ending position of a string, special characters, and professional terms; The information to be processed in the language file to be processed is marked respectively to obtain the text to be translated.

4. The method according to claim 1, wherein The text to be translated is input into the second model for translation to obtain a translation result in the target language type, including: Inputting the text to be translated into a second model, and translating a first text in the text to be translated based on the second model to obtain a second text in the target language; wherein the first text is the text content between a marker at the start position of a character string and a marker at the end position of the character string in the text to be translated; Translating the second text based on the second model to obtain a third text; wherein the language type of the third text is the same as the language type of the first text; The semantic similarity between the third text and the first text is determined, and if it is determined that the semantic similarity is greater than or equal to a preset threshold, the second text is determined to be a translation result of the target language type.

5. The method according to claim 4, characterized in that The second model is configured with a knowledge base; wherein the knowledge base includes: a professional terminology knowledge base, and / or a standard specification knowledge base.

6. The method according to claim 1, characterized in that Writing the translation result into the language file to be processed to obtain a processed language file includes: Inputting the text to be translated into a third model and performing position calibration processing to obtain target position information; the target position information is used to indicate the position where the translation result needs to be written in the language file to be processed; Based on the target location information, the translation result is written into the language file to be processed to obtain a processed language file.

7. The method according to any one of claims 1 to 6, characterized in that Get the language files to be processed, including: In response to receiving the code file, a language file to be processed is obtained from a project source code path corresponding to the code file; and the target language type corresponding to the language file to be processed is determined; wherein the code file is used to call the language file to be processed from the project source code path corresponding to the code file, so as to display text information in the language file to be processed on a preset interface.

8. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Adding the target language type to the language support list; wherein the language support list includes the language types supported by the processed language file; In response to a language type selected in the language support list, text information corresponding to the selected language type in the processed language file is called and displayed on a preset interface.

9. A model training method for language file processing, characterized in that: include: Obtaining a training data set, the training data set including language files to be trained in multiple language types, wherein a portion of text information in the language files to be trained is masked, and the language files to be trained are marked with a plurality of original labels, wherein the plurality of original labels represent a string start position, a string end position, special characters, and professional terms in the text information in the language files; The initial model is trained based on the training data set to obtain a first model; wherein the first model is the first model according to any one of claims 1 to 8.

10. The method according to claim 9, characterized in that Training the initial model based on the training data set to obtain a first model includes: Inputting the language file to be trained into the initial model, predicting a portion of text information in the language file to be trained that has been masked, and obtaining a feature vector; Based on the initial model, the feature vector is processed to obtain a predicted label of the language file to be trained; The initial model is trained based on the predicted labels and the original labels to obtain the first model.

11. A language file processing device, characterized in that: include: An acquisition module, configured to acquire a language file to be processed; wherein the language file to be processed includes text information in at least one language type for display on a preset interface; a processing module, configured to input the language file to be processed into a first model for processing to obtain a text to be translated; The processing module is further configured to input the text to be translated into a second model for translation to obtain a translation result in the target language type; and write the translation result into the language file to be processed to obtain a processed language file; wherein the processed language file is used to display text information in different language types on a preset interface.

12. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images