Method for training a machine translation model for generating pseudo-parallel data, method for acquiring pseudo-parallel data, and method for training a machine translation model

Abstract

To provide a pseudo parallel translation data generation device that, even if there is no parallel translation data of an adaptive field, is able to create pseudo parallel translation data for accurately executing machine translation in this adaptive field.SOLUTION: In a machine translation system 1000, a pseudo parallel translation data creating device uses parallel translation data D0(L1-L2) of another field, a first language data D0(L1) of another field, second language data D0(L2) of another field, first language data D0(R1) of an adaptive field, and second language data D0(R2) of the adaptive field to perform an optimization process on a language cross language model consisting of an input data embedding unit 2 and an XLM processing unit 3, and performs a parameter optimization process on an NMT model for creating pseudo parallel translation data consisting of the input data embedding unit 2 and a machine translation processing unit 5 after the optimization process. Processing that uses the machine translation model for creating the pseudo parallel translation data, acquired by this process, makes it possible to obtain pseudo parallel translation data of an adaptive field having no parallel translation data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to neural machine translation technology. [Background technology]

[0002] Neural machine translation (NMT) is a machine translation technology that uses a multi-layer neural network. A neural machine translation system consists of a neural network (encoder) that converts each word and entire sentence of the source language into a vector or tensor, and a neural network (decoder) that generates a sequence of words in the target language from the obtained vector or tensor, and learns a model of the neural network (optimizes parameters) based on bilingual data. The translation performance that can be achieved by a neural machine translation system is highly dependent on the size of the bilingual data for a given source language, target language, and target field. A large amount of bilingual data is required to build a practical neural machine translation system (at least hundreds of thousands of bilingual data are required, depending on the language pair and field). If only a small amount of bilingual data exists, it is difficult to achieve high-quality machine translation processing in a neural machine translation system.

[0003] When bilingual data exists for the language pair that is the subject of the desired machine translation (target language pair) and the same language pair in another field (a field other than the field that is the subject of machine translation), or when bilingual data exists for a different language pair in the same field, it may be possible to realize machine translation in the target field using such bilingual data by using machine translation field application technology or multilingual machine translation technology.

[0004] Furthermore, technologies are being developed that utilize monolingual data, which, unlike bilingual data, is cheap and available in large quantities.

[0005] In recent years, unsupervised machine translation techniques have also been proposed that build machine translation systems using only monolingual data, without using bilingual data from other fields or bilingual data for other language pairs (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Mikel Artetxe, Gorka Labaka, Eneko Agirre (2018). Unsupervised Statistical Machine Translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3632-3642. Summary of the Invention [Problem to be solved by the invention]

[0007] However, all of the above technologies are premised on the existence of bilingual data of several thousand sentences in the target language pair and the target domain, and if this premise does not hold, the machine translation system realized by the above technologies cannot achieve high machine translation performance. Also, in order to create the bilingual data of several thousand sentences in the target language pair and the target domain required by the above technologies, a non-negligible amount of human and financial costs are required.

[0008] As such, when adapting a neural machine translation system to a new field, a certain amount of bilingual data in the field (target field) that the neural machine translation system targets for machine translation is required. Without such bilingual data, it is difficult to improve the performance of the machine translation.

[0009] In view of the above problems, the present invention aims to provide a pseudo-parallel data generation device that generates pseudo-parallel data for performing accurate machine translation in a target field (the field targeted for machine translation) even when there is no bilingual data at all in the target field, a machine translation system that performs accurate machine translation in the target field using the pseudo-parallel data generated by the pseudo-parallel data generation device, and a training method for a machine translation model for generating pseudo-parallel data used in the machine translation system, a method for acquiring pseudo-parallel data, and a training method for a machine translation model. [Means for solving the problem]

[0010] A first invention for solving the above problem is a learning method for a machine translation model for generating pseudo-parallel data, which includes an initialization step and an optimization step.

[0011] The initialization step is A multi-field parallel translation data set Dsetp(L1-L2) including a plurality of parallel translation data consisting of other-field first language data, which is data in a first language in a field other than the target field, which is the field to be targeted for the pseudo parallel translation data to be generated, and other-field second language data, which is translation data in a second language of the other-field first language data; A multi-disciplinary monolingual data set Dsetm(L1) containing multiple data in the first language of a field other than the target field; A multi-disciplinary monolingual data set Dsetm(L2) ​​that contains multiple data in a second language from a field other than the target field; A monolingual data set Dsetm(R1) in the target field that contains multiple data in the first language of the target field; A monolingual data set Dsetm(R2) in the target field that contains multiple data in the second language of the target field; A learning process is performed on the machine translation model for generating pseudo-parallel data using the above, and the parameters set in the machine translation model for generating pseudo-parallel data after the learning process is performed are set as initial parameters.

[0012] The optimization step is For a machine translation model for generating pseudo-parallel data with the initial parameters set, (1) Auto-encoding process in which correct answer data is made the same as the input data and a learning process is performed for a machine translation model for generating pseudo-parallel data; (2) A zero-shot return machine translation process in which the output data of the pseudo-parallel data generating machine translation model for the input data (this "output data" is, for example, data in a language different from the input data and is zero-shot data (a field that has not been studied)) is input again into the pseudo-parallel data generating machine translation model, and a learning process is performed for the pseudo-parallel data generating machine translation model so that the output of the pseudo-parallel data generating machine translation model (this "output" is in the same language as the input data) is identical to the input data; (3) A supervised machine translation process in which either the first language data or the second language data included in the multi-disciplinary parallel data set Dsetp(L1-L2) is used as input to the machine translation model for generating pseudo-parallel data, and the other is used as correct answer data, to perform a learning process for the machine translation model for generating pseudo-parallel data; A learning process is performed using at least one of the above to obtain optimal parameters for a machine translation model for generating pseudo-parallel data.

[0013] In this method for learning a machine translation model for generating pseudo-parallel data, (1) A highly accurate bilingual data set from another field (Dsetp(L1-L2) (preferably large-scale (a large number of bilingual data))) (2) A first language data set from another field (Dsetm(L1)) (3) A second language data set from another field (Dsetm(L2)) (4) A first language data set in the target domain (Dsetm(R1)); (5) A second language dataset for the target domain (Dsetm(R2)); By using this to initialize a machine translation model for generating pseudo-parallel data (e.g., a neural network model) through a pre-learning process (processing in an initialization step) and then performing a parameter optimization process (processing in an optimization step), it is possible to train a machine translation model for generating pseudo-parallel data to generate pseudo-parallel data in the first and second languages ​​of the target field, even if there is no bilingual data at all in the target field (the field to be subjected to machine translation).

[0014] Then, even if no bilingual data exists in the target field (target field), the machine translation model for generating pseudo-parallel data after learning (has already been trained) can generate pseudo-parallel data in the target field. Furthermore, by training a machine translation model using the generated pseudo-parallel data in the target field and performing machine translation processing using the trained machine translation model, it is possible to perform machine translation in the target field (target field) with high accuracy even if no bilingual data exists in the target field.

[0015] The target of initialization by the pre-learning process may be only the part that acquires the distributed representation data from the input data. For example, when the machine translation model is composed of an embedding layer (for example, realized by an input data embedding unit) that acquires the distributed representation data from the input data and a neural network model for machine translation (neural network model for MT), a cross-lingual language model composed of an embedding layer (for example, realized by an input data embedding unit) that acquires the distributed representation data from the input data and a neural network model for a cross-lingual language model (neural network model for XLM, XLM: Cross-lingual language model) is set as the pre-learning process, and parameter optimization processing of the embedding layer and the cross-lingual language model is performed. Then, the machine translation model may be a model composed of an embedding layer (for example, realized by an input data embedding unit) after the optimization processing of the cross-lingual language model and a neural network model for MT, and the state after the optimization processing of the cross-lingual language model may be set as the initial state, and parameter optimization processing of the embedding layer and the machine translation model may be performed.

[0016] The second invention is a pseudo-parallel data acquisition method for acquiring pseudo-parallel data in a target field using a machine translation model for generating pseudo-parallel data acquired by the learning method of a machine translation model for generating pseudo-parallel data, which is the first invention, and includes a first machine translation step, a second machine translation step, and a pseudo-parallel data acquisition step.

[0017] The first machine translation step performs machine translation processing on data in a first language obtained from the multi-domain parallel translation data set Dsetp(L1-L2) using a machine translation model for generating pseudo-parallel translation data, setting the output to the second language of the target domain, to obtain pseudo-translated data in the second language of the target domain, which is data resulting from the machine translation of data in the first language of the other domain.

[0018] The second machine translation step performs machine translation processing on the second language data obtained from the multi-field parallel translation data set Dsetp(L1-L2) using a machine translation model for generating pseudo-parallel data, setting the output to the first language of the target field, to obtain pseudo-translated data in the first language of the target field, which is data resulting from the machine translation of the second language data in the other field.

[0019] The pseudo-translation data acquisition step acquires pseudo-translation data for the target field by matching the second language pseudo-translation data for the target field acquired in the first machine translation step with the first language pseudo-translation data for the target field acquired in the second machine translation step.

[0020] As a result, in this pseudo-parallel data acquisition method, even if no parallel data exists in the application destination field (target field), pseudo-parallel data in the application destination field can be acquired.

[0021] The first machine translation step and the second machine translation step may be executed in parallel.

[0022] The third invention is a pseudo-parallel data acquisition method for acquiring pseudo-parallel data in a target field using a machine translation model for generating pseudo-parallel data acquired by the learning method of the machine translation model for generating pseudo-parallel data of the first invention, and includes a monolingual data machine translation step and a pseudo-parallel data acquisition step.

[0023] The monolingual data machine translation step performs machine translation processing using a machine translation model for generating pseudo-parallel data on data in a first language obtained from the monolingual data set Dsetm(R1) in the target field or data in a second language obtained from the monolingual data set Dsetm(R2) in the target field, setting the output to the second language or the first language in the target field, to obtain pseudo-translated data in the second language of the target field, which is data resulting from the machine translation of data in the first language in the target field, or pseudo-translated data in the first language of the target field, which is data resulting from the machine translation of data in the second language in the target field.

[0024] The pseudo-bilingual data acquisition step obtains pseudo-bilingual data for the target field by matching the first language data or second language data in the target field that was used as input to the machine translation model for generating pseudo-bilingual data in the monolingual data machine translation step with the second language pseudo-translated data or first language pseudo-translated data for the target field obtained in the monolingual data machine translation step, or by matching the second language data in the target field that was used as input to the machine translation model for generating pseudo-bilingual data with the first language pseudo-translated data for the target field obtained in the monolingual data machine translation step.

[0025] As a result, in this pseudo-parallel data acquisition method, even if no parallel data exists in the application destination field (target field), pseudo-parallel data in the application destination field can be acquired.

[0026] A fourth invention is the second or third invention, further comprising a filtering step of obtaining a reliability indicating the accuracy of the results of the machine translation process for each sentence pair of the pseudo-parallel translation data in the application field obtained in the pseudo-parallel translation data obtaining step, and selecting and outputting only pseudo-parallel translation data including sentence pairs whose obtained reliability is equal to or greater than a predetermined value.

[0027] This ensures that pseudo-parallel data of a predetermined accuracy or higher is acquired in this pseudo-parallel data acquisition method.

[0028] It should be noted that the "reliability" is an index in which the higher the value, the higher the reliability. In addition, when the data obtained by the machine translation process (processing result data) is data of a word string or token, it is preferable that the reliability is assigned to each individual word or token.

[0029] A fifth invention is a machine translation model learning method for performing machine translation on data in a first language in a target field to obtain data in a second language, the method comprising a machine translation model learning step.

[0030] The machine translation model learning step involves learning the machine translation model using pseudo-translated data in the target field acquired by the pseudo-parallel data acquisition method which is any one of the second to fourth inventions, and a multi-field parallel translation data set Dsetp(L1-L2) which includes a plurality of parallel translation data consisting of other-field first language data which is data in a first language in a field other than the target field, and other-field second language data which is translation data in a second language of the other-field first language data.

[0031] As a result, in this machine translation model learning method, even if there is no bilingual data in the application destination field (target field), the machine translation model can be learned using the pseudo-translated data in the application destination field acquired by the pseudo-bilingual data acquisition method of any one of the second to fifth inventions. Then, by performing machine translation processing using the trained machine translation model, even if there is no bilingual data in the application destination field (target field), it is possible to perform machine translation in the application destination field with high accuracy.

[0032] A sixth invention is a machine translation device that performs machine translation processing using a trained machine translation model obtained by the training method for a machine translation model for generating pseudo-parallel data of the first invention, or a trained machine translation model obtained by the training method for a machine translation model of the fifth invention.

[0033] As a result, this machine translation device can perform machine translation processing using a trained machine translation model obtained by the training method for a machine translation model for generating pseudo-parallel data, which is the first invention, or a trained machine translation model obtained by the training method for a machine translation model, which is the fifth invention.

[0034] The present invention also includes a program for causing a computer to execute the learning method for a machine translation model for generating pseudo-parallel data, which is the first invention.

[0035] The present invention also includes a program for causing a computer to execute the pseudo-parallel translation data acquisition method according to any one of the second to fourth aspects of the present invention.

[0036] The present invention also includes a program for causing a computer to execute the machine translation model learning method, which is the fifth invention.

[0037] Furthermore, the present invention also includes a machine translation method for performing machine translation processing using a trained machine translation model obtained by the machine translation model training method, which is the fifth invention, and a program for causing a computer to execute the machine translation method. Effect of the Invention

[0038] According to the present invention, it is possible to realize a pseudo-parallel data generation device that generates pseudo-parallel data for performing accurate machine translation in a target field (the field targeted for machine translation) even when there is no parallel data at all in the target field, a machine translation system that uses the pseudo-parallel data generated by the pseudo-parallel data generation device to perform accurate machine translation in the target field, and a training method for a machine translation model for generating pseudo-parallel data used in the machine translation system, a method for acquiring pseudo-parallel data, and a training method for a machine translation model. [Brief description of the drawings]

[0039] [Figure 1] 1 is a schematic configuration diagram of a machine translation system 1000 according to a first embodiment. [Diagram 2] 1 is a schematic diagram showing the configuration of a pseudo-parallel translation data generation device 100 according to a first embodiment. [Diagram 3] 1 is a schematic configuration diagram of an input data acquisition unit 1, an input data embedding unit 2, and an XLM processing unit 3 of a pseudo-parallel translation data generation device 100 according to a first embodiment. [Figure 4] 1 is a schematic configuration diagram of an input data acquisition unit 1, an input data embedding unit 2, and a machine translation processing unit 5 of a pseudo-parallel translation data generation device 100 according to a first embodiment. [Diagram 5] 1 is a flowchart of a process executed by the machine translation system 1000. [Figure 6] 3 is a flowchart of a process executed by the pseudo-parallel translation data generation device 100. [Figure 7] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 8] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 9] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 10]3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 11] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 12] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 13] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 14] 3 is a diagram for explaining the processing executed by the pseudo-parallel translation data generation device 100. [Figure 15] FIG. 2 is a diagram showing an example of data acquired by the machine translation device MT1. [Figure 16] A diagram showing the CPU bus configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] [First embodiment] The first embodiment will be described below with reference to the drawings.

[0041] <1.1: Machine translation system configuration> FIG. 1 is a schematic configuration diagram of a machine translation system 1000 according to the first embodiment.

[0042] FIG. 2 is a schematic diagram of the pseudo-parallel translation data generation device 100 according to the first embodiment.

[0043] 3 is a schematic diagram of the input data acquisition unit 1, the input data embedding unit 2, and the XLM processing unit 3 of the pseudo-parallel translation data generation device 100 according to the first embodiment. Note that the first selector SEL1 is omitted in FIG.

[0044] 4 is a schematic diagram of the input data acquisition unit 1, the input data embedding unit 2, and the machine translation processing unit 5 of the pseudo-parallel translation data generation device 100 according to the first embodiment. Note that the first selector SEL1 is omitted in FIG.

[0045] As shown in FIG. 1, the machine translation system 1000 includes a bilingual data storage unit DBp(L1-L2), a first monolingual data storage unit DBm(L1), a second monolingual data storage unit DBm(L2), a third monolingual data storage unit DBm(R1), a fourth monolingual data storage unit DBm(R2), a pseudo-bilingual data generation device 100, a pseudo-bilingual data storage unit DB1, and a machine translation device MT1.

[0046] In the following description, the field targeted by the bilingual data generated by the pseudo-bilingual data generation device 100 is referred to as the "target field," and fields other than the target field are referred to as "other fields."

[0047] The bilingual data storage unit DBp(L1-L2) stores a bilingual data set Dsetp(L1-L2) including a plurality of bilingual data consisting of other-field first language data, which is data in a first language of a field (other field) other than the application target field, and other-field second language data, which is translation data of the other-field first language data in a second language. Then, the bilingual data storage unit DBp(L1-L2) reads out a predetermined bilingual data D0(L1-L2) stored in the bilingual data storage unit DBp(L1-L2) based on a read command from the pseudo-bilingual data generation device 100, and outputs the bilingual data D0(L1-L2) to the pseudo-bilingual data generation device 100. The bilingual data storage unit DBp(L1-L2) is realized by, for example, a database. It is preferable that the amount of bilingual data stored in the bilingual data storage unit DBp(L1-L2) is large (it is preferable that the bilingual data storage unit DBp(L1-L2) stores and holds a large amount of bilingual data).

[0048] The first monolingual data storage unit DBm(L1) stores a different field first language data set Dsetm(L1) including a plurality of different field first language data which is monolingual data in a first language in a field (other field) other than the application field. The first monolingual data storage unit DBm(L1) reads out monolingual data D0(L1) in a predetermined first language stored in the first monolingual data storage unit DBm(L1) based on a read command from the pseudo-parallel translation data generation device 100, and outputs the monolingual data D0(L1) to the pseudo-parallel translation data generation device 100. The first monolingual data storage unit DBm(L1) is realized by, for example, a database.

[0049] The second monolingual data storage unit DBm(L2) ​​stores a different field second language data set Dsetm(L2) ​​including a plurality of different field second language data which is monolingual data in a second language in a field (other field) other than the application field. The second monolingual data storage unit DBm(L2) ​​reads out monolingual data D0(L2) in a predetermined second language stored in the second monolingual data storage unit DBm(L2) ​​based on a read command from the pseudo-parallel data generation device 100, and outputs the monolingual data D0(L2) to the pseudo-parallel data generation device 100. The second monolingual data storage unit DBm(L2) ​​is realized by, for example, a database.

[0050] The third monolingual data storage unit DBm(R1) stores a destination field first language data set Dsetm(R1) including a plurality of destination field first language data which are monolingual data in a first language in the destination field. The third monolingual data storage unit DBm(R1) reads out monolingual data D0(R1) in a predetermined first language stored in the third monolingual data storage unit DBm(R1) based on a read command from the pseudo-parallel data generation device 100, and outputs the monolingual data D0(R1) to the pseudo-parallel data generation device 100. The third monolingual data storage unit DBm(R1) is realized by, for example, a database.

[0051] The fourth monolingual data storage unit DBm(R2) stores a destination field second language data set Dsetm(R2) including a plurality of destination field second language data which are monolingual data in the second language of the destination field. The fourth monolingual data storage unit DBm(R2) reads out monolingual data D0(R2) in a predetermined second language stored in the fourth monolingual data storage unit DBm(R2) based on a read command from the pseudo-parallel data generation device 100, and outputs the monolingual data D0(R2) to the pseudo-parallel data generation device 100. The fourth monolingual data storage unit DBm(R2) is realized by, for example, a database.

[0052] As shown in FIG. 2, the pseudo-parallel translation data generation device 100 includes a data input interface IF1, an input data acquisition unit 1, a first selector SEL1, an input data embedding unit 2, an XLM processing unit 3, a first output data evaluation unit 4, a machine translation processing unit 5, a second selector SEL2, a second output data evaluation unit 6, a first buffer Buf1, a second input data acquisition processing unit 7, and a filter processing unit 8.

[0053] The data input interface IF1 receives a control signal CTL1 from a control unit (not shown) that controls each functional unit of the pseudo-translation data generation device 100, and in accordance with the control signal CTL1, reads out specified data from any of the bilingual data storage unit DBp(L1-L2), the first monolingual data storage unit DBm(L1), the second monolingual data storage unit DBm(L2), the third monolingual data storage unit DBm(R1), and the fourth monolingual data storage unit DBm(R2), and outputs the read data to the input data acquisition unit 1 as data D1.

[0054] 3, the input data acquisition unit 1 includes a first input data acquisition processing unit 11, a masking processing unit 12, a correct answer data acquisition unit 13, and an input data output unit 14. The input data acquisition unit 1 includes a buffer (not shown) that can store and hold all or a part of the input data for a predetermined period of time, and can use the data stored and held in the buffer depending on the processing.

[0055] The first input data acquisition processing unit 11 receives data D1 output from the data input interface IF1 and a control signal CTL2 output from the control unit. In accordance with the control signal CTL2, the first input data acquisition processing unit 11 acquires (generates) data to be input to the masking processing unit 12 and the input data embedding unit 2 from the data D1. Specifically, the first input data acquisition processing unit 11 extracts from the input data D1 (for example, data of a subword string (or a word string), or data of a string of indexes corresponding to each subword (or each word)) (1) token data xi0 which is data representing a token (a token may represent a character string); token and (2) location data xi for identifying the location of the token. pos and (3) language data for identifying the language of the token. lang Then, the first input data acquisition processing unit 11 acquires the token data xi0 acquired as described above. token is output to the masking processor 12, and the position data xi pos and language data xi lang and are output to the first selector SEL1.

[0056] The masking processing unit 12 receives the token data xi0 output from the first input data acquisition processing unit 11. token and a control signal CTL2 output from the control unit. (1) When the control signal CTL2 instructs execution of a masking process, the masking processor 12 inputs token data xi0 token The masking process is performed on the token data xi token (2) When the control signal CTL2 does not instruct the execution of the masking process, the masking processor 12 outputs the input token data xi0 token The token data xi token to the first selector SEL1.

[0057] In addition, (1) token data xi output from the input data acquisition unit 1 to the first selector SEL1 token and (2) location data xipos and (3) language dataxi lang The data including these is represented as data D2a.

[0058] The correct data acquisition unit 13 receives data D1 output from the data input interface IF1 and a control signal CTL2 output from the control unit. In accordance with the control signal CTL2, the correct data acquisition unit 13 generates correct data D_correct from the data D1 to be used in the learning process of a cross-language language model (XLM) (a model consisting of the input data embedding unit 2 (corresponding to the embedding layer) and the neural network model 31 for XLM in the XLM processing unit 3) or a machine translation model (a neural network model consisting of the input data embedding unit 2 and the machine translation processing unit 5), and outputs the correct data D_correct to the first output data evaluation unit 4 and the second output data evaluation unit 6.

[0059] The input data output unit 14 receives the data D1 output from the data input interface IF1 and the control signal CTL2 output from the control unit. The input data output unit 14 outputs the input data D1 to the filter processing unit 8 as data D1_org in accordance with the control signal CTL2.

[0060] The first selector SEL1 receives data D2a output from the first input data acquisition processing unit 11 of the input data acquisition unit 1, data D2b output from the second input data acquisition processing unit 7, and a selection signal sel1 output from the control unit. The first selector SEL1 selects data D2a or data D2b in accordance with the selection signal sel1, and outputs it to the input data embedding unit 2 as data D3.

[0061] As shown in FIGS. 3 and 4, the input data embedding unit 2 includes a token embedding unit 21, a position embedding unit 22, and a language embedding unit .

[0062] The token embedding unit 21 embeds token data xi token Enter the token data you entered. tokenThe distributed representation data of xi' is obtained. token The token embedding unit 21 outputs the token data xi token By performing a matrix operation to obtain the distributed representation data, the distributed representation data xi' token For example, the token embedding unit 21 obtains the distributed expression data xi′ by performing the following matrix calculation: token In addition, we obtain the matrix W token Each element (corresponding to a weighting coefficient) is a parameter θ emb It is a part of the parameter θ emb is the parameter update data update(θ emb ) is updated. xi' token =xi token ·W token xiii token : A vector representing each token (input data) string (e.g., a 1×n1 matrix (n1-dimensional vector) (n1: natural number)) W token : A matrix for obtaining distributed representation data (e.g., an n1 × m1 matrix (n1, m1: natural numbers)) xi' token : Input data xi token Distributed representation data of (e.g., 1×m1 matrix (m1-dimensional vector) (m1: natural number)) The position embedding unit 22 embeds the position data xi pos Enter the position data xi pos The distributed representation data of xi' is obtained. pos The position embedding unit 22 outputs the position data xi pos By performing a matrix operation to obtain the distributed representation data, the distributed representation data xi' posFor example, the position embedding unit 22 performs the process using the following matrix operation to obtain the distributed representation data xi′ pos In addition, we obtain the matrix W pos Each element (corresponding to a weighting coefficient) is a parameter θ emb It is a part of the parameter θ emb is the parameter update data update(θ emb ) is updated. xi' pos =xi pos ·W pos xiii pos : A vector that represents the position of each token (input data) (e.g., a 1×n2 matrix (n2-dimensional vector) (n2: natural number)) W pos : A matrix for obtaining distributed representation data (e.g., an n2 × m2 matrix (n2, m2: natural numbers)) xi' pos : Input data xi pos Distributed representation data of (e.g., 1×m2 matrix (m2-dimensional vector) (m2: natural number)) The language embedding unit 23 embeds the language data xi lang Enter the language data xi lang The distributed representation data of xi' is obtained. lang The language embedding unit 23 outputs the language data xi lang By performing a matrix operation to obtain the distributed representation data, the distributed representation data xi' lang For example, the language embedding unit 23 performs the following matrix calculation to obtain the distributed expression data xi′ lang In addition, we obtain the matrix W lang Each element (corresponding to a weighting coefficient) is a parameter θ emb It is a part of the parameter θ emb is the parameter update data update(θ emb) is updated. xi' lang =xi lang ·W lang xiii lang : A vector that represents the language of each token (input data) (e.g., a 1×n3 matrix (n3-dimensional vector) (n3: natural number)) W lang : A matrix for obtaining distributed representation data (e.g., an n3 × m3 matrix (n3, m3: natural numbers)) xi' lang : Input data xi lang Distributed representation data of (e.g., 1×m3 matrix (m3-dimensional vector) (m3: natural number)) The input data embedding unit 2 outputs the distributed expression data acquired as described above to the XML processing unit 3 and the machine translation processing unit 5 as data D4.

[0063] As shown in FIG. 3, the XLM processing unit 3 includes a neural network model 31 for XLM.

[0064] The neural network model 31 for XLM is, for example, a neural network model disclosed in the following document A, and is realized by a neural network model that employs the architecture of the transformer model disclosed in the following document B. (Reference A): Alexis Conneau and Guillaume Lample (2019). Cross-Lingual Language Model Pretraining. In Proceedings of the 32nd Neural Information Processing Systems Conference (NeurIPS), pp. 7057-7067. (Literature B): Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin (2017). Attention is All You Need. In Proceedings of the 30th Neural Information Processing Systems Conference (NeurIPS), pp. 5998-6008. The neural network model for XLM 31 uses distributed representation data D4 (={xi' token, xi' pos, xi' lang}).

[0065] The neural network model for XLM 31 uses the parameter update data update(θ XLM ), the parameters of the neural network model for XLM 31 are updated based on the parameter update data update(θ emb ) and generate the parameter update data update(θ emb ) is output to the input data embedding unit 2.

[0066] Furthermore, the neural network model for XLM 31 executes XLM processing on the data D4 output from the input data embedding unit 2 to obtain data D5x. Then, the XLM processing unit 3 outputs the data D5x obtained as described above to the first output data evaluation unit 4.

[0067] As shown in FIG. 2, the first output data evaluation unit 4 receives the data D5x output from the XLM processing unit 3 and the correct data D_correct output from the input data acquisition unit 1. In order to evaluate the loss, the first output data evaluation unit 4 acquires data D_correct' for loss evaluation from the correct data D_correct for the data D5x output from the XLM processing unit 3, and acquires the loss from the data D5x and the data D_correct' (details will be described later). Then, the first output data evaluation unit 4 evaluates the parameters θ of the XLM neural network model 31 in the XLM processing unit 3 based on the loss for predetermined learning data (learning loss). XLM Data update(θ XLM ) and generate the data update(θ XLM ) is output to the XML processing unit 3.

[0068] The machine translation processor 5 includes a neural network model 51 for MT, as shown in FIG.

[0069] The MT neural network model 51 is an encoder / decoder type neural network model, for example, a neural network model that employs an encoder / decoder configuration based on the transformer model architecture disclosed in the above-mentioned document B.

[0070] The MT neural network model 51 uses distributed representation data D4 (={xi' token, xi' pos, xi' lang}) and a control signal CTL3 output from the control unit. The MT neural network model 51 outputs data in the language specified by the control signal CTL3.

[0071] In addition, the MT neural network model 51 uses the parameter update data update(θ MT ) based on which the parameters of the MT neural network model 51 are updated.

[0072] Furthermore, the MT neural network model 51 executes a machine translation process on the data D4 output from the input data embedding unit 2 to obtain data D5. The MT neural network model 51 executes the machine translation process so that data of a type specified by the control signal CTL3 is output as data D5. The type specified by the control signal CTL3 is any one of (1) L1 (other field, first language), (2) L2 (other field, second language), (3) R1 (target field, first language), and (4) R2 (target field, second language).

[0073] The machine translation processor 5 outputs data D5 acquired by the MT neural network model 51 to the second selector SEL2.

[0074] When the machine translation processing unit 5 obtains pseudo-parallel data, the machine translation processing unit 5 pairs data (e.g., data R2) obtained by performing machine translation processing on data input to the input data embedding unit 2 (e.g., data D3(L1) in a first language of another field) with data (e.g., data R1) obtained by performing machine translation processing on data input to the input data embedding unit 2 (data D3(L2) corresponding to a translation in a second language of another field of the data D3(L1) in the first language of another field), and outputs the paired data to the second selector SEL2 as data D5 (e.g., data D5(R1-R2) (parallel data D5(R1-R2) consisting of a pair of a machine-translated sentence in the first language of the target field (the result of the machine translation of the input data L2) and a machine-translated sentence in the second language of the target field (the result of the machine translation of the input data L1)).

[0075] The second selector SEL2 is a one-input, two-output switch that receives as input the data D5 output from the machine translation processor 5 and a selection signal sel2 output from the control unit. In accordance with the selection signal sel2, the second selector SEL2 either (1) outputs the data D5 as data D6a to the second output data evaluator 6 and the filter processor 8, or (2) outputs the data D5 as data D6b to the first buffer Buf1.

[0076] 2, the second output data evaluation unit 6 receives the data D6a output from the second selector SEL2 and the correct data D_correct output from the input data acquisition unit 1. In order to evaluate the loss, the second output data evaluation unit 6 acquires data D_correct' for loss evaluation from the correct data D_correct of the data D6a output from the machine translation processing unit 5, and acquires the loss from the data D6a and the data D_correct' (details will be described later). Then, the second output data evaluation unit 6 evaluates the parameters θ of the MT neural network model 51 of the machine translation processing unit 5 based on the loss for predetermined learning data (learning loss). MT Data update(θ MT )

[0077] Then, the second output data evaluation unit 6 calculates the data update(θ MT ) is output to the machine translation processor 5.

[0078] The first buffer Buf1 is a buffer for storing and holding the data D6b output from the second selector SEL2. The first buffer Buf1 outputs the stored data to the second input data acquisition processing unit 7 as data D6b'.

[0079] The second input data acquisition processing unit 7 is a functional unit similar to the first input data acquisition processing unit 11, and inputs data D6b' output from the first buffer Buf1, and acquires (generates) data from the data D6b' to be input to the input data embedding unit 2. Then, the second input data acquisition processing unit 7 outputs the acquired data to the first selector SEL1 as data D2b.

[0080] The filter processing unit 8 receives the data D6a output from the second selector SEL2, the data D1_org output from the input data acquisition unit 1, and the control signal CTL4 output from the control unit.

[0081] In addition, the filter processing unit 8 (1) assigns a reliability to each piece of bilingual data (pseudo-bilingual data) when the data D6a is bilingual data (pseudo-bilingual data) and performs filtering processing based on the reliability, and (2) assigns a reliability to the bilingual data (pseudo-bilingual data) obtained by pairing the machine-translated data with the data D1_org when the data D6a is machine-translated data, and performs filtering processing based on the reliability.

[0082] Then, the filter processing unit 8 outputs the filtered bilingual data as data Dpsd1 to the pseudo bilingual data storage unit DB1.

[0083] 1, the pseudo-parallel data storage unit DB1 is a storage unit for storing and holding data Dpsd1 output from the pseudo-parallel data generation device 100. Furthermore, the pseudo-parallel data storage unit DB1 reads out the stored data based on a read command from the machine translation device MT1, and outputs the read out data to the machine translation device MT1 as data Dpsd2.

[0084] 1, the machine translation device MT1 reads data from the bilingual data storage unit DBp(L1-L2) and / or the pseudo bilingual data storage unit DB1, and performs a learning process for a machine translation model using the read data.The machine translation device MT1 then performs a machine translation process using the learned model of the machine translation model acquired by the learning process.That is, the machine translation device MT1 inputs source language data Din_e, performs a machine translation process on the source language data Din_e, and acquires target language data Dout_j.

[0085] <1.2: Operation of machine translation system> The operation of the machine translation system 1000 configured as above will now be described.

[0086] FIG. 5 is a flowchart of the process executed by the machine translation system 1000.

[0087] FIG. 6 is a flowchart of the process (pre-learning process) executed by the pseudo-parallel translation data generation device 100.

[0088] 7 to 11 are diagrams for explaining the process (pre-learning process) executed by the pseudo-parallel translation data generation device 100. FIG.

[0089] 12 to 14 are diagrams for explaining the process (pseudo-translation data generation process) executed by the pseudo-translation data generation device 100. FIG.

[0090] The operation of the machine translation system 1000 will be described below with reference to the drawings.

[0091] (1.2.1: Pre-learning process) First, the pre-training process (pre-training process of the NMT model for generating pseudo-parallel data (model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5)) executed in the pseudo-parallel data generating device 100 of the machine translation system 1000 (step S1 in the flowchart of FIG. 5) will be explained separately as the training process of the XLM and the training process of the NMT model for generating pseudo-parallel data.

[0092] (1.2.1.1: XLM learning process) First, the learning process of XLM will be described.

[0093] The learning process of XLM is as shown in Figure 6. (A) Processing of a masked language model (MLM) (step S111); (B) Processing of the translation language model (TLM) (step S112); (C) Loss calculation process (step S113); (D) Parameter (θ XLM ,θ emb ) update process (step S114); It consists of:

[0094] As shown in the flowchart of FIG. 6, the processing in steps S111 to S114 is repeatedly executed until a termination condition is satisfied (loop processing (loop 1) S110 to S115 in FIG. 6).

[0095] (1.2.1.1A: Masked Language Model (MLM) Processing) In processing the masked language model (MLM), the pseudo-parallel translation data generation device 100 performs a learning process by treating the input to the input data embedding unit 2 as masked data and the correct answer data as original data (unmasked data), as shown in FIG.

[0096] ≪M1:MLM processing (L1.mask→L1)≫ Specifically, the data input interface IF1 reads out monolingual data D0(L1) in a first language in another field from the first monolingual data storage unit DBm(L1) and outputs the read monolingual data to the input data acquisition unit 1 as data D1(L1).

[0097] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts (1) token data xi0 from data D1 (data D1(L1) in a first language in another field). token (=xi0 token (L1)) and (2) location data xi for identifying the location of the token. pos (=xi pos (L1)) and (3) language data for identifying the language of the token. lang (=xi lang (L1)) and obtain.

[0098] Then, the input data acquisition unit 1 receives the token data xi0 token is output to the masking processor 12.

[0099] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0100] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token Then, the masking processing unit 12 performs a masking process to replace some of the tokens in the token data xi with masked tokens (for example, data (tokens) with a label "[mask]"). token to the first selector SEL1.

[0101] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos (L1)) is output to the first selector SEL1, and language data xi lang (=xi lang (L1)) is output to the first selector SEL1.

[0102] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(L1.mask)={xi token (L1),xi pos (L1),xi lang (L1)}) to the first selector SEL1.

[0103] In addition, parentheses are added to the end of the variable that indicates the data type. For example, xi token (L1) indicates that the data is derived from data in the first language (L1) in another field (hereinafter the same). Also, the notation ".mask" in the parentheses indicates that the data includes token data that has been masked, for example, "L1.mask" indicates that the data includes data that has been masked on token data in data derived from data in the first language (L1) in another field (hereinafter the same).

[0104] In accordance with the control signal CTL2, the correct data acquisition unit 13 generates correct data D_correct (=D_correct(L1)) from the data D1 (=D1(L1)) to be used in the learning process (XLM learning process) in the input data embedding unit 2 and the XLM processing unit 3. Then, the correct data acquisition unit 13 outputs the generated data to the first output data evaluation unit 4 as correct data D_correct (=D_correct(L1)).

[0105] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0106] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a(L1.mask) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3 (=D3(L1.mask)).

[0107] The token embedding unit 21 of the input data embedding unit 2 embeds token data xi token Enter the token data you entered. token The distributed representation data is obtained, for example, by executing a process corresponding to the following formula. xi' token =xi token ·W token xiii token : A vector representing each token (input data) string (e.g., a 1×n1 matrix (n1-dimensional vector) (n1: natural number)) W token : A matrix for obtaining distributed representation data (e.g., an n1 × m1 matrix (n1, m1: natural numbers)) xi' token : Input data xi token Distributed representation data of (e.g., 1×m1 matrix (m1-dimensional vector) (m1: natural number)) The position embedding unit 22 of the input data embedding unit 2 embeds the position data xi pos Enter the position data xipos The distributed representation data is obtained, for example, by executing a process corresponding to the following formula. xi' pos =xi pos ·W pos xiii pos : A vector that represents the position of each token (input data) (e.g., a 1×n2 matrix (n2-dimensional vector) (n2: natural number)) W pos : A matrix for obtaining distributed representation data (e.g., an n2 × m2 matrix (n2, m2: natural numbers)) xi' pos : Input data xi pos Distributed representation data of (e.g., 1×m2 matrix (m2-dimensional vector) (m2: natural number)) The language embedding unit 23 of the input data embedding unit 2 embeds the language data xi lang Enter the language data xi lang The distributed representation data is obtained, for example, by executing a process corresponding to the following formula. xi' lang =xi lang ·W lang xiii lang : A vector that represents the language of each token (input data) (e.g., a 1×n3 matrix (n3-dimensional vector) (n3: natural number)) W lang : A matrix for obtaining distributed representation data (e.g., an n3 × m3 matrix (n3, m3: natural numbers)) xi' lang : Input data xi lang Distributed representation data of (e.g., 1×m3 matrix (m3-dimensional vector) (m3: natural number)) The input data embedding unit 2 outputs the distributed expression data acquired as described above as data D4(L1.mask) to the XML processing unit 3. Note that the method of embedding the token string and language identifier and the method of embedding the position are not limited to the above, and other methods may be used.

[0108] The neural network model for XLM 31 of the XLM processing unit 3 executes XLM processing on the data D4(L1.mask) output from the input data embedding unit 2 to obtain data D5x (=D5x(L1.mask)). Then, the XLM processing unit 3 outputs the data D5x obtained as described above to the first output data evaluation unit 4.

[0109] The first output data evaluation unit 4 inputs the data D5x(L1.mask) output from the XLM processing unit 3 and the correct data D_correct (=D_correct(L1)) output from the input data acquisition unit 1. The first output data evaluation unit 4 acquires a loss from the data D5(L1.mask) that is the output of the XLM processing unit 3 and the correct data D_correct(L1).

[0110] ≪M2:MLM processing (L2.mask→L2)≫ Next, the data input interface IF1 of the pseudo-parallel translation data generating device 100 reads out the monolingual data D0(L2) in the second language of the other field from the second monolingual data storage unit DBm(L2), and outputs the read monolingual data as data D1(L2) to the input data acquiring unit 1. Then, the same process as above is performed. In other words, the input data to the input data embedding unit 2 is data D3(L2.mask), that is, D3(L2.mask) = {xi token (L2),xi pos (L2),xi lang (L2)} xiii token (L2): Token data xi0 of monolingual data D1(L2) read from the second monolingual data storage unit DBm(L2) token Masked token data obtained by masking process to replace some tokens in (L2) with masked tokens xiii pos (L2): Data (position data) indicating the position of token data of the monolingual data D1(L2) read from the second monolingual data storage unit DBm(L2) xiii lang(L2): Data (language data) indicating the language of the token of the monolingual data D1(L2) read from the second monolingual data storage unit DBm(L2) Then, the correct data is set as D_correct(L2) (=original data (unmasked data)) and the same processing as above is performed.

[0111] ≪M3:MLM processing (R1.mask→R1)≫ Next, the data input interface IF1 of the pseudo-parallel data generating device 100 reads out monolingual data D0(R1) in the first language of the application target field from the third monolingual data storage unit DBm(R1), and outputs the read monolingual data as data D1(R1) to the input data acquiring unit 1. Then, the same process as above is performed. In other words, the input data to the input data embedding unit 2 is data D3(R1.mask), that is, D3(R1.mask) = {xi token (R1),xi pos (R1),xi lang (R1)} xiii token (R1): Token data xi0 of monolingual data D1(R1) read from the third monolingual data storage unit DBm(R1) token Masked token data obtained by performing a masking process to replace some of the tokens in (R1) with masked tokens xiii pos (R1): Data (position data) indicating the position of token data of the monolingual data D1(R1) read from the third monolingual data storage unit DBm(R1) xiii lang (R1): Data (language data) indicating the language of the token data of the monolingual data D1(R1) read from the third monolingual data storage unit DBm(R1) Then, the correct data is set as D_correct(R1) (=original data (unmasked data)) and the same processing as above is performed.

[0112] ≪M4:MLM processing (R2.mask→R2)≫ Next, the data input interface IF1 of the pseudo-parallel translation data generating device 100 reads out the monolingual data D0(R2) in the second language of the application target field from the fourth monolingual data storage unit DBm(R2), and outputs the read monolingual data as data D1 to the input data acquiring unit 1. Then, the same process as above is performed. In other words, the input data to the input data embedding unit 2 is data D3(R2.mask), that is, D3(R2.mask) = {xi token (R2),xi pos (R2),xi lang (R2)} xiii token (R2): Token data xi0 of monolingual data D1(R2) read from the fourth monolingual data storage unit DBm(R2) token Masked token data obtained by masking process to replace some tokens in (R2) with masked tokens xiii pos (R2): Data (position data) indicating the position of token data of the monolingual data D1(R2) read from the fourth monolingual data storage unit DBm(R2) xiii lang (R2): Data (language data) indicating the language of the token data of the monolingual data D1(R2) read from the fourth monolingual data storage unit DBm(R2) Then, the correct data is set as D_correct(R2) (=original data (unmasked data)) and the same processing as above is performed.

[0113] (1.2.1.1B: Translation Language Model (TLM) Processing) Next, in processing the translation language model (TLM), the pseudo-parallel data generation device 100 performs a learning process by treating the input (parallel data) to the input data embedding unit 2 as masked data and the correct answer data as original data (unmasked data), as shown in FIG.

[0114] ≪T1: TLM processing ((L1-L2).mask→L1-L2)≫ Specifically, the data input interface IF1 reads out bilingual data D0(L1-L2) of other fields from the bilingual data storage unit DBp(L1-L2) and outputs the read bilingual data to the input data acquisition unit 1 as data D1(L1-L2).

[0115] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts (1) token data xi0 from data D1 (parallel data D1(L1-L2) (parallel data D1(L1-L2) which is a pair of a sentence in a first language of a different field and a translation of the sentence in a second language)). token (=xi0 token (L1-L2)) and (2) location data xi for identifying the location of the token. pos (=xi pos (L1-L2)), and (3) language data to identify the language of the token. lang (=xi lang (L1-L2)) is obtained.

[0116] Then, the input data acquisition unit 1 receives the token data xi0 token (L1-L2) is output to the masking processor 12.

[0117] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0118] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token A masking process is performed to replace some tokens of (L1-L2) with mask tokens (for example, data (tokens) in which the character string is set to an artificial token “[MASK]”). Then, the masking process unit 12 converts the token data after the masking process into token data xi tokenThe masked data L1-L2 (parallel data consisting of a sentence in a first language of another domain and its translation in a second language) is output to the first selector SEL1 as ((L1-L2).mask). The data obtained by performing masking processing on the parallel translation data L1-L2 (parallel data consisting of a sentence in a first language of another domain and its translation in a second language) is represented as "(L1-L2).mask" (the same applies below).

[0119] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos )(=xi pos (L1-L2)) is output to the first selector SEL1, and language data xi lang (=xi lang )(=xi lang (L1-L2) is output to the first selector SEL1.

[0120] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a(=D2a((L1-L2).mask)={xi token ((L1-L2).mask),xi pos (L1-L2),xi lang (L1-L2)}) to the first selector SEL1.

[0121] In accordance with the control signal CTL2, the correct data acquisition unit 13 generates correct data D_correct (=D_correct(L1-L2)) from the data D1 to be used in the learning process (XLM learning process) in the input data embedding unit 2 and the XLM processing unit 3. Then, the correct data acquisition unit 13 outputs the generated data to the first output data evaluation unit 4 as correct data D_correct(L1-L2).

[0122] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0123] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a((L1-L2).mask) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3((L1-L2).mask).

[0124] The token embedding unit 21 of the input data embedding unit 2 embeds token data xi contained in the data D3 ((L1-L2).mask). token (=xi token ((L1-L2).mask)) and enter the token data xi token Distributed representation data xi' token is obtained by performing the same process as described in the MLM process above.

[0125] The position embedding unit 22 of the input data embedding unit 2 embeds the position data xi pos (=xi pos (L1-L2)) and enter the position data xi pos Distributed representation data xi' pos is obtained by performing the same process as described in the MLM process above.

[0126] The language embedding unit 23 of the input data embedding unit 2 embeds the language data xi lang (=xi lang (L1-L2))) and input the language data xi lang Distributed representation data xi' lang is obtained by performing the same process as described in the MLM process above.

[0127] The input data embedding unit 2 outputs the distributed representation data acquired as described above to the XLM processing unit 3 as data D4((L1-L2).mask).

[0128] The neural network model for XLM 31 of the XLM processing unit 3 executes XLM processing on the data D4((L1-L2).mask) output from the input data embedding unit 2 to obtain data D5x (=D5x((L1-L2).mask)). Then, the XLM processing unit 3 outputs the data D5x((L1-L2).mask) obtained as described above to the first output data evaluation unit 4.

[0129] The first output data evaluation unit 4 inputs the data D5x((L1-L2).mask) output from the XLM processing unit 3 and the correct data D_correct(L1-L2) output from the input data acquisition unit 1. The first output data evaluation unit 4 acquires a loss from the data D5x((L1-L2).mask) output from the XLM processing unit 3 and the correct data D_correct(L1-L2).

[0130] ≪T2:TLM processing ((L2-L1).mask→L2-L1)≫ Next, the data input interface IF1 of the pseudo-parallel data generation device 100 reads out the parallel data D0 (L1-L2) of another field from the parallel data storage unit DBp (L1-L2), and outputs the parallel data obtained by swapping the first language data and the second language data of the read out parallel data as data D1 (L2-L1) to the input data acquisition unit 1.

[0131] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts (1) token data xi from data D1 (parallel data D1(L2-L1) (parallel data D1(L2-L1) consisting of a sentence in a second language of another field and a translation of the sentence in a first language)) token (=xi token (L2-L1)) and (2) location data xi to identify the location of the token. pos (=xi pos (L2-L1)), and (3) linguistic data to identify the language of the token. lang (=xi lang (L2-L1)) is obtained.

[0132] Then, the input data acquisition unit 1 receives the token data xi token (L2-L1) token data xi0 token The signal is output to the masking processor 12 as (L2-L1).

[0133] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0134] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token A masking process is performed to replace some tokens of (L2-L1) with mask tokens (for example, data (tokens) in which the character string is set to an artificial token “[MASK]”). Then, the masking process unit 12 converts the token data after the masking process into token data xi token The masked data L2-L1 (parallel data consisting of a sentence in a second language of another field and its translation in a first language) is output to the first selector SEL1 as ((L2-L1).mask). The data obtained by performing a masking process on the parallel translation data L2-L1 (parallel data consisting of a sentence in a second language of another field and its translation in a first language) is represented as "(L2-L1).mask" (the same applies below).

[0135] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos )(=xi pos (L2-L1)) is output to the first selector SEL1, and language data xi lang (=xi lang )(=xi lang (L2-L1)) is output to the first selector SEL1.

[0136] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a(=D2a((L2-L1).mask)={xi token ((L2-L1).mask),xi pos (L2-L1),xi lang(L2-L1)}) to the first selector SEL1.

[0137] In accordance with the control signal CTL2, the correct data acquisition unit 13 generates correct data D_correct (=D_correct(L2-L1)) from the data D1 to be used in the learning process (XLM learning process) in the input data embedding unit 2 and the XLM processing unit 3. Then, the correct data acquisition unit 13 outputs the generated data to the first output data evaluation unit 4 as correct data D_correct(L2-L1).

[0138] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0139] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a((L2-L1).mask) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3((L2-L1).mask).

[0140] The token embedding unit 21 of the input data embedding unit 2 embeds token data xi token (=xi token (L2-L1)) and enter the token data xi token Distributed representation data xi' token is obtained by performing the same process as described in the MLM process above.

[0141] The position embedding unit 22 of the input data embedding unit 2 embeds the position data xi pos (=xi pos (L2-L1)) and enter the position data xi pos Distributed representation data xi' pos is obtained by performing the same process as described in the MLM process above.

[0142] The language embedding unit 23 of the input data embedding unit 2 embeds the language data xi lang (=xi lang (L2-L1))) and input the language data xi lang Distributed representation data xi' lang is obtained by performing the same process as described in the MLM process above.

[0143] The input data embedding unit 2 outputs the distributed representation data acquired as described above to the XLM processing unit 3 as data D4((L2-L1).mask).

[0144] The neural network model for XLM 31 of the XLM processing unit 3 executes XLM processing on the data D4((L2-L1).mask) output from the input data embedding unit 2 to obtain data D5x (=D5x((L2-L1).mask)). The XLM processing unit 3 then outputs the data D5x((L2-L1).mask) obtained as described above to the first output data evaluation unit 4.

[0145] The first output data evaluation unit 4 inputs the data D5x((L2-L1).mask) output from the XLM processing unit 3 and the correct data D_correct(L2-L1) output from the input data acquisition unit 1. The first output data evaluation unit 4 acquires a loss from the data D5x((L2-L1).mask) output from the XLM processing unit 3 and the correct data D_correct(L2-L1).

[0146] (1.2.1.1C: Loss Calculation) After performing the above processes, that is, (A) MLM process (using four types of monolingual data) (step S111) and (B) TLM process (using L1-L2 bilingual data) (step S112), the first output data evaluation unit 4 performs a loss calculation process. When the loss calculation is performed for each M sentences (M sentences, M: natural number), the number of subwords included in the i-th sentence among the M sentences is set to N i (1≦i≦M), and the output data D5x(Xin )(Input data to X in jth (j: natural number, 1≦j≦N i The data corresponding to the subword of D5x((X in →X out ),i,j)("(X in →X out )" means that the input data is X in and the output data is X out (which indicates that the loss is smaller than the original loss), the first output data evaluation unit 4 obtains the loss from the data output from the XLM processing unit 3 and the correct answer data, as shown in the following formula.

number

number

number

number

[0147] Also, in the above formula, for example, D5x((L1.mask→L1),i,j) is an n1-dimensional real vector (a real vector indicating the probability that each dimensional data (vector element) is the corresponding subword), which is randomized using the softmax function (normalized so that the sum of each element of the real vector is "1"). And, D5x((L1.mask→L1),i,j) is the output data from the XLM processing unit 3 when the input data is L1.mask, and is data equivalent to the j-th subword of the i-th sentence.

[0148] In the above formula, loss(p, q) is a formula for calculating the cross entropy error, which makes it possible to quantify the difference between the probability distributions (probability distributions p, q in the above formula).

[0149] In this way, the first output data evaluation unit 4 obtains the loss by the process corresponding to the above formula.

[0150] (1.2.1.1D: Parameter update process) The first output data evaluation unit 4 evaluates the parameters θ of the neural network model 31 for XLM in the XLM processing unit 3 based on the loss (learning loss) calculated above for the predetermined learning data. XLM Data update(θ XLM ) and generate the data update(θ XLM ) is output to the XML processing unit 3.

[0151] The neural network model 31 for XLM in the XLM processing unit 3 performs the data update(θ XLM ) based on the parameter θ XLM Update.

[0152] In addition, the neural network model for XLM 31 updates the parameters of the neural network model for XLM 31, and then outputs parameter update data update(θemb ) and generate the parameter update data update(θ emb ) is output to the input data embedding unit 2.

[0153] The input data embedding unit 2 embeds parameter update data update(θ emb ), the parameters of the embedding layer (corresponding to the token embedding unit 21, the position embedding unit 22, and the language embedding unit 23 of the input data embedding unit 2) are updated. For example, the input data embedding unit 2 updates the parameter update data update(θ emb ), a transformation matrix (e.g., matrix W token , W pos , W lang ) to update the elements (values).

[0154] The pseudo-parallel data generation device 100 repeatedly executes the above process (the process of loop 1 in FIG. 6) until a predetermined end condition is satisfied. At that time, for example, the loss (test loss) calculated by the first output data evaluation unit 4 for adjustment data different from the data used for learning is referred to as an evaluation value.

[0155] Then, the pseudo-parallel translation data generation device 100 ends the XLM learning process when the end condition of the above process (the process of loop 1 in FIG. 6) is satisfied, thereby completing the initialization of the input data embedding unit 2.

[0156] That is, when the learning process of the XLM is completed, the parameters for acquiring the distributed representation data set in the input data embedding unit 2 (for example, the transformation matrix W token , W pos , W lang ) is set as the initial value (initial parameter) for the learning process of the NMT model for generating pseudo-parallel translation data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5).

[0157] In other words, in the pseudo-parallel data generation device 100, the state in which the parameters are set at the time the XLM learning process is completed (the state of the input data embedding unit 2) is set as the initial state of the learning process of the NMT model for generating pseudo-parallel data, and the learning process of the NMT model for generating pseudo-parallel data is started.

[0158] The end condition of the loop process (loop 1) in the flowchart of FIG. 6 is set, for example, as follows. (1) The loop process (loop 1) was executed a predetermined number of iterations. (2) In the learning process of the cross-language language model (XLM), the evaluation value in the first output data evaluation unit 4 did not show a change of a certain level or more (a value determined in advance). (3) In the learning process of the cross-language language model (XLM), the evaluation value in the first output data evaluation unit 4 falls below a predetermined value. (4) In the learning process of the cross-language language model (XLM), the evaluation value in the first output data evaluation unit 4 was not updated a predetermined number of times.

[0159] When the above termination condition is satisfied, the pseudo-parallel data generation device 100 determines that the XLM learning process is complete, and terminates the XLM learning process.

[0160] Then, with the parameters set to the above initial values ​​as described above, the pseudo-parallel data generation device 100 performs a learning process for an NMT model for generating pseudo-parallel data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5), which will be described below.

[0161] (1.2.1.2: Learning process of NMT model for generating pseudo-parallel data) Next, in the pseudo-parallel data generation device 100, a learning process (steps S120 to S126) of the NMT model for generating pseudo-parallel data is executed.

[0162] The training process for the NMT model for generating pseudo-parallel data is as shown in Figure 6. (A) Auto-encoding process (step S121); (B) Zero-shot folded machine translation processing (step S122); (C) Machine translation processing with teacher data (step S123); (D) Loss calculation process (step S124); (E) Parameter (θ MT ,θ emb ) update process (step S125); It consists of:

[0163] As shown in the flowchart in FIG. 6, the processing in steps S121 to S125 is repeatedly executed until a termination condition is satisfied (loop processing (loop 2) S120 to S126 in FIG. 6).

[0164] (1.2.1.2A: Self-encoding process) In the self-encoding process, the pseudo-parallel data generation device 100 performs a learning process (a learning process of the NMT model for generating pseudo-parallel data) so that output data identical to the input data to the input data embedding unit 2 is output, as shown in Fig. 8. In other words, in the self-encoding process, the correct answer data is set to the same data as the input data. In other words, in the self-encoding process, the pseudo-parallel data generation device 100 (1) The input to the input data embedding unit 2 is D3(L1), and the correct data of the output data D5(L1) of the machine translation processing unit 5 is D_correct(L1). (2) The input to the input data embedding unit 2 is D3(L2), and the correct data of the output data D5(L2) of the machine translation processing unit 5 is D_correct(L2). (3) The input to the input data embedding unit 2 is D3(R1), and the correct data of the output data D5(R1) of the machine translation processing unit 5 is D_correct(R1), and (4) The input to the input data embedding unit 2 is D3(R2), and the correct data of the output data D5(R2) of the machine translation processing unit 5 is D_correct(R2), and a learning process (parameter optimization process by self-encoding process) is executed.

[0165] <A1: Self-encoding process (L1→L1)> A case will be described in which the input to the input data embedding unit 2 is D3(L1) and the correct data of the output data D5(L1) from the machine translation processing unit 5 is D_correct(L1).

[0166] The data input interface IF1 reads out monolingual data D0(L1) in a first language in another field from the first monolingual data storage unit DBm(L1), and outputs the read monolingual data to the input data acquisition unit 1 as data D1 (=D1(L1)).

[0167] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts (1) token data xi0 from data D1 (data D1(L1) in a first language in another field) (data D1(L1) constituting a sentence in a first language in another field). token and (2) location data xi for identifying the location of the token. pos and (3) language data for identifying the language of the token. lang And, get.

[0168] Then, the input data acquisition unit 1 receives the token data xi0 token is output to the masking processor 12.

[0169] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 not to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0170] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token The token data xi token to the first selector SEL1.

[0171] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos(L1)) is output to the first selector SEL1, and language data xi lang (=xi lang )(=xi lang (L1)) is output to the first selector SEL1.

[0172] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(L1)={xi token (L1),xi pos (L1),xi lang (L1)}) to the first selector SEL1.

[0173] In accordance with the control signal CTL2, the correct data acquisition unit 13 generates correct data D_correct (=D_correct(L1)) to be used for the learning process of the NMT model for generating pseudo-parallel translation data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5) from the data D1 (=D1(L1)). Specifically, the correct data acquisition unit 13 sets the same data as the input data D2a(L1) to the first selector SEL1 as the correct data D_correct(L1). Then, the correct data acquisition unit 13 outputs the correct data D_correct(L1) generated as described above to the second output data evaluation unit 6.

[0174] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0175] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a(L1) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3(L1).

[0176] The token embedding unit 21 of the input data embedding unit 2 embeds token data xi token Enter the token data you entered. token Distributed representation data xi' tokenis obtained by performing the same process as described in the MLM process above.

[0177] The position embedding unit 22 of the input data embedding unit 2 embeds the position data xi pos Enter the position data xi pos Distributed representation data xi' pos is obtained by performing the same process as described in the MLM process above.

[0178] The language embedding unit 23 of the input data embedding unit 2 embeds the language data xi lang Enter the language data xi lang Distributed representation data xi' lang is obtained by performing the same process as described in the MLM process above.

[0179] The input data embedding unit 2 outputs the distributed expression data acquired as described above to the machine translation processing unit 5 as data D4(L1).

[0180] The MT neural network model 51 of the machine translation processing unit 5 uses the data D4(L1) (distributed representation data D4(L1)(={xi' token, xi' pos, xi' lang})) and a control signal CTL3 output from the control unit.

[0181] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output data of L1, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0182] The MT neural network model 51 executes machine translation processing on the data D4 (L1) output from the input data embedding unit 2 to obtain data D5 (L1). The MT neural network model 51 sets the type of output data to L1 (other field, first language) in accordance with the control signal CTL3.

[0183] The MT neural network model 51 outputs the data D5(L1) thus acquired to the second selector SEL2.

[0184] The control unit generates a selection signal sel2 that selects the terminal "0" of the second selector SEL2, and outputs the selection signal sel2 to the second selector SEL2.

[0185] The second selector SEL2 selects the terminal “0” in accordance with the selection signal sel2, and outputs the data D5 (data D5(L1)) output from the machine translation processing unit 5 to the second output data evaluation unit 6 as data D6a (data D6a(L1)).

[0186] The second output data evaluation unit 6 receives the data D5(L1) (=data D6a(L1)) output from the machine translation processing unit 5 via the second selector SEL2, and the correct data D_correct(L1) output from the input data acquisition unit 1. The second output data evaluation unit 6 acquires a loss from the data D5(L1) that is the output of the machine translation processing unit 5, and the correct data D_correct(L1).

[0187] <A2: Self-encoding process (L2→L2)> The pseudo-parallel data generation device 100 also executes the same process as above when the input to the input data embedding unit 2 is D3(L2) and the correct data of the output data D5(L2) of the machine translation processing unit 5 is D_correct(L2). That is, in the above process, the pseudo-parallel data generation device 100 replaces D3(L1) with D3(L2) and replaces the correct data D_correct(L1) with the correct data D_correct(L2), and executes the same process as above.

[0188] <A3: Self-encoding process (R1 → R1)> The pseudo-parallel data generation device 100 also executes the same process as above when the input to the input data embedding unit 2 is D3(R1) and the correct data of the output data D5(R1) of the machine translation processing unit 5 is D_correct(R1). That is, in the above process, the pseudo-parallel data generation device 100 replaces D3(L1) with D3(R1) and replaces the correct data D_correct(L1) with the correct data D_correct(R1), and executes the same process as above.

[0189] <A4: Self-encoding process (R2 → R2)> The pseudo-parallel data generation device 100 also executes the same process as above when the input to the input data embedding unit 2 is D3(R2) and the correct data of the output data D5(R2) of the machine translation processing unit 5 is D_correct(R2). That is, in the above process, the pseudo-parallel data generation device 100 replaces D3(L1) with D3(R2) and replaces the correct data D_correct(L1) with the correct data D_correct(R2), and executes the same process as above.

[0190] (1.2.1.2B: Zero-shot back translation processing) In zero-shot turn-back machine translation processing, as shown in Figures 9 and 10, the pseudo-parallel data generation device 100 (1) performs machine translation processing (machine translation processing using an NMT model for generating pseudo-parallel data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the neural network model for MT 51 of the machine translation processing unit 5)) on input data to the input data embedding unit 2 (first machine translation processing), outputting data in a language different from the input data (zero-shot machine translation data), and (2) performs machine translation processing (machine translation processing using an NMT model for generating pseudo-parallel data) on the output data (second machine translation processing), and performs a learning process so that data identical to the input data is output. In other words, in zero-shot turn-back machine translation processing, the correct answer data is set to be the same as the input data.

[0191] Specifically, in the zero-shot back machine translation process, the pseudo-parallel translation data generation device 100 performs the following processes (1) to (6). (1) L1 → R2 → L1: In the first machine translation process, the input to the input data embedding unit 2 is D3 (1) (L1), and the output from the machine translation processor 5 is D5 (1) (R2), and in the second machine translation process, the input to the input data embedding unit 2 is D3 (2) (R2)(=D5 (1) (R2)) and the output from the machine translation processor 5 is D5 (2) (L1), and the correct data is D_correct(L1).

[0192] The input data to the input data embedding unit 2 of the first machine translation process is D3 (1) (x) and the output data from the input data embedding unit 2 are D4 (1) (x) and the input data to the input data embedding unit 2 of the second machine translation process are D3 (2) (x) and the output data from the input data embedding unit 2 are D4 (2) This will be written as (x) (and so on). (2)R1→L2→R1: In the first machine translation process, the input to the input data embedding unit 2 is D3 (1) (R1), and the output from the machine translation processor 5 is D5 (1) (L2), and in the second machine translation process, the input to the input data embedding unit 2 is D3 (2) (L2)(=D5 (1) (L2)) and the output from the machine translation processor 5 is D5 (2) (R1), and the correct data is D_correct(R1). (3)R1→R2→R1: In the first machine translation process, the input to the input data embedding unit 2 is D3 (1) (R1), and the output from the machine translation processor 5 is D5 (1) (R2), and in the second machine translation process, the input to the input data embedding unit 2 is D3 (2) (R2)(=D5 (1)(R2)) and the output from the machine translation processor 5 is D5 (2) (R1), and the correct data is D_correct(R1). (4)L2→R1→L2: In the first machine translation process, the input to the input data embedding unit 2 is D3 (1) (L2), and the output from the machine translation processor 5 is D5 (1) (R1), and in the second machine translation process, the input to the input data embedding unit 2 is D3 (2) (R1)(=D5 (1) (R1)) and the output from the machine translation processor 5 is D5 (2) (L2), and the correct data is D_correct(L2). (5)R2→L1→R2: In the first machine translation process, the input to the input data embedding unit 2 is D3 (1) (R2), and the output from the machine translation processor 5 is D5 (1) (L1), and in the second machine translation process, the input to the input data embedding unit 2 is D3 (2) (L1)(=D5 (1) (L1)) and the output from the machine translation processor 5 is D5 (2) (R2), and the correct data is D_correct(R2). (6)R2→R1→R2: In the first machine translation process, the input to the input data embedding unit 2 is D3 (1) (R2), and the output from the machine translation processor 5 is D5 (1) (R1), and in the second machine translation process, the input to the input data embedding unit 2 is D3 (2) (R1)(=D5 (1) (R1)) and the output from the machine translation processor 5 is D5 (2) (R2), and the correct data is D_correct(R1).

[0193] <B1: Zero-shot folded machine translation processing (L1→R2→L1)> The first input to the input data embedding unit 2 is D3 (1) (L1), and the output from the first machine translation processor 5 is D5(1) (R2), and the second input to the input data embedding unit 2 is D3 (2) (R2)(=D5 (1) (R2)) and the output from the second machine translation processor 5 is D5 (2) (L1) and the correct data is D_correct(L1).

[0194] The data input interface IF1 reads out monolingual data D0(L1) in a first language of another field from the first monolingual data storage unit DBm(L1) and outputs the read monolingual data to the input data acquisition unit 1 as data D1 (=D1(L1)).

[0195] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts (1) token data xi0 from data D1 (data D1(L1) in a first language in another field) (data D1(L1) constituting a sentence in a first language in another field). token and (2) location data xi for identifying the location of the token. pos and (3) language data for identifying the language of the token. lang And, get.

[0196] Then, the input data acquisition unit 1 receives the token data xi0 token is output to the masking processor 12.

[0197] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 not to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0198] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token The token data xi token to the first selector SEL1.

[0199] The first input data acquisition processing unit 11 also acquires position data xi pos(=xi pos (L1)) is output to the first selector SEL1, and language data xi lang (=xi lang (L1)) is output to the first selector SEL1.

[0200] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(L1)={xi token (L1),xi pos (L1),xi lang (L1)}) to the first selector SEL1.

[0201] In accordance with the control signal CTL2, the correct data acquisition unit 13 generates correct data D_correct (=D_correct(L1)) to be used for the learning process of the NMT model for generating pseudo-parallel translation data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5) from the data D1 (=D1(L1)). Specifically, the correct data acquisition unit 13 sets the same data as the input data D2a(L1) to the first selector SEL1 as the correct data D_correct(L1). Then, the correct data acquisition unit 13 outputs the correct data D_correct(L1) generated as described above to the second output data evaluation unit 6.

[0202] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0203] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and transfers the data D2a (L1) output from the input data acquisition unit 1 to the data D3 (1) (L1) and outputs it to the input data embedding unit 2.

[0204] The token embedding unit 21 of the input data embedding unit 2 embeds data D3 (1) Token data xi included in (L1) token Enter the token data you entered. token Distributed representation data xi'token is obtained by performing the same process as described in the MLM process above.

[0205] The position embedding unit 22 of the input data embedding unit 2 embeds data D3 (1) Location data xi included in (L1) pos Enter the position data xi pos Distributed representation data xi' pos is obtained by performing the same process as described in the MLM process above.

[0206] The language embedding unit 23 of the input data embedding unit 2 embeds data D3 (1) Language data contained in (L1)xi lang Enter the language data xi lang Distributed representation data xi' lang is obtained by performing the same process as described in the MLM process above.

[0207] The input data embedding unit 2 embeds the distributed representation data acquired as above into data D4 (1) (L1) and output to the machine translation processor 5.

[0208] The MT neural network model 51 of the machine translation processing unit 5 is generated based on the data D4 output from the input data embedding unit 2. (1) (L1) (Distributed Representation Data D4 (1) (L1)(={xi' token, xi' pos, xi' lang})) and a control signal CTL3 output from the control unit.

[0209] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output the data of R2, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0210] The MT neural network model 51 uses the data D4 output from the input data embedding unit 2. (1) Machine translation is performed on (L1) to obtain data D5. (1)In addition, the MT neural network model 51 sets the type of output data to R2 (target field, second language) in accordance with the control signal CTL3.

[0211] The neural network model 51 for MT uses the data D5 obtained above. (1) (R2) is output to the second selector SEL2.

[0212] The control unit generates a selection signal sel2 that selects the terminal "1" of the second selector SEL2, and outputs the selection signal sel2 to the second selector SEL2.

[0213] The second selector SEL2 selects the terminal "1" in accordance with the selection signal sel2, and the data D5 (data D5 (1) (R2)) to the first buffer Buf1 as data D6b (data D6b (1) (R2)) (see Figure 9).

[0214] Next, the second input data acquisition processing unit 7 receives the data D6b' (data D6b (1) (R2)) and input data D6b' (data D6b (1) (R2)) to be input to the input data embedding unit 2 (={xi token (R2),xi pos (R2),xi lang Then, the second input data acquisition processing unit 7 acquires (generates) the acquired data as data D2b (data D2b (2) (R2)(={xi token (R2),xi pos (R2),xi lang (R2))) and output it to the first selector SEL1.

[0215] The control unit generates a selection signal sel1 that selects the terminal "1" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0216] The first selector SEL1 selects the terminal "1" in response to the selection signal sel1, and the data D2b output from the second input data acquisition processing unit 7 is (2) (R2) to data D3 (2) (R2) and outputs it to the input data embedding unit 2.

[0217] The token embedding unit 21 of the input data embedding unit 2 embeds data D3 (2) Token data included in (R2)xi token Enter the token data you entered. token Distributed representation data xi' token is obtained by performing the same process as described in the MLM process above.

[0218] The position embedding unit 22 of the input data embedding unit 2 embeds data D3 (2) Location data xi included in (R2) pos Enter the position data xi pos Distributed representation data xi' pos is obtained by performing the same process as described in the MLM process above.

[0219] The language embedding unit 23 of the input data embedding unit 2 embeds data D3 (2) Language data included in (R2)xi lang Enter the language data xi lang Distributed representation data xi' lang is obtained by performing the same process as described in the MLM process above.

[0220] The input data embedding unit 2 embeds the distributed representation data acquired as above into data D4 (2) (R2) and output to the machine translation processor 5.

[0221] The MT neural network model 51 of the machine translation processing unit 5 is generated based on the data D4 output from the input data embedding unit 2. (2) (R2) (Distributed Representation Data D4 (2) (R2)(={xi' token, xi' pos, xi' lang})) and a control signal CTL3 output from the control unit.

[0222] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output data of L1, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0223] The MT neural network model 51 uses the data D4 output from the input data embedding unit 2. (2) Machine translation is performed on (R2) to obtain data D5. (2) In addition, the MT neural network model 51 sets the type of output data to L1 (other field, first language) in accordance with the control signal CTL3.

[0224] The neural network model 51 for MT uses the data D5 obtained above. (2) (L1) is output to the second selector SEL2.

[0225] The control unit generates a selection signal sel2 that selects the terminal "0" of the second selector SEL2, and outputs the selection signal sel2 to the second selector SEL2.

[0226] The second selector SEL2 selects the terminal "0" in accordance with the selection signal sel2, and the data D5 output from the machine translation processor 5 is (2) (L1) to data D6a (2) (L1) and output it to the second output data evaluation unit 6.

[0227] The second output data evaluation unit 6 evaluates the data D6a output from the second selector SEL2. (2) (L1)(=D5 (2) The second output data evaluation unit 6 receives the data D5 output from the machine translation processing unit 5 and the correct answer data D_correct(L1) output from the input data acquisition unit 1. (2) (L1) and its correct answer data D_correct(L1).

[0228] <B2: Zero-shot folded machine translation processing (R1→L2→R1)> The first input to the input data embedding unit 2 is D3 (1) (R1), and the output from the first machine translation processor 5 is D5 (1) (L2), and the second input to the input data embedding unit 2 is D3 (2) (L2)(=D5 (1) (L2)) and the output from the second machine translation processing unit 5 is D5 (2) In the case where the correct data is D_correct(R1) and the correct answer data is D_correct(R1), the pseudo-parallel translation data generation device 100 executes the same process as above.

[0229] That is, in the above process, the pseudo-parallel translation data generating device 100 receives the first input D3 (1) (L1) to D3 (1) (R1) is output from the machine translation processing unit 5, D5 (1) (R2) to D5 (1) (L2), and the second input D3 to the input data embedding unit 2 (2) (R2) to D3 (2) (L2) is the output D5 from the machine translation processing unit 5 (2) (L1) to D5 (2) (R1), and replaces the correct data D_correct(L1) with the correct data D_correct(R1), and then performs the same processing as described above.

[0230] <B3: Zero-shot folded machine translation processing (R1→R2→R1)> The first input to the input data embedding unit 2 is D3 (1) (R1), and the output from the first machine translation processor 5 is D5 (1) (R2), and the second input to the input data embedding unit 2 is D3 (2) (R2)(=D5 (1) (R2)) and the output from the second machine translation processor 5 is D5 (2) In the case where the correct data is D_correct(R1) and the correct answer data is D_correct(R1), the pseudo-parallel translation data generation device 100 executes the same process as above.

[0231] That is, in the above process, the pseudo-parallel translation data generating device 100 receives the first input D3 (1) (L1) to D3 (1) (R1) is output from the machine translation processing unit 5, D5 (1) (R2) is also D5 (1) (R2), and the second input D3 to the input data embedding unit 2 (2) (R2) is also D3 (2) (R2), and the output from the machine translation processor 5 is D5. (2) (L1) to D5 (2) (R1), and replaces the correct data D_correct(L1) with the correct data D_correct(R1), and then performs the same processing as described above.

[0232] <B4: Zero-shot folded machine translation processing (L2→R1→L2)> The first input to the input data embedding unit 2 is D3 (1) (L2), and the output from the first machine translation processor 5 is D5 (1) (R1), and the second input to the input data embedding unit 2 is D3 (2) (R1)(=D5 (1) (R1)) and the output from the second machine translation processor 5 is D5 (2) In the case where the translated data is D_(L2) and the correct answer data is D_correct(L2), the pseudo-parallel translation data generation device 100 executes the same process as above.

[0233] That is, in the above process, the pseudo-parallel translation data generating device 100 receives the first input D3 (1) (L1) to D3 (1) (L2) is the output D5 from the machine translation processing unit 5 (1) (R2) to D5 (1) (R1), and the second input D3 to the input data embedding unit 2 (2) (R2) to D3 (2) (R1) is output from the machine translation processing unit 5, D5 (2) (L1) to D5 (2)(L2), and replaces the correct data D_correct(L1) with the correct data D_correct(L2), and then performs the same processing as described above.

[0234] <B5: Zero-shot folded machine translation processing (R2→L1→R2)> The first input to the input data embedding unit 2 is D3 (1) (R2), and the output from the first machine translation processor 5 is D5 (1) (L1), and the second input to the input data embedding unit 2 is D3 (2) (L1)(=D5 (1) (L1)) and the output from the second machine translation processor 5 is D5 (2) In the case where the correct data is D_correct(R2) and the correct answer data is D_correct(R2), the pseudo-parallel translation data generation device 100 executes the same process as above.

[0235] That is, in the above process, the pseudo-parallel translation data generating device 100 receives the first input D3 (1) (L1) to D3 (1) (R2) is the output D5 from the machine translation processing unit 5 (1) (R2) to D5 (1) (L1), and the second input D3 to the input data embedding unit 2 (2) (R2) to D3 (2) (L1) is output from the machine translation processing unit 5, D5 (2) (L1) to D5 (2) (R2), and replaces the correct data D_correct(L1) with the correct data D_correct(R2), and then performs the same processing as described above.

[0236] <B6: Zero-shot folded machine translation processing (R2 → R1 → R2)> The first input to the input data embedding unit 2 is D3 (1) (R2), and the output from the first machine translation processor 5 is D5 (1) (R1), and the second input to the input data embedding unit 2 is D3 (2) (R1)(=D5 (1) (R1)) and the output from the second machine translation processor 5 is D5(2) In the case where the correct data is D_correct(R2) and the correct answer data is D_correct(R2), the pseudo-parallel translation data generation device 100 executes the same process as above.

[0237] That is, in the above process, the pseudo-parallel translation data generating device 100 receives the first input D3 (1) (L1) to D3 (1) (R2) is the output D5 from the machine translation processing unit 5 (1) (R2) to D5 (1) (R1), and the second input D3 to the input data embedding unit 2 (2) (R2) to D3 (2) (R1) is output from the machine translation processing unit 5, D5 (2) (L1) to D5 (2) (R2), and replaces the correct data D_correct(L1) with the correct data D_correct(R2), and then performs the same processing as described above.

[0238] In this manner, the pseudo-parallel translation data generation device 100: (1) L1 → R2 → L1 (2) R1 → L2 → R1 (3) R1 → R2 → R1 (4) L2 → R1 → L2 (5) R2 → L1 → R2 (6) R2 → R1 → R2 Zero-shot backtracking machine translation processing is performed for the six patterns (six types) listed above.

[0239] (1.2.1.2C: Machine translation processing with teacher data) In the machine translation process with teacher data, the pseudo-parallel data generation device 100 performs a learning process so that the input data to the input data embedding unit 2 is data in one language of the parallel translation data D0(L1-L2) acquired from the parallel translation data storage unit DBp(L1-L2), and parallel translation data corresponding to the input data is output, as shown in Fig. 11. In other words, in the machine translation process with teacher data, the correct answer data is set based on the parallel translation data D0(L1-L2) read out from the parallel translation data storage unit DBp(L1-L2).

[0240] In the supervised machine translation process, the pseudo-parallel translation data generation device 100 (1) Input data embedding part 2 Let the input to be D3(L1) and the correct data be D_correct(L2), or (2) Input data embedding part 2 Let the input to be D3(L2) and the correct data be D_correct(L1), A learning process (parameter optimization process) is performed on the NMT model for generating pseudo-parallel translation data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5).

[0241] <C1: Supervised machine translation processing (L1 → L2)> A case will be described where the input to the input data embedding unit 2 is D3(L1) and the correct data is D_correct(L2).

[0242] The data input interface IF1 reads out the bilingual data D0(L1-L2) of other fields from the bilingual data storage unit DBp(L1-L2) and outputs the read bilingual data to the input data acquisition unit 1 as data D1(L1-L2).

[0243] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts data in the first language as data D1(L1) from the data D1 (multiple field parallel translation data D1(L1-L2)), and extracts the following from the first language data (language data) D1(L1) (data D1(L1) constituting a sentence in the first language in the other field): (1) token data xi0 token and (2) location data xi for identifying the location of the token. pos and (3) language data for identifying the language of the token. lang And, get.

[0244] Then, the input data acquisition unit 1 receives the token data xi0 tokenis output to the masking processor 12.

[0245] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 not to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0246] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token The token data xi token to the first selector SEL1.

[0247] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos (L1)) is output to the first selector SEL1, and language data xi lang (=xi lang (L1)) is output to the first selector SEL1.

[0248] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(L1)={xi token (L1),xi pos (L1),xi lang (L1)}) to the first selector SEL1.

[0249] The correct data acquisition unit 13 extracts data in the second language from the data D1 (other-field parallel translation data D1(L1-L2)) as D1(L2) (data equivalent to the parallel translation data of data D1(L1) (data in the first language of another field) extracted from the other-field parallel translation data D1(L1-L2)) in accordance with the control signal CTL2, and generates correct data D_correct (=D_correct(L2)) to be used in the learning process of the NMT model for generating pseudo-parallel translation data from the extracted data D1(L2). Then, the correct data acquisition unit 13 outputs the correct data D_correct (=D_correct(L2)) to the second output data evaluation unit 6.

[0250] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0251] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a(L1) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3(L1).

[0252] The token embedding unit 21 of the input data embedding unit 2 embeds token data xi token Enter the token data you entered. token Distributed representation data xi' token is obtained by performing the same process as described in the MLM process above.

[0253] The position embedding unit 22 of the input data embedding unit 2 embeds the position data xi pos Enter the position data xi pos Distributed representation data xi' pos is obtained by performing the same process as described in the MLM process above.

[0254] The language embedding unit 23 of the input data embedding unit 2 embeds the language data xi lang Enter the language data xi lang Distributed representation data xi' lang is obtained by performing the same process as described in the MLM process above.

[0255] The input data embedding unit 2 outputs the distributed expression data acquired as described above to the machine translation processing unit 5 as data D4(L1).

[0256] The neural network model 51 for MT in the machine translation processing unit 5 uses the distributed representation data D4(L1) (={xi' token, xi' pos, xi' lang}) and a control signal CTL3 output from the control unit.

[0257] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output L2 data, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0258] The MT neural network model 51 executes machine translation processing on the data D4 (L1) output from the input data embedding unit 2 to obtain data D5 (L2). The MT neural network model 51 sets the type of output data to L2 (another field, second language) in accordance with the control signal CTL3.

[0259] The MT neural network model 51 outputs the data D5(L2) acquired as described above to the second selector SEL2.

[0260] The control unit generates a selection signal sel2 that selects the terminal "0" of the second selector SEL2, and outputs the selection signal sel2 to the second selector SEL2.

[0261] The second selector SEL2 selects the terminal “0” in accordance with the selection signal sel2, and outputs the data D5 (data D5(L2)) output from the machine translation processing unit 5 to the second output data evaluation unit 6 as data D6a (data D6a(L2)).

[0262] The second output data evaluation unit 6 receives the data D5(L2) (=data D6a(L2)) output from the machine translation processing unit 5 via the second selector SEL2, and the correct data D_correct(L2) output from the input data acquisition unit 1. The second output data evaluation unit 6 acquires a loss from the data D5(L2) that is the output of the machine translation processing unit 5, and the correct data D_correct(L2).

[0263] <C2: Supervised machine translation processing (L2 → L1)> When the input to the input data embedding unit 2 is D3(L2) and the correct data is D_correct(L1), the pseudo-translation data generation device 100 executes the same process as above. That is, in the above process, the pseudo-translation data generation device 100 replaces D3(L1) with D3(L2) and replaces the correct data D_correct(L2) with the correct data D_correct(L1), and executes the same process as above.

[0264] (1.2.1.2D: Loss Calculation) The above process, i.e. (A) Auto-encoding process (step S121), (B) zero-shot folded machine translation processing (step S122); and (C) Machine translation processing with training data (step S123) After executing the above, the second output data evaluation unit 6 performs a loss calculation process. When the loss calculation is performed for each M sentences (M sentences, M: natural number), the number of subwords included in the i-th sentence among the M sentences is set to N i (1≦i≦M), and the output data D5(X in )(Input data to X in jth (j: natural number, 1≦j≦N i The data corresponding to the subword of D5((X in →X out ),i,j)("(X in →X out )" means that the input data is X in and the output data is X out (X in →X m →X out ),i,j)("(X in →X m →X out )" means that the input data is X in The first output data is X m and the second input is X m and the second output is X out(which indicates that the loss is 1 / 100)), the second output data evaluation unit 6 obtains the loss from the data output from the machine translation processing unit 5 and the correct answer data, as shown in the following formula.

number

number

number

number

number

[0265] Also, in the above formula, for example, D5((L1→L2),i,j) is an n1-dimensional real vector (a real vector indicating the probability that each dimensional data (vector element) is the corresponding subword), which is randomized by the softmax function (normalized so that the sum of each element of the real vector is "1"). And, D5((L1→L2),i,j) is the output data when the input data is L1 and the output data type from the machine translation processing unit 5 is L2, and is data corresponding to the j-th subword of the i-th sentence.

[0266] In the above formula, for example, D5((L1 → R2 → L1), i, j) is shown in a format that explicitly indicates the type of input data and the type of translation processing result data (output data), and (L1 → R2 → L1) indicates that in the first processing, data of type L1 is output as data of type R2, and in the second processing, the output of the first processing is used as input and data of type L1 is output.

[0267] In the above formula, loss(p, q) is a formula for calculating the cross entropy error, which makes it possible to quantify the difference between the probability distributions (probability distributions p, q in the above formula).

[0268] In this way, the second output data evaluation unit 6 obtains the loss by the process corresponding to the above formula.

[0269] (1.2.1.2E: Parameter update processing) The second output data evaluation unit 6 evaluates the parameters θ of the MT neural network model 51 of the XLM processing unit 3 based on the loss (learning loss) calculated above for the predetermined learning data. MT Data update(θ MT ) and generate the data update(θ MT ) is output to the machine translation processor 5.

[0270] The ML neural network model 51 of the machine translation processing unit 5 receives data update(θ MT) based on the parameter θ MT Update.

[0271] In addition, the MT neural network model 51 updates the parameters of the MT neural network model 51, and then outputs parameter update data update(θ emb ) and generate the parameter update data update(θ emb ) is output to the input data embedding unit 2.

[0272] The input data embedding unit 2 embeds parameter update data update(θ emb ), the parameters of the embedding layer (corresponding to the token embedding unit 21, the position embedding unit 22, and the language embedding unit 23 of the input data embedding unit 2) are updated. For example, the input data embedding unit 2 updates the parameter update data update(θ emb ), a transformation matrix (e.g., matrix W token , W pos , W lang ) to update the elements (values).

[0273] The pseudo-parallel data generation device 100 repeatedly executes the above process (the process of loop 2 in FIG. 6) until a predetermined end condition is satisfied. At that time, for example, the loss (test loss) calculated by the second output data evaluation unit 6 for adjustment data different from the data used for learning is referred to as an evaluation value.

[0274] Then, the pseudo-parallel data generation device 100 ends the learning process of the NMT model for generating pseudo-parallel data when the end condition of the above process (the process of loop 2 in FIG. 6) is satisfied.

[0275] The end condition of the loop process (loop 2) in the flowchart of FIG. 6 is set, for example, as follows. (1) The loop process (loop 2) was executed a predetermined number of iterations. (2) In the learning process of the NMT model for generating pseudo-parallel data, the evaluation value in the second output data evaluation unit 6 did not show a change of a certain level or more (a value determined in advance). (3) In the learning process of the NMT model for generating pseudo-parallel data, the evaluation value in the second output data evaluation unit 6 fell below a predetermined value. (4) In the learning process of the NMT model for generating pseudo-parallel data, the evaluation value in the second output data evaluation unit 6 was not updated a predetermined number of times.

[0276] If the above termination condition is satisfied, the pseudo-parallel data generation device 100 determines that the learning process of the NMT model for generating pseudo-parallel data is complete, and terminates the process.

[0277] In this manner, the pseudo-parallel data generation device 100 sets the parameters at the time of completing the pre-learning process as the initial state (state in which initial parameters are set (state in which the parameters of the input data embedding unit 2 (corresponding to the embedding layer) are optimized by XLM learning)) of the NMT model for generating pseudo-parallel data (model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5), and performs a learning process (parameter optimization process) of the NMT model for generating pseudo-parallel data by the above process.

[0278] (1.2.2: Pseudo-parallel data generation process) Next, the pseudo-parallel translation data generation process (step S2 in FIG. 4) will be described.

[0279] The pseudo-parallel data generation device 100 automatically generates pseudo-parallel data for the target field using an NMT model for generating pseudo-parallel data (a model realized by the input data embedding unit 2 (corresponding to the embedding layer) and the MT neural network model 51 of the machine translation processing unit 5) obtained by the above-mentioned pre-learning process.

[0280] Specifically, the pseudo-parallel data generation device 100 performs pseudo-parallel data generation processing using the following methods (1) and (2). (1) Method 1: Modifying bilingual data (L1-L2) from other fields for the target field (2) Method 2: Machine translation of monolingual data (R1 or R2) in the target field The pseudo-parallel translation data generation process using the above two methods will be described below.

[0281] (1.2.2.1: Pseudo-parallel data generation process (first method) (using parallel data from other fields)) First, the first method (using bilingual data from different fields) will be explained.

[0282] The data input interface IF1 reads out the bilingual data D0(L1-L2) of other fields from the bilingual data storage unit DBp(L1-L2) and outputs the read bilingual data to the input data acquisition unit 1 as data D1(L1-L2).

[0283] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts data of L1 from data D1 (multi-domain parallel translation data D1(L1-L2)) as data D1(L1), and extracts from the data D1(L1): (1) token data xi0 token (=xi0 token (L1)) and (2) location data xi for identifying the location of the token. pos (=xi pos (L1)) and (3) language data for identifying the language of the token. lang (=xi lang (L1)) and obtain.

[0284] Then, the input data acquisition unit 1 receives the token data xi0 token (L1) is output to the mask processing unit 12.

[0285] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 not to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0286] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token (L1) is the token data xi token (L1) to the first selector SEL1.

[0287] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos (L1)) is output to the first selector SEL1, and language data xi lang (=xi lang (L1)) is output to the first selector SEL1.

[0288] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(L1)={xi token (L1),xi pos (L1),xi lang (L1)}) to the first selector SEL1.

[0289] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0290] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a(L1) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3(L1).

[0291] The input data embedding unit 2 embeds (1) token data xi included in the data D3(L1). token (L1), (2) Position data xi pos (L1), (3) Language data xi lang From (L1), the distributed representation data D4(L1)(={xi' token (L1) , xi' pos (L1) , xi' lang (L1)}) and outputs the acquired distributed expression data D4(L1) to the machine translation processing unit 5.

[0292] The MT neural network model 51 of the machine translation processing unit 5 uses the data D4(L1) (distributed representation data D4(L1)(={xi' token (L1) , xi' pos (L1) , xi' lang (L1)}) and a control signal CTL3 output from the control unit.

[0293] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output the data of R2, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0294] The MT neural network model 51 executes machine translation processing on the data D4 (L1) output from the input data embedding unit 2, acquires data D5 (R2), and holds the data D5 (R2). The MT neural network model 51 sets the type of output data to R2 (target field, second language) in accordance with the control signal CTL3.

[0295] In addition, the first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts data of L2 from the data D1 (multi-disciplinary parallel translation data D1(L1-L2)) as data D1(L2), and from the data D1(L2), (1) token data xi0 token (=xi0 token (L2)) and (2) location data xi for identifying the location of the token. pos (=xi pos (L2)) and (3) language data to identify the language of the token. lang (=xi lang (L2)) and obtain.

[0296] Then, the input data acquisition unit 1 receives the token data xi0 token (L2) is output to the mask processing unit 12.

[0297] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 not to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0298] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token (L2) is the token data xi token (L2) to the first selector SEL1.

[0299] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos (L2)) is output to the first selector SEL1, and language data xi lang (=xi lang (L2)) is output to the first selector SEL1.

[0300] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(L2)={xi token (L2),xi pos (L2),xi lang (L2)}) to the first selector SEL1.

[0301] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0302] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a(L2) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3(L2).

[0303] The input data embedding unit 2 embeds (1) token data xi included in the data D3(L2). token (L2), (2) Position data xi pos (L2), (3) Language Dataxi lang From (L2), the distributed representation data D4(L2)(={xi'token (L2) , xi' pos (L2) , xi' lang (L2)}) and outputs the acquired distributed expression data D4(L2) to the machine translation processing unit 5.

[0304] The MT neural network model 51 of the machine translation processing unit 5 uses the data D4(L2) (distributed representation data D4(L2)(={xi' token (L2) , xi' pos (L2) , xi' lang (L2)})) and a control signal CTL3 output from the control unit.

[0305] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output the data of R1, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0306] The MT neural network model 51 executes machine translation processing on the data D4 (L2) output from the input data embedding unit 2 to obtain data D5 (R1). The MT neural network model 51 sets the type of output data to R1 (target field, first language) in accordance with the control signal CTL3.

[0307] The machine translation processor 5 pairs the data D5(R2) and the data D5(R1) acquired by the above process, and outputs the paired data D5(R2-R1) to the second selector SEL2.

[0308] The control unit generates a selection signal sel2 that selects the terminal "0" of the second selector SEL2, and outputs the selection signal sel2 to the second selector SEL2.

[0309] The second selector SEL2 selects the terminal "0" in accordance with the selection signal sel2, and outputs the data D5 (data D5(R2-R1)) output from the machine translation processing unit 5 to the filter processing unit 8 as data D6a(R2-R1) (see FIG. 12).

[0310] The control unit generates a control signal CTL4 that instructs the filter processing unit 8 to execute a filter process on the pseudo-parallel translation data of R2-R1, and outputs the control signal to the filter processing unit 8.

[0311] The filter processor 8 performs a filter process on the input data D6a (R2-R1) (pseudo bilingual data of R2 and R1) in accordance with the control signal CTL4.

[0312] Here, the pseudo-parallel data (D6a(R2-R1)) for the target field obtained by the above process is data obtained by processing using a neural network model trained to output parallel data for the target field for which there is no training data, and therefore the quality of the parallel data may be low.

[0313] Therefore, the filter processing unit 8 performs a filter process on the pseudo-parallel data (D6a(R2-R1)) of the application destination field obtained by the above process. For example, the filter processing unit 8 assigns a reliability to each sentence pair of the pseudo-parallel data (D6a(R2-R1)) of the application destination field obtained by the above process, and performs filtering based on the assigned reliability.

[0314] For example, it is possible to refer to the original L1-L2 bilingual data (bilingual data D0(L1-L2) acquired from the bilingual data storage unit DBp(L1-L2)). More specifically, the filter processing unit 8 calculates the reliability of the sentence pair bl-b2 of the pseudo bilingual data of the application field R1-R2 acquired by machine translating the sentence pair al-a2 in the bilingual data L1-L2(D0(L1-L2)) based on the similarity between al and bl and the similarity between a2 and b2, and judges whether the reliability is higher than a predetermined threshold Th. Note that the filter processing unit 8 is capable of acquiring the sentence pair al-a2 in the bilingual data L1-L2(D0(L1-L2)) from the bilingual data storage unit DBp(L1-L2) (for example, acquiring it via the data input interface IF1).

[0315] The filter processing unit 8 then obtains, by filtering, only the pseudo-parallel data (R1-R2) whose reliability calculated as above is higher than a predetermined threshold value Th, and outputs the pseudo-parallel data obtained by filtering to the pseudo-parallel data storage unit DB1 as pseudo-parallel data Dpsd1(R1-R2, #1).

[0316] (1.2.2.2: Pseudo-parallel data generation process (second method) (using monolingual data in the target field)) Next, the second method (using monolingual data in the target field) will be explained.

[0317] As shown in FIG. 13, the data input interface IF1 reads out monolingual data D0(R1) in the first language of the target field from the third monolingual data storage unit DBm(R1), and outputs the read out data in the first language to the input data acquisition unit 1 as data D1(R1).

[0318] The first input data acquisition processing unit 11 of the input data acquisition unit 1 extracts, from the data D1 (data D1(R1) of the first language in the application field) and the data (language data) D1(R1) of the first language (data D1(R1) constituting a sentence in the first language in the application field), (1) token data xi0 token (=xi0token (R1)) and (2) location data xi to identify the location of the token. pos (=xi pos (R1)) and (3) language data for identifying the language of the token. lang (=xi lang (R1)) and obtain.

[0319] Then, the input data acquisition unit 1 receives the token data xi0 token (R1) is output to the masking processor 12.

[0320] A control unit (not shown) generates a control signal CTL2 that instructs the masking processing unit 12 of the input data acquiring unit 1 not to execute masking processing, and outputs the control signal CTL2 to the masking processing unit 12.

[0321] In response to the control signal CTL2, the masking processor 12 processes the token data xi0 input from the first input data acquisition processor 11. token (R1) is the token data xi token (R1) is output to the first selector SEL1.

[0322] The first input data acquisition processing unit 11 also acquires position data xi pos (=xi pos (R1)) is output to the first selector SEL1, and language data xi lang (=xi lang (R1)) is output to the first selector SEL1.

[0323] That is, the input data acquisition unit 1 converts the data acquired as described above into data D2a (=D2a(R1)={xi token (R1),xi pos (R1),xi lang (R1)}) to the first selector SEL1.

[0324] The control unit generates a selection signal sel1 that selects the terminal "0" of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1.

[0325] The first selector SEL1 selects the terminal "0" in accordance with the selection signal sel1, and outputs the data D2a(R1) output from the input data acquisition unit 1 to the input data embedding unit 2 as data D3(R1).

[0326] The input data embedding unit 2 embeds (1) token data xi included in the data D3 (R1). token ,(2) Position data xi pos , (3) Language Dataxi lang From the distributed representation data D4(R1)(={xi' token (R1) , xi' pos (R1) , xi' lang (R1)}) and outputs the acquired distributed expression data D4(R1) to the machine translation processing unit 5.

[0327] The neural network model 51 for MT in the machine translation processing unit 5 uses the distributed representation data D4(R1) (={xi' token (R1) , xi' pos (R1) , xi' lang (R1)} and a control signal CTL3 output from the control unit.

[0328] The control unit generates a control signal CTL3 that instructs the MT neural network model 51 to output the data of R2, and outputs the control signal CTL3 to the machine translation processing unit 5.

[0329] The MT neural network model 51 executes machine translation processing on the data D4 (R1) output from the input data embedding unit 2 to obtain data D5 (R2). The MT neural network model 51 sets the type of output data to R2 (target field, second language) in accordance with the control signal CTL3.

[0330] The MT neural network model 51 outputs the data D5(R2) acquired as described above to the second selector SEL2.

[0331] The control unit generates a selection signal sel2 that selects the terminal "0" of the second selector SEL2, and outputs the selection signal sel2 to the second selector SEL2.

[0332] The second selector SEL2 selects the terminal “0” in accordance with the selection signal sel2, and outputs the data D5 (data D5(R2)) output from the machine translation processing unit 5 to the filter processing unit 8 as data D6a(R2) (see FIG. 13).

[0333] The control unit generates a control signal CTL4 that instructs the filter processing unit 8 to execute a filter process on the pseudo-parallel translation data of R1-R2, and outputs the control signal to the filter processing unit 8.

[0334] In addition, the input data acquisition unit 1 outputs the language data D0(R1) of the destination field (data used as input for the machine translation process) acquired from the third monolingual data storage unit DBm(R1) to the filter processing unit 8 as data D1_org(R1).

[0335] The filter processing unit 8 then obtains pseudo-translation data (R1-R2) for the application target field by matching the input (1) data D1_org(R1) (data used as input for machine translation processing) with (2) data D6a(R2) (machine translated data of data D3(R1)).The filter processing unit 8 then performs filtering on the pseudo-translation data (R1-R2) in accordance with the control signal CTL4.

[0336] Here, the pseudo-parallel data (R1-R2) for the target field obtained by the above process is data obtained by processing using a neural network model trained to output word data for the target field for which there is no training data, and therefore may be of low quality as parallel data.

[0337] Therefore, the filter processing unit 8 performs a filter process on the pseudo-parallel data (R1-R2) of the application target field obtained by the above process. The filter processing unit 8 assigns a reliability to the pseudo-parallel data (R1-R2) of the application target field obtained by the above process, for example, by using a technology related to machine translation quality estimation disclosed in the following document D. (Document D): Lucia Specia, Carolina Scarton, and Gustavo Henrique Paetzold (2018). Quality Estimation for Machine Translation. Morgan & Claypool. Then, the filter processing unit 8 performs filtering based on the assigned reliability. For example, the filter processing unit 8 obtains by filtering only the pseudo-parallel data (R1-R2) whose reliability calculated as above is higher than a predetermined threshold Th. The filter processing unit 8 outputs the pseudo-parallel data obtained by the filtering to the pseudo-parallel data storage unit DB1 as pseudo-parallel data Dpsd1(R1-R2, #2).

[0338] In the above, data D1_org is data in the first language of the target field (data of R1 (=D1_org(R1))) and data D3(R1) is translated data obtained by machine translation processing as data D5(R2) (data D6a(R2)) in the second language of the target field (as in the case of Figure 13), but as shown in Figure 14, data D1_org may be data in the second language of the target field (data of R2 (=D1_org(R2))) and data D3(R2) is translated data obtained by machine translation processing as data D5(R1) (data D6a(R2)) in the second language of the target field, and pseudo-translation data (data Dpsd1(R1-R2, #2)) may be generated by processing similar to that described above.

[0339] <Summary> As described above, in the pseudo-parallel translation data generation device 100, (1) Large-scale (large number of bilingual data) and highly accurate bilingual data from other fields (bilingual data stored in the bilingual data storage unit DBp(L1-L2)), (2) first language data of another field (monolingual data stored in the first monolingual data storage unit DBm(L1)); (3) second language data of another field (monolingual data stored in the second monolingual data storage unit DBm(L2)); (4) First language data of the application field (monolingual data stored in the third monolingual data storage unit DBm(R1)); (5) second language data of the application field (monolingual data stored in the fourth monolingual data storage unit DBm(R2)); Then, in the pseudo-parallel translation data generation device 100, the input data embedding unit 2 (corresponding to the embedding layer) after the optimization process (after the pre-learning process) of the cross-language language model (XLM) (a model realized by the input data embedding unit 2 (corresponding to the embedding layer)) (parameters obtained by optimizing the XLM (for example, the transformation matrix W token , W pos , W lang A parameter optimization process is performed on the NMT model for generating pseudo-parallel data, with the state of the NMT model for generating pseudo-parallel data being set as the initial state, the NMT model being made up of the input data embedding unit 2) in which the above is set, and the neural network model for MT 51 of the machine translation processing unit 5 before learning. As a result, the pseudo-parallel data generating device 100 can acquire a model (trained model) that can generate pseudo-parallel data (R1-R2) in the first and second languages ​​of the application target field, even if there is no bilingual data at all in the application target field (the field targeted for machine translation).

[0340] (1.2.3: Translation model learning process) Next, the translation model learning process (step S3 in FIG. 5) will be described.

[0341] The machine translation device MT1 has an NMT model and performs machine translation using the NMT model. The machine translation device MT1 performs a learning process of a machine translation model (NMT model) for performing machine translation in a target field, using pseudo-parallel data generated by the pseudo-parallel data generation device 100 and stored in the pseudo-parallel data storage unit DB1.

[0342] Specifically, for example, the machine translation device MT1 performs a learning process of a machine translation model (NMT model) for performing machine translation in the application target field by any one of the following methods (1) to (3). (1) Fine adjustment method: The machine translation device MT1 acquires the bilingual data D0'(L1-L2) of the other field from the bilingual data storage unit DBp(L1-L2), and performs a learning process of the NMT model of the machine translation device MT1 using the bilingual data D0'(L1-L2) of the other field. After that, the machine translation device MT1 reads out the pseudo-bilingual data (R1-R2 bilingual data) of the application destination field generated by the pseudo-bilingual data generation device 100 from the pseudo-bilingual data storage unit DB1 as pseudo-bilingual data Dpsd2(R1-R2), and performs a learning process (fine-tuning of parameters) of the NMT model of the machine translation device MT1 using the read out pseudo-bilingual data Dpsd2(R1-R2) of the application destination field. (2) Data Mixing: The machine translation device MT1 acquires the bilingual data D0' (L1-L2) of the other field from the bilingual data storage unit DBp (L1-L2), and acquires the pseudo bilingual data Dpsd2 (R1-R2) of the application field from the pseudo bilingual data storage unit DB1. Then, the machine translation device MT1 generates data in which the bilingual data D0' (L1-L2) of the other field (L1-L2) and the pseudo bilingual data Dpsd2 (R1-R2) of the application destination field (R1-R2) are mixed, and the NMT model of the machine translation device MT1 is trained by the generated data. Note that when the bilingual data D0' (L1-L2) of the other field (L1-L2) and the pseudo bilingual data Dpsd2 (R1-R2) of the application destination field (R1-R2) are mixed, the two types of bilingual data may be distinguished by tags. Furthermore, before mixing two types of bilingual data, the mixing ratio of the bilingual data may be changed by oversampling or undersampling one or both of them. (3) Data Mixture Fine Tuning: The machine translation device MT1 acquires the bilingual data D0'(L1-L2) of the other field from the bilingual data storage unit DBp(L1-L2), and performs a learning process of the NMT model of the machine translation device MT1 using the bilingual data D0'(L1-L2) of the other field. After that, the machine translation device MT1 generates data that mixes the bilingual data D0'(L1-L2) that is the same as the bilingual data used in the learning process of the NMT model of the machine translation device MT1 with the pseudo-bilingual data (R1-R2) of the application destination field (the pseudo-bilingual data Dpsd2(R1-R2) of the application destination field read from the pseudo-bilingual data storage unit DB1). Then, the machine translation device MT1 performs a learning process (fine-tuning of parameters) of the NMT model of the machine translation device MT1 using the generated data (data that mixes two types of bilingual data). When mixing the bilingual data D0'(L1-L2) from another field (L1-L2) with the pseudo bilingual data Dpsd2(R1-R2) from the target field (R1-R2), the two types of bilingual data may be distinguished by tags. Also, before mixing the two types of bilingual data, the mixing ratio of the bilingual data may be changed by oversampling or undersampling one or both of them.

[0343] The machine translation device MT1 performs a learning process of the NMT model by any one of the above methods (1) to (3) and acquires optimal parameters of the NMT model. Then, the machine translation device MT1 performs machine translation in the application destination field using the NMT model (trained model) in which the optimal parameters are set. That is, as shown in FIG. 1, when the machine translation device MT1 receives source language data Din_e in the application destination field, it performs machine translation processing on the data Din_e and acquires target language data Dout_j (machine translated data).

[0344] Fig. 15 shows an example of data acquired by the machine translation device MT1. The top part of Fig. 15 shows the original text (source language data), and the bottom part of Fig. 15 shows the machine translation data acquired by the machine translation system 1000 of the present invention. For comparison, the middle part of Fig. 15 shows the result data of the machine translation process of the conventional technology (the result data of the machine translation by the NMT model acquired without learning with the bilingual data of the application target field). The output data in the bottom part of Fig. 15 is the data (machine translation result data) when Japanese-English bilingual data (bilingual data other than the application target field) is used as the bilingual data of the other field, and the application target field is the SNS (Social Networking Service) field.

[0345] As can be seen from FIG. 15, the translation data obtained by machine translation without a target field (conventional machine translation) is not translated correctly, but the machine translation data obtained by the machine translation system 1000 of the present invention is better than the conventional machine translation.

[0346] <Summary> In this way, in the machine translation system 1000, even if there is no bilingual data in the application destination field (target field), the pseudo-bilingual data generation device 100 can generate pseudo-bilingual data in the application destination field, and the NMT model of the machine translation device MT1 can be trained using the pseudo-bilingual data generated by the pseudo-bilingual data generation device 100. Furthermore, in the machine translation system 1000, by performing machine translation processing using the NMT model trained from the pseudo-bilingual data generated by the pseudo-bilingual data generation device 100, it is possible to perform accurate machine translation in the application destination field (target field) even if there is no bilingual data in the application destination field (target field).

[0347] In the machine translation system 1000, no regular bilingual data of the target field is used in training the XLM, training the machine translation model for generating pseudo-bilingual data, or training the NMT model of the target field by the machine translation device MT1. In this respect, the machine translation system 1000 differs greatly from the prior art. Note that in the machine translation system 1000, the most reliable part of the pseudo-bilingual data may be extracted and used as bilingual data for adjustment.

[0348] Furthermore, the machine translation system 1000 is extremely effective when there is no bilingual data in the application destination field (target field) but there is a large amount of bilingual data in other fields and monolingual data in the application destination field.

[0349] [Other embodiments] In the machine translation system 1000, pseudo-parallel data generation device 100, and machine translation device MT1 described in the above embodiments, each block may be individually integrated into a single chip using a semiconductor device such as an LSI, or may be integrated into a single chip to include some or all of the blocks.

[0350] Although we refer to it as LSI here, depending on the level of integration, it may also be called IC, system LSI, super LSI, or ultra LSI.

[0351] The method of integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. It is also possible to use a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI.

[0352] In addition, a part or all of the processing of each functional block in each of the above embodiments may be realized by a program. And a part or all of the processing of each functional block in each of the above embodiments is performed by a central processing unit (CPU) in a computer. Also, the programs for performing each processing are stored in a storage device such as a hard disk or ROM, and are executed in the ROM or by being read into the RAM.

[0353] Each process in the above embodiment may be realized by hardware, or may be realized by software (including cases where it is realized together with an OS (operating system), middleware, or a predetermined library).Furthermore, it may be realized by a mixed process of software and hardware.

[0354] For example, when each functional unit of the above embodiment is realized by software, each functional unit may be realized by software processing using the hardware configuration shown in FIG. 16 (e.g., a hardware configuration in which a CPU (which may be a GPU), a ROM, a RAM, an input unit, an output unit, a communication unit, a memory unit (e.g., a memory unit realized by an HDD, SSD, etc.), an external media drive, etc. are connected via a bus).

[0355] Furthermore, when each functional unit of the above embodiment is realized by software, the software may be realized by using a single computer having the hardware configuration shown in FIG. 16, or may be realized by distributed processing using multiple computers.

[0356] Furthermore, the order of execution of the processing methods in the above-described embodiments is not necessarily limited to that described in the above-described embodiments, and the order of execution can be changed without departing from the gist of the invention.

[0357] The scope of the present invention includes a computer program for causing a computer to execute the above-mentioned method, and a computer-readable recording medium having the program recorded thereon. Examples of computer-readable recording media include flexible disks, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, large-capacity DVDs, next-generation DVDs, and semiconductor memories.

[0358] The computer program is not limited to one recorded on the recording medium, but may be one transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, or the like.

[0359] In the description of this specification and the claims, "optimization" refers to bringing the system into the best state, and a parameter for "optimizing" a system (model) refers to a parameter when the value of the objective function of the system is the optimal value. The "optimum value" is a maximum value when the system is in a better state as the value of the objective function of the system increases, and is a minimum value when the system is in a better state as the value of the objective function of the system decreases. The "optimum value" may also be an extreme value. The "optimum value" may also be one that allows for a predetermined error (measurement error, quantization error, etc.), and may be a value included in a predetermined range (a range that can be considered to have converged sufficiently).

[0360] The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications are possible without departing from the gist of the invention. [Explanation of symbols]

[0361] 1000 Machine Translation Systems 100 Pseudo-parallel data generator 1. Input data acquisition section 11 First input data acquisition processing unit 12 Masking processing section 2. Input data embedding section 21 Token embedding section 22 Position embedding section 23 Language embedding section 3 XLM Processing Section 31 Neural network model for XLM 5 Machine translation processing section 51 Neural network model for MT 8 Filter Processing Section MT1 Machine Translation Device DBp(L1-L2) Parallel data storage unit DBm(L1) First monolingual data storage unit (for monolingual data in other fields (first language)) DBm(L2) ​​Second monolingual data storage unit (for monolingual data in other fields (second language)) DBm(R1) Third monolingual data storage unit (for monolingual data in the application field (first language)) DBm(R2) 4th monolingual data storage unit (for monolingual data in the application field (2nd language)) DB1 Pseudo-parallel data storage

Claims

1. A learning method for a machine translation model for generating pseudo-parallel data, which can perform learning processing by setting parameters and includes an input data embedding unit and a machine learning processing unit, comprising: A multi-field parallel translation data set Dsetp(L1-L2) including a plurality of parallel translation data consisting of other-field first language data, which is data in a first language in a field other than the application field, which is the field targeted by the pseudo parallel translation data to be generated, and other-field second language data, which is translation data in a second language of the other-field first language data; A multi-field monolingual data set Dsetm(L1) including a plurality of data in a first language in a field other than the field of application; A multi-field monolingual data set Dsetm(L2) ​​including a plurality of data in a second language in a field other than the field of application; A monolingual data set Dsetm(R1) in a target field including a plurality of data in a first language in the target field; A monolingual data set Dsetm(R2) in the target field including a plurality of data in a second language in the target field; A learning process can be performed by setting parameters using the above, and for the XLM model including the input data embedding unit and the XLM processing unit, (A) a learning process using a masking process, which is a process of learning such that the loss between the correct answer data and the output of the XLM model is reduced by using masked data obtained by masking a part of monolingual input data, which is data included in the other field monolingual data set Dsetm(L1), the other field monolingual data set Dsetm(L2), the adaptation field monolingual data set Dsetm(R1), and the adaptation field monolingual data set Dsetm(R2), as input and using the monolingual input data as correct answer data; (B) a learning process using supervised data, which is a process of learning such that one of the other field first language data and the other field second language data of the bilingual data included in the other field bilingual data set Dsetp(L1-L2) is used as an input, the other data is used as correct answer data, and a loss between the correct answer data and the output of the XLM model is reduced. and an initialization step of acquiring optimal parameters of the XLM model by performing the above-mentioned steps, and setting the optimal parameters set in the input data embedding unit of the XLM model in which the optimal parameters have been set as initial parameters of the input data embedding unit of the pseudo-parallel data generating machine translation model; The pseudo-parallel data generating machine translation model includes the input data embedding unit in which the initial parameters are set and the machine learning processing unit, (1) a self-encoding process in which correct answer data is made the same as input data and a learning process is performed on the machine translation model for generating pseudo-parallel translation data; (2) a zero-shot return machine translation process in which output data of the pseudo-parallel data generating machine translation model for input data is input again to the pseudo-parallel data generating machine translation model, and a learning process of the pseudo-parallel data generating machine translation model is performed so that the output of the pseudo-parallel data generating machine translation model becomes identical to the input data; (3) A supervised machine translation process in which either the first language data or the second language data included in the multi-domain parallel translation data set Dsetp(L1-L2) is used as an input to the machine translation model for generating pseudo-parallel translation data, and the other is used as correct answer data to perform a learning process for the machine translation model for generating pseudo-parallel translation data; an optimization step of acquiring optimal parameters of the machine translation model for generating pseudo-parallel data by performing a learning process using at least one of the above; Equipped with The pseudo-parallel data generating machine translation model is The device is capable of outputting a type of data designated by a control signal, and is configured to output any one of (1) the other field first language data, (2) the other field second language data, (3) data in the first language of the application field, and (4) data in the second language of the application field, designated by the control signal. A method for training a machine translation model for generating pseudo-parallel data.

2. A pseudo-parallel data acquisition method for acquiring pseudo-parallel data in a target field by using a machine translation model for generating pseudo-parallel data acquired by the method for learning a machine translation model for generating pseudo-parallel data according to claim 1, comprising: a first machine translation step of performing machine translation processing on the first language data acquired from the other field parallel translation data set Dsetp(L1-L2) using the machine translation model for generating pseudo-parallel translation data, the output of which is set by the control signal to be data in the second language of the application target field, to acquire second language pseudo-translated data in the application target field, which is machine translation result data of the first language data in the other field; a second machine translation step of performing machine translation processing on the second language data acquired from the other field parallel translation data set Dsetp(L1-L2) using the machine translation model for generating pseudo-parallel translation data, the output of which is set by the control signal to be data in the first language of the application target field, to acquire first language pseudo-translated data in the application target field, which is machine translation result data of the other field second language data; a pseudo-translation data acquisition step of acquiring pseudo-translation data of the target field by associating the second language pseudo-translation data of the target field acquired in the first machine translation step with the first language pseudo-translation data of the target field acquired in the second machine translation step; A pseudo-parallel data acquisition method comprising:

3. A pseudo-parallel data acquisition method for acquiring pseudo-parallel data in a target field by using a machine translation model for generating pseudo-parallel data acquired by the method for learning a machine translation model for generating pseudo-parallel data according to claim 1, comprising: a monolingual data machine translation step of acquiring second language pseudo-translated data in the target field which is data resulting from the machine translation of the first language data in the target field, or first language pseudo-translated data in the target field which is data resulting from the machine translation of the second language data in the target field, by performing machine translation processing using the machine translation model for generating pseudo-translation data, which is set by the control signal to output data in the second language in the target field, or data in the first language in the target field, for data in the first language acquired from the monolingual data set Dsetm(R1) in the target field, or data in the second language acquired from the monolingual data set Dsetm(R2) in the target field; a pseudo-parallel data acquisition step of acquiring pseudo-parallel data of the target field by associating the first language data of the target field, which is used as an input of the pseudo-parallel data generating machine translation model in the monolingual data machine translation step, with the second language pseudo-translated data of the target field acquired in the monolingual data machine translation step, or by associating the second language data of the target field, which is used as an input of the pseudo-parallel data generating machine translation model, with the first language pseudo-translated data of the target field acquired in the monolingual data machine translation step; A pseudo-parallel data acquisition method comprising:

4. a filtering step of obtaining a reliability indicating the accuracy of a result of a machine translation process for each sentence pair of the pseudo-parallel translation data of the application target field obtained in the pseudo-parallel translation data obtaining step, and selecting and outputting only the pseudo-parallel translation data including sentence pairs for which the obtained reliability is equal to or greater than a predetermined value; 4. The pseudo-parallel translation data acquisition method according to claim 2 or 3.

5. A machine translation model capable of performing a learning process by setting parameters, comprising a learning method for said machine translation model for performing machine translation on data in a first language in a target field to obtain data in a second language, comprising: Pseudo-translation data for a target field acquired by the pseudo-parallel data acquisition method according to any one of claims 2 to 4; A multi-field parallel translation data set Dsetp(L1-L2) including a plurality of parallel translation data consisting of a first language data of a field other than the application field, and a second language data of a field other than the application field, which is a translation data of the first language data of the other field into a second language; to the machine translation model, (A) a learning process in which the other field first language data of the bilingual data included in the other field bilingual data set Dsetp(L1-L2) is used as an input, the other field second language data of the bilingual data included in the other field bilingual data set Dsetp(L1-L2) is used as correct answer data, and a loss between the correct answer data and an output of the machine translation model is reduced; (B) a learning process in which data in the first language of the target field included in the pseudo-translated data is used as input, data in the second language of the target field included in the pseudo-translated data is used as correct answer data, and loss between the correct answer data and the output of the machine translation model is reduced; a machine translation model learning step of performing a learning process in which optimal parameters of the machine translation model are obtained by performing the above step, and the optimal parameters are set in the machine translation model to obtain a trained machine translation model; A method for training a machine translation model comprising:

6. A machine translation device that performs machine translation processing using a trained machine translation model acquired by the method for training a machine translation model for generating pseudo-parallel data according to claim 1, or a trained machine translation model acquired by the method for training a machine translation model according to claim 5.

Images