Systems and methods for training translation models using source enhanced training examples

Abstract

Systems and methods for training a translation model based on a first text sequence in a first language, a second text sequence in a second language different from the first language, and tags of sources based on the second text sequence. In some examples, the tag may include an Internet domain, an Internet sub-domain, a uniform resource locator, a website name, or an IP address. In some examples, the tag may also indicate a source of the first text sequence. In some examples, each given training sample may be automatically generated by sampling a first text sequence from a first page of a given Internet domain, sampling a second text sequence from a second page of the given Internet domain, and generating a tag based on all or part of source data of the second page.

Description

Background Art

[0001] The quality of the translation produced by the neural machine translation model may be affected by the quantity and quality of the data used to train the model. Unfortunately, although various automatic methods can be used to collect a large amount of training data, it is difficult to ensure the quality of such data, and manual supervision is usually required. For example, the system can be configured to crawl the Internet to identify a collection of pages published in multiple languages ​​(for example, pages from domains en.website.com and es.website.com can have the same content published in English and Spanish respectively) and separate the corresponding text sequences from which training examples can be generated. However, the training examples from some websites or web pages may be relatively high or low in quality, depending on various factors, such as whether the translation is created or supervised by a human translator, whether the translation is more concise or more lengthy, etc. Similarly, the training examples from some websites or web pages may use specific jargon, which makes them more desirable or less desirable for training a given translation model (for example, web pages for certain regions may use regional specific dialects, and web pages for scientific or legal content may use terms with different meanings in non-scientific or non-legal contexts, etc.). Summary of the invention

[0002] The present technology relates to a system and method for training a translation model using source-enhanced training examples so that the model can learn to associate a specific translation style with the source of each example. For example, in some aspects of the technology, a translation model can be trained based on a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on the source of the second text sequence. In some aspects, the label can include an Internet domain, an Internet subdomain, a uniform resource locator ("URL"), a website name, or an IP address related to the source of the second text sequence. Similarly, in some aspects, the label can also indicate the source of the first text sequence. In addition, in some aspects of the technology, each given training example in a plurality of training examples can be automatically generated by sampling a first text sequence from a first page of a given Internet domain, sampling a second text sequence from a second page of a given Internet domain, and generating a label based on the source of the second text sequence and / or the first text sequence (e.g., all or part of the URL, Internet domain, Internet subdomain, website name, or IP address of the first and / or second page).

[0003] The present technology can therefore generate a translation model that can be prompted during inference to imitate the translation of a particular high-quality or otherwise desired source by including only the label of the source and the input text sequence. These high-quality or desired sources can be identified after training by repeatedly feeding a validation set of examples to the trained translation model using different labels and comparing the quality of the resulting translations (e.g., using automatic quality indicators, human scorers, or a combination thereof). In this way, the present technology can reduce or eliminate the amount of filtering required for a given training data set, thus enabling the use of a large synthetic training sample data set that is automatically collected, generated, and / or filtered to train the translation model. Similarly, the present technology can be used to generate a translation model that can be flexibly and efficiently "adjusted" to imitate different translation qualities and / or styles by simply changing which source labels are used during inference. The present technology can therefore solve the following technical problem: how to control the output of a translation model trained on multiple sources or domains so as to generate translations based on the characteristics of a particular source or domain of interest. In addition, in various example implementations, this can be achieved by training only a single model (rather than training one or more models per domain of interest), thus reducing technical complexity and computational costs.

[0004] In one aspect, the present disclosure describes a computer-implemented method comprising training a translation model, wherein the training comprises: (1) for each given first training example in a plurality of training examples, the given training example comprising a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence: using the translation model, generating a predicted text sequence based at least in part on the first text sequence and the label for the given training example; and using one or more processors of a processing system, comparing the predicted text sequence with the second text sequence to generate a loss value for the given training example; and (2) using one or more processors, modifying one or more parameters of the translation model based at least in part on the loss value generated for each training example in the plurality of training examples. In some aspects, the label comprises an Internet domain. In some aspects, the label comprises an Internet subdomain. In some aspects, the label comprises a uniform resource locator. In some aspects, the label comprises a website name. In some aspects, the label comprises an IP address. In some aspects, the label also indicates a source of the first text sequence. In some aspects, the source of the first text sequence is in a first subdomain of a given internet domain, and the source of the second text sequence is in a second subdomain of a given internet domain. In some aspects, the method further comprises generating, using one or more processors, each given training example in a plurality of training examples by: sampling the first text sequence from a first page of a given internet domain; sampling the second text sequence from a second page of a given internet domain; and generating a label based on all or part of a uniform resource locator of the second page. In some aspects, the method further comprises generating, using one or more processors, each given training example in a plurality of training examples by: sampling the first text sequence from a first page of a given internet domain; sampling the second text sequence from a second page of a given internet domain; and generating a label based on all or part of an IP address of the second page.

[0005] In another aspect, the present disclosure describes a computer program product comprising computer readable instructions which, when executed by a processing system, cause the processing system to perform any of the methods described in the preceding paragraphs.

[0006] In another aspect, the present disclosure describes a processing system comprising: (1) a memory storing a translation model; and (2) one or more processors coupled to the memory and configured to train the translation model according to a training method, the training method comprising: (a) for each given training example in a plurality of training examples, the given training example comprising a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence: using the translation model, generating a predicted text sequence based at least in part on the first text sequence and the label for the given training example; and comparing the predicted text sequence with the second text sequence to generate a loss value for the given training example; and (b) modifying one or more parameters of the translation model based at least in part on the loss value generated for each training example in the plurality of training examples. In some aspects, the one or more processors are configured to train the translation model according to the training method, wherein each given training example comprises a label, the label comprising an Internet domain. In some aspects, the one or more processors are configured to train the translation model according to the training method, wherein each given training example comprises a label, the label comprising an Internet subdomain. In some aspects, one or more processors are configured to train a translation model according to a training method, wherein each given training example includes a label, which includes a uniform resource locator. In some aspects, one or more processors are configured to train a translation model according to a training method, wherein each given training example includes a label, which includes a website name. In some aspects, one or more processors are configured to train a translation model according to a training method, wherein each given training example includes a label, which includes an IP address. In some aspects, one or more processors are configured to train a translation model according to a training method, wherein each given training example includes a label indicating the source of the first text sequence and the source of the second text sequence. In some aspects, one or more processors are further configured to generate each given training example in a plurality of training examples by: sampling a first text sequence from a first page of a given Internet domain; sampling a second text sequence from a second page of a given Internet domain; and generating a label based on all or part of the uniform resource locator of the second page. In some aspects, the one or more processors are further configured to generate each given training example in the plurality of training examples by: sampling a first text sequence from a first page of a given Internet domain; sampling a second text sequence from a second page of the given Internet domain; and generating a label based on all or part of an IP address of the second page.

[0007] In another aspect, the present disclosure describes a processing system comprising: (1) a memory storing a translation model; and (2) one or more processors coupled to the memory and configured to generate a predicted translation of an input text sequence based on the input text sequence and a label using the translation model, wherein the translation module has been trained to generate the predicted translation according to a training method, the training method comprising: (a) for each given training example in a plurality of training examples, the given training example comprising a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence: using the translation model, generating a predicted text sequence based at least in part on the first text sequence and the label for the given training example; and comparing the predicted text sequence with the second text sequence to generate a loss value for the given training example; and (b) modifying one or more parameters of the translation model based at least in part on the loss value generated for each of the plurality of training examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a functional diagram of an example system in accordance with aspects of the present disclosure.

[0009] Figure 2 is a functional diagram of an example system in accordance with aspects of the present disclosure.

[0010] Figure 3 is a flow chart illustrating how exemplary training examples may be generated based on pages of a website according to aspects of the present disclosure.

[0011] Figure 4 Exemplary methods for training a translation model according to aspects of the present disclosure are described.

[0012] Figure 5 Exemplary methods for generating a plurality of training examples according to aspects of the present disclosure are described. DETAILED DESCRIPTION

[0013] The present technology will now be described with respect to the following exemplary systems and methods.Common reference numerals between the figures depicted and described below are intended to identify like features. Example System

[0014] Figure 1A high-level system diagram 100 of an exemplary processing system 102 for performing the methods described herein is shown. The processing system 102 may include one or more processors 104 and a memory 106 storing instructions 108 and data 110. As further described below, the instructions 108 and the data 110 may include a translation model. In addition, the data 110 may store training examples to be used in training the translation model (e.g., training examples used in pre-training, training, or fine-tuning), training signals and / or loss values ​​generated during training, and / or predicted text sequences generated by the translation model.

[0015] The processing system 102 can reside on a single computing device. For example, the processing system 102 can be a server, a personal computer, or a mobile device, and the translation model can therefore be local to the single computing device. Similarly, the processing system 102 can reside on a cloud computing system or other distributed system. In this case, the translation model can be distributed across two or more different physical computing devices. For example, the processing system may include a first computing device that stores layers 1-n of a translation model having m layers, and a second computing device that stores layers nm of the translation model. In this case, the first computing device can be a device (e.g., a personal computer, a mobile phone, a tablet computer, etc.) with less memory and / or processing power than the memory and / or processing power of the second computing device, or vice versa. Similarly, in some aspects of the technology, the processing system may include one or more computing devices that store the translation model, and one or more separate computing devices configured to collect and / or generate training examples (e.g., as described below with respect to Figure 5 In addition, in some aspects of the technology, data used by the translation model (e.g., training data, labels used during inference, etc.) can be stored on a different computing device than the translation model.

[0016] Further in this regard, Figure 2A high-level system diagram 200 is shown in which the exemplary processing system 102 just described is distributed across two computing devices 102a and 102b, each of which may include one or more processors (104a, 104b) and memory (106a, 106b) storing instructions (108a, 108b) and data (110a, 110b). The processing system 102, including the computing devices 102a and 102b, is shown as communicating with one or more websites and / or remote storage systems including a website 204 and a remote storage system 212 via one or more networks 202. In this example, the website 204 includes one or more servers 206a-206n. Each of the servers 206a-206n may have one or more processors (e.g., 208) and associated memory (e.g., 210) storing instructions and data (including the content of one or more web pages). Likewise, although not shown, the remote storage system 212 may also include one or more processors and memory storing instructions and data. In some aspects of the technology, the processing system 102, including the computing devices 102a and 102b, can be configured to retrieve data from one or more of the website 204 and / or the remote storage system 212 for use in training the translation model. For example, in some aspects, the first computing device 102a can be configured to retrieve training examples from the remote storage system 212 for use in pre-training, training, or fine-tuning the translation model hosted on the first computing device 102a and / or the second computing device 102b. Similarly, in some aspects, the first computing device 102a can be configured to store the translation model, while the second computing device 102b can be configured to collect data from the website 204 and generate training examples based on the retrieved data for use in training the translation model (e.g., as described below with respect to Figure 5 In addition, in this case, the second computing device 102b can be configured to store one or more of the generated training examples on the remote storage system 212 for retrieval by the first computing device 102a.

[0017] The processing system described herein can be implemented on any type of computing device (such as any type of general-purpose computing device, server, or collection thereof), and can further include other components typically present in a general-purpose computing device or server. Likewise, the memory of such a processing system can be of any non-transitory type capable of storing information accessible by a processor of the processing system. For example, the memory can include non-transitory media such as hard drives, memory cards, optical disks, solid-state, tape storage, etc. A computing device suitable for the roles described herein can include different combinations of the foregoing, whereby different portions of instructions and data are stored on different types of media.

[0018] In all cases, the computing devices described herein may further include any other components commonly used in conjunction with computing devices such as user interface subsystems. The user interface subsystem may include one or more user inputs (e.g., a mouse, keyboard, stylus, touch screen, and / or microphone) and one or more electronic displays (e.g., a monitor with a screen or any other electrical device operable to display information). Output devices such as speakers, lights, and vibration, pulse, or tactile elements other than electronic displays may also be included in the computing devices described herein.

[0019] The one or more processors included in each computing device may be any conventional processor, such as a commercially available central processing unit ("CPU"), a graphics processing unit ("GPU"), a tensor processing unit ("TPU"), and the like. Alternatively, one or more processors may be dedicated devices, such as ASICs or other hardware-based processors. Each processor may have multiple cores capable of operating in parallel. The processor, memory, and other elements of a single computing device may be stored in a single physical housing, or may be distributed between two or more housings. Similarly, the memory of a computing device may include a hard drive or other storage medium located in a housing different from the housing of the processor (such as in an external database or networked storage device). Therefore, references to processors or computing devices will be understood to include a collection of processors or computing devices or memories that may or may not operate in parallel, as well as references to one or more servers of a load-balanced server farm or a cloud-based system.

[0020] The computing device described herein can store instructions (such as machine code) that can be directly executed by a processor or instructions (such as scripts) that are indirectly executed. The computing device can also store data, which can be retrieved, stored or modified by one or more processors according to the instructions. The instructions can be stored on a computing device readable medium as a computing device code. In this regard, the terms "instructions" and "programs" can be used interchangeably herein. Instructions can also be stored in an object code format to be directly processed by a processor, or stored in any other computing device language, which includes a collection of scripts or independent source code modules that are interpreted or compiled in advance on demand. For example, the programming language can be C#, C++, JAVA or another computer programming language. Similarly, any component of an instruction or program can be implemented with a computer scripting language (such as JavaScript, PHP, ASP or any other computer scripting language). In addition, any of these components can be implemented using a combination of a computer programming language and a computer scripting language. Example Method

[0021] Figure 3 3 is a flowchart 300 showing how exemplary training examples may be generated based on pages of a website according to aspects of the present disclosure. Figure 3 In the example, suppose the website in question is the one described above. Figure 2 2. Assume further that website 204 includes two web pages 302a and 302b. In this example, web page 302a is from URL "http: / / en.website.com / " and includes text in English, while web page 302b is from URL "http: / / es.website.com / " and includes corresponding text in Spanish. Thus, web pages 302a and 302b are from different subdomains of the same root domain (website.com).

[0022] Figure 3 Also shown is a training example 304 that can be generated from the content of web pages 302a and 302b. In this case, training example 304 includes a first text sequence that includes a sentence from web page 302a stated in English, "This page is available in other languages ​​(This page has other language versions)", a second text sequence includes a corresponding sentence from web page 302b stated in Spanish, "Esta página está disponible enotros idiomas (This page has other language versions)", and a label that includes a portion of the URL of web page 302b. It can be understood that a second training example can also be generated, in which the sentence from web page 302b is the "first text sequence", the sentence from web page 302a is the "second text sequence", and the label includes a portion of the URL of web page 302a.

[0023] Although Figure 3 The label in the example of uses the full domain name of the web page 302b, but the label can be based on any suitable information about the source of the web page 302b. For example, in some aspects, the label of the training example 304 can include the full URL of the web page 302b (e.g., http: / / es.website.com / ), the domain and / or subdomain of the web page 302b (e.g., "es.website.com", "website.com", "es", "website", or "com"), the name of the website (e.g., "Website"), the IP address of the web page 302b, and / or any other suitable information about the source of the web page 302b.

[0024] Likewise, although not Figure 3As reflected in the example of , in some aspects of the technology, instead of or in addition to information based on the source of the second text sequence, the label can also include information about the source of the first text sequence. For example, in some aspects, the label of the training example 304 can include information about the source of the web page 302a, such as the full URL of the web page 302a (e.g., http: / / en.website.com / ), the domain and / or subdomain of the web page 302a (e.g., "en.website.com", "website.com", "en", "website", or "com"), the name of the website (e.g., "Website"), the IP address of the web page 302a, and / or any other suitable information related to the source of the web page 302a.

[0025] Furthermore, in some aspects, the labels of training examples 304 may include information that is not directly related to the source of web pages 302a and 302b. For example, where web pages 302a and 302b are obtained from a website or a curated collection of web pages related to a particular topic (e.g., artificial intelligence, law, sports, etc.), the labels of training examples 304 may include (alone or in addition to other source information) information related to the topic.

[0026] The labels may be included in the training examples 304 in any suitable manner and format. For example, in some aspects of the technology, the labels may be appended or attached to the input sequence as vector embeddings, tokenized text, or raw text (thus requiring no additional preprocessing or special vocabulary). In this regard, where training examples have been collected from sources with similar domain names, then including the raw text of the domain name in each label may increase the likelihood that the translation model will infer similarities between the training examples for those domains.

[0027] although Figure 3 The example assumes that training examples 304 will be generated based on text collected from web pages 302a and 302b, but it will be appreciated that training examples may be generated from any suitable source available in more than one language (e.g., a book, a user manual, an advertisement, song lyrics, etc.). Thus, as an example, training examples may be generated from a first sequence of text collected from a book and a second corresponding sequence of text collected from a translated copy of the book, where labels indicate source-based information, such as the title of the book, the title of the translated copy, the author's name, the translator's name, etc.

[0028] Figure 4 An exemplary method 400 for training a translation model according to aspects of the present disclosure is set forth.

[0029] In step 402, a processing system (e.g., Figure 1 or Figure 2The processing system 102 of the embodiment of the present invention selects a given training example from a plurality of training examples, the given training example comprising a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence. The plurality of training examples can come from any suitable source or collection of sources. For example, the plurality of training examples can include training examples from an existing database of training data, artificially generated or artificially supervised examples, and / or synthetically generated (e.g., based on Figure 5 The tag may also include any suitable information about the source of the second text sequence, including the above information about Figure 3 Any of the options discussed for training example 304 .

[0030] In addition, despite Figure 4 Not reflected in the example, but in some aspects of the technology, instead of or in addition to the information based on the source of the second text sequence, the tag may also include other information, as described above with respect to Figure 3 In this regard, in some aspects, instead of or in addition to information based on the source of the second text sequence, the label may also include information about the source of the first text sequence. Similarly, in some aspects, instead of or in addition to information based on the source of the second text sequence, the label may also include information that is not directly related to the source of the first text sequence or the second text sequence (e.g., the subject area or group to which the training example belongs).

[0031] In step 404, the processing system uses the translation model to translate the first text sequence and the label (e.g., Figure 3 The processing system may use a translation model of any suitable type, architecture, and number of parameters to achieve this, including a translation model based on a Transformer architecture, a long short-term memory ("LSTM") architecture, a recurrent neural network architecture ("RNN"), a convolutional neural network ("CNN") architecture, and / or any suitable mixture of these architectures. For example, in some aspects of the technology, the translation model may be a deep LSTM network with multiple encoder and decoder layers (e.g., a 6-layer LSTM encoder and an 8-layer LSTM decoder, an 8-layer LSTM encoder and an 8-layer LSTM decoder, etc.). Similarly, in some aspects of the technology, the translation model may be based on a hybrid architecture, such as an architecture using a transformer as an encoder and an RNN as a decoder (e.g., a 12-layer transformer encoder and a 2-layer RNN decoder).

[0032] In addition, the translation model can directly or indirectly generate the predicted text sequence based on the first text sequence and label of the given training example. Thus, for example, the processing system or translation model can be configured to initially process the first text sequence and / or label to generate a modified version thereof (e.g., a lemma version of the first text sequence and / or label, a vector based on the first text sequence and / or label, etc.). In this case, the translation model can generate the predicted text sequence based on the modified version of the first text sequence and / or label (e.g., a lemma version, a vector, etc.).

[0033] In step 406, the processing system compares the predicted text sequence with a second text sequence of a given training example (e.g., Figure 3 The processing system may perform such a comparison and use any suitable loss function to generate the loss value in any suitable manner. For example, in some aspects of the technology, the processing system may be configured to compare the predicted text sequence to the second text sequence using a "hard distillation" method that evaluates how similar each text string is to another text string. Similarly, in some aspects, the processing system may be configured to use a connectionist temporal classification loss ("CTC loss") or a cross entropy loss to compare the predicted text sequence to the second text sequence.

[0034] In step 408, the processing system determines whether there are additional training examples in the batch. In this regard, the plurality of training examples may be divided into multiple batches, or remain intact, in which case there will be a single "batch" containing each of the plurality of first training examples. In either case, as indicated by the "yes" arrow, if the processing system determines that there are additional training examples in the batch, it will proceed to step 410. In step 410, the processing system will select the next given training example from the batch and then repeat steps 404 to 408 for the newly selected training example. This process will then be repeated for each next given training example in the batch until the processing system determines at step 408 that there are no additional training examples in the batch and therefore proceeds to step 412 (as indicated by the "no" arrow).

[0035] As shown in step 412, after the loss value has been generated for each given training example in the batch (in step 406), the processing system modifies one or more parameters of the translation model based at least in part on the generated loss values. The processing system can be configured to modify the one or more parameters in any suitable manner and at any suitable interval based on these generated loss values. For example, an optimization routine such as stochastic gradient descent can be applied to the generated loss values ​​to determine the parameter modifications. In some aspects of the technology, each "batch" can include a single training example, so that the processing system will perform a back-propagation step in which the processing system modifies one or more parameters of the translation model each time a loss value is generated. Similarly, in the case where each "batch" includes two or more training examples, the processing system can be configured to combine the generated loss values ​​into an aggregate loss value (e.g., by summing or averaging multiple loss values) and modify one or more parameters of the translation model based on the aggregate loss value.

[0036] In step 414, the processing system determines whether there are additional batches in the plurality of training examples. In the event that the plurality of training examples have not been split, and therefore there is a single "batch" containing each of the plurality of training examples, the determination in step 414 will automatically be "no", and the method 400 will then end as shown in step 418. However, in the event that the plurality of training examples have been split into two or more batches, the processing system will follow the "yes" arrow to step 416 to select the next given training example from the plurality of training examples. This will then begin another set of passes through steps 404 to 408 for each training example in the next batch, and make another modification to one or more parameters of the translation model in step 412. This process will continue until there are no additional batches remaining, at which point the processing system will follow the "no" arrow to step 418.

[0037] Although method 400 is shown as ending in step 418 once all of the training examples in the plurality of training examples have been used to adjust the parameters of the translation model, it will be understood that method 400 may be repeated any suitable number of times using the same plurality of training examples until each of the predicted text sequences is sufficiently close to their corresponding second text sequences in each training example. In this regard, in some aspects of the technology, the processing system may be configured to repeat method 400 for a plurality of training examples a predetermined number of times. Further, in some aspects, the processing system may be configured to aggregate all loss values ​​generated during a given pass through method 400 and determine whether to repeat method 400 for a plurality of training examples based on the aggregated loss value. For example, in some aspects of the technology, if the aggregated loss value of the most recent pass through method 400 is greater than a predetermined threshold, the processing system may be configured to repeat method 400 for a plurality of training examples. Likewise, in some aspects, the processing system may be configured to use gradient descent and thus repeat method 400 for a plurality of training examples until the aggregated loss value for a given pass through method 400 is equal to or greater than the aggregated loss value in the pass before it.

[0038] As described above, once the translation model has been trained according to method 400, different labels can be used for testing to determine which labels make the trained translation model produce the highest quality results for a given validation set. For example, if the trained translation model is intended to be used for translation between English and French, a validation set for the language pairing can be obtained (e.g., from a benchmark translation data set, from one or more representative websites or books, etc.). Similarly, if the trained translation model is intended to perform translations in a certain subject area, a validation set can be obtained from a source in the subject area (e.g., a website about the subject, a book about the subject, etc.). Then, the examples of the validation set can be repeatedly fed to the translation model to generate translations using each different label in the candidate label set. The translation set of each candidate label can then be evaluated for quality and compared to identify which labels make the translation model produce the most desired results. These quality assessments can be performed in any suitable manner, such as using any known automatic quality indicators (e.g., BLEU, BLEURT, ROUGE, BERTscore), comparison with the target translation (e.g., if an example from a benchmark training set of target translations including each input text sequence is used), an evaluation by a human grader, or a combination thereof.

[0039] Figure 5 An exemplary method 500 for generating a plurality of training examples according to aspects of the present disclosure is described. In this regard, in some aspects of the technology, Figure 5 An exemplary method can be used to generate Figure 4 Multiple training examples referenced in method 400.

[0040] In step 502, a processing system (e.g., Figure 1 or Figure 2 The processing system 102, Figure 4 The processing system of the method 400 of the present invention and the like) samples a first text sequence from a first page of a given Internet domain (for example, a first text sequence sampled from web page 302a to generate Figure 3 The processing system may perform this sampling in any suitable manner. For example, in some aspects of the technology, the processing system may sample the first text sequence directly from the first page. Likewise, in some aspects, the processing system may download the first page (or a portion thereof), and may then sample the first text sequence from the downloaded copy or portion of the first page.

[0041] In step 504, the processing system samples a second text sequence from a second page of a given Internet domain (e.g., a second text sequence sampled from web page 302b to generate Figure 3 Here again, the processing system may perform this sampling in any suitable manner. For example, in some aspects of the technology, the processing system may sample the second text sequence directly from the second page. Likewise, in some aspects, the processing system may download the second page (or a portion thereof), and may then sample the second text sequence from the downloaded copy or portion of the second page.

[0042] In step 506, the processing system generates a tag based on the source of the second text sequence (e.g., a tag generated based on the URL of the web page 302b to generate Figure 3 training examples 304). As described above for Figure 3 As discussed above with respect to training examples 304 of the processing system, the processing system may generate labels based on any suitable information about the source of the second text sequence, including the information described above with respect to Figure 3 Thus, in some aspects of the technology, the processing system can generate a label based on all or part of the following: the URL of the second page (e.g., "http: / / es.website.com / ", "es.website.com", "website.com", "es", "website", or "com"), the name of the website (e.g., "Website"), the IP address of the second page, and / or any other suitable information related to the source of the second page.

[0043] In addition, despite Figure 5 Not reflected in the example, but in some aspects of the technology, instead of or in addition to the information based on the source of the second text sequence, the tag may also include other information, as described above with respect to Figure 3As discussed above with respect to training examples 304 of the present invention. In this regard, in some aspects, instead of or in addition to information based on the source of the second text sequence, the label may also include information about the source of the first text sequence. For example, the label may include all or part of the following: the URL of the first page (e.g., "http: / / en.website.com / ", "en.website.com", "website.com", "es", "website", or "com"), the name of the website (e.g., "Website"), the IP address of the second page, and / or any other suitable information related to the source of the first page. Similarly, in some aspects, instead of or in addition to information based on the source of the second text sequence, the label may also include information that is not directly related to the source of the first text sequence or the second text sequence (e.g., the subject area or group to which the training example belongs).

[0044] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description of the exemplary systems and methods should be made by way of illustration rather than by way of limitation of the subject matter defined by the claims. In addition, the examples described herein and the provision of clauses expressed as "such as," "including," "comprising," etc. should not be interpreted as limiting the subject matter of the claims to specific examples; rather, these examples are intended to illustrate only some of the many possible embodiments. Further, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A computer-implemented method comprising: Training a translation model, wherein the training comprises: For each given training example in a plurality of training examples, the given training example includes a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence: Using the translation model, generating a predicted text sequence based at least in part on the first text sequence and the label of the given training example; and comparing the predicted text sequence with the second text sequence using one or more processors of a processing system to generate a loss value for the given training example; and One or more parameters of the translation model are modified using the one or more processors based at least in part on the loss value generated for each training example in the plurality of training examples.

2. The method of claim 1, wherein: The tag includes an Internet domain.

3. The method according to claim 1 or claim 2, wherein: The label includes an Internet subdomain.

4. The method according to any one of claims 1 to 3, wherein: The tag includes a uniform resource locator.

5. The method according to any one of claims 1 to 4, wherein: The label includes the website name.

6. The method according to any one of claims 1 to 5, wherein: The label includes an IP address.

7. The method according to any one of claims 1 to 6, wherein: The tag also indicates a source of the first text sequence.

8. The method according to any one of claims 1 to 7, wherein: The source of the first text sequence is in a first subdomain of a given Internet domain, and the source of the second text sequence is in a second subdomain of the given Internet domain.

9. The method according to any one of claims 1 to 8, further comprising: Using the one or more processors, generating each given training example in the plurality of training examples by: sampling the first text sequence from a first page of a given Internet domain; sampling the second text sequence from a second page of the given Internet domain; and The tag is generated based on all or part of a uniform resource locator of the second page.

10. The method according to any one of claims 1 to 9, further comprising: Using the one or more processors, generating each given training example in the plurality of training examples by: sampling the first text sequence from a first page of a given Internet domain; sampling the second text sequence from a second page of the given Internet domain; and The tag is generated based on all or part of the IP address of the second page.

11. A processing system comprising: A memory storing a translation model; as well as One or more processors, the one or more processors being coupled to the memory and configured to train the translation model according to a training method, the training method comprising: For each given training example in a plurality of training examples, the given training example includes a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence: Using the translation model, generating a predicted text sequence based at least in part on the first text sequence and the label of the given training example; and comparing the predicted text sequence with the second text sequence to generate a loss value for the given training example; as well as One or more parameters of the translation model are modified based at least in part on the loss value generated for each training example in the plurality of training examples.

12. The processing system of claim 11, wherein: The one or more processors are configured to train the translation model according to the training method, wherein each given training example comprises a label, wherein the label comprises an Internet domain.

13. A processing system as claimed in claim 11 or claim 12, wherein: The one or more processors are configured to train the translation model according to the training method, wherein each given training example comprises a label, wherein the label comprises an Internet subdomain.

14. The processing system according to any one of claims 11 to 13, wherein: The one or more processors are configured to train the translation model according to the training method, wherein each given training example includes a label, wherein the label includes a uniform resource locator.

15. The processing system according to any one of claims 11 to 14, wherein: The one or more processors are configured to train the translation model according to the training method, wherein each given training example includes a label, wherein the label includes a website name.

16. The processing system of any one of claims 11 to 15, wherein: The one or more processors are configured to train the translation model according to the training method, wherein each given training example comprises a label, wherein the label comprises an IP address.

17. The processing system of any one of claims 11 to 16, wherein: The one or more processors are configured to train the translation model according to the training method, wherein each given training example comprises a label indicating a source of a first text sequence and the source of the second text sequence.

18. The processing system of any one of claims 11 to 17, wherein: The one or more processors are further configured to generate each given training example in the plurality of training examples by: sampling the first text sequence from a first page of a given Internet domain; sampling the second text sequence from a second page of the given Internet domain; and The tag is generated based on all or part of a uniform resource locator of the second page.

19. The processing system of any one of claims 11 to 18, wherein: The one or more processors are further configured to generate each given training example in the plurality of training examples by: sampling the first text sequence from a first page of a given Internet domain; sampling the second text sequence from a second page of the given Internet domain; and The tag is generated based on all or part of the IP address of the second page.

20. A processing system comprising: A memory storing a translation model; as well as one or more processors coupled to the memory and configured to generate a predicted translation of the input text sequence based on the input text sequence and the label using the translation model, Wherein, the translation model has been trained to generate the predicted translation according to a training method, the training method comprising: For each given training example in a plurality of training examples, the given training example includes a first text sequence in a first language, a second text sequence in a second language different from the first language, and a label based on a source of the second text sequence: Using the translation model, generating a predicted text sequence based at least in part on the first text sequence and the label of the given training example; and comparing the predicted text sequence with the second text sequence to generate a loss value for the given training example; and One or more parameters of the translation model are modified based at least in part on the loss value generated for each training example in the plurality of training examples.

21. A computer program product comprising computer readable instructions which, when executed by a processing system, cause the processing system to perform the method of any one of claims 1 to 10.

Images