Method and system for generating cross-language texts with enhanced combinatorial generalization capabilities
By generating context for the training dataset and introducing random perturbations, the combinatorial generalization ability of neural network models is improved, solving the problem of low efficiency in existing technologies and achieving more accurate cross-language text generation.
Patent Information
- Application Number
- CN202410813598.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-06-24
AI Technical Summary
Existing sequence-to-sequence models based on neural networks have limitations in their combinatorial generalization ability, leading to prediction errors when faced with unfamiliar language combinations. Furthermore, existing methods for increasing training and testing overhead are inefficient, limiting the application scope of these models.
By generating context for sentence pairs in the training dataset and introducing random perturbations during training, calculating combinatorial similarity, and determining an appropriate set of contexts, the model's combinatorial generalization ability is improved.
Without changing the model framework and training method, the model's combinatorial generalization ability is significantly improved, enabling it to handle novel combinatorial inputs more accurately.
Smart Images

Figure CN118378636B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of natural language processing, and in particular to a method and system for generating cross-language texts for enhancing combinatorial generalization capabilities. Background Art
[0002] Compositional generalization aims to enable the model to understand and infer the meaning of unknown combinations based on limited known knowledge. It is a sub-research problem of out-of-distribution generalization. For example, the model has learned the translation of red car and small dog using training data, and enabling the model to decompose and infer the translation of small car and red dog is the research scope of compositional generalization. In recent years, sequence-to-sequence models based on neural networks, such as Transformer, have made great progress in natural language processing. However, it is also found that whether it is task-specific or based on the support of pre-trained models, the model still has great limitations in its compositional generalization ability. That is to say, although the model may have seen a variety of combinations in training, it often makes wrong predictions when facing language combinations that have not been seen in training during the inference stage.
[0003] Faced with such defects, the most direct approach of researchers is to introduce some new modules or new training algorithms in order to improve the combinatorial generalization ability of the model. For example, the Dangle-Transformer model dynamically re-encodes the source semantic representation at each decoding to achieve the purpose of decoupling the representation. The Edge-Transformer model models the traditional scalar attention mechanism as a vector, thereby enhancing the reasoning and combinatorial generalization ability of the model to a certain extent. Another way, such as the MAML-Transformer model, is to change the training paradigm from supervised learning to gradient-based meta-learning to enhance the model's out-of-distribution generalization ability.
[0004] However, technical methods such as Dangle-Transformer or Edge-Transformer that introduce additional modules or mechanisms will significantly increase the cost of training and testing. For example, the computational complexity of the above two models is O (n 3), the efficiency of the model is significantly lower than the original Transformer model when processing long sequences, thus limiting the application space of the model. Although the MAML-Transformer model does not change the model framework, it still needs to construct a data sampling method that conforms to the meta-learning paradigm during training. The optimization process is more complicated, involving quadratic differentials, and the improvement of the final combined generalization performance is also relatively limited. Therefore, no technical solution has been found in the prior art that can significantly increase the combined generalization ability of the model without changing the original model framework and excessively increasing the cost of model training. Summary of the invention
[0005] The present application is provided to solve the above defects in the prior art. A method and system for generating cross-language text with enhanced combination generalization capability is needed, which can adapt the learning model to more novel combination inputs, thereby having stronger combination generalization capability when generating cross-language text for source language text.
[0006] According to a first solution of the present application, a method for generating cross-language text for enhancing combinatorial generalization capability is provided, comprising receiving, by a processor, a source language text of a cross-language text to be generated input by a user; generating a target cross-language text based on the received source language text using a trained first learning model, wherein the first learning model is trained based on a training data set containing source language text-cross-language text sentence pairs, wherein at least some of the sentence pairs are associated with contexts generated based on the training data set; and when generating a context for a first sentence pair in the training data set: performing a first random perturbation on a first-level language unit in the source language text of the first sentence pair to generate a first source language text set; determining a plurality of second sentence pairs in the training data set, and performing a first random perturbation on a first-level language unit in the source language text of each second sentence pair to generate a corresponding second source language text set; calculating a combined similarity between the first sentence pair and each second sentence pair based on the first source language text set and each second source language text set, and determining a set of at least some of the second sentence pairs as a context associated with the first sentence pair based on the combined similarity.
[0007] According to a second solution of the present application, a system for generating cross-language texts for enhancing the generalization ability of combinations is provided, comprising an interface configured to obtain a source language text of the cross-language text to be generated input by a user. The system also comprises a processor configured to perform various operations of the method for generating cross-language texts for enhancing the generalization ability of combinations according to various embodiments of the present application.
[0008] According to a third embodiment of the present application, a non-temporary computer-readable storage medium is provided, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, various operations of the method for generating cross-language text for enhancing combinatorial generalization capabilities according to various embodiments of the present application are implemented.
[0009] The methods, systems and media for generating cross-language texts with enhanced combination generalization capabilities provided in various embodiments of the present application generate contexts for training sentence pairs based on the original training data set, so that the learning model can see multiple possibilities of combination during training and efficiently learn the inherent rules of combination, thereby significantly improving the combination generalization capability of the model without having to change the original architecture and training method of the model and without excessively increasing the complexity and overhead of model training, so that when the model receives a source language text containing a novel combination, it can generate a more accurate and reasonable target cross-language text for it.
[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.
[0011] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0013] Figure 1 A schematic flow chart of a method for generating cross-language text for enhancing combination generalization capability according to an embodiment of the present application is shown.
[0014] Figure 2 A schematic flow chart of a method for generating cross-language text for enhancing combinatorial generalization capability according to another embodiment of the present application is shown.
[0015] Figure 3 A schematic diagram showing some components of a system for generating cross-language texts for enhancing combinatorial generalization capabilities according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solution and advantages of the embodiment of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described in conjunction with the drawings of the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all of the embodiments. Based on the described embodiment of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0017] Unless otherwise defined, the technical terms or scientific terms used in this application shall have the common meanings understood by persons with ordinary skills in the field to which this application belongs. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, but do not exclude other elements or objects.
[0018] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. Words such as "include" or "comprise" and similar words mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also covering other elements. The execution order of the various steps in the method described in this application in conjunction with the accompanying drawings is not intended to be limiting. As long as it does not affect the logical relationship between the various steps, several steps can be integrated into a single step, a single step can be decomposed into multiple steps, and the execution order of the various steps can be changed according to specific needs.
[0019] It should also be understood that the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.
[0020] In order to keep the following description of the embodiments of the present application clear and concise, the present application omits detailed descriptions of known functions and known components.
[0021] Figure 1 A schematic flow chart of a method for generating cross-language text for enhancing combination generalization capability according to an embodiment of the present application is shown.
[0022] like Figure 1As shown, in step 101, the processor may receive the source language text of the cross-language text to be generated input by the user. It should be noted that the "source language text" here may be, for example, a specific language text to be translated into a cross-language text, which may be a sentence or a paragraph, and the length of the text and the specific language of the source language and the cross-language are not specifically limited in this application. In other embodiments, the source language text may also be a sentence containing a query task, and the cross-language text is the target language text as the answer to the query task. It can be understood that, usually, in a sentence or a paragraph as the source language text, there are language units of different levels, for example, as mentioned above, "red" and "car" are two word-level language units, which can form an adjective + noun combination "red car"; and "Whodirected Inception" is a combination of words of different parts of speech that conform to the "who do something" template, and so on. It is worth noting that the combination of words may not be limited to the combination of adjectives + nouns, but may also be a combination supported by any other grammatical rules such as verbs + nouns. In addition, the level of the language unit is not limited to the word level, but can also be, for example, a combination of phrases, etc. This application does not impose any specific restrictions on this.
[0023] In step 102, based on the received source language text, a target cross-language text is generated using a trained first learning model, wherein the first learning model is trained based on a training data set comprising source language text-cross-language text sentence pairs, wherein at least some of the sentence pairs are associated with a context generated based on the training data set.
[0024] When generating context for a specific sentence pair in the training dataset, such as the first sentence pair, you can follow Figure 1 The operations of steps 1021 to 1024 shown are performed.
[0025] First, in step 1021, a first random perturbation is performed on the first level language units in the source language text of the first sentence pair to generate a first source language text set.
[0026] Just as an example, for example, the first statement It indicates that, represents the source language text in the first sentence pair, It represents the cross-language text in the first sentence pair. conduct The first random perturbation generates The first source language text set of source language texts can be represented as:
[0027]
[0028] Then, in step 1022, for example, multiple second sentence pairs may be determined in the training data set, and a first random perturbation may be performed on the first-level language units in the source language text of each second sentence pair to generate a corresponding second source language text set.
[0029] For example, for the second sentence pair determined , through conduct The first random perturbation generates The second source language text set of source language texts can be expressed as:
[0030]
[0031] in, The specific value of The same or different, this application does not limit this.
[0032] Next, in step 1023, based on the first source language text set and each second source language text set , calculate the combined similarity between the first sentence pair and each second sentence pair. As an example, when calculating the combined similarity, for example, a similarity-based retrieval method can be used. Specifically, any applicable pre-trained language model that can generate sentence representations, such as ELMO, BERT, etc., can be used to and Each source language text in is mapped to a real number vector used to represent the semantics of the corresponding source language text, thereby generating The set of real vectors and contains The set of real vectors , and The real vector in is calculated as follows:
[0033] ,as well as,
[0034]
[0035] in, represents the pre-trained language model. Then, for example, an algorithm such as cosine similarity is used to calculate Each real vector in The similarity of each real vector in is expressed as real vector and For example, the cosine similarity between the two can be calculated as follows:
[0036]
[0037] in, Representation vector The transpose of Represents the modulus of the orientation vector.
[0038] Then, by taking the mean similarity of each pair of vectors, we finally get With each The set similarity value of is used as the combined similarity between the first sentence pair and each second sentence pair.
[0039] In other embodiments, any applicable vector similarity and set similarity calculation method may be used, which is not listed in detail in this application.
[0040] Then, in step 1024, at least a set of second sentence pairs may be further determined as contexts associated with the first sentence pairs based on the calculated combined similarity. For example, a threshold may be preset to include second sentence pairs with a combined similarity higher than the preset threshold into the context set.
[0041] In some embodiments, an associated context may be generated for each source language text-cross-language text sentence pair in the training data set according to the above steps 1021-1024, or a context may be selectively generated for some sentence pairs. As an example only, for example, the phased test results of the first learning model may be analyzed and evaluated, and according to the evaluation results, relevant combination categories with poor generalization performance may be selected, and a larger number of contexts may be generated for the sentence pairs corresponding to such combinations, etc. The present application does not limit the specific generalization performance evaluation method, etc.
[0042] According to the method of the embodiment of the present application, without changing the standard model framework such as Transformer, RNN, etc., by adding context to the source language text-cross-language text sentence pairs in the training data set, the first learning model can use the information in the context associated with the sentence pairs to better learn the mapping relationship between input and output in a parameter space-guided manner during training, so that the model can dynamically infer model parameters according to the context, and make full use of the combination rules in the context information, so that after the training is completed, the changing parameters can be used to adapt to any novel input. Not only that, the present application also specifically introduces random perturbations in the process of determining the context information, which can make the determined context have the desired similarity in the context of more diverse combinations, so that more accurate and more applicable sentence pairs can be selected as the learning process of the associated context-assisted model, and the learning efficiency is higher. At the same time, the combined generalization ability of the model is further improved, so that the trained first learning model can give more accurate and reasonable outputs when facing novel inputs.
[0043] In some embodiments, the first level of language unit may be, for example, a word or a phrase, and the first random perturbation includes synonym / phrase replacement, near-synonym / phrase replacement, and antonym / phrase replacement. In addition, the perturbation may not be limited to a single level of language unit, but may be performed on multiple levels of the above-mentioned words, phrases, and phrases.
[0044] In some embodiments, performing a first random perturbation on the first-level language units in the source language text of the first sentence pair to generate a first source language text set specifically includes: performing corresponding synonym / phrase replacement, near-synonym / phrase replacement, or antonym / phrase replacement on the selected words or phrases without destroying the original syntactic structure and semantic rationality of the source language text of the first sentence pair.
[0045] As an example only, in the case where the source language text of the first sentence pair contains "red car", the first random perturbation such as synonym / near-synonym replacement can be performed multiple times on the word-level language unit "red", and / or, the first random perturbation such as synonym / near-synonym replacement can be performed on "car", so as to form a first source language text set. On the other hand, any applicable relatively fast and low-computation similarity screening method can be used to select multiple second sentence pairs from the training data set. As an example only, the source language text in the selected second sentence pairs can include combinations such as "small dog", and then the source language text of each second sentence pair can be subjected to a first random perturbation similar to the first sentence pair to generate a corresponding second source language text set.
[0046] In some embodiments, based on the first source language text set and each second source language text set, the combined similarity between the first sentence pair and each second sentence pair is calculated, and based on the combined similarity, a set of at least part of the second sentence pairs is determined as the context associated with the first sentence pair. Any applicable method can be used. As an example only, for example, the average value of the similarity between each source language text in the first source language text set and each source language text in the second source language text set corresponding to each second sentence pair can be first calculated, and the average value can be used as the combined similarity between the first sentence pair and the second sentence pair, and then, the set of second sentence pairs whose combined similarity is higher than a preset second threshold value can be further determined as the context of the first sentence pair. More specifically, when calculating the similarity between each source language text, a measurement index such as cosine similarity and Euclidean distance can be used, and the specific calculation method is not repeated here.
[0047] In some embodiments, a specific method for determining a plurality of second sentence pairs in the training data set may include, for example, calculating a coarse correlation between other sentence pairs in the training data set except the first sentence pair and the first sentence pair, and determining a sentence pair with a coarse correlation higher than a first threshold as a second sentence pair. Among them, the specific method for calculating the coarse correlation may, for example, use the BM2.5 algorithm, the cosine similarity of the SentenceBERT sentence representation, or use the KNN nearest neighbor algorithm, or the LSH algorithm, etc., and the first threshold may be set in association with the acceptable training overhead of the first learning model, so as to specifically set the first threshold by selecting an appropriate number of second sentence pairs.
[0048] In some embodiments, when a sentence pair is associated with a context, the associated context can be spliced with the sentence pair and used as a whole to train the first learning model. In the practice of this application, it is found that without changing the structure of the first learning model at all, the sentence pair and the associated context are simply spliced in a way that the separator is in between, so that the first learning model can better learn the internal logic of the mapping relationship between the source language text input and the cross-language text output in the sentence pair, thereby greatly improving the combination generalization performance of the first learning model, for example, making the generated target cross-language text more consistent with the query task contained in the source language text, and the combination more reasonable, etc. This is not attempted in other prior arts before this application, and the performance improvement brought about is also unexpected.
[0049] Figure 2 A schematic flow chart of a method for generating cross-language text for enhancing combinatorial generalization capability according to another embodiment of the present application is shown.
[0050] Figure 2The steps in the preceding sequence of the process shown and related to the determination of the context of the first statement can be Figure 1 Similar to step 1021-step 1021, based on determining the associated context for the first sentence pair, in step 201, a second random perturbation including asymmetric random perturbation of the first-level language units and / or symmetric random perturbation of the first-level language units can be further performed on at least part of the second sentence pairs in the context.
[0051] Among them, the asymmetric random perturbation of the language units of the first level can specifically include: randomly selecting any word or phrase from the source language text or cross-language text in a given second sentence pair, for example, by querying a knowledge base to find its synonyms / synonymous phrases, and replacing the word or phrase with a synonym or synonymous phrase. It is worth noting that when the randomly selected word or phrase is from the source language text, it does not replace the corresponding word or phrase in the corresponding cross-language text. Conversely, when the randomly selected word or phrase is from the cross-language text, it does not replace the corresponding word or phrase in the source language text. As described above, it is a simple and quick way to implement random perturbation of language units, and it enables the first learning model to effectively learn the combination features based on semantic substitution (perturbation) independence.
[0052] The symmetrical random perturbation of the first-level language unit refers to randomly selecting any word or phrase from the source language text in the given second sentence pair, and replacing the word or phrase and the aligned word or phrase of the word or phrase in the cross-language text. As an example only, the symmetrical replacement of words or phrases can be achieved by means of alignment tools or querying bilingual dictionaries.
[0053] Then, in step 202, the second randomly perturbed context may be concatenated with the first sentence pair and used to train the first learning model.
[0054] By adding asymmetric or symmetric random perturbations to the context, the first learning model can see and learn the characteristics and rules of the combination from more perspectives without increasing the demand for actual parallel corpus training data. In particular, the perturbations can enable the model to understand the inherent laws of semantic relevance and irrelevance more efficiently and deeply, thereby further improving its combinatorial generalization ability when faced with novel combinations.
[0055] In other embodiments, whether it is the random perturbation of the first sentence pair and the second sentence pair before determining the context, or the perturbation of the sentence pair in the determined context, it is not limited to the perturbation at the text level, and similar perturbations can also be performed at the embedding level after generating the corresponding real number vector (that is, the vector representation of the sentence or statement pair, also called embedding). The effect that can be achieved is similar to that of perturbing the sentence pair, and the present application does not impose any limitation on this.
[0056] In some embodiments, when the first learning model is trained based on a training data set containing source language text-cross-language text sentence pairs, unlike other existing sequence-sequence training in which the loss of each token is calculated, in the embodiment of the present application, the target cross-language text is constructed as a whole and the loss function is calculated. The exemplary loss function is It can be calculated as follows:
[0057]
[0058] in, Represents a set containing j-1 elements After defining the loss function, the parameters of the first learning model can be optimized by mini-batch based stochastic gradient descent:
[0059]
[0060] in is the set of model parameters, is the learning rate. Mini-batches are iteratively extracted from the training data to update the parameters until the loss function converges and remains unchanged, and the training is considered complete.
[0061] By calculating and converging the loss function in the above manner, the training of the first learning model can be more focused on improving the combination generalization ability.
[0062] According to an embodiment of the present application, a system for generating cross-language texts with enhanced combination generalization capabilities is also provided. Figure 3 A schematic diagram showing some components of a system for generating cross-language texts for enhancing combinatorial generalization capabilities according to an embodiment of the present application is shown.
[0063] like Figure 3 As shown, the system 300 at least includes an interface 301 and a processor 302. The interface 301 may be configured to obtain a source language text of a cross-language text to be generated that is input by a user.
[0064] In some embodiments, the interface 301 may include a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transmission adapter (such as an optical fiber, USB 3.0, a thunderbolt interface, etc.), a wireless network adapter (such as a WiFi adapter), a telecommunications (3G, 4G / LTE, etc.) adapter, etc., and the present application does not limit this. The system 300 may transmit the source language text of the cross-language text to be generated, etc., which is input by the user, to other parts such as the processor 302 through the interface 301.
[0065] In some embodiments, the interface 301 may also receive, for example, a trained first learning model from a learning network training device (not shown), which are not listed one by one in this application.
[0066] In some embodiments, the processor 302 may be configured to perform various operations of the method for generating cross-language text for enhancing the generalization capability of the combination described in various embodiments of the present application. In some embodiments, the processor 302 may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.
[0067] In some other embodiments, the system 300 may further include a display (not shown), which may be configured to display, for example, a source language text and a generated cross-language text target cross-language text, etc.
[0068] According to an embodiment of the present application, a non-transitory computer-readable storage medium is also provided, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, various operations of the method for generating cross-language text for enhancing the combinatorial generalization capability described in various embodiments of the present application are implemented.
[0069] In some embodiments, the above-mentioned non-transitory computer-readable medium can be a read-only memory (ROM), random access memory (RAM), phase change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), flash disk or other forms of flash memory, cache, register, static memory, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, cassette tape or other magnetic storage device, or any other possible non-transitory medium used to store information or instructions that can be accessed by a computer device.
[0070] According to the method base, system and medium for generating cross-language text for enhancing the generalization ability of combination according to the embodiment of the present application, the context is generated for the training sentence pair based on the original training data set, so that the learning model can see multiple possibilities of combination during training and efficiently learn the inherent rules of combination, thereby achieving the combination generalization ability of the model without having to change the original architecture and training method of the model, and without excessively increasing the complexity and overhead of model training. In addition, by adding asymmetric or symmetric random perturbations to the context, the first learning model can see and learn the characteristics and rules of the combination from more angles without increasing the demand for actual parallel corpus training data, especially the perturbations can make the model more efficient and more deeply understand the inherent laws of semantic relevance and irrelevance, thereby further improving its combination generalization ability when facing novel combinations, and generating more accurate and reasonable target cross-language texts for it.
[0071] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the execution of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the claims and the full scope of their equivalents.
[0072] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the claims and the full scope of equivalent forms granted by these claims.
[0073] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
Claims
1. A method for generating cross-language texts for enhancing combinatorial generalization capability, characterized in that: This includes, by the processor: Receiving source language text of a cross-language text to be generated input by a user; Based on the received source language text, a target cross-language text is generated using a trained first learning model, wherein the first learning model is trained based on a training data set comprising source language text-cross-language text sentence pairs, wherein at least some of the sentence pairs are associated with a context generated based on the training data set; and When generating a context for a first sentence pair in a training data set: performing a first random perturbation on a first-level language unit in a source language text of the first sentence pair to generate a first source language text set, wherein the first-level language unit is a word or a phrase, and the first random perturbation includes synonym / phrase replacement, near-synonym / phrase replacement, and antonym / phrase replacement. Performing a first random perturbation on a first-level language unit in a source language text of the first sentence pair to generate a first source language text set specifically includes: performing corresponding synonym / phrase replacement, near-synonym / phrase replacement, or antonym / phrase replacement on the selected word or phrase without destroying the original syntactic structure and semantic rationality of the source language text of the first sentence pair; A plurality of second sentence pairs are determined in the training data set, and a first random perturbation is performed on a first-level language unit in a source language text of each second sentence pair to generate a corresponding second source language text set; based on the first source language text set and each second source language text set, a combined similarity between the first sentence pair and each second sentence pair is calculated, and based on the combined similarity, a set of at least part of the second sentence pairs is determined as a context associated with the first sentence pair.
2. The method according to claim 1, characterized in that Based on the first source language text set and each second source language text set, calculating the combined similarity between the first sentence pair and each second sentence pair, and determining at least a part of the set of second sentence pairs as the context associated with the first sentence pair based on the combined similarity specifically includes: Calculating an average value of similarities between each source language text in the first source language text set and each source language text in the second source language text set corresponding to each second sentence pair, and using the average value as the combined similarity between the first sentence pair and the second sentence pair; A set of second sentence pairs whose combined similarity is higher than a second threshold is determined as the context of the first sentence pair.
3. The method according to claim 1 or 2, characterized in that: Determining a plurality of second sentence pairs in the training data set specifically includes: The rough correlation between the first sentence pair and other sentence pairs in the training data set except the first sentence pair is calculated, and the sentence pair whose rough correlation is higher than the first threshold is determined as the second sentence pair.
4. The method according to claim 1 or 2, characterized in that: The method further comprises: in the case where the sentence pair is associated with a context, The associated context is concatenated with the sentence pair and then used to train the first learning model.
5. The method according to claim 1, characterized in that The method further comprises: After determining the associated context for the first sentence pair, performing a second random perturbation including an asymmetric random perturbation of the first-level language unit and / or a symmetric random perturbation of the first-level language unit on at least part of the second sentence pairs in the context; The second randomly perturbed context is concatenated with the first sentence pair and used to train the first learning model.
6. The method according to claim 5, characterized in that The asymmetric random perturbation of the language unit at the first level specifically includes: randomly selecting any word or phrase from the source language text or the cross-language text in the given second sentence pair, and replacing the word or phrase with a synonym or a synonym phrase; The symmetric random perturbation of the first-level language unit specifically includes: randomly selecting any word or phrase from the source language text in the given second sentence pair, and replacing the word or phrase and the aligned word or phrase of the word or phrase in the cross-language text.
7. The method according to claim 1 or 2, characterized in that: The method further comprises: When the first learning model is trained based on a training data set containing source language text-cross-language text sentence pairs, the target cross-language text is constructed as a whole and a loss function is calculated.
8. A system for generating cross-language texts with enhanced combinatorial generalization capability, characterized in that: include: The interface is configured to: obtain the source language text of the cross-language text to be generated input by the user; A processor configured to: execute the method for generating cross-language text for enhancing combinatorial generalization capability according to any one of claims 1-7.
9. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed by a processor, the method for generating cross-language text for enhancing combinatorial generalization capability according to any one of claims 1 to 7 is implemented.
