Cross-language information sorting method and device based on semantic space clustering prompt, equipment and storage medium

Abstract

The invention discloses a cross-language information sorting method and device based on semantic space clustering prompt, equipment and a storage medium, a prompt template of an unlimited language is initialized by averagely pooling a source language prompt template and a target language prompt template embedding vector, and a progressive multi-language transcoding method is introduced. Language bias in the model learning process is reduced; in the answer engineering stage, an answer space construction method based on clustering is provided, and a unified multi-language answer space is constructed through semantic space clustering guided by semantic consistency, so that multi-language prompt learning answers are clustered, and then information sorting categories are obtained. The method solves the problem of poor cross-language information sorting performance caused by inconsistent answer spaces in prompt learning of different languages in the prior art. And the cross-language scene-oriented information sorting task performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence in the direction of computer technology, and in particular to technical fields such as natural language processing and deep learning. Specifically, a method, device, equipment and storage medium for cross-language information sorting based on semantic space clustering prompts are provided. Background Art

[0002] Cross-language information sorting aims to process multilingual text data and sort it into corresponding subject categories. The cross-language information sorting system needs to have cross-language processing capabilities, map different languages ​​to a unified semantic representation space, thereby achieving knowledge transfer from high-resource languages ​​to low-resource languages, and semantic mapping and alignment between different languages. It can recognize and understand text content in different languages ​​and sort it into relevant categories or topics. Currently, cross-language information sorting has been applied in many fields, including social media analysis, news classification, text archiving and retrieval, and sentiment analysis.

[0003] In recent years, prompt learning technology has been widely used in various tasks and can effectively improve the performance of natural language processing tasks. The inventors of this application have found in long-term research and development that when prompt learning technology is directly used for cross-language information sorting tasks, it is difficult to fully activate the knowledge learned in the pre-training model due to the inconsistency of the answer space between the source language and the target language, resulting in a decrease in the performance of cross-language information sorting. To solve this problem, the present invention proposes a cross-language information sorting method based on semantic space clustering prompts, which unifies the answer space of different languages ​​through a semantic clustering method, thereby improving the performance of cross-language information sorting tasks. Summary of the invention

[0004] The purpose of the present invention is to provide a cross-language information sorting method, device, equipment and storage medium based on semantic space clustering prompts. The method includes two stages: prompt engineering and answer engineering. In the prompt engineering stage, the prompt template of the non-restricted language is initialized by averaging the source language and target language prompt template embedding vectors, and a progressive multi-language transcoding method is introduced to reduce the language bias in the model learning process; in the answer engineering stage, the method proposes a clustering-based answer space construction method, and constructs a unified multi-language answer space through semantic space clustering guided by semantic consistency, thereby clustering the answers of multi-language prompt learning, and then obtaining information sorting categories, which improves the information sorting task performance for cross-language scenarios. This method solves the problem of poor cross-language information sorting performance caused by inconsistent answer spaces of different languages ​​in prompt learning in the prior art.

[0005] The cross-language information sorting method based on semantic space clustering prompts described in the present invention is carried out according to the following steps:

[0006] a. Construct a cross-language prompt template for the information sorting task, wherein the cross-language prompt template is a preset template in the source language Preset templates with target language Average pooling result of embedded features;

[0007] b. Perform progressive multilingual transcoding on the text to be sorted, and combine it with the prompt template in step a to generate input variables for prompt learning; the progressive multilingual transcoding is to convert the words in the source language text to be sorted into multiple languages ​​according to the progressive probability r n = r1+(n-1)×d is replaced with a synonym in the target language, and the synonym is searched according to a preset bilingual dictionary;

[0008] c. Input the input variables in step b into the cross-language pre-training model for masked language modeling training and generate prediction features. The input variables are passed through the pre-training model to obtain output features. The prediction features are feature vectors at the [MASK] position in the output features.

[0009] d. Mapping the predicted features to a preset multilingual answer clustering space, and giving a sorting result of the text to be sorted according to the clustering result. The multilingual answer clustering space takes the source language center word of the sorting category as the clustering center, clusters words in different languages ​​through a semantic consistency check, and generates an answer clustering space.

[0010] In step d, the multilingual answer clustering space is composed of a linear layer, a normalization layer and an activation layer. The predicted features are mapped to the multilingual answer clustering space to obtain clustering results and output corresponding sorting labels.

[0011] A device for a cross-language information sorting method based on semantic space clustering prompts, the device comprises a progressive multi-language transcoding module, a prompt engineering module, a model training module and an answer engineering module, wherein:

[0012] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, and generate input text for multilingual transcoding;

[0013] Prompt engineering module: used to construct the cross-language prompt template, and splice the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0014] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0015] Answer engineering module: used to map the prediction features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0016] The progressive multilingual transcoding module includes: a bilingual dictionary construction unit for constructing a source language-target language bilingual dictionary;

[0017] A synonym replacement unit, used to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability;

[0018] The prompt engineering module includes: a prompt template construction unit, which is used to averagely pool the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template;

[0019] An input variable generating unit, used for splicing the prompt template with the input text generated by the progressive multi-language transcoding module to generate an input variable for prompt learning;

[0020] The model training module includes: a multilingual word embedding unit, which is used to convert the multilingual text in the input variable into a word embedding representation feature;

[0021] An encoder unit, for encoding word embedding representation features, understanding and analyzing the semantic information of the input variable;

[0022] The decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0023] The answer engineering module includes: a multilingual answer clustering unit, which is used to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0024] The label mapping unit is used to map clustering features into information sorting labels to generate prediction results.

[0025] An electronic device, comprising: at least one processor; at least one GPU computing card, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card, so that the at least one processor or the at least one GPU computing card can execute the cross-language information sorting method based on semantic space clustering prompts as described in any one of claims 1-3.

[0026] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the cross-language information sorting method based on semantic space clustering prompts according to any one of claims 1 to 3.

[0027] The present invention provides a method, device, equipment and storage medium for cross-language information sorting based on semantic space clustering prompts, the method comprising:

[0028] a. Creating a multi-language prompt template, generating a prompt template for a non-restricted language by calculating the embedding average of the source language prompt template and the target language prompt template; this step aims to create a prompt template that can be used in different languages ​​to reduce the model's dependence on a specific language; wherein, generating a prompt template for a non-restricted language is specifically as follows:

[0029] Constructing prompt templates of the source language and the target language respectively, and adding a [MASK] prediction symbol at the answer prediction position to obtain the embedded features of the two templates, and calculating the average of the two features through an average pooling operation as the prompt template for model learning, that is, the prompt template of the non-restrictive language;

[0030] b. Progressive multilingual transcoding, which is used to replace words in the source language with synonyms in the target language with a progressive probability during the training phase; by gradually increasing the language transcoding probability according to a predetermined progressive ratio, the adaptability of the model to the target language is enhanced and the overfitting of the model to the source language is reduced; the progressive multilingual transcoding is specifically:

[0031] For each source language-target language pair, a bilingual dictionary Dict(s, t) is first constructed, and different transcoding probabilities p are given. The words in the source language text to be sorted are selected with the probabilities, and replaced with synonyms of the words in the target language based on the bilingual dictionary Dict(s, t);

[0032] c. Construct a unified multilingual answer space, integrate answer candidates in different languages ​​through semantic consistency-guided clustering, cluster words with the same or similar meanings in different languages ​​to form a cross-language answer space; in the clustering process, select a topic center word as the cluster center, and learn the cluster space through the K-nearest neighbor (KNN) clustering method; the answer candidate words are specifically:

[0033] For each sorting topic, a topic center word is selected as the cluster center, clustering is performed based on word-level semantic vectors in the source language and the target language, and n answer candidate words are generated for each sorting topic for the multilingual answer space construction;

[0034] d. A clustering-based answer mapper, which can learn how to map the model-predicted embedding to a specific information sorting category label based on the multilingual clustered answer space; the learning process relies on a context initialization method, which is based on word-level pre-labeling of answer candidate words on unlabeled corpus, and trains the answer mapper in a self-supervised manner, thereby mapping the multilingual clustered answer space to a linear classification layer, so that it learns to classify the output features of the semantic clustering module; the answer mapper includes: a linear layer, a normalization layer, and an activation layer; the context initialization method is specifically:

[0035] For each source language-target language pair, an unlabeled corpus is constructed, and each sample is traversed. According to the answer candidate words described in step c, if a candidate word appears in the sample, the word is labeled as the corresponding sorting topic, thereby generating a self-supervised labeled corpus. Based on the corpus, the answer mapper is initialized to fully learn the context information of the candidate word.

[0036] The present invention provides a device for cross-language information sorting based on semantic space clustering prompts, the device comprising:

[0037] Prompt engineering module: used to construct the cross-language prompt template, and splice the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0038] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, and generate input text for multilingual transcoding;

[0039] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0040] Answer engineering module: used to map the prediction features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0041] Furthermore, the prompt engineering module includes:

[0042] A prompt template construction unit, used for performing average pooling on the embedded features of the source language preset template and the target language preset template, thereby generating a cross-language prompt template;

[0043] Furthermore, the progressive multi-language transcoding module includes:

[0044] Bilingual dictionary building unit, used to build source language-target language bilingual dictionary;

[0045] A synonym replacement unit, used to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability;

[0046] An input variable generating unit, used for splicing the prompt template with the input text generated by the progressive multi-language transcoding module to generate an input variable for prompt learning;

[0047] Furthermore, the model training module includes:

[0048] Multilingual word embedding unit, used to convert multilingual text in input variables into word embedding representation features;

[0049] An encoder unit, for encoding word embedding representation features, understanding and analyzing the semantic information of the input variable;

[0050] The decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0051] Furthermore, the answer engineering module includes:

[0052] A multilingual answer clustering unit, used to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0053] The label mapping unit is used to map clustering features into information sorting labels to generate prediction results.

[0054] Another aspect of the present invention provides an electronic device, the device comprising:

[0055] At least one processor; at least one GPU computing card; and,

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to execute the method described in any one of claims 1-3.

[0058] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 3.

[0059] According to yet another aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the embodiments of the present application.

[0060] The technology of the present application can complete the cross-language information sorting task and improve the performance of cross-language information sorting.

[0061] The beneficial effects of the present invention are as follows: in view of the problem of poor cross-language information sorting performance of traditional prompt learning, the present invention proposes a cross-language information sorting method based on semantic space clustering prompts, which unifies the answer spaces of different languages ​​through the semantic clustering method, thereby improving the performance of the cross-language information sorting task.

[0062] In order to achieve the above and related purposes, one or more aspects of the present invention include features that will be described in detail later and are particularly pointed out in the claims. The following description and the accompanying drawings describe some exemplary aspects of the present invention in detail. However, these aspects indicate only some of the various ways in which the principles of the present invention can be used. In addition, the present invention is intended to include all of these aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flow chart of cross-language information sorting based on semantic space clustering prompts provided by the present invention;

[0064] Figure 2 A structural diagram of a cross-language information sorting model based on semantic space clustering prompts provided by the present invention;

[0065] Figure 3 A flow chart of a progressive multi-language transcoding method provided by the present invention;

[0066] Figure 4 A flow chart for constructing a multi-language answer clustering space provided by the present invention;

[0067] Figure 5 A flowchart of an information sorting answer mapping method provided by the present invention;

[0068] Figure 6 A schematic diagram of the structure of a cross-language information sorting device based on semantic space clustering prompts provided by the present invention;

[0069] Figure 7 The present invention is a block diagram of an electronic device for cross-language information sorting based on semantic space clustering prompts. DETAILED DESCRIPTION

[0070] The following is a further detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0071] Example

[0072] The cross-language information sorting method based on semantic space clustering prompts described in the present invention is carried out according to the following steps:

[0073] a. Construct a cross-language prompt template for the information sorting task, wherein the cross-language prompt template is a preset template in the source language Preset templates with target language The average pooling result of the embedded features; in this step, let [MASK] be the answer mask text, and let the preset template of the source language s be T s , The source language prompt template word sequence, then the template can be expressed as Suppose the preset template of the target language t is T t , The template word sequence is the target language prompt, then the template can be expressed as The multi-language template is the average pooling result of the embedding features of the above two templates;

[0074] b. Perform progressive multilingual transcoding on the text to be sorted, and combine it with the prompt template in step a to generate input variables for prompt learning; the progressive multilingual transcoding is to convert the words in the source language text to be sorted into multiple languages ​​according to the progressive probability r n = r1+(n-1)×d is replaced with a synonym in the target language, and the synonym is searched according to a preset bilingual dictionary; wherein, r 1 is the initial replacement probability, d is the probability increase at each step, and the probability of progressive multilingual transcoding at the nth step can be expressed as r n =r1+(n-1)×d;

[0075] c. Input the input variables in step b into the cross-language pre-training model for masked language modeling training and generate prediction features. The input variables are passed through the pre-training model to obtain output features. The prediction features are feature vectors at the [MASK] position in the output features.

[0076] d. Mapping the predicted features to a preset multilingual answer clustering space, and giving a sorting result of the text to be sorted according to the clustering result. The multilingual answer clustering space takes the source language center word of the sorting category as the clustering center, clusters words in different languages ​​through a semantic consistency check, and generates an answer clustering space.

[0077] In step d, the multilingual answer clustering space is composed of a linear layer, a normalization layer and an activation layer. The predicted features are mapped to the multilingual answer clustering space to obtain clustering results and output corresponding sorting labels.

[0078] A device for a cross-language information sorting method based on semantic space clustering prompts, the device comprises a progressive multi-language transcoding module, a prompt engineering module, a model training module and an answer engineering module, wherein:

[0079] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, and generate input text for multilingual transcoding;

[0080] Prompt engineering module: used to construct the cross-language prompt template, and splice the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0081] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0082] Answer engineering module: used to map the prediction features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0083] The progressive multilingual transcoding module includes: a bilingual dictionary construction unit for constructing a source language-target language bilingual dictionary;

[0084] A synonym replacement unit, used to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability;

[0085] The prompt engineering module includes: a prompt template construction unit, which is used to averagely pool the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template;

[0086] An input variable generating unit, used for splicing the prompt template with the input text generated by the progressive multi-language transcoding module to generate an input variable for prompt learning;

[0087] The model training module includes: a multilingual word embedding unit, which is used to convert the multilingual text in the input variable into a word embedding representation feature;

[0088] An encoder unit, for encoding word embedding representation features, understanding and analyzing the semantic information of the input variable;

[0089] The decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0090] The answer engineering module includes: a multilingual answer clustering unit, which is used to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0091] A label mapping unit, used to map clustering features into information sorting labels to generate prediction results;

[0092] like Figure 1 As shown, this embodiment is applicable to the case of implementing cross-language information sorting for texts in different languages, and is performed by a cross-language information sorting device, which can be implemented by software and / or hardware, see Figure 1 , the cross-language information sorting methods provided include:

[0093] S110, build multilingual unannotated corpus, cross-language information sorting dataset, source language-target language bilingual dictionary:

[0094] In one embodiment, the multilingual unlabeled corpus is composed of source language and target language text corpus; the cross-language information sorting data set is composed of source language and target language information sorting data, each data includes a text segment and a corresponding sorting category; the source language-target language bilingual dictionary is composed of synonyms in a one-to-one format, and if a word corresponds to multiple words in another language, the first synonym is selected by default in the embodiment;

[0095] S120, calculate the average vector of the source language and target language prompt templates, generate a language-independent prompt template, and add the [MASK] predictor:

[0096] In one embodiment, the language-independent prompt template is obtained by calculating the average vector of the source language and the target language prompt templates;

[0097] S130, based on the bilingual dictionary, the information sorting data set is progressively transcoded into multiple languages, and is spliced ​​with the prompt template and input into the multilingual pre-training model:

[0098] In one embodiment, for each source language sorting data, some words are replaced with synonyms in the target language according to progressive probability, and the replacement words are retrieved through a bilingual dictionary. The replaced text is spliced ​​with the prompt template to form the input of the model;

[0099] S140. Construct answer clustering semantic space based on multilingual word vector similarity:

[0100] In one embodiment, a set of answer words is constructed for an answer category through the similarity of multilingual word vectors, and the model is guided to learn the answer word clustering semantic space; in the clustering space, each answer category has an independent cluster center;

[0101] S150, map the model output vector to the answer clustering semantic space, and calculate its distance to the center of each answer cluster:

[0102] In one embodiment, the output vector of the model is mapped to the answer cluster semantic space by a linear layer, and the Euclidean distance between the output vector and the center of each answer cluster is calculated;

[0103] S150: The answer mapper outputs the information sorting result:

[0104] In one embodiment, the answer mapper obtains a sorting result according to the distance from the answer cluster center;

[0105] Figure 2 It is a cross-language information sorting model structure diagram based on semantic space clustering prompts. The illustrated information sorting model mainly includes:

[0106] S210, source language prompt template:

[0107] In one embodiment, the source language prompt template is composed of the source language prompt word and the [MASK] predictor;

[0108] For example, in the information sorting task, the source language (English) prompt template is “A[MASK]document”;

[0109] S220, Target language prompt template:

[0110] In one embodiment, the target language prompt template is composed of a target language prompt word and a [MASK] predictor;

[0111] For example, in the information sorting task, the target language (Chinese) prompt template is “a [MASK] information”;

[0112] S230, multi-language prompt template:

[0113] In one embodiment, the multi-language prompt template is obtained by averaging the source language prompt template and the target language prompt template;

[0114] S240, progressive multi-language transcoding module:

[0115] In one embodiment, a bilingual dictionary Dict(s, t) is first constructed, and for each source language sample to be sorted, a different transcoding probability p is given, and a word in the source language text to be sorted is selected based on the probability, and replaced with a synonym of the word in the target language based on the bilingual dictionary Dict(s, t), to generate a transcoded input text;

[0116] For example, assuming that the transcoding probability is R = {r1, r2, ..., r l}, then one of the probabilities r n It can be obtained by the following formula:

[0117] r n =r1+(n-1)d

[0118] In the formula, n represents the serial number of the current probability, and d represents the preset progressive probability increment;

[0119] S250, multi-language pre-trained model:

[0120] In one embodiment, the multilingual pre-trained model is a pre-trained language model of a Transformer structure, supporting multilingual text input;

[0121] Exemplarily, the multilingual pre-training model may be XLM, XLMR, mBERT, etc.;

[0122] S260, multi-language answer clustering module:

[0123] In one embodiment, the multilingual answer clustering module is composed of a linear layer, a normalization layer, and an activation layer, and the module maps the output vector of the pre-trained model at the [MASK] position to a preset multilingual answer clustering semantic space;

[0124] S270, information sorting answer mapper:

[0125] In one embodiment, the information sorting answer mapper calculates the Euclidean distance between the model output vector and the center of each answer cluster, and outputs the information sorting result according to the distance;

[0126] Figure 3 It is a flowchart of a progressive multi-language transcoding method, and the specific process includes:

[0127] S310: For each source language-target language pair, construct a bilingual dictionary:

[0128] In one embodiment, for each source language-target language pair, a bilingual dictionary Dict(s, t) is first constructed, in which each source language word corresponds to a target language word. If a word has multiple synonyms, the first target language word is selected as the synonym by default.

[0129] S320, calculating a progressive transcoding probability vector:

[0130] In one embodiment, the transcoding probability is assumed to be R = {r1, r2, ..., r l}, then the probability r n It can be obtained by the following formula:

[0131] r n =r1+(n-1)d

[0132] In the formula, n represents the serial number of the current probability, and d represents the preset progressive probability increment;

[0133] S330: Input source language samples to be selected, and replace source language words with target language synonyms according to different transcoding probabilities:

[0134] In one embodiment, given different transcoding probabilities p, words in the source language text to be sorted are selected with the probabilities, and replaced with synonyms of the words in the target language based on the bilingual dictionary Dict(s, t);

[0135] Figure 4 This is a flowchart of constructing a multilingual answer clustering space provided in an embodiment of the present application. The specific process includes:

[0136] S410. In the multilingual word vector space, taking the sorting category word as the center, calculate the m words closest to it and establish an answer set:

[0137] In one embodiment, multilingual word vectors are based on semantics, so the Cosine distance between word vectors can represent the semantic similarity of two words. This method takes the sorting category as the central word and finds the keyword set belonging to a certain sorting category by calculating the Cosine similarity. Among them, the Cosine similarity can be obtained by the following formula:

[0138]

[0139] Among them, v i Represents the word vector of the i-th word, v j Represents the word vector of the jth word;

[0140] Optionally, the number of keywords in each category can be adjusted according to the usage scenario. In the embodiment of the present application, the number of keywords in each category is 50;

[0141] Optionally, the similarity calculation algorithm may be Euclidean distance, Chebyshev distance, cosine similarity, Pearson correlation coefficient, etc. In the embodiment of the present application, Cosine distance is used;

[0142] Optionally, the pre-trained word vector can be adjusted according to the usage scenario. In the embodiment of the present application, the word vector adopts fast Text;

[0143] S420. According to the answer set, automatically label the words in the set with pseudo-labels in the unlabeled corpus:

[0144] In one embodiment, each unlabeled sample is traversed to check whether it contains words in the keyword set, and if so, it is labeled as a corresponding sorting category, i.e., a pseudo label;

[0145] S430, keeping the pre-trained model parameters fixed, and using the pseudo-label data to train the clustering module so that it learns the answer clustering semantic space:

[0146] Input the samples with pseudo labels into the pre-trained model, keep the model parameters fixed, and only train the clustering module to learn to cluster word-level semantic vectors, that is, learn the answer clustering semantic space;

[0147] Figure 5 The present invention is a flowchart of an information sorting answer mapping method, and the specific process includes:

[0148] S510, mapping the output vector of the pre-trained model to the preset answer clustering semantic space:

[0149] In one embodiment, the answer clustering module is composed of a linear layer, a normalization layer, and an activation layer, and the output vector of the pre-trained model is input into the answer clustering module;

[0150] S520, cluster the output vector to obtain the mapped vector features:

[0151] In one embodiment, the output vector is clustered by the answer clustering semantic space, and the output of the answer clustering module is obtained;

[0152] S530, obtain the final sorting category through the linear layer according to the vector feature:

[0153] In one embodiment, the distance between the output of the answer clustering module and the center of each cluster is calculated to obtain the final sorting category. This method simplifies the process into linear layer classification.

[0154] Figure 6 This is a schematic diagram of the structure of a cross-language information sorting device based on semantic space clustering prompts, see Figure 6 The cross-language information sorting device based on semantic space clustering prompts provided in this embodiment includes: a prompt engineering module, a progressive multi-language transcoding module, a model training module and an answer engineering module, wherein:

[0155] Prompt engineering module: used to construct the cross-language prompt template, and splice the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0156] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, and generate input text for multilingual transcoding;

[0157] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0158] Answer engineering module: used to map the prediction features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0159] Furthermore, the progressive multi-language transcoding module includes:

[0160] Bilingual dictionary building unit, used to build source language-target language bilingual dictionary;

[0161] The synonym replacement unit is used to replace the words in the source language text to be sorted with the synonyms of the words in the target language according to progressive probability.

[0162] Furthermore, the prompt engineering module includes:

[0163] A prompt template construction unit, used for performing average pooling on the embedded features of the source language preset template and the target language preset template, thereby generating a cross-language prompt template;

[0164] The input variable generating unit is used to concatenate the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning.

[0165] Furthermore, the model training module includes:

[0166] Multilingual word embedding unit, used to convert multilingual text in input variables into word embedding representation features;

[0167] An encoder unit, for encoding word embedding representation features, understanding and analyzing the semantic information of the input variable;

[0168] The decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0169] Furthermore, the answer engineering module includes:

[0170] A multilingual answer clustering unit, used to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0171] The label mapping unit is used to map clustering features into information sorting labels to generate prediction results.

[0172] The present invention also provides an electronic device and a readable storage medium;

[0173] like Figure 7 As shown, it is a block diagram of an electronic device for cross-language information sorting based on semantic space clustering prompts; the electronic device refers to various modern electronic digital computers, including personal computers, portable computers, and various server devices. The components and their interconnections and functions shown in this article are only examples;

[0174] like Figure 7 As shown, the electronic device includes: one or more multi-core processors, one or more GPU computing cards, and a memory. In order to make the electronic devices interactive, they should also include: input devices and output devices. Various devices are interconnected and communicated through a bus;

[0175] The memory is the non-transitory computer-readable storage medium provided by the present invention; wherein the memory stores instructions executable by at least one multi-core processor or at least one GPU computing card, so that the cross-language text representation method of the fused word alignment adapter module provided by the present application is executed. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to cause a computer to execute the cross-language text representation method of the fused word alignment adapter module provided by the present application;

[0176] Input devices provide and receive control signals input by users into electronic devices, including keyboards that generate digital or character information and mice used to control devices to generate other key signals. Output devices provide feedback information from user electronic devices, including displays that print execution results or processes.

[0177] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A cross-language information sorting method based on semantic space clustering prompts, characterized in that: Follow these steps: a. Construct a cross-language prompt template for the information sorting task, wherein the cross-language prompt template is a preset template in the source language Preset templates with target language Average pooling result of embedded features; b. Perform progressive multilingual transcoding on the text to be sorted, and combine it with the prompt template in step a to generate input variables for prompt learning; the progressive multilingual transcoding is to convert the words in the source language text to be sorted into multiple languages ​​according to the progressive probability r n = r1+(n-1)×d is replaced with a synonym in the target language, and the synonym is searched according to a preset bilingual dictionary; c. Input the input variables in step b into the cross-language pre-training model to train the MLM model and generate prediction features. The input variables are passed through the pre-training model to obtain output features. The prediction features are feature vectors at the [MASK] position in the output features. d. Mapping the predicted features to a preset multilingual answer clustering space, and giving a sorting result of the text to be sorted according to the clustering result. The multilingual answer clustering space takes the source language center word of the sorting category as the clustering center, clusters words in different languages ​​through a semantic consistency check, and generates an answer clustering space.

2. The cross-language information sorting method based on semantic space clustering prompts according to claim 1 is characterized in that: In step d, the multilingual answer clustering space is composed of a linear layer, a normalization layer and an activation layer. The predicted features are mapped to the multilingual answer clustering space to obtain clustering results and output corresponding sorting labels.

3. A device for the cross-language information sorting method based on semantic space clustering prompts according to claim 1, characterized in that The device consists of a progressive multi-language transcoding module, a prompt engineering module, a model training module and an answer engineering module, wherein: Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, and generate input text for multilingual transcoding; Prompt engineering module: used to construct the cross-language prompt template, and splice the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning; Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features; Answer engineering module: used to map the prediction features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

4. The device according to claim 3, characterized in that The progressive multilingual transcoding module includes: a bilingual dictionary construction unit for constructing a source language-target language bilingual dictionary; A synonym replacement unit, used to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability; The prompt engineering module includes: a prompt template construction unit, which is used to averagely pool the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template; An input variable generating unit, used for splicing the prompt template with the input text generated by the progressive multi-language transcoding module to generate an input variable for prompt learning; The model training module includes: a multilingual word embedding unit, which is used to convert the multilingual text in the input variable into a word embedding representation feature; An encoder unit, for encoding word embedding representation features, understanding and analyzing the semantic information of the input variable; The decoder unit is used to decode the output features of the encoder unit to generate prediction features. The answer engineering module includes: a multilingual answer clustering unit, which is used to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features; The label mapping unit is used to map clustering features into information sorting labels to generate prediction results.

5. An electronic device, wherein: include: at least one processor; At least one GPU computing card and a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card, so that the at least one processor or the at least one GPU computing card can execute the cross-language information sorting method based on semantic space clustering prompts according to any one of claims 1-3.

6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the cross-language information sorting method based on semantic space clustering prompts according to any one of claims 1 to 3.

Images