Cross-language information sorting method and device based on semantic space clustering hints, equipment and storage medium

Abstract

The application discloses a cross-language information sorting method and device based on semantic space clustering prompts, equipment and a storage medium. The method initializes the prompt template of the non-restricted language by averaging the source language and target language prompt template embedding vectors, and introduces a progressive multilingual transcoding method to reduce language bias in the model learning process. In the answer engineering stage, a clustering-based answer space construction method is proposed. A unified multilingual answer space is constructed through semantic space clustering guided by semantic consistency, so as to realize the clustering of the answers of multilingual prompt learning, and then obtain the information sorting category. The method solves the problem of poor cross-language information sorting performance caused by inconsistent answer spaces in prompt learning of different languages in the prior art. The information sorting task performance for the cross-language scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence in the field of computer technology, and in particular to technical fields such as natural language processing and deep learning. Specifically, a method, apparatus, device, and storage medium for cross-language information sorting based on semantic space clustering prompts are provided. Background Art

[0002] Cross-language information sorting aims to process multilingual text data and sort it into relevant thematic categories. A cross-language information sorting system needs to possess cross-lingual processing capabilities, mapping different languages ​​into a unified semantic representation space to achieve knowledge transfer from high-resource languages ​​to low-resource languages, and semantic mapping and alignment between different languages. It must be able to recognize and understand text content in different languages ​​and sort it into relevant categories or topics. Currently, cross-language information sorting has been applied in a variety of fields, including social media analysis, news classification, text archiving and retrieval, and sentiment analysis.

[0003] In recent years, prompt learning technology has been widely used in various tasks and can effectively improve the performance of natural language processing tasks. During long-term research and development, the inventors of this application discovered that when prompt learning technology is directly applied to cross-language information sorting tasks, due to the inconsistency of the answer spaces of the source language and the target language, it is difficult to fully activate the knowledge learned in the pre-trained model, resulting in a decrease in cross-language information sorting performance. To solve this problem, the present invention proposes a cross-language information sorting method based on semantic space clustering prompts. This method unifies the answer spaces of different languages ​​through semantic clustering, thereby improving the performance of cross-language information sorting tasks. Summary of the Invention

[0004] The present invention aims to provide a method, apparatus, device, and storage medium for cross-language information sorting based on semantic space clustering prompts. The method includes two phases: prompt engineering and answer engineering. In the prompt engineering phase, the prompt template for the non-restricted language is initialized by averaging the source and target language prompt template embedding vectors, and a progressive multilingual transcoding method is introduced to reduce language bias during model learning. In the answer engineering phase, the method proposes a clustering-based answer space construction method. This method uses semantic consistency-guided semantic space clustering to construct a unified multilingual answer space, thereby clustering the answers to multilingual prompt learning and obtaining information sorting categories, thereby improving the performance of information sorting tasks in cross-language scenarios. This method addresses the problem of poor cross-language information sorting performance in the prior art, which is caused by inconsistent answer spaces in different languages ​​during prompt learning.

[0005] The cross-language information sorting method based on semantic space clustering hints described in the present invention is performed according to the following steps:

[0006] a. Construct a cross-language prompt template for the information sorting task. The cross-language prompt template is a preset template in the source language. Preset templates with target language Average pooling result of embedded features;

[0007] b. Perform progressive multilingual transcoding on the text to be sorted and combine it with the prompt template in step a to generate input variables for prompt learning; the progressive multilingual transcoding is to convert the words in the source language text to be sorted into words according to the progressive probability r n = r1+(n-1)×d is replaced with a synonym in the target language, and the synonym is searched according to a preset bilingual dictionary;

[0008] c. Input the input variables described in step b into the cross-language pre-trained model for masked language modeling training and generate prediction features. After the input variables are trained by the pre-trained model, output features are obtained. The prediction features are the feature vectors at the [MASK] position in the output features.

[0009] d. Mapping the predicted features to a preset multilingual answer clustering space, and providing a sorting result for the text to be sorted based on the clustering result. The multilingual answer clustering space uses the central word of the source language of the sorting category as the cluster center, clusters words in different languages ​​through a semantic consistency test, and generates an answer clustering space.

[0010] In step d, the multilingual answer clustering space is composed of a linear layer, a normalization layer, and an activation layer. The predicted features are mapped to the multilingual answer clustering space to obtain clustering results and output corresponding sorting labels.

[0011] A device for a cross-language information sorting method based on semantic space clustering prompts, comprising a progressive multilingual transcoding module, a prompt engineering module, a model training module, and an answer engineering module, wherein:

[0012] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, generating input text for multilingual transcoding;

[0013] Prompt Engineering Module: used to construct the cross-language prompt template and concatenate the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0014] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0015] Answer engineering module: used to map the predicted features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0016] The progressive multilingual transcoding module includes: a bilingual dictionary construction unit for constructing a source language-target language bilingual dictionary;

[0017] A synonym replacement unit, configured to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability;

[0018] The prompt engineering module includes: a prompt template construction unit for performing average pooling on the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template;

[0019] an input variable generating unit, configured to concatenate the prompt template with the input text generated by the progressive multilingual transcoding module to generate input variables for prompt learning;

[0020] The model training module includes: a multilingual word embedding unit for converting multilingual text in input variables into word embedding representation features;

[0021] An encoder unit, configured to encode word embedding representation features and understand and analyze the semantic information of the input variable;

[0022] A decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0023] The answer engineering module includes: a multilingual answer clustering unit, configured to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0024] The label mapping unit is used to map clustering features into information sorting labels and generate prediction results.

[0025] An electronic device, comprising: at least one processor; at least one GPU computing card; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor or the at least one GPU computing card so as to enable the at least one processor or the at least one GPU computing card to execute the cross-language information sorting method based on semantic space clustering cues as described in any one of claims 1-3.

[0026] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the cross-language information sorting method based on semantic space clustering prompts according to any one of claims 1 to 3.

[0027] The present invention provides a method, apparatus, device, and storage medium for cross-language information sorting based on semantic space clustering prompts, the method comprising:

[0028] a. Create a multilingual prompt template. Generate a language-neutral prompt template by calculating the average embedding value of the source language prompt template and the target language prompt template. This step aims to create a prompt template that can be used across different languages ​​to reduce the model's reliance on specific languages. Specifically, generating a language-neutral prompt template involves:

[0029] Construct prompt templates for the source language and target language respectively, and add a [MASK] predictor at the answer prediction position to obtain the embedded features of the two templates. Calculate the average of the two features through an average pooling operation and use it as the prompt template for model learning, that is, the prompt template for the non-restricted language.

[0030] b. Progressive multilingual transcoding, which replaces words in the source language with synonyms in the target language with a progressive probability during the training phase. By gradually increasing the language transcoding probability according to a predetermined progressive ratio, the model's adaptability to the target language is enhanced and overfitting to the source language is reduced. The progressive multilingual transcoding is specifically as follows:

[0031] For each source language-target language pair, a bilingual dictionary Dict(s, t) is first constructed. Given different transcoding probabilities p, words in the source language text to be sorted are selected based on the probabilities and replaced with synonyms of the words in the target language based on the bilingual dictionary Dict(s, t).

[0032] c. Construct a unified multilingual answer space, integrate answer candidates from different languages ​​through semantic consistency-guided clustering, cluster words with the same or similar meanings in different languages ​​to form a cross-language answer space; in the clustering process, select a topic center word as the cluster center, and learn the cluster space using the K-nearest neighbor (KNN) clustering method; the answer candidate words are specifically:

[0033] For each sorting topic, a topic center word is selected as the cluster center, and clustering is performed based on word-level semantic vectors in the source language and the target language. For each sorting topic, n answer candidate words are generated for the multilingual answer space construction;

[0034] d. A clustering-based answer mapper that learns how to map the model-predicted embeddings to specific information sorting category labels based on the multilingual clustered answer space. The learning process relies on a context initialization method that pre-labels answer candidates at the word level on unlabeled corpus and trains the answer mapper in a self-supervised manner, thereby mapping the multilingual clustered answer space to a linear classification layer, enabling it to learn to classify the output features of the semantic clustering module. The answer mapper includes a linear layer, a normalization layer, and an activation layer. The context initialization method is specifically as follows:

[0035] For each source-target language pair, we construct an unannotated corpus. We iterate through each sample and, based on the answer candidate terms described in step c, annotate each candidate term with the corresponding sorting topic, thereby generating a self-supervised annotated corpus. Initializing the answer mapper based on this corpus allows us to fully learn the contextual information of the candidate terms.

[0036] The present invention provides a device for cross-language information sorting based on semantic space clustering prompts, the device comprising:

[0037] Prompt Engineering Module: used to construct the cross-language prompt template and concatenate the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0038] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, generating input text for multilingual transcoding;

[0039] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0040] Answer engineering module: used to map the predicted features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0041] Furthermore, the prompt engineering module includes:

[0042] A prompt template construction unit, configured to average-pool the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template;

[0043] Furthermore, the progressive multi-language transcoding module includes:

[0044] Bilingual dictionary construction unit, used to construct source language-target language bilingual dictionary;

[0045] A synonym replacement unit, configured to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability;

[0046] an input variable generating unit, configured to concatenate the prompt template with the input text generated by the progressive multilingual transcoding module to generate input variables for prompt learning;

[0047] Furthermore, the model training module includes:

[0048] Multilingual word embedding unit, used to convert multilingual text in input variables into word embedding representation features;

[0049] An encoder unit, configured to encode word embedding representation features and understand and analyze the semantic information of the input variable;

[0050] A decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0051] Furthermore, the answer engineering module includes:

[0052] a multilingual answer clustering unit, configured to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0053] The label mapping unit is used to map clustering features into information sorting labels and generate prediction results.

[0054] Another aspect of the present invention provides an electronic device, the device comprising:

[0055] At least one processor; at least one GPU computing card; and,

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to execute the method described in any one of claims 1-3.

[0058] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 3.

[0059] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the embodiments of the present application.

[0060] The technology of this application can complete the cross-language information sorting task and improve the performance of cross-language information sorting.

[0061] The beneficial effects of the present invention are as follows: in response to the problem of poor cross-language information sorting performance of traditional prompt learning, the present invention proposes a cross-language information sorting method based on semantic space clustering prompts, which unifies the answer spaces of different languages ​​through the semantic clustering method, thereby improving the performance of cross-language information sorting tasks.

[0062] To achieve the above and related ends, one or more aspects of the present invention include features that will be described in detail below and particularly pointed out in the claims. The following description and the accompanying drawings set forth certain exemplary aspects of the present invention in detail. However, these aspects are merely indicative of the various ways in which the principles of the present invention may be employed. Furthermore, the present invention is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flow chart of cross-language information sorting based on semantic space clustering hints provided by the present invention;

[0064] Figure 2 A structural diagram of a cross-language information sorting model based on semantic space clustering hints provided by the present invention;

[0065] Figure 3 A flowchart of a progressive multi-language transcoding method provided by the present invention;

[0066] Figure 4 A flowchart for constructing a multilingual answer clustering space provided by the present invention;

[0067] Figure 5 A flowchart of an information sorting answer mapping method provided by the present invention;

[0068] Figure 6 A schematic diagram of the structure of a cross-language information sorting device based on semantic space clustering prompts provided by the present invention;

[0069] Figure 7 This is a block diagram of an electronic device for cross-language information sorting based on semantic space clustering prompts according to the present invention. DETAILED DESCRIPTION

[0070] The following detailed description of the embodiments of the present invention is provided in conjunction with the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.

[0071] Example

[0072] The cross-language information sorting method based on semantic space clustering hints described in the present invention is performed according to the following steps:

[0073] a. Construct a cross-language prompt template for the information sorting task. The cross-language prompt template is a preset template in the source language. Preset templates with target language The average pooling result of the embedded features; in this step, let [MASK] be the answer mask text, and let the preset template of the source language s be T s , The source language prompt template word sequence, the template can be expressed as Assume that the preset template of the target language t is T t , Prompt the template word sequence for the target language, then the template can be expressed as The multilingual template is the average pooling result of the embedding features of the above two templates;

[0074] b. Perform progressive multilingual transcoding on the text to be sorted and combine it with the prompt template in step a to generate input variables for prompt learning; the progressive multilingual transcoding is to convert the words in the source language text to be sorted into words according to the progressive probability r n = r1+(n-1)×d is replaced by a synonym in the target language, and the synonym is searched according to a preset bilingual dictionary; wherein, r 1 is the initial replacement probability, d is the probability increase at each step, and the probability of progressive multilingual transcoding at step n can be expressed as r n =r1+(n-1)×d;

[0075] c. Input the input variables described in step b into the cross-language pre-trained model for masked language modeling training and generate prediction features. After the input variables are trained by the pre-trained model, output features are obtained. The prediction features are the feature vectors at the [MASK] position in the output features.

[0076] d. Mapping the predicted features to a preset multilingual answer clustering space, and providing a sorting result for the text to be sorted based on the clustering result. The multilingual answer clustering space uses the central word of the source language of the sorting category as the cluster center, clusters words in different languages ​​through a semantic consistency test, and generates an answer clustering space.

[0077] In step d, the multilingual answer clustering space is composed of a linear layer, a normalization layer, and an activation layer. The predicted features are mapped to the multilingual answer clustering space to obtain clustering results and output corresponding sorting labels.

[0078] A device for a cross-language information sorting method based on semantic space clustering prompts, comprising a progressive multilingual transcoding module, a prompt engineering module, a model training module, and an answer engineering module, wherein:

[0079] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, generating input text for multilingual transcoding;

[0080] Prompt Engineering Module: used to construct the cross-language prompt template and concatenate the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0081] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0082] Answer engineering module: used to map the predicted features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0083] The progressive multilingual transcoding module includes: a bilingual dictionary construction unit for constructing a source language-target language bilingual dictionary;

[0084] A synonym replacement unit, configured to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability;

[0085] The prompt engineering module includes: a prompt template construction unit for performing average pooling on the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template;

[0086] an input variable generating unit, configured to concatenate the prompt template with the input text generated by the progressive multilingual transcoding module to generate input variables for prompt learning;

[0087] The model training module includes: a multilingual word embedding unit for converting multilingual text in input variables into word embedding representation features;

[0088] An encoder unit, configured to encode word embedding representation features and understand and analyze the semantic information of the input variable;

[0089] A decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0090] The answer engineering module includes: a multilingual answer clustering unit, configured to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0091] The label mapping unit is used to map clustering features into information sorting labels and generate prediction results;

[0092] like Figure 1 As shown, this embodiment is applicable to the case of implementing cross-language information sorting for texts in different languages, and is performed by a cross-language information sorting device, which can be implemented by software and / or hardware, see Figure 1 , the cross-language information sorting methods provided include:

[0093] S110. Constructing multilingual unannotated corpora, cross-language information sorting datasets, and source-target language bilingual dictionaries:

[0094] In one embodiment, the multilingual unannotated corpus consists of source language and target language text corpora; the cross-language information sorting dataset consists of source language and target language information sorting data, each data piece including a text segment and a corresponding sorting category; the source language-target language bilingual dictionary consists of synonyms in a one-to-one format; if a word corresponds to multiple words in the other language, the first synonym is selected by default in this embodiment;

[0095] S120. Calculate the average vector of the source language and target language prompt templates, generate a language-independent prompt template, and add the [MASK] predictor:

[0096] In one embodiment, the language-independent prompt template is obtained by calculating the average vector of the source language and target language prompt templates;

[0097] S130: Perform progressive multilingual transcoding on the information sorting dataset based on the bilingual dictionary, combine it with the prompt template, and input it into the multilingual pre-training model:

[0098] In one embodiment, for each source language sorted data item, some words are replaced with synonyms in the target language according to a progressive probability. The replacement words are retrieved from a bilingual dictionary. The replaced text is then concatenated with the prompt template to form the input of the model.

[0099] S140. Constructing answer clustering semantic space based on multilingual word vector similarity:

[0100] In one embodiment, a set of answer words is constructed for an answer category based on the similarity of multilingual word vectors, and the model is guided to learn a clustering semantic space of answer words; in this clustering space, each answer category has an independent cluster center;

[0101] S150. Map the model output vector to the answer cluster semantic space and calculate its distance to the center of each answer cluster:

[0102] In one embodiment, the output vector of the model is mapped to the answer cluster semantic space by a linear layer, and the Euclidean distance between the output vector and the center of each answer cluster is calculated;

[0103] S150: The answer mapper outputs the information sorting result:

[0104] In one embodiment, the answer mapper obtains a sorting result based on the distance from the answer cluster center;

[0105] Figure 2 This is a cross-language information sorting model structure diagram based on semantic space clustering prompts. The illustrated information sorting model mainly includes:

[0106] S210, source language prompt template:

[0107] In one embodiment, the source language prompt template is composed of the source language prompt word and the [MASK] predictor;

[0108] For example, in the information sorting task, the source language (English) prompt template is “A[MASK]document”;

[0109] S220, Target language prompt template:

[0110] In one embodiment, the target language prompt template is composed of the target language prompt word and the [MASK] predictor;

[0111] For example, in the information sorting task, the target language (Chinese) prompt template is “a [MASK] information”;

[0112] S230, multi-language prompt template:

[0113] In one embodiment, the multi-language prompt template is obtained by averaging the source language prompt template and the target language prompt template;

[0114] S240, progressive multi-language transcoding module:

[0115] In one embodiment, a bilingual dictionary Dict(s, t) is first constructed. For each source language sample to be sorted, a different transcoding probability p is given. A word in the source language text to be sorted is selected based on the probability. Based on the bilingual dictionary Dict(s, t), the word is replaced with a synonym in the target language to generate the transcoded input text.

[0116] For example, assuming that the transcoding probability is R={r1, r2, ..., r l}, then one of the probabilities r n It can be obtained by the following formula:

[0117] r n =r1+(n-1)d

[0118] In the formula, n represents the serial number of the current probability, and d represents the preset progressive probability increment;

[0119] S250, multilingual pre-training model:

[0120] In one embodiment, the multilingual pre-trained model is a pre-trained language model with a Transformer structure, which supports multilingual text input;

[0121] Exemplarily, the multilingual pre-training model may be XLM, XLMR, mBERT, etc.;

[0122] S260, multilingual answer clustering module:

[0123] In one embodiment, the multilingual answer clustering module is composed of a linear layer, a normalization layer, and an activation layer. The module maps the output vector of the pre-trained model at the [MASK] position to a preset multilingual answer clustering semantic space;

[0124] S270, information sorting answer mapper:

[0125] In one embodiment, the information sorting answer mapper calculates the Euclidean distance between the model output vector and the center of each answer cluster, and outputs the information sorting result according to the distance;

[0126] Figure 3 This is a flowchart of a progressive multilingual transcoding method. The specific process includes:

[0127] S310: Build a bilingual dictionary for each source language-target language pair:

[0128] In one embodiment, for each source language-target language pair, a bilingual dictionary Dict(s, t) is first constructed. In this dictionary, each source language word corresponds to a target language word. If a word has multiple synonyms, the first target language word is selected as the synonym by default.

[0129] S320: Calculate the progressive transcoding probability vector:

[0130] In one embodiment, let the transcoding probability be R={r1, r2, ..., r l}, then the probability r n It can be obtained by the following formula:

[0131] r n =r1+(n-1)d

[0132] In the formula, n represents the serial number of the current probability, and d represents the preset progressive probability increment;

[0133] S330: Input source language samples to be selected, and replace source language words with target language synonyms according to different transcoding probabilities:

[0134] In one embodiment, given different transcoding probabilities p, words in the source language text to be sorted are selected based on the probabilities and replaced with synonyms of the words in the target language based on the bilingual dictionary Dict(s, t);

[0135] Figure 4 This is a flowchart of constructing a multilingual answer clustering space provided by an embodiment of the present application. The specific process includes:

[0136] S410. In the multilingual word vector space, with the sorting category word as the center, calculate the m words closest to it and establish an answer set:

[0137] In one embodiment, multilingual word vectors are semantically based, so the Cosine distance between word vectors can represent the semantic similarity between two words. This method uses the sorting category as the central word and finds the set of keywords belonging to a certain sorting category by calculating the Cosine similarity. The Cosine similarity can be obtained by the following formula:

[0138]

[0139] Among them, v i Represents the word vector of the i-th word, v j Represents the word vector of the jth word;

[0140] Optionally, the number of keywords in each category can be adjusted according to the usage scenario. In the embodiment of the present application, the number of keywords in each category is 50;

[0141] Optionally, the similarity calculation algorithm may be Euclidean distance, Chebyshev distance, cosine similarity, Pearson correlation coefficient, etc. In the embodiment of the present application, Cosine distance is used;

[0142] Optionally, the pre-trained word vector can be adjusted according to the usage scenario. In the embodiment of this application, the word vector adopts fast text;

[0143] S420: Based on the answer set, automatically label the words in the set with pseudo-labels in the unlabeled corpus:

[0144] In one embodiment, each unlabeled sample is traversed to check whether it contains a word in the keyword set. If so, it is labeled with the corresponding sorting category, i.e., a pseudo label.

[0145] S430: Keep the pre-training model parameters fixed and use the pseudo-label data to train the clustering module so that it learns the answer clustering semantic space:

[0146] Input the pseudo-labeled samples into the pre-trained model, keep the model parameters fixed, and only train the clustering module to learn to cluster word-level semantic vectors, that is, learn the answer clustering semantic space;

[0147] Figure 5 This is a flow chart of an information sorting answer mapping method, the specific process includes:

[0148] S510: Map the output vector of the pre-trained model to the preset answer clustering semantic space:

[0149] In one embodiment, the answer clustering module is composed of a linear layer, a normalization layer, and an activation layer, and the output vector of the pre-trained model is input into the answer clustering module;

[0150] S520: Cluster the output vector to obtain the mapped vector features:

[0151] In one embodiment, the output vector is clustered by the answer clustering semantic space and the output of the answer clustering module is obtained;

[0152] S530. Obtain the final sorting category based on the vector features through the linear layer:

[0153] In one embodiment, the distance between the output of the answer clustering module and the center of each cluster is calculated to obtain the final sorting category. This method simplifies the process into linear layer classification.

[0154] Figure 6 This is a schematic diagram of the structure of a cross-language information sorting device based on semantic space clustering prompts, see Figure 6 The cross-language information sorting device based on semantic space clustering prompts provided in this embodiment includes: a prompt engineering module, a progressive multilingual transcoding module, a model training module and an answer engineering module, wherein:

[0155] Prompt Engineering Module: used to construct the cross-language prompt template and concatenate the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning;

[0156] Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, generating input text for multilingual transcoding;

[0157] Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features;

[0158] Answer engineering module: used to map the predicted features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

[0159] Furthermore, the progressive multi-language transcoding module includes:

[0160] Bilingual dictionary construction unit, used to construct source language-target language bilingual dictionary;

[0161] The synonym replacement unit is used to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability.

[0162] Furthermore, the prompt engineering module includes:

[0163] A prompt template construction unit, configured to average-pool the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template;

[0164] An input variable generating unit is configured to concatenate the prompt template with the input text generated by the progressive multilingual transcoding module to generate input variables for prompt learning.

[0165] Furthermore, the model training module includes:

[0166] Multilingual word embedding unit, used to convert multilingual text in input variables into word embedding representation features;

[0167] An encoder unit, configured to encode word embedding representation features and understand and analyze the semantic information of the input variable;

[0168] A decoder unit is used to decode the output features of the encoder unit to generate prediction features.

[0169] Furthermore, the answer engineering module includes:

[0170] a multilingual answer clustering unit, configured to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features;

[0171] The label mapping unit is used to map clustering features into information sorting labels and generate prediction results.

[0172] The present invention also provides an electronic device and a readable storage medium;

[0173] like Figure 7 The block diagram of an electronic device for cross-language information sorting based on semantic spatial clustering cues is shown. The electronic device refers to a variety of modern electronic digital computers, including personal computers, portable computers, and various server devices. The components, their interconnections, and their functions shown in this article are for illustrative purposes only.

[0174] like Figure 7 As shown, the electronic device includes: one or more multi-core processors, one or more GPU computing cards, and memory. In order to enable electronic devices to interact, it should also include: input devices and output devices. Various devices are interconnected and communicated through a bus;

[0175] The memory is the non-transitory computer-readable storage medium provided by the present invention; wherein the memory stores instructions executable by at least one multi-core processor or at least one GPU computing card, so that the cross-language text representation method of the fused word alignment adapter module provided by the present application is executed. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to cause a computer to execute the cross-language text representation method of the fused word alignment adapter module provided by the present application;

[0176] Input devices provide and receive user input into electronic devices. These include keyboards that generate numeric or character information and mice used to control devices and generate other key signals. Output devices provide feedback to users about electronic devices and include displays that print execution results or processes.

[0177] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A cross-language information sorting method based on semantic space clustering prompts, characterized by: Follow these steps: a. Construct a cross-language prompt template for the information sorting task. The cross-language prompt template is a preset template in the source language. Preset templates with target language Average pooling result of embedded features; b. Perform progressive multilingual transcoding on the text to be sorted and combine it with the prompt template in step a to generate input variables for prompt learning; the progressive multilingual transcoding is to convert the words in the source language text to be sorted into words according to the progressive probability r n = r1+(n-1)×d is replaced with a synonym in the target language, and the synonym is searched according to a preset bilingual dictionary; c. Input the input variables described in step b into the cross-language pre-training model to train the MLM model and generate prediction features. After the input variables are trained by the pre-training model, output features are obtained. The prediction features are the feature vectors at the [MASK] position in the output features. d. Mapping the predicted features to a preset multilingual answer clustering space, and providing a sorting result for the text to be sorted based on the clustering result. The multilingual answer clustering space uses the central word of the source language of the sorting category as the cluster center, clusters words in different languages ​​through a semantic consistency test, and generates an answer clustering space.

2. The cross-language information sorting method based on semantic space clustering prompts according to claim 1 is characterized in that: In step d, the multilingual answer clustering space is composed of a linear layer, a normalization layer, and an activation layer. The predicted features are mapped to the multilingual answer clustering space to obtain clustering results and output corresponding sorting labels.

3. A device for the cross-language information sorting method based on semantic space clustering prompts according to claim 1, characterized in that The device consists of a progressive multilingual transcoding module, a prompt engineering module, a model training module, and an answer engineering module, among which: Progressive multilingual transcoding module: used to replace words in the source language text to be sorted with synonyms in the target language according to progressive probability, generating input text for multilingual transcoding; Prompt Engineering Module: used to construct the cross-language prompt template and concatenate the prompt template with the input text generated by the progressive multi-language transcoding module to generate input variables for prompt learning; Model training module: used to substitute the information to be sorted into the input variables and input them into the cross-language pre-training model for training, understand and analyze the semantics of the input variables, and generate prediction features; Answer engineering module: used to map the predicted features to the multilingual answer clustering space, obtain clustering results, and output corresponding sorting labels.

4. The device according to claim 3, characterized in that The progressive multilingual transcoding module includes: a bilingual dictionary construction unit for constructing a source language-target language bilingual dictionary; A synonym replacement unit, configured to replace words in the source language text to be sorted with synonyms of the words in the target language according to progressive probability; The prompt engineering module includes: a prompt template construction unit for performing average pooling on the embedded features of the source language preset template and the target language preset template to generate a cross-language prompt template; an input variable generating unit, configured to concatenate the prompt template with the input text generated by the progressive multilingual transcoding module to generate input variables for prompt learning; The model training module includes: a multilingual word embedding unit for converting multilingual text in input variables into word embedding representation features; An encoder unit, configured to encode word embedding representation features and understand and analyze the semantic information of the input variable; A decoder unit, configured to decode the output features of the encoder unit to generate prediction features; The answer engineering module includes: a multilingual answer clustering unit, configured to map the prediction features output by the model training module to the multilingual answer clustering space to obtain cluster features; The label mapping unit is used to map clustering features into information sorting labels and generate prediction results.

5. An electronic device, wherein: include: at least one processor; At least one GPU computing card and a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to execute the cross-language information sorting method based on semantic space clustering prompts according to any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the cross-language information sorting method based on semantic space clustering prompts according to any one of claims 1 to 3.

Images