Cross - language abstract dataset construction method and system based on filtering and text enhancement

Abstract

The present invention relates to the field of cross - language processing, and particularly to a method and system for constructing a cross - language summary dataset based on filtering and text enhancement. By using a filtering strategy, the original monolingual summary dataset is filtered from the perspectives of characters and semantics, and the filtered data is converted into a cross - language summary dataset through a translation system; and a self - attention mechanism and a masked language model are used to perform dynamic synonym replacement on the cross - language summary dataset and generate new cross - language summary text data. The present invention can comprehensively remove low - quality monolingual summary samples, greatly improve the quality of the dataset, realize the similarity measurement of non - parallel texts, and increase the data scale on the premise of ensuring quality, and can obtain a high - quality and large - scale cross - language summary dataset at a low cost.

Description

Technical Field

[0001] The present invention relates to the technical field of cross - language data processing in the application of artificial intelligence in the new generation of information technology, and particularly relates to a method and system for constructing a cross - language summary dataset based on filtering and text enhancement. Background Art

[0002] Cross - lingual Summarization (CLS) refers to converting the text of one language into a summary of another language, enabling people to quickly and effectively obtain information from texts in unfamiliar languages. Its research methods have evolved from pipeline methods to end - to - end methods, and end - to - end methods are introducing deep learning models such as Transformer. A large amount of work shows that the quality and scale of labeled data directly affect the performance of deep learning models. Therefore, both the quality and scale of CLS datasets are extremely important.

[0003] Currently, researchers have constructed some CLS datasets through collection methods and conversion methods. Among them, the most representative is the NCLS dataset constructed using the conversion method. The datasets obtained by the collection method have high quality, but also high costs, so their scales are generally small. The conversion method generates CLS datasets from other task datasets, with low costs and guaranteed scale. However, the datasets obtained by the conversion method usually contain more low - quality samples, seriously affecting the performance of related research methods. There are two reasons for this phenomenon: one is the error of the source dataset. For example, in the sub - dataset Zh2EnSum of the NCLS dataset, which is sourced from the LCSTS dataset, due to the characteristics of microblog texts, some of its summaries are overly general in summarizing the original text. The other is the error of the conversion system, such as translation errors, etc. Therefore, how to construct a dataset with high quality and large scale at low cost is an important challenge in CLS research. Summary of the Invention

[0004] To this end, the present invention provides a method and system for constructing a cross - language summary dataset based on filtering and text enhancement, to solve the problems of uneven sample quality and low data scale existing in existing cross - language summary datasets.

[0005] According to the design scheme provided by the present invention, a method for constructing a cross - language summary dataset based on filtering and text enhancement is provided, including:

[0006] Filtering the original monolingual summary dataset from the perspectives of characters and semantics using a filtering strategy, and converting the filtered dataset into a cross - language summary dataset through a translation system;

[0007] Using a self - attention mechanism and a masked language model to perform dynamic synonym replacement on the text of the cross - language summary dataset and generate new text.

[0008] As a method for constructing a cross - language abstract dataset based on filtering and text enhancement in the present invention, further, a filtering strategy is used to filter the original monolingual abstract dataset from the perspectives of characters and semantics, including: analyzing and filtering the original monolingual abstract dataset from the perspectives of characters and semantics by using irrelevant word statistics, keyword statistics, and semantic measurement strategies respectively.

[0009] As a method for constructing a cross - language abstract dataset based on filtering and text enhancement in the present invention, further, an irrelevant word statistics strategy is used to analyze and filter the original monolingual abstract dataset, including:

[0010] First, the abstract words that do not appear in the text in the original monolingual abstract dataset are regarded as irrelevant words, and the proportion of irrelevant words in the total number of abstract words in the abstract is calculated.

[0011] Then, the sample pairs with a proportion greater than the first preset threshold are filtered out.

[0012] As a method for constructing a cross - language abstract dataset based on filtering and text enhancement in the present invention, further, a keyword statistics strategy is used to analyze and filter the original monolingual abstract dataset, including:

[0013] First, keywords of the text in the original monolingual abstract dataset are extracted from the semantic perspective through the Word2Vec word vector model, and the proportion of words belonging to the text keywords in the total number of abstract words in the abstract is calculated.

[0014] Then, the sample pairs with a proportion less than the second preset threshold are filtered out.

[0015] As a method for constructing a cross - language abstract dataset based on filtering and text enhancement in the present invention, further, keywords of the text in the original monolingual abstract dataset are extracted from the semantic perspective through the Word2Vec word vector model, including: First, the text is encoded through the Word2Vec word vector model, and a word representation sequence is obtained; then, all the words in the word representation sequence are clustered, the cluster centers are used as the main keywords, and the Euclidean distances between other words and the cluster centers are calculated; then, the p words closest to the cluster center are selected as keywords, where p is a preset integer.

[0016] As a method for constructing a cross - language abstract dataset based on filtering and text enhancement in the present invention, further, a semantic measurement strategy is used to analyze and filter the original monolingual abstract dataset, including:

[0017] First, BERT word embeddings are used to obtain the word representation sequences of the text and the reference abstract in the original monolingual abstract dataset.

[0018] Then, the BERT-Whiteing method is used to transform the word representation sequence into isotropic, and the cosine similarity is used to measure the similarity between the reference summary and the original text sample data from the semantic perspective, and the sample pairs with similarity less than the third preset threshold are filtered out.

[0019] As a method for constructing a cross-lingual summary dataset based on filtering and text enhancement according to the present invention, further, the BERT-Whiteing method is used to transform the word representation sequence into isotropic, including: First, obtain the text representation vectors of the word representation sequences in both the text and the reference summary, and unify the text representation vectors of the two; Then, perform whitening operation on the unified data, retain h principal components, and obtain the transformed text representation vectors; Then, split the text representation vectors according to the text and the reference summary to calculate the similarity between the two.

[0020] As a method for constructing a cross-lingual summary dataset based on filtering and text enhancement according to the present invention, further, the self-attention mechanism and the masked language model are used to perform dynamic synonym replacement on the text of the cross-lingual summary dataset and generate new text, including:

[0021] First, use the self-attention of the RoBERTa model and combine the context information to mask the words in the text of the cross-lingual summary sample pair to obtain the masked text sequence;

[0022] Then, use the masked language model pre-training task of the RoBERTa model to predict the mask of the masked text sequence to obtain new text;

[0023] Then, by combining the new text with the reference summary, a new cross-lingual summary sample pair is generated.

[0024] Further, the present invention also provides a cross-lingual summary dataset construction system based on filtering and text enhancement, including: a data filtering module and a data enhancement module, wherein,

[0025] The data filtering module is used to filter the original monolingual summary dataset from the character and semantic perspectives by using a filtering strategy, and convert the filtered data into a cross-lingual summary dataset through a translation system;

[0026] The data enhancement module is used to perform dynamic synonym replacement on the cross-lingual summary dataset by using the self-attention mechanism and the masked language model and generate new cross-lingual summary text data.

[0027] The beneficial effects of the present invention:

[0028] By adopting multiple strategies to filter and remove low-quality monolingual abstract samples, the present invention can comprehensively and thoroughly remove low-quality monolingual abstract samples, greatly improve the quality of the dataset, and achieve the similarity measurement of non-parallel texts; and use the self-attention mechanism and masked language model to expand cross-lingual abstract samples, increasing the data scale while ensuring the quality, so as to obtain a high-quality and large-scale cross-lingual abstract dataset at a low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the construction process of a cross-lingual abstract dataset based on filtering and text enhancement in the embodiment;

[0030] Figure 2 It is a schematic diagram of the text enhancement process in the embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the drawings and technical solutions.

[0032] In the embodiments of this case, aiming at the problems of uneven sample quality and low data scale in the existing cross-lingual abstract dataset, a method for constructing a cross-lingual abstract dataset based on filtering and text enhancement is provided, which includes the following contents:

[0033] Use filtering strategies to filter the original monolingual abstract dataset from the perspectives of characters and semantics, and convert the filtered data into a cross-lingual abstract dataset through a translation system;

[0034] Use the self-attention mechanism and masked language model to perform dynamic synonym replacement on the text of the cross-lingual abstract dataset and generate new text.

[0035] The sub-dataset Zh2EnSum of the NCLS dataset, which is derived from the LCSTS dataset. Due to the characteristics of Weibo texts, some of its abstracts are too general in summarizing the original text. As shown in Table 1, the underlined words indicate the keywords that appear in both the text and the abstract, the bold words appear in the abstract but not in the text and are irrelevant to the text content, belonging to noise, and the words under the wavy line appear in the abstract but not in the text, yet reflect the key information. This results in low-quality cross-lingual abstract samples, which is not conducive to model learning.

[0036] Table 1 Sample examples of the LCSTS dataset

[0037]

[0038] In the embodiments of this case, refer to Figure 1As shown, to ensure quality, multiple strategies are used to filter out low-quality samples from the source dataset. To meet the scale requirement, a text augmentation algorithm based on a pre-trained model is used to expand the samples for the cross-lingual summary dataset. In the embodiments of this case, to accurately measure the generalization of the summary to the text content in the monolingual summary dataset, irrelevant word statistics, keyword statistics, and semantic measurement strategies can be used to analyze and filter the original monolingual summary dataset from the character and semantic perspectives respectively.

[0039] Furthermore, in the embodiments of this case, the irrelevant word statistics strategy is used to analyze and filter the original monolingual summary dataset, including:

[0040] First, the summary words that do not appear in the text in the original monolingual summary dataset are regarded as irrelevant words, and the proportion of irrelevant words in the total number of summary words is calculated;

[0041] Then, the sample pairs with a proportion greater than the first preset threshold are filtered out.

[0042] Summary words that do not appear in the original text as irrelevant words will, to a certain extent, affect the learning effect of the model. Therefore, in the embodiments of this case, the proportion of irrelevant words in the summary is calculated to measure the amount of text content contained in the summary from the character perspective. If the proportion of irrelevant words is too high, it means that there are too many words in the summary that do not appear in the original text, and the samples should be filtered.

[0043] Specifically, given the text X = {x1,..., x i ,..., x m} of a monolingual summary sample and its reference summary Y = {y1,..., y j ,..., y n}, m is the number of text words, n is the number of reference summary words, n < m, x i and y j represent the i-th word of X and the j-th word of Y respectively. Then the irrelevant word proportion r A is:

[0044]

[0045] where |·| represents the total number of set elements.

[0046] Furthermore, in the embodiments of this case, keyword statistics are used to analyze and filter the collected original monolingual summary dataset, including:

[0047] First, the keyword statistics are used to analyze and filter the original monolingual summary dataset, including:

[0048] Then, the sample pairs with a proportion less than the second preset threshold are filtered out.

[0049] A good summary should contain many original keywords. Word embedding can reflect the semantic relationship between words in high-dimensional space and is a good choice for measuring semantic similarity to introduce semantic information. The K-means algorithm can cluster similar objects into the same cluster. Therefore, in the embodiment of this case, a word clustering method based on Word2Vec is used to extract text keywords from a semantic perspective, and then the proportion of words in the summary that are text keywords to the words in the summary is calculated, reflecting the amount of key information in the text contained in the summary from a character perspective. If the ratio is too low, it means that the summary contains too many non-keywords and the sample should be filtered.

[0050] Specifically, given a text X and its reference summary Y, we can first encode X through the Word2Vec word vector to obtain a word representation sequence X = {x1, ..., x i ,...,x m Then, all words are clustered by k-means algorithm, with the cluster center as the main keyword, the Euclidean distance between other words and the cluster center is calculated, and the p words closest to the cluster center are selected as keywords, and the keyword set C = {c1, ..., c p}. Then the abstract word ratio r B for:

[0051]

[0052] where |·| represents the total number of elements in the set.

[0053] Furthermore, in this embodiment, semantic metrics are used to analyze and filter the collected original monolingual summary dataset, including:

[0054] First, the BERT word embedding model is used to obtain the word representation sequence of the text and reference summaries in the original monolingual summary dataset;

[0055] Then, the BERT-Whiteing method is used to transform the word representation sequence into isotropic, and the cosine similarity is used to measure the similarity between the reference summary and the original sample data from a semantic perspective, and sample pairs with similarity less than a third preset threshold are filtered out.

[0056] Among them, the principal component analysis method is used to transform the word representation sequence into an isotropic one. First, the text representation vectors of the word representation sequence in both the original sample data and the reference summary can be obtained, and the text representation vectors of the two can be unified; then, the unified data is whitened, h principal components are retained, and the transformed text representation vector is obtained; then, the text representation vector is split according to the original sample data and the reference summary to calculate the similarity between the two.

[0057] A good abstract should be semantically similar to the original text. Contextual word embeddings of pre-trained models such as BERT have brought leapfrog progress to text representation. However, due to the anisotropy problem, cosine similarity cannot be used to measure the similarity of text embeddings based on BERT. BERT-Whiteing solves this problem by simply whitening (i.e., principal component analysis) the embedding vectors to make them isotropic. Therefore, in the embodiments of this case, text embeddings based on BERT-Whiteing and cosine similarity can be used to measure the similarity between the abstract and the text from a semantic perspective. If the cosine similarity is too small, it indicates that the similarity between the abstract and the text is too low, and the samples should be filtered.

[0058] Specifically, given text X and its reference abstract Y, the word representation sequences of X and Y, X = {x1,..., x i ,..., x m} and Y = {y1,..., y j ,..., y n}, can be obtained respectively through the word embeddings of BERT, and then their text representation vectors x' and y' can be obtained; then x' and y' are uniformly denoted as z', and is whitened, and h principal components are retained to obtain The specific process is shown in Algorithm 1, where N is the total number of samples; finally, is split into Then the cosine similarity r C of x' and y' is:

[0059]

[0060] where cos(·) represents calculating the cosine similarity of two vectors.

[0061]

[0062] As a preferred embodiment, further, a self-attention mechanism and a masked language model are used to perform dynamic synonym replacement on the cross-lingual abstract dataset and generate new cross-lingual abstract text data, including:

[0063] First, the self-attention of the RoBERTa model is used and combined with context information to mask the words of the text of the cross-lingual abstract sample pair to obtain a masked text sequence;

[0064] Next, the masked text sequence mask is predicted using the masked language model pre-training task of the RoBERTa model to obtain a new text;

[0065] Then, by combining the new text with the reference abstract, a new cross-lingual abstract sample pair is generated.

[0066] The self-attention mechanism can capture the dependencies between words. The masked language model pre-training tasks of self-encoding pre-trained models such as BERT and RoBERTa can predict words in combination with the context. Therefore, in the embodiments of this case, the self-attention of the pre-trained model and the masked language model pre-training task are used to perform dynamic synonym replacement on words, thereby generating new text.

[0067] Specifically, given the text of the CLS dataset and its reference abstract First, the self-attention of RoBERTa can be used to select words for masking, obtaining Then, the masked language model pre-training task of RoBERTa is used to predict the mask, obtaining new text Finally, combine X src′ with Y tgt to form a new CLS sample. The process is as Figure 2 shown, where in the new text box, the underlined gray font indicates that the prediction result is different from the original text, and the non-underlined gray text indicates that the prediction result is the same as the original text.

[0068] Furthermore, based on the above method, the embodiments of the present invention also provide a cross-lingual abstract dataset construction system based on filtering and text enhancement, including: a data filtering module and a data enhancement module, where

[0069] The data filtering module is used to filter the original monolingual abstract dataset from the perspectives of characters and semantics using a filtering strategy, and convert the filtered data into a cross-lingual abstract dataset through a translation system;

[0070] The data enhancement module is used to perform dynamic synonym replacement on the cross-lingual abstract dataset using the self-attention mechanism and the masked language model and generate new cross-lingual abstract text data.

[0071] To verify the effectiveness of the solution in this case, the following further explanation is made in combination with experimental data:

[0072] The NCLS dataset is used, which includes the English-Chinese CLS dataset En2ZhSum and the Chinese-English CLS dataset Zh2EnSum. The used tokenization algorithm is the BPE algorithm (Byte-Pair Encoding). Randomly sample one-sixth (60,781 samples) of the En2ZhSum training set and one-half (846,857 samples) of the Zh2EnSum training set as sub-training sets, and use the data enhancement module on them to obtain enhanced sub-training sets, with the data volumes reaching 115,589 and 1,424,296 respectively.

[0073] LCSTS is a Chinese abstract dataset sourced from Sina Weibo, consisting of three parts: Part_I, Part_II, and Part_III. The authors scored the samples in Part_II and Part_III to evaluate the relevance between the abstracts and the texts. 2,196,263 samples after deduplication in Part_I and 195 samples with a score of 5 and after deduplication in Part_III were used as the original samples for constructing the En2Zh_Sum dataset.

[0074] And the following comparison algorithms are set: TETran, which first uses a Transformer-based machine translation model to translate the source language text and then uses the LexRank algorithm to abstract the result. TLTran, which first uses a Transformer-based monolingual abstract model to abstract the source language text and then uses a Transformer-based machine translation model to translate the abstract into the target language. GETran and GLTran, which use Google Translator to replace the translation models in TETran and TLTran respectively. NCLS, which uses the NCLS dataset for supervised training of Transformer. NCLS-MT, which conducts multi-task learning of cross-lingual abstract and machine translation on Transformer. NCLS-MS, which conducts multi-task learning of cross-lingual abstract and monolingual abstract on Transformer. XNLG, which fine-tunes the XNLG model using the NCLS dataset. ATS, which sums the output distribution of Transformer and the translation probability distribution of an external bilingual dictionary with weights to generate an abstract. MLPT, which first pre-trains a cross-lingual abstract model with various supervised and unsupervised tasks and then fine-tunes it using the NCLS dataset. RL-XSIM, which uses cross-lingual semantic similarity as the reward function of reinforcement learning to jointly optimize the model with the traditional objective function of cross-lingual abstract. MCLAS, which concatenates the outputs of monolingual abstract and cross-lingual abstract for training as a new task. CSC, which unifies the machine translation and cross-lingual abstract tasks using the compression ratio and encodes it into the semantic representation of the text.

[0075] The pre-trained model BART achieved the best performance at that time in monolingual abstract, so this paper selects the multilingual pre-trained model mBART as the basic framework for cross-lingual abstract, making full use of its powerful semantic understanding, cross-lingual alignment, and text generation capabilities. Combining the method of this case, the following three experimental groups can be obtained: mBART, which directly uses mBART for cross-lingual abstract; mBART ft , which fine-tunes mBART on the sub-training set; (mBART+PTA) ft , which fine-tunes mBART on the enhanced sub-training set.

[0076] The BPE algorithm is used to segment tokens and construct the vocabulary, and all English is lowercased; the roberta-base model and mbart-large-cc25 model of the Huggingface-transformers project are used as the implementations of RoBERTa and mBART; in the English-Chinese CLS, the input and output sequence lengths are set to 550 and 100 respectively, and in the Chinese-English CLS, the input and output sequence lengths are set to 80 and 60 respectively; the AdamW optimizer is used for parallel training on 2 NVIDIA RTX A6000 GPUs, and the fine-tuning ends after about 100,000 iterations of training.

[0077] When constructing the En2Zh_Sum dataset, by translating the source text of the LCSTS dataset instead of the target summary, the error introduced to the reference summary is avoided, which may affect the learning effect of the model. And the Baidu Translate API is used as the conversion system to ensure the translation quality. Specifically, the jieba library in Python is used for Chinese word segmentation, the Word2Vector word vectors of the gensim library and the kmeans algorithm of the sklearn library are used to implement the word clustering method based on Word2Vec, the bert-base-uncased model of the Huggingface-transformers project and the BERT-whitening code of the NLP-Series-sentence-embeddings project are used for BERT word embedding and whitening, and the average word vectors of all words in the first layer and the last layer of the BERT word vectors are used as the text representation vectors.

[0078] To select the optimal pre-trained model for the text augmentation algorithm (PTA) based on pre-trained models, the "cloze" performance of 5 classic pre-trained models, such as BERT, ELECTRA, ERNIE, RoBERTA, and ALBERT, is also tested. Specifically, the electra-base-discriminator model, ernie-2.0-base-en model, and albert-base-v2 model of the Huggingface-transformers project are used as the implementations of ELECTRA, ERNIE, and ALBERT respectively.

[0079] The cross - language summarization performance is evaluated using standard ROUGE metrics, specifically using the rouge - metric library in Python. It should be noted that the standard ROUGE metrics can only evaluate English summaries. Therefore, when evaluating Chinese summaries, special processing is carried out, that is, the summary is segmented at the character granularity and then concatenated with space characters. In the experiment of selecting the most suitable pre - trained model, the average accuracy of predicting words equal to the masked words can be used to measure the prediction ability of the model.

[0080] Fine - tune mBART on the enhanced sub - training set and compare the results with those of many cross - language summarization related studies on the complete training set. The experimental results are shown in Table 2. Among them, represents the previous best performance, * represents the result of fine - tuning MBART on the sub - training set, and the bold numerical values represent the result of fine - tuning MBART on the enhanced sub - training set.

[0081] Table 2 ROUGE F1 scores (%) on the test set

[0082]

[0083] As can be seen from the above table, whether it is English - to - Chinese cross - language summarization or Chinese - to - English cross - language summarization, the performance of directly applying mBART is not ideal. This shows that even if the pre - trained model has strong performance, it cannot be directly applied to cross - language summarization without learning specific data. The mBART fine - tuned on the sub - training set improves ROUGE - 1, ROUGE - 2 and ROUGE - L by 18.77, 13.2 and 15.84 respectively in English - to - Chinese cross - language summarization, and improves ROUGE - 1, ROUGE - 2 and ROUGE - L by 1.42, 0.11 and 4.98 respectively in Chinese - to - English cross - language summarization. This shows that the pre - trained model can significantly improve the performance of the cross - language summarization system. And the mBART fine - tuned on the enhanced sub - training set has better performance. It improves ROUGE - 1, ROUGE - 2 and ROUGE - L by 19.83, 15.4 and 17.4 respectively in English - to - Chinese cross - language summarization, and improves ROUGE - 1, ROUGE - 2 and ROUGE - L by 1.49, 0.31 and 4.99 respectively in Chinese - to - English CLS. This fully shows that the PTA algorithm can generate high - quality cross - language summary samples, improve the model learning effect, and indirectly prove the quality of the En2Zh_Sum dataset.

[0084] Use the solution of this case to construct the English - to - Chinese cross - language summary dataset En2Zh_Sum from the Chinese summary dataset LCSTS. The number of samples is shown in Table 3. To simply and intuitively verify the quality of En2Zh_Sum, one - seventh (397,688 samples) of its training set is randomly sampled to fine - tune mBART and test its performance on its test set. The results are shown in Table 4.

[0085] Table 3 Number of Samples in the En2Zh_Sum Dataset

[0086]

[0087] Table 4 ROUGE F1 Scores (%) on the En2Zh_Sum Test Set

[0088]

[0089] As can be seen from the above table, the En2Zh_Sum dataset constructed in this case enables the cross-lingual summarization model to achieve good performance with only partial samples, directly demonstrating the quality of the En2Zh_Sum dataset and the usability and effectiveness of the proposed cross-lingual summarization construction method.

[0090] The above experimental data further shows that in this case, a multi-strategy filtering algorithm is used to comprehensively filter low-quality samples from the character and semantic perspectives to supervise the quality; the self-attention mechanism of the pre-trained model and the pre-trained task are combined to perform dynamic synonym replacement based on the context to supervise the scale. This can not only comprehensively and thoroughly filter low-quality samples, but also flexibly and effectively increase the scale of the dataset, thereby obtaining a high-quality and large-scale cross-lingual summarization dataset at a low cost.

[0091] Unless otherwise specifically stated, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0092] Each embodiment in this specification is described in a progressive manner. The key points of each embodiment are the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0093] The units and method steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art can use different methods to implement the described functions for each specific application, but such implementation is not considered to exceed the scope of the present invention.

[0094] Those of ordinary skill in the art can understand that all or part of the steps in the above methods can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disc, etc. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Correspondingly, each module / unit in the above embodiments can be implemented in the form of hardware or in the form of a software function module. The present invention is not limited to any specific form of combination of hardware and software.

[0095] Finally, it should be noted that the above embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for constructing a cross - language abstract dataset based on filtering and text enhancement, characterized in that, Including: Analyze and filter the original monolingual abstract dataset from the character and semantic perspectives using the stop-word statistics, keyword statistics, and semantic measurement strategies respectively, and convert the filtered data into a cross-lingual abstract dataset through a translation system. Among them, analyzing and filtering the original monolingual abstract dataset using the semantic measurement strategy includes: First, use the BERT word embedding model to obtain the word representation sequences of the text and the reference abstract in the original monolingual abstract dataset. Then, use the BERT-Whiteing method to transform the word representation sequences into isotropic ones, and use the cosine similarity to measure the similarity between the reference abstract and the input text from the semantic perspective, and filter out the sample pairs with a similarity less than the third preset threshold. Use the self-attention mechanism and the masked language model to perform dynamic synonym replacement on the cross-lingual abstract dataset and generate new cross-lingual abstract text data. Among them, use the self-attention of the RoBERTa model and combine the context information to select the words of the text of the cross-lingual abstract sample pair to be masked, and obtain the masked text sequence. Use the masked language model pre-training task of the RoBERTa model to predict the mask of the masked text sequence to obtain new text, so as to generate a new cross-lingual abstract sample pair by combining the new text with the reference abstract.

2. The method for constructing a cross - language abstract dataset based on filtering and text enhancement according to claim 1, wherein, Analyze and filter the original monolingual abstract dataset using the stop-word statistics strategy, including: First, take the abstract words that do not appear in the text in the original monolingual abstract dataset as stop words, and calculate the proportion of stop words in the total number of abstract words in the abstract. Then, filter out the sample pairs with a proportion greater than the first preset threshold.

3. The method for constructing a cross - language abstract dataset based on filtering and text enhancement according to claim 1, wherein Analyze and filter the original monolingual abstract dataset using the keyword statistics strategy, including: First, extract the keywords of the text in the original monolingual abstract dataset from the semantic perspective through the Word2Vec word vector model, and calculate the proportion of the words belonging to the text keywords in the total number of abstract words in the abstract. Then, filter out the sample pairs with a proportion less than the second preset threshold.

4. The method for constructing a cross - language abstract dataset based on filtering and text enhancement according to claim 3, wherein, Extract the keywords of the text in the original monolingual abstract dataset from the semantic perspective through the Word2Vec word vector model, including: First, encode the text through the Word2Vec word vector model and obtain the word representation sequence. Then, cluster all the words in the word representation sequence, take the cluster center as the main keyword, and calculate the Euclidean distance between other words and the cluster center. Then, select the p words closest to the cluster center as keywords, where p is a preset integer.

5. The method for constructing a cross - language abstract dataset based on filtering and text enhancement according to claim 1, wherein, Use the BERT-Whiteing method to transform the word representation sequence into isotropic, including: First, obtain the text representation vectors of the word representation sequences of both the text and the reference abstract, and unify the text representation vectors of both. Then, perform whitening operation on the unified data, retain h principal components, and obtain the transformed text representation vectors. Then, split the text representation vectors according to the text and the reference abstract to calculate the similarity between the two.

6. A cross-lingual summarization dataset construction system based on filtering and text enhancement, characterized in that Including: a data filtering module and a data augmentation module, where A data filtering module, which is used to filter the original monolingual abstract dataset from the character and semantic perspectives by using irrelevant word statistics, keyword statistics, and semantic measurement strategies respectively, and convert the remaining sample pairs into cross-language abstract sample pairs through a translation system; among them, analyzing and filtering the original monolingual abstract dataset by using the semantic measurement strategy includes: first, using a BERT word embedding model to obtain the word representation sequences of the text and the reference abstract in the original monolingual abstract dataset; then, using the BERT-Whiteing method to make the word representation sequences isotropic, and using the cosine similarity to measure the similarity between the reference abstract and the input text from the semantic perspective, and filtering out the sample pairs with a similarity less than the third preset threshold. A data augmentation module, which is used to perform dynamic synonym replacement on the text of the cross-language abstract sample pairs by using the self-attention mechanism and the masked language model and generate new cross-language abstract sample pairs. Among them, using the self-attention of the RoBERTa model and combining the context information to select the words of the text of the cross-language abstract sample pairs for masking to obtain the masked text sequence; using the masked language model pre-training task of the RoBERTa model to predict the mask of the masked text sequence to obtain new text, so as to generate new cross-language abstract sample pairs by combining the new text with the reference abstract.

7. An electronic device, characterized in that, It includes a memory and a processor, and the processor and the memory complete communication with each other through a bus; the memory stores program instructions executable by the processor, and the processor can execute the method steps described in any one of claims 1 to 5 by calling the program instructions.

Images