Data enhancement method and device, electronic device, and storage medium

Abstract

The embodiments of the present invention provide a data enhancement method and device, an electronic device, and a storage medium, which relate to the field of artificial intelligence technology. The data enhancement method includes: obtaining an original text sample and inputting it into a pre-trained topic model, calculating the contribution value of each topic word in each sentence to the text sentence, and then obtaining a set of words to be replaced based on the contribution value of the topic word to the text sentence, and then selecting candidate words from the pre-trained word vector set, and finally replacing the words to be replaced with the candidate words to obtain a data enhanced text sample. The topic distribution probability information corresponding to each sentence in the original text sample is obtained by using the topic model, so as to well measure the contribution value of each word in the sentence to the topic of the text sentence, and complete the data enhancement under the condition that the sentence topic distribution is not affected. At the same time, with the help of pre-trained word vectors, words with similar semantics to the words to be replaced are selected as replacement words to maximize the semantic information of the sentence.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data enhancement method and device, an electronic device, and a storage medium. Background Art

[0002] Data augmentation is a data processing method commonly used in fields such as image processing and speech processing. For example, in image processing, researchers often use operations such as rotation and cropping to enhance and enrich data samples. Unlike image and speech processing, augmenting text data cannot simply rely on text transformations such as permuting or discarding words. This is because the word order in a text is governed by strict syntax and semantics, and these simple operations can result in loss of semantic information.

[0003] In related technologies, data augmentation methods used for text classification tasks mainly include: data augmentation based on synonym tables, data augmentation based on back-translation, or data augmentation based on pre-trained word vectors. However, data augmentation based on synonym tables cannot effectively adapt to texts in specific fields; data augmentation based on back-translation requires the use of additional translation models, and large amounts of bilingual corpus are also severely lacking in specific fields; and when using pre-trained word vectors for data augmentation, there is also the problem of how to select appropriate words to replace. Summary of the Invention

[0004] The main purpose of the embodiments of the present invention is to propose a data enhancement method and device, an electronic device and a storage medium, which can improve the accuracy of text sample data enhancement and expand its scope of application.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present invention provides a data enhancement method, comprising:

[0006] Obtaining an original text sample to be enhanced; wherein the original text sample includes at least one text sentence, and the original text sample includes at least one subject word;

[0007] Inputting the original text sample into a pre-trained topic model to obtain topic distribution probability information corresponding to each text sentence, wherein the topic model is a hidden Dirichlet allocation topic model;

[0008] Calculating a contribution value of each of the subject words to the text sentence according to the subject distribution probability information;

[0009] Calculating the replacement probability of the subject word according to the contribution value of the subject word, and selecting a to-be-replaced word from the text sentence according to the replacement probability to obtain a to-be-replaced word set;

[0010] Screening candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced;

[0011] The candidate word is used to replace the word to be replaced to obtain a data-enhanced text sample.

[0012] In some embodiments, before inputting the original text sample into the pre-trained topic model, the method further includes:

[0013] Obtaining a training sample set in a preset domain, wherein the training sample set includes unlabeled training text samples and corresponding probability labels;

[0014] Inputting the training text sample into the initial topic model, and obtaining the predicted topic distribution probability of the training text sample according to a preset number of topics;

[0015] Calculate the loss value based on the predicted topic distribution probability and the corresponding probability label;

[0016] The model weights of the initial topic model are adjusted according to the loss value using a loss function until the loss function meets a convergence condition, and the topic model is obtained by training.

[0017] In some embodiments, the topic distribution probability information includes: topic distribution probabilities of topic words and topic distribution probabilities of text sentences, and calculating the contribution value of each topic word to the text sentence based on the topic distribution probability information includes:

[0018] Calculate the topic distribution probability of the text sentence according to the first formula;

[0019] Calculate the topic distribution probability of the topic words;

[0020] Multiplying a preset smoothing parameter, the topic distribution probability, and the topic distribution probability to obtain the contribution value;

[0021] Among them, the first formula is:

[0022]

[0023] Among them, ω i Represents the subject word, s=s(ω1,ω2,...,ω i ,...,ω N ) represents a text sentence, which contains N keywords, p(t|ω i ) represents the topic distribution probability of the topic word, and p(t|s) represents the topic distribution probability of the text sentence.

[0024] In some embodiments, calculating the replacement probability of the subject word based on the contribution value of the subject word, and selecting the to-be-replaced word from the text sentence based on the replacement probability to obtain a to-be-replaced word set includes:

[0025] Calculating the replacement probability of the subject word according to the contribution value of the subject word to the text sentence;

[0026] The word to be replaced is obtained by sampling according to the preset number of replacement words and the replacement probability;

[0027] The to-be-replaced words are used to form the to-be-replaced word set.

[0028] In some embodiments, calculating the replacement probability of the subject word based on the contribution value of the subject word to the text sentence includes:

[0029] Calculate the maximum contribution value of all topic words in the text sentence;

[0030] Calculating the difference between the contribution value of each of the subject words and the maximum contribution value, and summing all the differences to obtain a total contribution value;

[0031] The ratio between each difference and the total contribution value is calculated to obtain the replacement probability of the subject word.

[0032] In some embodiments, before filtering candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced, the method further includes:

[0033] Obtain training text samples in a preset field;

[0034] Using the Word2vec tool to train the training text sample to obtain a pre-trained word vector;

[0035] The word vector set is formed using the pre-trained word vectors.

[0036] In some embodiments, the step of selecting candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced includes:

[0037] Calculating the distance between the to-be-replaced word in the to-be-replaced word set and the pre-trained word vector in the word vector set in the vector space;

[0038] Sorting the distances to obtain a distance sorting result;

[0039] According to the distance sorting result, a preset number of words are selected from the word vector set as the candidate words, and the position distribution of the candidate words in the word vector set obeys a geometric distribution.

[0040] To achieve the above-mentioned object, the second aspect of the present invention provides a text sample data enhancement device, comprising:

[0041] A sample acquisition module, configured to acquire an original text sample to be enhanced; wherein the original text sample includes at least one text sentence, and the original text sample includes at least one keyword;

[0042] a topic distribution probability calculation module, configured to input the original text sample into a pre-trained topic model to obtain topic distribution probability information corresponding to each of the text sentences, wherein the topic model is a hidden Dirichlet allocation topic model;

[0043] A contribution value calculation module, configured to calculate a contribution value of each of the subject words to the text sentence based on the subject distribution probability information;

[0044] a to-be-replaced word selection module, configured to calculate a replacement probability of the subject word according to the contribution value of the subject word, and select a to-be-replaced word from the text sentence according to the replacement probability to obtain a to-be-replaced word set;

[0045] A candidate word selection module is used to select candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced;

[0046] The data enhancement module is used to replace the to-be-replaced word with the candidate word to obtain a data-enhanced text sample.

[0047] To achieve the above-mentioned object, a third aspect of the present invention provides an electronic device, comprising:

[0048] at least one memory;

[0049] at least one processor;

[0050] at least one program;

[0051] The program is stored in the memory, and the processor executes the at least one program to implement the method of the present invention as described in the first aspect above.

[0052] To achieve the above-mentioned objective, a fourth aspect of the present invention provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute:

[0053] As described in the first aspect above.

[0054] The data enhancement method and device, electronic device, and storage medium proposed in the embodiments of the present invention obtain a raw text sample, input the raw text sample into a pre-trained topic model, and obtain topic distribution probability information corresponding to each sentence in the raw text sample. The topic model is a topic model based on LDA hidden Dirichlet allocation. The contribution value of each topic word in each sentence to the text sentence is calculated based on the topic distribution probability information. The replacement probability of the topic word is then calculated based on the contribution value of the topic word to the text sentence. The replacement probability is used to select a to-be-replaced word in the text sentence to obtain a to-be-replaced word set. Then, a word similar to the to-be-replaced word in the to-be-replaced word set is selected from the pre-trained word vector set as a candidate word. Finally, the candidate word is used to replace the to-be-replaced word to obtain a data enhanced text sample. This embodiment uses the topic model to obtain the topic distribution probability information corresponding to each sentence in the raw text sample, thereby effectively measuring the contribution value of each word in the sentence to the text sentence topic. It can ensure that data enhancement is completed without affecting the sentence topic distribution. At the same time, with the help of pre-trained word vectors, a word with a semantic similarity to the to-be-replaced word can be selected as a replacement word, thereby maximizing the semantic information of the sentence. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 4 is a flow chart of a data enhancement method provided by an embodiment of the present invention.

[0056] Figure 2 It is a partial flow chart of a data enhancement method provided by another embodiment of the present invention.

[0057] Figure 3 This is a partial flow chart of a data enhancement method provided by another embodiment of the present invention.

[0058] Figure 4 This is a partial flow chart of a data enhancement method provided by another embodiment of the present invention.

[0059] Figure 5 This is a partial flow chart of a data enhancement method provided by another embodiment of the present invention.

[0060] Figure 6 This is a flowchart of a data enhancement method provided by another embodiment of the present invention.

[0061] Figure 7 This is a structural block diagram of a text sample data enhancement device provided by another embodiment of the present invention.

[0062] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0064] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0066] First, some terms involved in this invention are analyzed:

[0067] Latent Dirichlet Allocation (LDA) model: The LDA model is an unsupervised machine learning technique and a model used in text semantic analysis to infer the topic distribution of documents. The topic of each document in a document collection can be given in the form of a probability distribution. After analyzing a number of documents to extract the topic distribution, topic clustering or text classification can be performed based on the topic distribution. The LDA model is a bag-of-words model that assumes that documents are independent of each other and that the words in a document are independent of each other. The bag-of-words model only considers whether a word appears in a document, regardless of the order in which it appears in the document. For example, in the bag-of-words model, "I like you" and "You like me" are equivalent.

[0068] Natural Language Processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. It integrates linguistics, computer science, and mathematics. Simply put, computers accept input in the form of natural language and perform a series of internal operations, including processing and calculations, using human-defined algorithms to simulate human understanding of natural language and return the desired results. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question-answering, and knowledge graphs.

[0069] Gensim is an open-source, third-party Python toolkit for unsupervised learning of latent topic vector representations from raw, unstructured text. It supports a variety of topic modeling algorithms, including TF-IDF, LSA, LDA, and word2vec, supports streaming training, and provides APIs for common tasks such as similarity calculation and information retrieval.

[0070] Word2Vec: Word2Vec is a word embedding method in the field of natural language processing (NLP). It is a language model used to generate word vectors. This model learns semantic knowledge from large amounts of text in an unsupervised manner and is widely used in natural language processing. Word embedding converts uncomputable, unstructured words into computable, structured vectors.

[0071] Gibbs sampling: is a special Markov chain algorithm that is often used to solve a series of problems including matrix decomposition and tensor decomposition. It is also called alternating conditional sampling. The word "alternating" means that Gibbs sampling is an iterative algorithm, and the corresponding variables are used alternately during the iteration process. In addition, the word "conditional" is added because the core of Gibbs sampling is Bayesian theory, which revolves around prior knowledge and observation data, and uses observation values ​​as conditions to infer the posterior distribution.

[0072] Data augmentation is a common data processing method widely used in image and speech processing. For example, in the image field, researchers often use simple operations such as rotation and cropping to process image data to achieve data enhancement and enrich data samples. This data augmentation method has been proven to effectively improve the model's generalization ability on test data. Unlike in the image and speech fields, when enhancing text data, simple text transformations such as permuting the order or discarding some words cannot be used. This is because the word order in the text forms a strict syntax and semantics, and these simple operations will result in the loss of semantic information.

[0073] In the text domain, the best data augmentation method is to manually rewrite sentences, but considering the scale of the dataset, this method is impractical and costly. In the related art, data augmentation methods used for text classification tasks mainly include: data augmentation based on synonym tables, data augmentation based on back-translation (such as translating Chinese into English and then translating English back into Chinese), or data augmentation based on pre-trained word vectors. However, data augmentation based on synonym tables generally uses existing public, general-purpose vocabulary to replace words, which cannot effectively adapt to text in specific fields, such as finance or medicine. Building a synonym dictionary from scratch is also very costly. Data augmentation based on back-translation requires the use of additional translation models to implement, and the data is also severely lacking in large amounts of bilingual corpus in specific fields. Using pre-trained word vectors for data augmentation is a compromise between the first two methods, as it can utilize unlabeled text in a specific field. However, there is a problem with this method: how to select which words in the text to replace, because these selected words must minimize the semantic impact of the sentence, otherwise the classification model performance will not be effectively improved.

[0074] Based on this, an embodiment of the present invention provides a data enhancement method and device, an electronic device, and a storage medium, which use a topic model to obtain the topic distribution probability information corresponding to each sentence in the original text sample, thereby well measuring the contribution value of each word in the sentence to the text sentence topic, and can ensure that data enhancement is completed without affecting the sentence topic distribution. At the same time, with the help of pre-trained word vectors, words with similar semantics to the word to be replaced can be selected as replacement words, thereby maximizing the semantic information of the sentence.

[0075] The embodiments of the present invention provide a data enhancement method and apparatus, an electronic device, and a storage medium, which are specifically described through the following embodiments. First, the data enhancement method in the embodiments of the present invention is described.

[0076] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0077] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0078] The data enhancement method provided by the embodiment of the present invention relates to the field of artificial intelligence technology, and in particular to the field of data mining technology. The data enhancement method provided by the embodiment of the present invention can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application that implements the data enhancement method, etc., but is not limited to the above forms.

[0079] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0080] Figure 1 is an optional flowchart of the data enhancement method provided by an embodiment of the present invention. Figure 1 The method may include but is not limited to steps S110 to S160.

[0081] Step S110: obtaining an original text sample to be enhanced.

[0082] Text classification is a fundamental task in natural language processing. Currently, deep learning models such as CNNs, RNNs, and Transformers have achieved promising results in tasks like news classification and sentiment analysis. However, these deep learning models require large-scale, high-quality annotated data, which is often difficult to obtain or very expensive to annotate in real-world scenarios.

[0083] In one embodiment, the original text sample includes at least one text sentence, and the original text sample includes at least one keyword. A sentence may contain different keywords or may not contain any keywords. In addition, the original text sample is a labeled sample in a preset field, wherein the preset field can be selected according to actual needs, and the labeled text sample is enhanced to obtain a large-scale enhanced labeled text related to the preset field. The labeling information in the text sample in the preset field can be obtained not only by automatic labeling based on a semi-supervised classification model, but also by manual labeling. Since the accuracy of the category label of the manually labeled labeled text is higher, the labeling accuracy of the labeled text can be improved.

[0084] In one embodiment, after obtaining the original text sample, it is necessary to perform text preprocessing on it to obtain a preprocessed text corpus, wherein the text preprocessing includes but is not limited to: removing illegal characters, removing stop words, removing redundant words, and word segmentation.

[0085] In one embodiment, a preset dictionary can be used to perform word segmentation on the original text sample, and stop words can be removed from each sample text in the sample set. Specifically, the preset dictionary can be a custom dictionary corresponding to a preset field, which can include multiple pre-defined word segments. When performing word segmentation on the original text sample, the custom dictionary can be used to perform word segmentation on the original text sample. For example, the custom dictionary can be used to obtain word segments from the original text sample that match each word in the custom dictionary, thereby decomposing the original text sample into multiple word segments that match the word segments in the custom dictionary, thereby improving the accuracy of word segmentation on the original text sample.

[0086] In addition, in order to improve search efficiency during information retrieval, some characters or words will be automatically filtered out before or after processing natural language data (or text). These characters or words are called stop words. Stop words can be roughly divided into two categories: one is to use very widely, even some words that are too frequent, such as "I", "Is", etc.; the other is words that are not very meaningful in the text. These words include modal particles, adverbs, prepositions, conjunctions, etc., which usually have no clear meaning themselves and have only a certain effect when put into a complete sentence, such as the common "of", "in", "and", "next", etc. Therefore, in the present embodiment, after the original text sample is segmented, some meaningless words such as modal particles, interjections or illegal characters can be removed by removing stop words. By performing feature analysis on a large number of text samples in a preset field, the importance values ​​of different metric words are calculated, such as by calculating the importance value through methods such as tf-idf and information gain, a stop word set for the preset field is generated according to the importance value, and the stop word set for the general field is obtained by combining the stop words in the general field. The stop word removal in this embodiment is to delete the words in the stop word set from the word sequence after word segmentation.

[0087] In one embodiment, after the original text sample is segmented, redundant words can also be removed. Redundant word filtering refers to removing semantically redundant words. Each word obtained after segmenting the two Chinese texts is matched against a preset semantic template. The matched semantic templates are used to identify superordinate and subordinate words that appear simultaneously in a sentence. Superordinate words are identified as semantically redundant and further filtered out. The semantic templates for redundant words are not specifically limited herein and can be calculated using existing or well-known calculation methods in the art, as long as they are applicable to the present invention.

[0088] Step S120: input the original text sample into a pre-trained topic model to obtain topic distribution probability information corresponding to each text sentence in the original text sample.

[0089] In one embodiment, the topic model is a hidden Dirichlet allocation topic model, and the tool used is the Gensim tool. In this embodiment, the hidden Dirichlet allocation topic model is used to obtain the probability distribution of each basic word (for example, each topic word obtained by word segmentation) in the original text sample after preprocessing to the topic, and the probability distribution of the original text sample to the topic. It is understandable that the hidden Dirichlet allocation topic model can be calculated using existing or well-known calculation methods in the field, as long as it can be applied to the present invention. The calculation of the hidden Dirichlet allocation topic model can include multiple methods, for example, it can be calculated with a single training thread of a processor, it can also be calculated with multiple training threads of multiple processors, or even distributed computing.

[0090] In one embodiment, the hidden Dirichlet allocation topic model is first trained, referring to Figure 2 The process of training the hidden Dirichlet allocation topic model includes but is not limited to steps S210 to S240:

[0091] Step S210: Obtain a training sample set of a preset domain.

[0092] In one embodiment, the training sample set includes unlabeled training text samples in a preset domain and corresponding probability labels. It is understandable that the above text preprocessing operations can also be performed on the training text samples, where the text preprocessing includes but is not limited to: removing illegal characters and word segmentation.

[0093] Step S220: input the training text sample into the initial topic model, and obtain the predicted topic distribution probability of the training text sample according to the preset number of topics.

[0094] Step S230: Calculate the loss value based on the predicted topic distribution probability and the corresponding probability label.

[0095] In one embodiment, a preset number of topics is first set. This number can be determined based on prior knowledge or actual needs and is not specifically limited herein. A training text sample is then input into an initial topic model for iterative processing. The initial topic model is calculated to obtain a predicted topic distribution probability for the training text sample. The predicted topic distribution probability includes the probability distribution from the base words of the training text sample to topics and the probability distribution from the training text sample to topics.

[0096] In one embodiment, the probability distribution of basic words to topics is a word-to-topic matrix, where the rows of the matrix are words and the columns are implicitly calculated topics. The probability distribution of training text samples to topics is a text-to-topic matrix, where the rows of the matrix are each training text sample and the columns are implicitly calculated topics. The word vector of the topic is the column vector of the matrix in the word-to-topic matrix. These matrices are initially random values, and through Gibbs sampling iterations, each value of the matrix is ​​gradually optimized to obtain the predicted topic distribution probability. Ultimately, the word-to-topic matrix can be used to obtain word clustering to guide keywords. In this embodiment, the loss value is calculated based on the predicted topic distribution probability and the corresponding probability label.

[0097] Step S240: Using the loss function, adjust the model weights of the initial topic model according to the loss value until the loss function meets the convergence condition, and the topic model is trained.

[0098] In one embodiment, after iteration, a loss function is used to determine whether the basic LDA model has reached convergence based on the loss value, thereby adjusting the model weights of the initial topic model until the loss function meets the convergence condition, thereby training to obtain a topic model. For example, the convergence condition can be the number of iterations. If the convergence condition is not met, the initial topic model is calculated for each training text sample. If the convergence condition is met, the topic model is trained to obtain.

[0099] In one embodiment, the topic distribution probability information corresponding to each sentence in the original text sample is obtained based on the pre-trained topic model, wherein the topic distribution probability information includes: the topic distribution probability of the topic words and the topic distribution probability of the sentences.

[0100] Step S130 , calculating the contribution value of each topic word in each sentence to the text sentence according to the topic distribution probability information.

[0101] In one embodiment, the topic distribution probability of the text sentence is calculated according to the first formula, and then the topic distribution probability of the topic word is calculated. Finally, the preset smoothing parameter, the topic distribution probability, and the topic distribution probability are multiplied according to the second formula to obtain the contribution value.

[0102] Specifically, the first formula is expressed as:

[0103]

[0104] The second formula is expressed as:

[0105]

[0106] Among them, ω i Represents the subject word, s=s(ω1,ω2,...,ω i ,...,ω N ) represents a text sentence, which contains N keywords. Indicates the contribution value of the topic word to the text sentence, p(t|ω i ) represents the topic distribution probability of the topic word, p(t|s) represents the topic distribution probability of the text sentence, and τ represents the smoothing parameter.

[0107] In one embodiment, the smoothing parameter τ is used to control the smoothness of the word replacement probability, and can generally be set to 0.75.

[0108] In step S140 , the replacement probability of the subject word is calculated based on the contribution value of the subject word, and words to be replaced are selected from the text sentence based on the replacement probability to obtain a set of words to be replaced.

[0109] In one embodiment, the contribution value of each subject word in the sentence to the subject of the text sentence is obtained, and the replacement probability of the subject word is further calculated based on the contribution value of the subject word to the text sentence, and the replacement probability is used to select the text sentence to be replaced to obtain a set of words to be replaced. In addition, if a text sentence does not contain a subject word, the contribution value of the subject word to the text sentence is zero. Figure 3 Step S140 includes but is not limited to steps S141 to S143:

[0110] Step S141 : Calculate the replacement probability of the subject word according to the contribution value of the subject word in the text sentence.

[0111] In one embodiment, the maximum contribution value of all subject words in the text sentence is calculated according to the third formula, and then the total contribution value obtained by subtracting the maximum contribution value from the contribution value of all subject words is calculated according to the fourth formula. Finally, the replacement probability of the subject word is calculated according to the fifth formula using the contribution value, the maximum contribution value, and the total contribution value of the subject word to the text sentence.

[0112] Calculate the maximum contribution value of all topic words in the text sentence, expressed as: Then calculate the difference between the contribution value of each keyword and the maximum contribution value, and sum up all the differences to get the total contribution value. The total contribution value is expressed as: Finally, the ratio between each difference and the total contribution value is calculated to obtain the replacement probability of the subject word. The replacement probability of the subject word is expressed as:

[0113]

[0114] in, Indicates the contribution value of the subject word to the text sentence, represents the replacement probability of the subject word, M represents the maximum contribution value of all subject words in the text sentence, and Z represents the total contribution value of all subject words minus the maximum contribution value.

[0115] In one embodiment, the contribution value of the subject word to the text sentence topic The higher the probability of its keyword replacement The lower it is, the higher the contribution value of a keyword to the topic of a sentence, and it will not be considered for replacement. This method can ensure the topic significance of the sentence in the preset field.

[0116] Step S142: sampling to obtain words to be replaced according to the preset number of replacement words and replacement probability.

[0117] Step S143: Using the words to be replaced, a set of words to be replaced is formed.

[0118] In one embodiment, a preset number r of replacement words is first randomly selected, where the preset number r of replacement words follows a geometric distribution, which is expressed as:

[0119] P[r]~p r

[0120] P(X=r)=p(1-p) r-1

[0121] In one embodiment, the geometric distribution is a discrete probability distribution defined as: the probability of obtaining the first success after k trials in n Bernoulli trials, i.e., the probability of success after the kth trial fails. In a Bernoulli trial, the probability of success is p. In this embodiment, p can be 0.5, but this is not specifically limited here.

[0122] After obtaining the preset number r of replacement words, random sampling is performed based on the contribution value of the subject words obtained above to the text sentence topic to obtain r words as to-be-replaced words, and the obtained to-be-replaced words are combined to obtain a to-be-replaced word set.

[0123] Step S150 , screening candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced.

[0124] In one embodiment, the pre-trained word vector set is trained using the Word2vec tool. Word2Vec is a tool that Google open-sourced in 2013 that represents words as real-valued vectors. It uses the idea of ​​deep learning to simplify the processing of the text content of training text samples into vector operations in a K-dimensional vector space through training. The similarity in the vector space can be used to represent the semantic similarity of the training text samples. Figure 4 The steps of pre-training to obtain a word vector set include but are not limited to steps S410 to S430:

[0125] Step S410: Obtain training text samples in a preset field.

[0126] In one embodiment, the training text samples may utilize training text samples from a preset domain used in training a latent Dirichlet allocation topic model.

[0127] Step S420: Use the Word2vec tool to train the training text samples to obtain pre-trained word vectors.

[0128] Step S430: Use pre-trained word vectors to form a word vector set.

[0129] In one embodiment, after training, pre-trained word vectors related to a preset domain are obtained. The word vector dimension setting during training can be determined according to the specific corpus size and is not specifically limited here.

[0130] In one embodiment, after obtaining the word vector set, words similar to the words to be replaced in the word vector set are selected as candidate words. Figure 5 Step S150 includes but is not limited to steps S151 to S153:

[0131] Step S151 , calculating the distance in the vector space between the words to be replaced in the word set to be replaced and the pre-trained word vectors in the word vector set.

[0132] In one embodiment, a distance matrix can be obtained by calculating the distance between the to-be-replaced words in the to-be-replaced word set and the pre-trained word vectors in the word vector set. The vector distance can be determined using a method for determining the distance between two vectors in existing or future technologies, which is not limited in this application. For example, the distance can be calculated using the Euclidean distance formula or the cosine distance formula.

[0133] In one embodiment, the distance between the words to be replaced in the word set and the pre-trained word vectors in the word vector set is specifically expressed in the distance matrix as follows: ij, i and j represent the row and column of the distance matrix, respectively. Since in this embodiment, synonyms of the word to be replaced are selected from the word vector set by distance, and whether two words are synonyms is related to the distance between the two word vectors in the matrix, if the distance between the two word vectors is within a preset distance, it can be determined that the two words are synonyms. Therefore, in this embodiment, synonyms of the word to be replaced can be found quickly and easily through the distance matrix, thereby improving the reliability of replacing the word to be replaced.

[0134] Step S152: sort the distances to obtain a distance sorting result.

[0135] In one embodiment, the distances in the distance matrix are sorted to obtain a distance sorting result, and then a preset distance is used to determine which pre-trained word vectors in the word vector set are synonymous with the words to be replaced in the word set to be replaced.

[0136] Step S153: Select a preset number of words from the word vector set as candidate words based on the distance sorting result.

[0137] In one embodiment, based on the distance sorting result, pre-trained word vectors in the word vector set whose distance between two word vectors in the distance matrix is ​​within a preset distance are selected as candidate words corresponding to the word to be replaced, for example, there are k pre-trained word vectors that meet the candidate word criteria.

[0138] In one embodiment, if there are many candidate words in the word vector set, the required number of candidate words can be selected based on the position distribution of the candidate words. For example, the position s of the candidate words in the word vector set can be set to follow a geometric distribution, which can be expressed as:

[0139] P[r]~p r

[0140] The value of q can be 0.5, which is not specifically limited here.

[0141] Step S160: Replace the word to be replaced with the candidate word to obtain a data-enhanced text sample.

[0142] In one embodiment, one or more updated sentences can be obtained by replacing the words to be replaced in a sentence with candidate words. Each sentence in the original text sample is updated accordingly to obtain one or more data-enhanced text samples. It is understood that the same sentence can be enhanced multiple times, with the specific number varying across datasets. For example, the same sentence can be enhanced two to four times. After each sentence in the original text sample is enhanced, one or more data-enhanced annotated corpus, i.e., data-enhanced text samples, is obtained.

[0143] In addition, refer to Figure 6, is a flow chart of a data enhancement method provided in one embodiment of the present application.

[0144] Step S600: Acquire a large number of unlabeled text samples in a preset field as training text samples.

[0145] Step S610 , performing text preprocessing on the training text sample, wherein the text preprocessing includes but is not limited to: removing illegal characters, removing stop words, removing redundant words, and word segmentation.

[0146] Step S620: Using the training text samples to train the initial topic model to obtain a topic model.

[0147] Step S630: Use the Word2vec tool to train the training text samples to obtain a pre-trained word vector set.

[0148] Step S640: Using the labeled text samples in the preset domain as original text samples to perform text sample data enhancement.

[0149] Step S650 , performing text preprocessing on the original text sample, wherein the text preprocessing includes but is not limited to: removing illegal characters, removing stop words, removing redundant words, and segmenting words.

[0150] Step S660, calculate the contribution value of each keyword in the original text sample to the text sentence, specifically: use the topic model obtained in step S620 to obtain the topic distribution probability information corresponding to each sentence in the original text sample, and calculate the contribution value of each keyword in each sentence to the text sentence based on the topic distribution probability information

[0151] Step S670, obtaining a set of words to be replaced based on the contribution value of the subject word to the text sentence, specifically: calculating the replacement probability of the subject word based on the contribution value of the subject word to the text sentence, and using the replacement probability to select the text sentence to replace the word to obtain the set of words to be replaced.

[0152] Step S680 , using the word vector set obtained in step S630 , select candidate words therefrom. Specifically, select words in the word vector set that are similar to the words to be replaced in the word set to be replaced as candidate words.

[0153] Step S690: obtaining a data-enhanced text sample, specifically: replacing the word to be replaced with the candidate word.

[0154] The data enhancement method provided by an embodiment of the present invention obtains an original text sample, inputs the original text sample into a pre-trained topic model, obtains topic distribution probability information corresponding to each sentence in the original text sample, and the topic model is a topic model based on LDA hidden Dirichlet allocation. The contribution value of each topic word in each sentence to the text sentence is calculated according to the topic distribution probability information, and then the replacement probability of the topic word is calculated according to the contribution value of the topic word to the text sentence. The replacement probability is used to select the to-be-replaced word in the text sentence to obtain a to-be-replaced word set, and then selects words similar to the to-be-replaced word in the to-be-replaced word set from the pre-trained word vector set as candidate words. Finally, the candidate words are used to replace the to-be-replaced words to obtain a data enhanced text sample.

[0155] This embodiment uses a topic model to obtain the topic distribution probability information corresponding to each sentence in the original text sample, thereby effectively measuring the contribution value of each word in the sentence to the text sentence topic, and can ensure that data enhancement is completed without affecting the sentence topic distribution. At the same time, with the help of pre-trained word vectors, words with similar semantics to the word to be replaced can be selected as replacement words, thereby maximizing the semantic information of the sentence.

[0156] In addition, the embodiment of the present invention also provides a text sample data enhancement device, which can implement the above data enhancement method, referring to Figure 7 , the device comprises:

[0157] The sample acquisition module 710 is configured to acquire an original text sample to be enhanced; wherein the original text sample includes at least one text sentence and at least one keyword;

[0158] Topic distribution probability calculation module 720 is used to input the original text sample into the pre-trained topic model to obtain the topic distribution probability information corresponding to each text sentence. The topic model is a hidden Dirichlet allocation topic model;

[0159] Contribution value calculation module 730, for calculating the contribution value of each topic word to the text sentence based on the topic distribution probability information;

[0160] The to-be-replaced word selection module 740 is configured to calculate the replacement probability of the subject word based on the contribution value of the subject word, and select the to-be-replaced word from the text sentence based on the replacement probability to obtain a to-be-replaced word set;

[0161] A candidate word selection module 750 is configured to select candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced;

[0162] The data enhancement module 760 is used to replace the words to be replaced with the candidate words to obtain data enhanced text samples.

[0163] In one embodiment, the topic distribution probability information includes: topic distribution probability of topic words and topic distribution probability of sentences. The contribution value of each topic word to the text sentence in the contribution value calculation module 730 is expressed as:

[0164]

[0165]

[0166] Among them, ω i Represents the subject word, s=s(ω1,ω2,...,ω i ,...,ω N ) represents a text sentence, which contains N keywords. Indicates the contribution value of the topic word to the text sentence, p(t|ω i ) represents the topic distribution probability of the topic word, p(t|s) represents the topic distribution probability of the text sentence, and τ represents the smoothing parameter.

[0167] In one embodiment, the to-be-replaced word selection module 740 is further configured to calculate the replacement probability of the subject word based on the contribution value of the subject word to the text sentence, and then sample the to-be-replaced words based on the preset number of replacement words and the replacement probability, and finally use the to-be-replaced words to form a to-be-replaced word set.

[0168] In one embodiment, the candidate word selection module 750 is also used to calculate the distance in the vector space between the words to be replaced in the set of words to be replaced and the pre-trained word vectors in the pre-trained word vector set, and then sort the distances to obtain a distance sorting result. Finally, based on the distance sorting result, a preset number of words are selected from the word vector set as candidate words, and the position distribution of the candidate words in the word vector set obeys a geometric distribution.

[0169] The specific implementation of the text sample data enhancement device of this embodiment is basically the same as the specific implementation of the above-mentioned data enhancement method, and will not be repeated here.

[0170] An embodiment of the present invention further provides an electronic device, including:

[0171] at least one memory;

[0172] at least one processor;

[0173] at least one program;

[0174] The program is stored in the memory, and the processor executes the at least one program to implement the data enhancement method of the present invention. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0175] See also Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0176] The processor 801 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0177] The memory 802 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 802 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called by the processor 801 to execute the data enhancement method of the embodiment of the present invention;

[0178] Input / output interface 803, used to implement information input and output;

[0179] Communication interface 804, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0180] Bus 805 , which transmits information between various components of the device (e.g., processor 801 , memory 802 , input / output interface 803 , and communication interface 804 );

[0181] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0182] An embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned data enhancement method.

[0183] The data enhancement method, text sample data enhancement device, electronic device, and storage medium proposed in the embodiments of the present invention use a topic model to obtain topic distribution probability information corresponding to each sentence in the original text sample, thereby effectively measuring the contribution value of each word in the sentence to the text sentence topic, and can ensure that data enhancement is completed without affecting the sentence topic distribution. At the same time, with the help of pre-trained word vectors, words with similar semantics to the word to be replaced can be selected as replacement words, thereby maximizing the semantic information of the sentence.

[0184] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0185] The non-transient software program and non-transient computer executable program provided in this embodiment can be used to perform the following steps: by obtaining an original text sample, inputting the original text sample into a pre-trained topic model to obtain the topic distribution probability information corresponding to each sentence in the original text sample, the topic model is a topic model based on LDA hidden Dirichlet allocation, and then calculating the contribution value of each topic word in each sentence to the text sentence based on the topic distribution probability information, and then calculating the replacement probability of the topic word based on the contribution value of the topic word to the text sentence, and using the replacement probability to select the text sentence to be replaced The word is obtained to obtain a set of words to be replaced, and then, in the word vector set obtained in pre-training, a word similar to the word to be replaced in the set of words to be replaced is selected as a candidate word, and finally, the candidate word is used to replace the word to be replaced to obtain a data enhanced text sample.

[0186] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0187] It will be understood by those skilled in the art that Figure 1-5 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0188] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0189] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0190] The terms "first," "second," "third," "fourth," and the like (if any) in the description of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or apparatus.

[0191] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0192] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0195] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store programs.

[0196] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the invention is not limited thereby. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the invention should be within the scope of the invention.

Claims

1. A data enhancement method, characterized in that: The method comprises: Obtaining an original text sample to be enhanced; wherein the original text sample includes at least one text sentence, and the original text sample includes at least one subject word; Inputting the original text sample into a pre-trained topic model to obtain topic distribution probability information corresponding to each of the text sentences, wherein the topic model is a hidden Dirichlet allocation topic model, and the topic distribution probability information includes: topic distribution probability of topic words and topic distribution probability of text sentences; Calculating the topic distribution probability of the text sentence according to the first formula; calculating the topic distribution probability of the subject word; multiplying a preset smoothing parameter, the topic distribution probability of the subject word, and the topic distribution probability of the text sentence to obtain a contribution value; Calculating the replacement probability of the subject word according to the contribution value of the subject word to the text sentence, sampling to obtain words to be replaced according to a preset number of replacement words and the replacement probability, and forming a set of words to be replaced using the words to be replaced; Screening candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced; Replacing the word to be replaced with the candidate word to obtain a data-enhanced text sample; The method for calculating the replacement probability of the subject word based on the contribution value of the subject word to the text sentence includes: calculating the maximum contribution value of all subject words in the text sentence; calculating the difference between the contribution value of each subject word and the maximum contribution value, and summing up all the differences to obtain the total contribution value; calculating the ratio between each difference and the total contribution value to obtain the replacement probability of the subject word.

2. The data enhancement method according to claim 1, wherein: Before inputting the original text sample into the pre-trained topic model, the method further includes: Obtaining a training sample set in a preset domain, wherein the training sample set includes unlabeled training text samples and corresponding probability labels; Inputting the training text sample into the initial topic model, and obtaining the predicted topic distribution probability of the training text sample according to a preset number of topics; Calculate the loss value based on the predicted topic distribution probability and the corresponding probability label; The model weights of the initial topic model are adjusted according to the loss value using a loss function until the loss function meets a convergence condition, and the topic model is obtained by training.

3. The data enhancement method according to claim 1, wherein: The first formula is: in, Indicates the subject word, represents a text sentence, wherein the text sentence contains N keywords, represents the topic distribution probability of the topic word, Represents the topic distribution probability of a text sentence.

4. The data enhancement method according to claim 1, wherein: Before the step of filtering candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced, the step further includes: Obtain training text samples in a preset field; Using the Word2vec tool to train the training text sample to obtain a pre-trained word vector; The word vector set is formed using the pre-trained word vectors.

5. The data enhancement method according to any one of claims 1 to 4, characterized in that: The step of screening the candidate words from the pre-trained word vector set according to the words to be replaced in the set of words to be replaced includes: Calculating the distance between the to-be-replaced word in the to-be-replaced word set and the pre-trained word vector in the word vector set in the vector space; Sorting the distances to obtain a distance sorting result; According to the distance sorting result, a preset number of words are selected from the word vector set as the candidate words, and the position distribution of the candidate words in the word vector set obeys a geometric distribution.

6. A text sample data enhancement device, characterized in that: include: A sample acquisition module, configured to acquire an original text sample to be enhanced; wherein the original text sample includes at least one text sentence, and the original text sample includes at least one keyword; a topic distribution probability calculation module, configured to input the original text sample into a pre-trained topic model to obtain topic distribution probability information corresponding to each of the text sentences, wherein the topic model is a hidden Dirichlet allocation topic model, and the topic distribution probability information includes: topic distribution probability of a topic word and topic distribution probability of a text sentence; a contribution value calculation module, configured to calculate the topic distribution probability of a text sentence according to the first formula; calculate the topic distribution probability of a topic word; and multiply a preset smoothing parameter, the topic distribution probability of the topic word, and the topic distribution probability of the text sentence to obtain a contribution value; A to-be-replaced word selection module is configured to calculate the replacement probability of the subject word based on the contribution value of the subject word to the text sentence, sample to-be-replaced words based on a preset number of replacement words and the replacement probability, and use the to-be-replaced words to form a to-be-replaced word set; A candidate word selection module is used to select candidate words from a pre-trained word vector set based on the words to be replaced in the set of words to be replaced; A data enhancement module, configured to replace the word to be replaced with the candidate word to obtain a data enhanced text sample; The method for calculating the replacement probability of the subject word based on the contribution value of the subject word to the text sentence includes: calculating the maximum contribution value of all subject words in the text sentence; calculating the difference between the contribution value of each subject word and the maximum contribution value, and summing up all the differences to obtain the total contribution value; calculating the ratio between each difference and the total contribution value to obtain the replacement probability of the subject word.

7. An electronic device, characterized in that: include: at least one memory; at least one processor; The memory stores a computer program, and the processor executes the computer program to implement: The method according to any one of claims 1 to 5.

8. A storage medium, wherein the storage medium is a computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to cause a computer to execute: The method according to any one of claims 1 to 5.

Images