A data processing method, apparatus, and storage medium based on prompting learning

Abstract

This invention discloses a data processing method, apparatus, and storage medium based on cue-based learning. It can extract samples from an unlabeled first corpus to obtain sample corpus; label each sample in the sample corpus to obtain labeled second corpus; train a model based on cue-based learning and ensemble learning using the second corpus to obtain a data classification model; and use the data classification model to classify unlabeled target data, obtaining classification results to clean the target data based on the classification results. The method of this invention can be used for data denoising, solves the problem of low statistical generalization ability with small samples, and makes application tasks more diversified; it lowers the application threshold of pre-trained models by using pre-trained models from a pre-trained model library, saving hardware costs; and it can promote the development and application of NLP.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a data processing method, apparatus, and storage medium based on cue-based learning. Background Technology

[0002] Data denoising is a crucial aspect of artificial intelligence. The presence of noisy data not only increases the amount of data but also the computational load, requiring more computer memory and incurring higher computational costs, and increasing computational errors. This is especially true for linear algorithms, which rely on iteration to find the optimal solution. If the data contains a large amount of noise, it will significantly impact the convergence speed and even negatively affect the accuracy of training generative models. Summary of the Invention

[0003] The problem this invention aims to solve includes how to denoise unlabeled data and filter out the required data. However, due to the diversity of unlabeled data, there is no practical and usable technical solution for automatic data denoising in the prior art.

[0004] To address the aforementioned technical challenges, such as how to denoise unlabeled data and filter out the required data, this invention is proposed. Embodiments of this invention provide a data processing method, apparatus, and storage medium based on cue-based learning.

[0005] According to another aspect of the present invention, a data processing method based on cue-based learning is provided, characterized in that the method includes:

[0006] Samples are extracted from the unannotated first corpus to obtain sample corpus;

[0007] Each sample in the sample corpus is labeled to obtain a labeled second corpus;

[0008] The second corpus was used to train a model based on prompting learning and ensemble learning to obtain a data classification model;

[0009] The unlabeled target data is classified using the data classification model to obtain classification results, and the target data is cleaned based on the classification results.

[0010] Optionally, the sample extraction from the unannotated first corpus to obtain sample corpus includes:

[0011] Obtain the first unlabeled corpus, and cluster the first corpus using a clustering algorithm to obtain the clustering results;

[0012] The clustering results are stratified and sampled according to a preset sampling ratio to obtain sample corpus.

[0013] Optionally, the training of a model based on cue-based learning and ensemble learning using the second corpus to obtain a data classification model includes:

[0014] Obtain multiple pre-trained models and multiple prompt templates;

[0015] The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data;

[0016] Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model.

[0017] Optionally, the method further includes:

[0018] Select multiple pre-trained models from the pre-trained model library and multiple prompt templates from the template library.

[0019] Optionally, the process of converting the second corpus according to the data format required by each prompt template to obtain training data includes:

[0020] For the input text x, use the function f corresponding to the prompt template. prompt (x), transform x into the prompt form x`, that is: x` = f prompt (x), where the operation of the function includes: using a template, which is a piece of natural language and contains two empty positions: the position [x] for filling the input x and the position [z] for generating the answer text z; and filling the input x into the position [x].

[0021] Optionally, the method further includes:

[0022] With the pre-trained model frozen, the content within the preset identifier in the training data is masked.

[0023] Optionally, the process of training a model based on the training data using cue-based learning and ensemble learning to obtain the data classification model includes:

[0024] The pre-trained prompt model is used to predict the masked content, and a strong classifier is constructed using the bagging method, with multiple prompt models serving as weak classifiers, to obtain the data classification model.

[0025] According to another aspect of the present invention, a storage medium is also provided, the storage medium including a stored program, wherein the methods described above are executed by a processor when the program is running.

[0026] According to another aspect of the present invention, a data processing apparatus based on cue-based learning is provided, the apparatus comprising:

[0027] The sample corpus acquisition module is used to extract samples from the unannotated first corpus in order to obtain sample corpus.

[0028] The annotation module is used to annotate each sample in the sample corpus to obtain an annotated second corpus;

[0029] The data classification model acquisition module is used to train a model based on prompting learning and ensemble learning using the second corpus in order to obtain a data classification model.

[0030] The data classification module is used to classify unlabeled target data using the data classification model, obtain classification results, and clean the target data based on the classification results.

[0031] According to another aspect of the present invention, a data processing apparatus based on cue-based learning is provided, the apparatus comprising:

[0032] Processor; and

[0033] A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps:

[0034] Samples are extracted from the unannotated first corpus to obtain sample corpus;

[0035] Each sample in the sample corpus is labeled to obtain a labeled second corpus;

[0036] The second corpus was used to train a model based on prompting learning and ensemble learning to obtain a data classification model;

[0037] The unlabeled target data is classified using the data classification model to obtain classification results, and the target data is cleaned based on the classification results.

[0038] This invention provides a data processing method, apparatus, and storage medium based on cue-based learning. It extracts samples from an unlabeled first corpus to obtain sample corpus; labels each sample in the sample corpus to obtain labeled second corpus; trains a model based on cue-based learning and ensemble learning using the second corpus to obtain a data classification model; and uses the data classification model to classify unlabeled target data, obtaining classification results to clean the target data based on the classification results. This method can be used for data denoising, solves the problem of low statistical generalization ability with small samples, and makes application tasks more diversified; it lowers the application threshold of pre-trained models by using pre-trained models from a pre-trained model library, saving hardware costs; and it can promote the development and application of NLP.

[0039] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0040] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0041] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing the method described in Embodiment 1 of the present invention;

[0042] Figure 2 This is a flowchart of the data processing method 200 based on cue learning according to the first aspect of Embodiment 1 of the present invention;

[0043] Figure 3 This is an overall architecture diagram of data processing according to Embodiment 1 of the present invention;

[0044] Figure 4 This is a schematic diagram of the structure of the data processing device 400 based on prompting learning according to Embodiment 2 of the present invention;

[0045] Figure 5 This is a schematic diagram of the structure of the data processing device 500 based on prompting learning according to Embodiment 3 of the present invention. Detailed Implementation

[0046] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0047] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0048] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0049] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0050] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0051] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0052] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0053] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0054] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0055] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0056] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0057] The embodiments of this invention can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Well-known examples of terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0058] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0059] In recent years, NLP technology has developed rapidly, especially with the emergence of BERT, which has ushered in a new round of development in the field. Since BERT, fine-tuning of pre-trained models has become a standard practice in the entire field. However, starting with GPT-3, a new paradigm has begun to attract attention and become increasingly popular: prompting. Prompt-tuning leverages large models purely through prompts. Prompt-tuning defines its own prompt for each task, concatenates it to the data as input, and simultaneously freezes the pre-trained model for training. Without adding extra layers, it can be seen that the performance improves as the model size increases, eventually catching up with the performance of fine-tuning.

[0060] The prompt function is highly sensitive to noise and confused data. This characteristic can be leveraged to detect noise in the data and iteratively optimize it. The most direct example is the data predicted by the prompt method that is not in the label set. Since the model's prediction performance reflects the training samples, the occurrence of predictions not in the label set indicates class confusion in a certain category of the training samples. Therefore, after predicting all training samples, identifying these types of samples allows us to determine what adjustments need to be made. Besides detecting class confusion, we can also use the error between the true and predicted labels of the training samples to find indistinguishable marginal samples.

[0061] Example 1

[0062] According to this embodiment, a method embodiment of a data processing method based on prompting learning is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0063] The method embodiments provided in this example can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a cue-based learning data processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, processing devices such as GPUs, microprocessors (MCUs), or programmable logic devices (FPGAs),), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports in the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0064] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of the present invention, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0065] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data processing method based on prompting learning in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned data processing method based on prompting learning of the application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0066] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0067] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0068] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1This is only one instance of a specific particular instance and is intended to illustrate the types of components that may exist in the aforementioned computer device (or mobile device).

[0069] In the above operating environment, according to the first aspect of this embodiment, a data processing method based on prompting learning is provided. Figure 2 A flowchart illustrating the method is shown below. (Refer to...) Figure 2 As shown, the method includes:

[0070] Step 201: Samples are extracted from the unannotated first corpus to obtain sample corpus.

[0071] Optionally, the sample extraction from the unannotated first corpus to obtain sample corpus includes:

[0072] Obtain the first unlabeled corpus, and cluster the first corpus using a clustering algorithm to obtain the clustering results;

[0073] The clustering results are stratified and sampled according to a preset sampling ratio to obtain sample corpus.

[0074] In embodiments of the present invention, the first step is to acquire a large amount of non-labeled corpus (i.e., unlabeled first corpus). Then, a clustering algorithm is used to cluster all the corpus to obtain the clustering results. Next, stratified sampling is performed on the clustering results, and a small amount of data is extracted according to a set sampling ratio to obtain sample corpus. The number of sample corpus samples is much smaller than the number of first corpus samples.

[0075] Step 202: Label each sample in the sample corpus to obtain a labeled second corpus.

[0076] In an embodiment of the present invention, after obtaining the sample corpus, each sample in each sample corpus is labeled with positive and negative examples, and the labeled corpus is used as the test corpus for model training (i.e., the labeled second corpus).

[0077] For example, suppose we have a non-standard corpus D with m samples. After clustering, we get k categories. Then, we perform stratified sampling on these k categories to get n samples. n is much smaller than m. The sample corpus set is defined as X, denoted as X = {x1, x2, ..., xn}. We then label X, marking those that meet the requirements as positive examples and those that do not as negative examples, resulting in a labeled Y = {y1, y2, y3, ..., yn}.

[0078] Step 203: Use the second corpus to train a model based on prompting learning and ensemble learning to obtain a data classification model.

[0079] Optionally, the training of a model based on cue-based learning and ensemble learning using the second corpus to obtain a data classification model includes:

[0080] Obtain multiple pre-trained models and multiple prompt templates;

[0081] The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data;

[0082] Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model.

[0083] Optionally, the method further includes:

[0084] Select multiple pre-trained models from the pre-trained model library and multiple prompt templates from the template library.

[0085] Optionally, the process of converting the second corpus according to the data format required by each prompt template to obtain training data includes:

[0086] For the input text x, use the function f corresponding to the prompt template. prompt (x), transform x into the prompt form x`, that is: x` = f prompt (x), where the operation of the function includes: using a template, which is a piece of natural language and contains two empty positions: the position [x] for filling the input x and the position [z] for generating the answer text z; and filling the input x into the position [x].

[0087] Optionally, the method further includes:

[0088] With the pre-trained model frozen, the content within the preset identifier in the training data is masked.

[0089] Optionally, the process of training a model based on the training data using cue-based learning and ensemble learning to obtain the data classification model includes:

[0090] The pre-trained prompt model is used to predict the masked content, and a strong classifier is constructed using the bagging method, with multiple prompt models serving as weak classifiers, to obtain the data classification model.

[0091] Prompting: A prompt is a piece of text or a set of vectors added to the input, allowing the model to perform masked language modeling (MLM) based on the input and the added prompt. In traditional supervised learning systems for NLP tasks, the input x is usually text data, and the output y is predicted based on the model P(y|x;θ). However, a major problem with supervised learning is that training the model P(y|x;θ) requires supervised data for the training task, but finding suitable data is difficult. In NLP, prompt-based learning methods attempt to circumvent this problem by learning an LM that models the probability P(x;θ) of the text x itself and uses that probability to predict y, thereby reducing or eliminating the need for large supervised datasets for training the model. Prompt-tuning methods can be used for text classification tasks by constructing a label vocabulary, transforming the classification problem into a problem of predicting words related to the labels. Therefore, such a problem can be solved by constructing a template containing [MASK], and then having the MLM (masked language model) predict the word at the [MASK] position. Thus, the classification task was transformed into a mask language modeling problem.

[0092] Bagging (bootstrap aggregating) is a technique that reduces generalization error by combining several models. The main idea is to train several different models separately, and then let all models vote on the output of the test examples.

[0093] like Figure 3 As shown, in the embodiments of the present invention, a large amount of non-standard corpus is acquired using the data acquisition and clustering module, and the entire corpus is clustered using a clustering algorithm; the clustering results are stratified and sampled using the data sampling and labeling module, and a small amount of data is extracted for labeling positive and negative examples. The labeled data is used as the test corpus for model training; the Prompt+Bagging model training module is used to complete the training of the model based on Prompt+Bagging. This scheme combines the principles of prompting learning and ensemble learning algorithms to obtain a strong classifier through training on a small scale of data; after obtaining the strong classifier based on Prompt and Bagging, the non-standard data prediction module is used to perform classification prediction on the entire corpus of data. The positive examples in the classification are the cleaned data, thereby achieving data cleaning.

[0094] The Prompt+Bagging model training module contains seven smaller functional units:

[0095] Model Manager: Used to implement functions such as adding, deleting, and selecting models;

[0096] Pre-trained model library: used to store pre-trained models;

[0097] Template Designer: Used to design new templates for specific tasks;

[0098] Template Manager: Used for adding, deleting, and combining templates;

[0099] Template library: Used to store pre-designed templates;

[0100] Data converter: Used to transform data corpora into the data format of selected templates;

[0101] Model trainer: Used to freeze the pre-trained model parameters, train on the transformed data, and obtain the optimal prompt model.

[0102] In an embodiment of the present invention, the specific steps for model training include:

[0103] S1. Based on the labeled second corpus, select multiple pre-trained models. These models can be publicly available pre-trained models, pre-trained models accumulated during the task, or pre-trained models recently trained using relevant domain datasets. For example, pre-trained models could be ALBert, Roberta, TinyBert, etc.

[0104] S2: Select multiple prompt templates. The selected templates and models are paired to provide prompts for learning weak classifiers. All classifiers are then used in a bagging process to construct a strong classifier.

[0105] S3. The second corpus is format-converted according to the data format required by each prompt template to obtain training data.

[0106] This involves processing the text into a template data format. For the input text x, there is a function f. prompt (x), transforming x into the prompt form x`, that is:

[0107] x`=f prompt (x),

[0108] This function typically performs two steps: 1. Use a template, which is usually a piece of natural language and contains two empty positions: [x] for filling in the input x and [z] for generating the answer text z; 2. Fill the input x into the position [x].

[0109] S4. Perform model training. During model training, freeze the pre-trained model, perform strong classifier training based on prompt and Bagging, obtain the optimal model, and acquire the data classification model.

[0110] Continuing with the example above, suppose a positive sample x1 corresponds to label y1, and a negative sample x2 corresponds to label y2. The model training process includes:

[0111] 1. Select a pre-trained model. This involves selecting multiple pre-trained models from the model library.

[0112] 2. Select a template or design a prompt template.

[0113] Template 1: [x1]{belongs to}[y1];

[0114] Template 2: [x1]{is a type of}[y1];

[0115] Template 3: [x1]{is contained in}[y1];

[0116] Template 4: [x2]{not belonging to}[y2];

[0117] Template 5: [x2]{is not a type of}[y2];

[0118] Template 6: [x2] is not contained within [y2];

[0119] 3. Perform data processing according to the template. X is processed into Z, and the number of elements in Z is the number of elements in X multiplied by the number of templates selected.

[0120] 4. Conduct model training.

[0121] With the pre-trained model frozen, the parts filled with "{}" in the processed corpus are masked. Specifically, the data is processed according to the prompt template, and the "{}" parts are filled with "[MASK]". The trained prompt model is used to predict the "[MASK]" content, and a bagging method is used to obtain a strong classifier from multiple prompt models as weak classifiers.

[0122] Step 204: Use the data classification model to classify the unlabeled target data and obtain the classification results, so as to clean the target data according to the classification results.

[0123] In an embodiment of the present invention, after obtaining a strong classifier based on Prompt and Bagging, i.e. a data classification model, the data classification model is used to classify and predict unlabeled data to determine positive and negative data. By removing negative data, data denoising and cleaning can be achieved.

[0124] As mentioned in the background section, data denoising is a crucial aspect of artificial intelligence. It impacts many algorithms during model training, increasing not only the amount of data but also computational load, computer memory, and computational expenses, as well as computational errors. Linear algorithms, in particular, rely on iteration to find the optimal solution; if the data contains significant noise, it will greatly affect the convergence speed and even negatively impact the accuracy of the trained model.

[0125] To address the problems existing in the aforementioned background technology, this embodiment obtains sample corpus by extracting samples from an unlabeled first corpus; labels each sample in the sample corpus to obtain labeled second corpus; trains a strong classifier based on prompt and Bagging using the second corpus to obtain a data classification model; and classifies the unlabeled target data using the data classification model to obtain classification results, thereby cleaning the target data based on the classification results.

[0126] Therefore, the technical problems that can be solved in this way include: 1) Existing methods are costly, inefficient and difficult to manually annotate; 2) Existing statistical methods have low statistical generalization ability and also require manual labor; 3) The number of parameters in pre-trained models is huge; 4) The utilization efficiency of pre-trained models is low.

[0127] The method of this invention can solve the problem of low generalization ability in small sample statistics, making application tasks more diversified; by using pre-trained models in the pre-trained model library, the application threshold of pre-trained models is reduced, saving hardware costs; and it can promote the development and application of NLP.

[0128] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium 104 is provided. The storage medium 104 includes a stored program, wherein, when the program is executed, the method described in any of the above embodiments is performed by a processor.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0131] Example 2

[0132] Figure 4 A data processing apparatus 400 based on cue-based learning according to this embodiment is shown, which corresponds to the method described according to the first aspect of Embodiment 1. Reference Figure 4 As shown, the device 400 includes:

[0133] The sample corpus acquisition module 401 is used to extract samples from the unannotated first corpus in order to obtain sample corpus.

[0134] The annotation module 402 is used to annotate each sample in the sample corpus to obtain an annotated second corpus;

[0135] The data classification model acquisition module 403 is used to train a model based on prompting learning and ensemble learning using the second corpus in order to acquire a data classification model.

[0136] The data classification module 404 is used to classify unlabeled target data using the data classification model, obtain classification results, and clean the target data based on the classification results.

[0137] Optionally, the sample corpus acquisition module 401 extracts samples from the unannotated first corpus to obtain sample corpus, including:

[0138] Obtain the first unlabeled corpus, and cluster the first corpus using a clustering algorithm to obtain the clustering results;

[0139] The clustering results are stratified and sampled according to a preset sampling ratio to obtain sample corpus.

[0140] Optionally, the data classification model acquisition module 403 uses the second corpus to train a model based on prompting learning and ensemble learning to obtain a data classification model, including:

[0141] Obtain multiple pre-trained models and multiple prompt templates;

[0142] The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data;

[0143] Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model.

[0144] Optionally, the data classification model acquisition module 403 further includes:

[0145] Select multiple pre-trained models from the pre-trained model library and multiple prompt templates from the template library.

[0146] Optionally, the data classification model acquisition module 403 performs format conversion on the second corpus according to the data format required by each prompt template to obtain training data, including:

[0147] For the input text x, use the function f corresponding to the prompt template. prompt (x), transform x into the prompt form x`, that is: x` = f prompt (x), where the operation of the function includes: using a template, which is a piece of natural language and contains two empty positions: the position [x] for filling the input x and the position [z] for generating the answer text z; and filling the input x into the position [x].

[0148] Optionally, the data classification model acquisition module 403 further includes:

[0149] With the pre-trained model frozen, the content within the preset identifier in the training data is masked.

[0150] Optionally, the data classification model acquisition module 403 acquires the data classification model by performing model training based on cue-based learning and ensemble learning on the training data, including:

[0151] The pre-trained prompt model is used to predict the masked content, and a strong classifier is constructed using the bagging method, with multiple prompt models serving as weak classifiers, to obtain the data classification model.

[0152] Therefore, according to this embodiment, samples can be extracted from an unlabeled first corpus to obtain sample corpus; each sample in the sample corpus is labeled to obtain labeled second corpus; a model based on cue learning and ensemble learning is trained using the second corpus to obtain a data classification model; the data classification model is used to classify the unlabeled target data to obtain classification results, and the target data is cleaned based on the classification results. The method of this invention can be used for data denoising, solves the problem of low statistical generalization ability for small samples, and makes application tasks more diversified; by using pre-trained models in a pre-trained model library, the application threshold of pre-trained models is lowered, saving hardware costs; and it can promote the development and application of NLP.

[0153] Example 3

[0154] Figure 5 A data processing apparatus 500 based on cue-based learning according to this embodiment is shown, which corresponds to the method described according to the first aspect of Embodiment 1. Reference Figure 5 As shown, the device 500 includes: a processor 510; and a memory 520 connected to the processor 510, for providing the processor 510 with instructions to process the following steps: extracting samples from an unlabeled first corpus to obtain sample corpus; labeling each sample in the sample corpus to obtain labeled second corpus; training a model based on cue learning and ensemble learning using the second corpus to obtain a data classification model; classifying unlabeled target data using the data classification model to obtain classification results, so as to clean the target data according to the classification results.

[0155] Optionally, the sample extraction from the unannotated first corpus to obtain sample corpus includes:

[0156] Obtain the first unlabeled corpus, and cluster the first corpus using a clustering algorithm to obtain the clustering results;

[0157] The clustering results are stratified and sampled according to a preset sampling ratio to obtain sample corpus.

[0158] Optionally, the training of a model based on cue-based learning and ensemble learning using the second corpus to obtain a data classification model includes:

[0159] Obtain multiple pre-trained models and multiple prompt templates;

[0160] The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data;

[0161] Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model.

[0162] Optionally, the method further includes:

[0163] Select multiple pre-trained models from the pre-trained model library and multiple prompt templates from the template library.

[0164] Optionally, the process of converting the second corpus according to the data format required by each prompt template to obtain training data includes:

[0165] For the input text x, use the function f corresponding to the prompt template. prompt (x), transform x into the prompt form x`, that is: x` = f prompt (x), where the operation of the function includes: using a template, which is a piece of natural language and contains two empty positions: the position [x] for filling the input x and the position [z] for generating the answer text z; and filling the input x into the position [x].

[0166] Optionally, the method further includes:

[0167] With the pre-trained model frozen, the content within the preset identifier in the training data is masked.

[0168] Optionally, the process of training a model based on the training data using cue-based learning and ensemble learning to obtain the data classification model includes:

[0169] The pre-trained prompt model is used to predict the masked content, and a strong classifier is constructed using the bagging method, with multiple prompt models serving as weak classifiers, to obtain the data classification model.

[0170] Therefore, according to this embodiment, samples can be extracted from an unlabeled first corpus to obtain sample corpus; each sample in the sample corpus is labeled to obtain labeled second corpus; a model based on cue learning and ensemble learning is trained using the second corpus to obtain a data classification model; the data classification model is used to classify the unlabeled target data to obtain classification results, and the target data is cleaned based on the classification results. The method of this invention can be used for data denoising, solves the problem of low statistical generalization ability for small samples, and makes application tasks more diversified; by using pre-trained models in a pre-trained model library, the application threshold of pre-trained models is lowered, saving hardware costs; and it can promote the development and application of NLP.

[0171] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0172] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0174] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0175] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0176] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0177] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data processing method based on cue-based learning, characterized in that, The method includes: Samples are extracted from the unannotated first corpus to obtain sample corpus; Each sample in the sample corpus is labeled to obtain a labeled second corpus; The second corpus was used to train a model based on prompting learning and ensemble learning to obtain a data classification model; The data classification model is used to classify unlabeled target data and obtain classification results, so as to clean the target data based on the classification results; The step of training a model based on prompting learning and ensemble learning using the second corpus to obtain a data classification model includes: Obtain multiple pre-trained models and multiple prompt templates; The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data; Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model; The method further includes: With the pre-trained model frozen, the content within the preset identifier in the training data is masked. The step of training a model based on the training data using cue-based learning and ensemble learning to obtain the data classification model includes: Multiple prompt models are trained by predicting masked content using a pre-trained model. Then, a strong classifier is constructed using the bagging method, based on the multiple prompt models as weak classifiers, to obtain the data classification model.

2. The method according to claim 1, characterized in that, The sample corpus is obtained by extracting samples from the unannotated first corpus, including: Obtain the first unlabeled corpus, and cluster the first corpus using a clustering algorithm to obtain the clustering results; The clustering results are stratified and sampled according to a preset sampling ratio to obtain sample corpus.

3. The method according to claim 1, characterized in that, The process of obtaining multiple pre-trained models and multiple prompt templates includes: Select multiple pre-trained models from the pre-trained model library and multiple prompt templates from the template library.

4. The method according to claim 1, characterized in that, The step of converting the second corpus according to the data format required by each prompt template to obtain training data includes: For the input text x, use the function corresponding to the prompt template. Transform x into the prompt form x`, that is: The operation of this function includes: using a template, which is a piece of natural language and contains two empty positions: the position [x] for filling the input x and the position [z] for generating the answer text z; and filling the input x into the position [x].

5. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the method described in any one of claims 1 to 4 is performed by a processor.

6. A data processing device based on cue-based learning, characterized in that, The device includes: The sample corpus acquisition module is used to extract samples from the unannotated first corpus in order to obtain sample corpus. The annotation module is used to annotate each sample in the sample corpus to obtain an annotated second corpus; The data classification model acquisition module is used to train a model based on prompting learning and ensemble learning using the second corpus in order to obtain a data classification model. The data classification module is used to classify unlabeled target data using the data classification model, obtain classification results, and clean the target data based on the classification results. The data classification model acquisition module utilizes the second corpus to train a model based on prompting learning and ensemble learning to obtain a data classification model, including: Obtain multiple pre-trained models and multiple prompt templates; The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data; Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model; The data classification model acquisition module further includes: With the pre-trained model frozen, the content within the preset identifier in the training data is masked. The data classification model acquisition module, based on the training data, performs model training based on cue-based learning and ensemble learning to acquire the data classification model, including: Multiple prompt models are trained by predicting masked content using a pre-trained model. Then, a strong classifier is constructed using the bagging method, based on the multiple prompt models as weak classifiers, to obtain the data classification model.

7. A data processing device based on cue-based learning, characterized in that, The device includes: Processor; and A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Samples are extracted from the unannotated first corpus to obtain sample corpus; Each sample in the sample corpus is labeled to obtain a labeled second corpus; The second corpus was used to train a model based on prompting learning and ensemble learning to obtain a data classification model; The data classification model is used to classify unlabeled target data and obtain classification results, so as to clean the target data based on the classification results; The step of training a model based on prompting learning and ensemble learning using the second corpus to obtain a data classification model includes: Obtain multiple pre-trained models and multiple prompt templates; The second corpus is format-converted according to the data format required by each prompt template in order to obtain training data; Freeze the pre-trained model, and train the model based on the training data using cue-based learning and ensemble learning to obtain the data classification model; The memory is also used to provide the processor with instructions to perform the following processing steps: With the pre-trained model frozen, the content within the preset identifier in the training data is masked. The step of training a model based on the training data using cue-based learning and ensemble learning to obtain the data classification model includes: Multiple prompt models are trained by predicting masked content using a pre-trained model. Then, a strong classifier is constructed using the bagging method, based on the multiple prompt models as weak classifiers, to obtain the data classification model.

Images