Semi-supervised document classification method and system based on generative model GPT enhancement

Abstract

The invention provides a semi-supervised document classification method and system based on generative model GPT enhancement, and relates to the field of statistical classification, and the method comprises the steps: obtaining an initial first document set, guiding a preset generative artificial intelligence model through a prompt sentence to generate an enhanced sample for the first document set, and obtaining a second document set; inputting the first document set and the second document set into a self-supervised learning model for training to obtain an initial classifier; and performing fine adjustment on the initial classifier through a few-sample learning method to obtain a target classifier. According to the method, the initial classifier is finely adjusted by using the few-sample learning method, few-sample learning is performed through the training set, so that the target classifier can predict new-class samples, the few-sample learning method is applied to document classification research, dependence on label documents is reduced, and the classification efficiency is improved. A new class data classification mode can be mined from existing basic class data, and an effective technical scheme is provided for solving the problem of sparse label data in enterprise documents.

Description

Technical Field

[0001] The present application relates to the technical field of statistical classification, and in particular to a semi-supervised document classification method and system based on generative model GPT enhancement. Background Art

[0002] Various types of documents provide rich information for the innovation management and strategic management of enterprises. Existing document classification methods in enterprises mainly rely on keyword matching technology and supervised machine learning methods. However, since these methods require a lot of efforts from experts and data annotators, it is still a challenge to classify large-scale professional technical documents. At present, the latest advances in generative artificial intelligence, such as GPT-4, have greatly improved the capabilities of natural language processing tasks. However, due to the large scale and high computational requirements of generative artificial intelligence, it is also challenging to directly use it for knowledge document classification. Summary of the invention

[0003] In response to the shortcomings of the existing technology, the present application provides a semi-supervised document classification method and system based on the generative model GPT enhancement, which solves the technical problems of low efficiency and high cost faced by enterprises in the process of classification and management of large-scale documents.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a semi-supervised document classification method based on a generative model GPT enhancement, the method comprising: obtaining an initial first document set, the first document set comprising a plurality of unlabeled document samples; setting a prompt sentence, guiding a preset generative artificial intelligence model through the prompt sentence to generate an enhanced sample for each document sample in the first document set, to obtain a second document set; inputting the first document set and the second document set into a preset self-supervised learning model for training, to obtain an initial classifier; obtaining a third document set comprising a plurality of labeled annotated samples, and dividing the third document set to obtain a training set and a test set; introducing a prototype network based on the training set, and fine-tuning the initial classifier through a few-sample learning method, to obtain a target classifier; testing the target classifier according to the test set to evaluate the performance of the target classifier; when the test result of the target classifier meets the preset indicator threshold, obtaining a fourth document set to be classified, and classifying the fourth document set through the target classifier.

[0006] According to the first aspect of the embodiment of the present application, the aforementioned prompt sentence may specifically include: slightly rewriting the document sample while keeping the semantics unchanged; performing synonymous replacement on the document sample while keeping the semantics unchanged; elaborating the document sample in one sentence; rearranging the structure of the document sample, changing the order, and keeping the semantics unchanged.

[0007] According to the first aspect of the embodiment of the present application, the aforementioned self-supervised learning model includes a document encoder. The aforementioned inputting the first document set and the second document set into the preset self-supervised learning model for training to obtain an initial classifier may specifically include the following steps: inputting the token sequence corresponding to the first document set and the second document set into the document encoder to generate sample hidden representation information; mapping the sample hidden representation information to the space for performing hierarchical supervised contrastive learning loss through a preset projection network, and discarding the projection network; wherein the sample hidden representation information includes multiple positive sample pairs and multiple negative sample pairs; based on the index information of the sample, performing self-supervised contrastive learning loss training on the sample hidden representation information to obtain an initial classifier.

[0008] According to the first aspect of the embodiment of the present application, the aforementioned document encoder is a RoBERT-large pre-trained language model, which includes multiple bidirectional Transformer encoders.

[0009] According to the first aspect of the embodiment of the present application, the aforementioned Token sequence satisfies the expression: S = {[CLS], w1, w2, ..., w n ,[SEP]}; where S represents the Token sequence, [CLS] represents the unique marker at the beginning, [SEP] is the unique marker at the end, and w i Represents the i-th character token in the sequence, 1≤i≤n.

[0010] According to the first aspect of the embodiment of the present application, the aforementioned fine-tuning of the initial classifier by a few-sample learning method based on the training set to obtain a target classifier can specifically include the following steps: in each iteration of the scenario training task, N classes are randomly selected from the training set, and each class provides K samples to construct a support set; wherein the prototype of each class belongs to the average value of the support sample vector representation of the class; a query set containing N classes is constructed, and each class provides Q samples as a query set; for each query sample, the Euclidean distance between its vector representation and the prototype of each class is calculated; the obtained Euclidean distance is converted into the category probability distribution of the query sample through the softmax activation function to ensure that the sum of the probabilities is 1; the cross entropy loss function is used to quantify the difference between the predicted probability and the actual label; by minimizing the loss, the text encoder in the initial classifier is trained to a probability distribution that matches the true label to obtain the target classifier.

[0011] According to the first aspect of the embodiment of the present application, the aforementioned cross entropy loss function satisfies the expression:

[0012]

[0013] Where, L FSL represents the cross entropy loss, Represents the true label, if the sample The true category of is c, then Otherwise, it is 0, Q is the number of samples corresponding to each class in the query set, and N is the number of classes in the query set.

[0014] In a second aspect, an embodiment of the present application provides a semi-supervised document classification system based on a generative model GPT enhancement, the system comprising a first acquisition module, a generation module, a training module, a second acquisition module, a fine-tuning module, a testing module and a classification module; wherein the first acquisition module is used to acquire an initial first document set, the first document set comprising a plurality of unlabeled document samples; the generation module is used to set a prompt sentence, and guide a preset generative artificial intelligence model to generate an enhanced sample for each document sample in the first document set through the prompt sentence to obtain a second document set; the training module is used to input the first document set and the second document set into a preset self-supervised learning model for training to obtain an initial classifier; the second acquisition module is used to acquire a third document set comprising a plurality of labeled annotated samples, and divide the third document set to obtain a training set and a test set; the fine-tuning module is used to fine-tune the initial classifier based on the training set through a few-sample learning method to obtain a target classifier; the testing module is used to test the target classifier according to the test set to evaluate the performance of the target classifier; the classification module is used to acquire a fourth document set to be classified when the test result of the target classifier meets the preset indicator threshold, and classify the fourth document set through the target classifier.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the semi-supervised document classification method based on generative model GPT enhancement in the aforementioned first aspect is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the semi-supervised document classification method based on generative model GPT enhancement in the aforementioned first aspect is implemented.

[0017] This application provides a semi-supervised document classification method and system based on the generative model GPT enhancement. Compared with the prior art, it has the following beneficial effects:

[0018] This application applies a generative artificial intelligence model to document classification, and trains the target classifier in two stages and tests it. In the first training stage, self-supervised comparative learning is performed. First, by designing prompt sentences, the generative artificial intelligence is guided to generate enhanced samples for the large-scale unlabeled data corresponding to the first document set, and then self-supervised comparative learning is performed to optimize the feature representation of the document in the representation space, aiming to use large-scale unlabeled data to learn high-quality feature representations, providing a technical basis for in-depth mining of knowledge documents. In the second training stage, this application uses the initial classifier obtained by fine-tuning the few-sample learning method, introduces the prototype network, and Few-sample learning is performed on the annotated samples of a limited number of basic classes in the training set, so that the target classifier can predict new class samples, thereby improving the performance of document classification. This application applies the few-sample learning method to document classification research, reduces the dependence on labeled documents, and can mine new class data classification patterns from the existing basic class data; after obtaining the target classifier, the performance of the target classifier is evaluated by a test set containing labeled annotated samples. After passing the evaluation, the fourth document set to be classified is formally batch processed by the target classifier. This application provides an effective technical solution to solve the problem of sparse labeled data in corporate documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flowchart of a semi-supervised document classification method based on the generative model GPT enhancement provided in an embodiment of the present application;

[0021] Figure 2 yes Figure 1 An exemplary flow chart of S130;

[0022] Figure 3 yes Figure 1 An exemplary flow chart of S150;

[0023] Figure 4 It is a flowchart of another semi-supervised document classification method based on the generative model GPT enhancement provided in an embodiment of the present application;

[0024] Figure 5 It is a structural diagram of a semi-supervised document classification system based on the generative model GPT enhancement provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0028] The embodiments of the present application provide a semi-supervised document classification method and system based on the generative model GPT enhancement, which solves the problems of low efficiency and high cost faced by enterprises in the process of classifying and managing large-scale documents, and realizes efficient classification management of large-scale documents.

[0029] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0030] Various documents provide rich information for the innovation management and strategic management of enterprises. Among the related technologies, enterprise document classification methods mainly rely on keyword matching technology and supervised machine learning methods. However, since these methods require a lot of efforts from experts and data annotators, it is still a challenge to classify large-scale professional technical documents.

[0031] At present, the latest advances in generative AI, such as GPT-4, have greatly improved the capabilities of natural language processing tasks. However, due to the large scale and high computational requirements of generative AI, it is also challenging to directly use it for knowledge document classification.

[0032] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] The following first introduces a semi-supervised document classification method based on the generative model GPT enhancement provided in an embodiment of the present application.

[0034] The present application provides a flow chart of a semi-supervised document classification method based on the generative model GPT enhancement, please refer to Figure 1 and Figure 4 The method may include the following steps S110-S170.

[0035] S110: Obtain an initial first document set, where the first document set includes a plurality of document samples without labels.

[0036] S120, setting a prompt sentence, and guiding the preset generative artificial intelligence model to generate an enhanced sample for each document sample in the first document set through the prompt sentence, so as to obtain a second document set. It can be understood that the second document set corresponds to the first document set, the document samples in the first document set are anchor document samples, and the enhanced samples in the second document set are positive document samples.

[0037] S130 , inputting the first document set and the second document set into a preset self-supervised learning model for training to obtain an initial classifier.

[0038] S140: Obtain a third document set including a plurality of labeled samples, and divide the third document set into a training set and a test set.

[0039] S150. Based on the training set, a prototype network is introduced, and the initial classifier is fine-tuned through a few-sample learning method to obtain a target classifier.

[0040] S160. Test the target classifier according to the test set to evaluate the performance of the target classifier.

[0041] S170 . When the test result of the target classifier meets the preset indicator threshold, obtain a fourth document set to be classified, and classify the fourth document set through the target classifier.

[0042] The above is a specific implementation method of the semi-supervised document classification method based on the generative model GPT enhancement provided in the embodiment of the present application. The present application applies the generative artificial intelligence model to document classification, and trains the target classifier in two stages and performs testing; in the first training stage, self-supervised comparative learning is performed. First, by designing prompt sentences, the generative artificial intelligence is guided to generate enhanced samples for the large-scale unlabeled data corresponding to the first document set, and then self-supervised comparative learning is performed to optimize the feature representation of the document in the representation space, aiming to use large-scale unlabeled data to learn high-quality feature representations, providing a technical basis for in-depth mining of knowledge documents.

[0043] Furthermore, in the second training stage, the present application uses the initial classifier obtained by fine-tuning the few-sample learning method, introduces the prototype network, and performs few-sample learning through annotated samples of a limited number of basic classes in the training set, so that the target classifier can predict new class samples, thereby improving the performance of document classification.

[0044] It should be noted that the present application applies the few-sample learning method to document classification research, reduces the dependence on labeled documents, and can mine new class data classification patterns from existing basic class data; after obtaining the target classifier, the performance of the target classifier is evaluated by a test set containing labeled annotated samples, and after passing the evaluation, the fourth document set to be classified is formally batch processed by the target classifier. The present application provides an effective technical solution to the problem of sparse labeled data in corporate documents.

[0045] In some embodiments, the present application uses simple and easy-to-understand prompt sentences, that is, using a few sentences to explain in detail the document sample to be converted, to guide the generative artificial intelligence model to generate enhanced samples. In one example, the aforementioned prompt sentence can specifically include: slightly rewriting the document sample while keeping the semantics unchanged; performing synonymous replacement on the document sample while keeping the semantics unchanged; elaborating the document sample in detail in one sentence; rearranging the structure of the document sample, changing the order, and keeping the semantics unchanged.

[0046] In some embodiments, please refer to Figure 2 The self-supervised learning model includes a document encoder. The first document set and the second document set are input into a preset self-supervised learning model for training to obtain an initial classifier, that is, the aforementioned S130 may specifically include the following steps:

[0047] S210: Input the Token sequences corresponding to the first document set and the second document set into the document encoder to generate sample hidden representation information.

[0048] S220, mapping the sample hidden representation information to a space for performing hierarchical supervised contrastive learning loss through a preset projection network, and discarding the projection network; wherein the sample hidden representation information includes multiple positive sample pairs and multiple negative sample pairs.

[0049] S230, based on the index information of the sample, perform self-supervised contrastive learning loss training on the hidden representation information of the sample to obtain an initial classifier.

[0050] In the embodiment of the present application, it can be understood that the aforementioned first document set is a large-scale unlabeled data set D A ={x1,x2,...,x M}, using generative artificial intelligence g(·) for each document sample x i ∈D A Both generate positive samples with similar semantics but different forms That is, enhanced samples; formally, the input samples can be regarded as samples randomly drawn from the basic distribution, and the positive samples can be regarded as variants sampled from the conditional probability of the generative artificial intelligence model, denoted as:

[0051]

[0052] It should be noted that the invention introduces a learnable nonlinear transformation between the representation layer and the contrastive learning loss to significantly improve the quality of the learned representation. A tiny projection network Proj(·) is used in the training process to map the sequence representation to the space where the hierarchical supervised contrastive learning loss is performed. After that, the projection network Proj(·) is discarded and the document encoder and the sequence representation r are used. S Perform classification tasks.

[0053] It should also be noted that this application is based on self-supervised contrastive learning loss. For each sample x i , there are positive sample pairs And several other negative sample pairs, hidden representation of positive sample pairs This application uses a multi-layer perceptron with one hidden layer to obtain the representation vector:

[0054] z i =W (2) ReLU(W (1) h i )

[0055]

[0056] Among them, z i and is x i and The vector representation of represents the weight matrix and ReLU(·) is an activation function.

[0057] In a batch of documents, let i∈A≡{1,2,...,B} be the index of the anchor sample, which is the multiple original document samples in the first document set. In the self-supervised contrastive learning loss, the loss function can be specifically expressed as follows:

[0058]

[0059] where sim(·) is the cosine similarity function, j is all the indices in a batch except i, and τ represents the temperature hyperparameter in self-supervised contrastive learning.

[0060] The total loss is the average loss of all samples and satisfies the expression:

[0061]

[0062] In one example, the aforementioned document encoder is a RoBERT-large pre-trained language model, which includes multiple bidirectional Transformer encoders.

[0063] In the embodiment of the present application, it can be understood that the present application selects the RoBERT-large pre-trained language model f θ (·) As a document encoder, it dynamically generates embedding vectors in the semantic space. A major advantage of the RoBERT-large model is its powerful language representation and feature extraction capabilities. It pre-trains multiple bidirectional Transformer encoders through a "masked language model" to capture richer contextual information and improve the model's performance in natural language processing tasks.

[0064] In one example, the aforementioned Token sequence satisfies the expression: S = {[CLS], w1, w2, ..., w n ,[SEP]}; where S represents the Token sequence, [CLS] represents the unique marker at the beginning, [SEP] is the unique marker at the end, and w i Represents the i-th character token in the sequence, 1≤i≤n.

[0065] For each token, the document encoder generates a hidden representation:

[0066] H=f θ (x) = (r [CLS] ,r1,r2,...,r n ,r [SEP] )

[0067] in, d1 represents the dimension of Token embedding, [CLS] is used to represent the entire Token sequence, expressed as h = r [CLS] .

[0068] In some embodiments, please refer to Figure 3 , the above-mentioned fine-tuning of the initial classifier based on the training set by a few-sample learning method to obtain a target classifier, that is, the above-mentioned S150 may specifically include the following steps:

[0069] S310. In each scenario training task iteration, N classes are randomly selected from the training set, and each class provides K samples to construct a support set; wherein the prototype of each class belongs to the average value of the support sample vector representation of that class.

[0070] S320: Build a query set containing N classes, each class providing Q samples.

[0071] S330: For each query sample, calculate the Euclidean distance between its vector representation and each class prototype.

[0072] S340, converting the obtained Euclidean distance into the category probability distribution of the query sample through a softmax activation function to ensure that the sum of the probabilities is 1.

[0073] S350. Use the cross entropy loss function to quantify the difference between the predicted probability and the actual label; by minimizing the loss, train the text encoder in the initial classifier into a probability distribution that matches the true label to obtain the target classifier.

[0074] In the embodiments of the present application, it can be understood that the purpose of fine-tuning the initial classifier by the few-sample learning method is to use limited labeled samples of basic classes to fine-tune the pre-trained document encoder so that it performs better in the classification task.

[0075] This stage uses the prior knowledge learned from the initial pre-training to enable it to understand and generalize document data. This patent uses a prototype network, one of the few-shot learning strategies, to fine-tune the document encoder. The prototype network is very effective in few-shot learning scenarios because it is able to create a prototype for each category based on a small number of labeled samples. By using these prototypes, the network can compare new samples with the closest category prototypes in the embedding space, thereby efficiently classifying new samples. This fine-tuning stage ensures that the document encoder not only retains the rich prior knowledge obtained in the pre-training stage, but also quickly adapts to the specific characteristics of the new classification task, thereby improving overall performance and robustness.

[0076] It should be noted that in the process of fine-tuning the initial classifier through the few-shot learning method, this application constructs an N-way K-shot scenario training task; given a labeled sample of a basic class with a label, the key in the few-shot learning setting is to construct an N-way K-shot training task. In each iteration of the scenario training task, N classes are first randomly selected from the training data set, and each class provides K samples to construct a support set in, Support for the sample, Represents its corresponding label. For each category, its prototype is the average value of the support sample vector representation belonging to that category:

[0077]

[0078] in, represents the i-th support sample belonging to the c-th class.

[0079] This application needs to build a query set containing N classes, each class provides Q samples as a query set in, represents a query sample, represents its corresponding label, and For each query sample Compute the Euclidean distance between its vector representation and each class prototype:

[0080]

[0081] This application uses the softmax activation function to convert the obtained distance into the category probability distribution of the query sample to ensure that the sum of the probabilities is 1. The calculation formula is as follows:

[0082]

[0083] in, Represents a query sample The probability distribution on category c, the training goal of this stage is to minimize the loss between the predicted probability distribution of the query sample and its true label. This application uses the cross entropy loss function to quantify the difference between the predicted probability and the actual label. By minimizing this loss, the document encoder will be trained to generate a probability distribution that closely matches the true label, thereby improving classification accuracy.

[0084] It should be noted that during the testing phase, this application will randomly generate multiple scenario tasks from the test set, which are designed to mimic the situations encountered by the model during the training phase; these scenario tasks generated from the target dataset are essential for evaluating the model's ability to generalize to new categories. For each scenario task, N prototypes are calculated, each of which provides K samples, and the categories of Q unlabeled samples are predicted.

[0085] In one example, the cross entropy loss function satisfies the expression:

[0086] Where, L FSL represents the cross entropy loss, Represents the true label, if the sample The true category of is c, then Otherwise, it is 0, Q is the number of samples corresponding to each class in the query set, and N is the number of classes in the query set.

[0087] In some embodiments, the present application provides a semi-supervised document classification system 400 based on the generative model GPT enhancement, such as Figure 5 As shown, the system 400 may include the following modules:

[0088] A first acquisition module 410 is used to acquire an initial first document set, where the first document set includes a plurality of document samples without labels;

[0089] A generation module 420 is used to set a prompt sentence, and guide a preset generative artificial intelligence model to generate an enhanced sample for each document sample in the first document set through the prompt sentence to obtain a second document set;

[0090] A training module 430 is used to input the first document set and the second document set into a preset self-supervised learning model for training to obtain an initial classifier;

[0091] A second acquisition module 440 is used to acquire a third document set including a plurality of labeled samples, and divide the third document set into a training set and a test set;

[0092] A fine-tuning module 450 is used to fine-tune the initial classifier based on the training set by a few-shot learning method to obtain a target classifier;

[0093] A testing module 460 is used to test the target classifier according to the test set to evaluate the performance of the target classifier;

[0094] The classification module 470 is used to obtain a fourth document set to be classified and classify the fourth document set through the target classifier when the test result of the target classifier meets the preset indicator threshold.

[0095] According to an embodiment of the present application, any multiple modules among the first acquisition module 410, the generation module 420, the training module 430, the second acquisition module 440, the fine-tuning module 450, the test module 460 and the classification module 470 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0096] In some embodiments, the aforementioned training module may be specifically used for:

[0097] Input the token sequences corresponding to the first document set and the second document set into the document encoder to generate sample hidden representation information;

[0098] The sample hidden representation information is mapped to a space for performing hierarchical supervised contrastive learning loss through a preset projection network, and the projection network is discarded; wherein the sample hidden representation information includes multiple positive sample pairs and multiple negative sample pairs;

[0099] Based on the index information of the sample, the hidden representation information of the sample is trained through self-supervised contrastive learning loss to obtain the initial classifier.

[0100] In some embodiments, the fine-tuning module 450 may be specifically used for:

[0101] In each scenario training task iteration, N classes are randomly selected from the training set, and each class provides K samples to construct a support set; where the prototype of each class belongs to the average value of the support sample vector representation of that class;

[0102] Construct a query set containing N classes, each class providing Q samples;

[0103] For each query sample, calculate the Euclidean distance between its vector representation and each class prototype;

[0104] The obtained Euclidean distance is converted into the category probability distribution of the query sample through the softmax activation function to ensure that the sum of the probabilities is 1;

[0105] The cross entropy loss function is used to quantify the difference between the predicted probability and the actual label; by minimizing the loss, the text encoder in the initial classifier is trained to a probability distribution that matches the true label to obtain the target classifier.

[0106] Figure 5 Each module in the system shown has the functions of implementing S110-S170 in the aforementioned semi-supervised document classification method enhanced by the generative model GPT, and can achieve its corresponding technical effects. For the sake of concise description, they will not be repeated here.

[0107] In some embodiments, the present application provides an electronic device, the structure diagram of the electronic device is as follows Figure 6 shown.

[0108] The electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0109] Specifically, the processor 510 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0110] The memory 520 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 520 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 520 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 520 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 520 is a non-volatile solid-state memory.

[0111] The memory 520 may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, typically, the memory 520 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the semi-supervised document classification methods based on the generative model GPT enhancement in the above embodiments.

[0112] The processor 510 implements any one of the semi-supervised document classification methods enhanced by the generative model GPT in the above embodiments by reading and executing the computer program instructions stored in the memory 520.

[0113] In one example, the electronic device may further include a communication interface 530 and a bus 500. Figure 6 As shown, the processor 510, the memory 520, and the communication interface 530 are connected via a bus 500 and communicate with each other.

[0114] The communication interface 530 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0115] Bus 500 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 500 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0116] In addition, in combination with the semi-supervised document classification method based on the generative model GPT enhancement in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the semi-supervised document classification methods based on the generative model GPT enhancement in the above embodiment is implemented.

[0117] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0118] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0119] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0120] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0121] In summary, compared with the prior art, this application has the following beneficial effects:

[0122] 1. This application applies a generative artificial intelligence model to document classification, trains the target classifier in two stages, and tests it; in the first training stage, self-supervised comparative learning is performed to optimize the feature representation of the document in the representation space by concentrating positive sample pairs and pushing away negative sample pairs.

[0123] 2. This application designs prompt sentences to guide generative artificial intelligence to generate enhanced samples for the large-scale unlabeled data corresponding to the first document set, and then conducts self-supervised comparative learning, using large-scale unlabeled data to learn high-quality feature representations, providing a technical foundation for in-depth mining of knowledge documents.

[0124] 3. In the second training stage, this application uses the initial classifier obtained by fine-tuning the few-shot learning method, introduces the prototype network, and performs few-shot learning through a limited number of labeled samples of basic classes in the training set, so that the target classifier can predict new class samples, thereby improving the performance of document classification. The few-shot learning method is applied to document classification research, which reduces the dependence on labeled documents and can mine new class data classification patterns from existing basic class data; it provides a solution to the problem of sparse labeled data in enterprise knowledge documents.

[0125] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-supervised document classification method based on generative model GPT enhancement, characterized in that: include: Acquire an initial first document set, wherein the first document set includes a plurality of document samples without labels; Setting a prompt sentence, and using the prompt sentence to guide a preset generative artificial intelligence model to generate an enhanced sample for each document sample in the first document set, to obtain a second document set; Inputting the first document set and the second document set into a preset self-supervised learning model for training to obtain an initial classifier; Obtaining a third document set including a plurality of labeled samples, and dividing the third document set into a training set and a test set; Introducing a prototype network based on the training set, fine-tuning the initial classifier by a few-shot learning method, and obtaining a target classifier; Testing the target classifier according to the test set to evaluate the performance of the target classifier; When the test result of the target classifier meets the preset indicator threshold, a fourth document set to be classified is obtained, and the fourth document set is classified by the target classifier.

2. The semi-supervised document classification method based on the generative model GPT enhancement as claimed in claim 1, characterized in that: The prompt sentence includes: Slightly rewrite the documentation sample, keeping the semantics intact; Perform synonymous replacement on document samples while keeping the semantics unchanged; Elaborate on the document sample in one sentence; Rearrange the structure of document samples, changing their order while keeping the semantics unchanged.

3. The semi-supervised document classification method based on the generative model GPT enhancement as claimed in claim 1, characterized in that: The self-supervised learning model includes a document encoder; The step of inputting the first document set and the second document set into a preset self-supervised learning model for training to obtain an initial classifier includes: Inputting the Token sequences corresponding to the first document set and the second document set into the document encoder to generate sample hidden representation information; Mapping the sample hidden representation information to a space that performs hierarchical supervised contrastive learning loss through a preset projection network, and discarding the projection network; wherein the sample hidden representation information includes a plurality of positive sample pairs and a plurality of negative sample pairs; Based on the index information of the sample, a self-supervised contrastive learning loss training is performed on the hidden representation information of the sample to obtain an initial classifier.

4. The semi-supervised document classification method based on the generative model GPT enhancement as claimed in claim 3, characterized in that: The document encoder is a RoBERT-large pre-trained language model, and the RoBERT-large pre-trained language model includes multiple bidirectional Transformer encoders.

5. The semi-supervised document classification method based on the generative model GPT enhancement as claimed in claim 3, characterized in that: The Token sequence satisfies the expression: S={[CLS],w1,w2,...,w n ,[SEP]} In the formula, S represents the token sequence, [CLS] represents the unique token of the beginning, [SEP] represents the unique token of the end, and w i Represents the i-th character token in the sequence, 1≤i≤n.

6. The semi-supervised document classification method based on the generative model GPT enhancement according to any one of claims 1 to 5, characterized in that: The method of fine-tuning the initial classifier based on the training set by a few-sample learning method to obtain a target classifier includes: In each scenario training task iteration, N classes are randomly selected from the training set, and each class provides K samples to construct a support set; wherein the prototype of each class belongs to the average value of the support sample vector representation of the class; Construct a query set containing N classes, each class providing Q samples; For each query sample, calculate the Euclidean distance between its vector representation and each class prototype; The obtained Euclidean distance is converted into a category probability distribution of the query sample by a softmax activation function to ensure that the sum of the probabilities is 1; A cross entropy loss function is used to quantify the difference between the predicted probability and the actual label; by minimizing the loss, the text encoder in the initial classifier is trained to a probability distribution matching the true label to obtain a target classifier.

7. The semi-supervised document classification method based on the generative model GPT enhancement as claimed in claim 6, characterized in that: The cross entropy loss function satisfies the expression: Where, L FSL represents the cross entropy loss, Represents the true label, if the sample The true category of is c, then Otherwise it is 0, Q is the number of samples corresponding to each class in the query set, and N is the number of classes in the query set.

8. A semi-supervised document classification system based on generative model GPT enhancement, characterized in that: include: A first acquisition module, used to acquire an initial first document set, wherein the first document set includes a plurality of document samples without labels; A generation module, used to set a prompt sentence, and guide a preset generative artificial intelligence model to generate an enhanced sample for each document sample in the first document set through the prompt sentence to obtain a second document set; A training module, used for inputting the first document set and the second document set into a preset self-supervised learning model for training to obtain an initial classifier; A second acquisition module is used to acquire a third document set including a plurality of labeled samples, and divide the third document set into a training set and a test set; A fine-tuning module, used for fine-tuning the initial classifier based on the training set by a few-sample learning method to obtain a target classifier; A testing module, used for testing the target classifier according to the test set to evaluate the performance of the target classifier; The classification module is used to obtain a fourth document set to be classified and classify the fourth document set through the target classifier when the test result of the target classifier meets a preset indicator threshold.

9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the semi-supervised document classification method based on generative model GPT enhancement as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the semi-supervised document classification method based on the generative model GPT enhancement as described in any one of claims 1 to 7 is implemented.

Images