Text classification method based on bidirectional long short-term memory network prompt learning

Abstract

The invention belongs to the field of deep learning and machine vision, and discloses a method for prompting learning text classification based on a bidirectional long short-term memory network, and the specific technical scheme is as follows: constructing multi-category text data and performing data cleaning processing; constructing a batch sample sequence, and calculating a word frequency-inverse frequency value to represent a text; the method comprises the following steps: establishing a bidirectional long-short-term memory network, learning forward and backward dependency relationships of a sequence, processing data at a specific time point, outputting and combining a preamble and subsequent information vector, and combining a regularization technology dropout to balance a model over-fitting risk brought by increasing the number of network layers; capturing interaction information between input text pairs by using a prompt learning mode to obtain higher prediction precision, and calculating relevancy measurement; according to the method, the relevancy between the test sample and each training sample is estimated by utilizing relevancy measurement, so that classification prediction is completed, the method has remarkable advantages in a text classification task, and the classification accuracy and efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the fields of deep learning and machine vision, and particularly relates to a method for text classification based on prompt learning of bidirectional long short-term memory networks. Background Art

[0002] Text classification algorithms have evolved from rule-based methods to machine learning algorithms and then to deep learning models. In the early days, text classification mainly relied on expert systems and simple statistical methods, but was limited by scale and flexibility. With the development of machine learning, especially the application of algorithms such as support vector machines (SVM) and random forests, the accuracy and adaptability of text classification have been significantly improved. Entering the era of deep learning, models such as convolutional neural networks (CNN), recurrent neural networks (RNN) and their variants have greatly improved the performance of text classification, especially when dealing with large-scale and complex data sets.

[0003] Text classification algorithms still face many difficulties. First of all, the complexity of language is a major challenge. Natural language has characteristics such as ambiguity, polysemy and context-dependence, which makes it difficult to accurately understand the meaning of text. Secondly, the design of the evaluation function is also a key issue. How to design a suitable evaluation function to accurately evaluate the performance of the model is a process that needs to be continuously explored and optimized. In addition, data imbalance and bias are also issues that need to be concerned about in text classification. The quality of the training data directly affects the classification performance, and data bias and imbalance may lead to a decline in the performance of the model on specific categories. To solve these difficulties, researchers have continuously explored new algorithms and technologies. Summary of the Invention

[0004] To solve the technical problems existing in the prior art, the present invention provides a text classification method based on prompt learning of bidirectional long short-term memory networks. By converting the task of text classification with few annotations into a task of estimating text relevance, and using a prompt learning model as the relevance metric, it can capture the interaction information between input text pairs. This method enables the model to more accurately understand the semantic relationship between texts, thereby improving the prediction accuracy of text classification.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: A text classification method based on prompt learning of bidirectional long short-term memory networks, and the specific steps are as follows:

[0006] Step S1: Collect various types of texts, including 11 categories such as finance, real estate, home furnishing, education, technology, society, fashion, current politics, sports, games, and entertainment, and use the text D t as the training data, and a sample is represented as d=(X d , y d ), where X dRepresents sample text, y d Represents its label.

[0007] Step S2: Read the sample text data under each category for word segmentation, including data preprocessing operations such as removing punctuation marks, part-of-speech screening, deduplication, and removing stop words.

[0008] Step S3: Use TF-IDF to represent the text vector, that is, calculate the TF-IDF values of each word to form a vector. The dimension of this vector is the size of the vocabulary, and the value reflects the importance of the word in the document, and construct a batch of sample sequences. Multiply the TF value of each word by its IDF value to obtain the TF-IDF value of the word in the document. The calculation process is as follows.

[0009]

[0010] TF~IDF(t,d,D) = TF(t,d) × IDF(t,D)

[0011] Among them, DF represents the word frequency, IDF represents the inverse document frequency, TF~IDF(t,d,D) represents the term frequency-inverse document frequency, t represents the word, d represents the number of documents containing the word, D represents the total number of documents in the document set. To prevent the denominator from being zero, usually 1 is added to the number of documents containing the word t.

[0012] Step S4: Construct a sample feature sequence, including a batch of 11 samples, each sample has N time steps, and the feature dimension of each time step is X. Set the input dimension format as batch_first = True, and the shape of the input data should be [11, N, X], which is used as the input.

[0013] Step S5: Construct a bidirectional long short-term memory neural network, and obtain the key information in the sequence through the selective input and selective forgetting mechanisms, and forget the information irrelevant to the current prediction task.

[0014] Step S6: The LSTM unit with a memory function includes an input gate, an output gate, a forget gate, and a memory cell block. The input gate determines the amount of information transmitted to the memory cell, and is used to control the current input value x i How much data is retained in C t Thereby realizing the update of the cell state C t The output gate determines how much information in the memory cell is transmitted to the current output. The forget gate controls the memory cell and is used to determine the memory and forgetting of the memory cell, and determines how much data of the memory cell at the previous moment will be transmitted to the present.

[0015] Receives an input vector at each time step i and updates its hidden state h t , and the calculation process can be as follows:

[0016] h t = f(W h * [h t-1 , x t + b h )

[0017] where W h and b h are the weight matrix and bias term of the hidden layer respectively, f is the activation function, usually tanh; [h t-1 , x t represents the vector formed by concatenating the hidden state of the previous moment and the input vector of the current moment.

[0018] Step S7: The forget gate mechanism effectively improves the model's ability to handle long-term dependence problems by dynamically adjusting the content in the memory cells and only retaining the information useful for subsequent tasks. The calculation process is as follows:

[0019] f t = σ(W f * [h t-1 , x t + b f )

[0020] C t = f t * C t-1

[0021] where f t is the output of the forget gate, representing the proportion of information retained in the memory cells; σ is the sigmoid function used to map the output value between 0 and 1; W f and b f are the weight matrix and bias vector of the forget gate respectively; [h t-1 , x t represents the vector formed by connecting the hidden state of the previous time step and the input of the current time step, and C t represents the cell state corresponding to the next time step.

[0022] Step S8: The input gate mechanism controls the inflow of new information and plays an important role in updating the memory cell state. The calculation process is as follows:

[0023] I t = σ(W I * [h t-1 , x t + b I )

[0024]

[0025] where I iis the output of the input gate, representing the proportion of new information allowed to pass through at the current time step; σ is the sigmoid function used to map the output value between 0 and 1; tanh represents the activation function; W I and b I represent the weight matrix and bias vector of the input gate respectively; W C and b C represent the weight matrix and bias vector of the current memory cell state respectively; [h t-1 , x t represents the vector formed by concatenating the hidden state of the previous time step and the input of the current time step.

[0026] Step 9: The output gate controls the amount of information output from the memory cell. The LSTM network can process sequence data more effectively and generate accurate outputs. The calculation process is as follows:

[0027] o t = σ(W o *[h t-1 , x t +b o )

[0028] h t = o t *tanh(C t )

[0029] Among them, o t is the output of the output gate, representing the proportion of new information allowed to pass through at the current time step; σ is the sigmoid function used to map the output value between 0 and 1; tanh represents the activation function; W o and b o represent the weight matrix and bias vector of the output gate; [h t-1 , x t represents the vector formed by concatenating the hidden state of the previous time step and the input of the current time step.

[0030] Step S10: Apply the Dropout optimization strategy to the fully connected layer in the LSTM memory unit. By randomly removing some neurons and their connections during training, the dependence of the neural network on specific neurons is reduced, and the generalization ability of the model is improved. The calculation method is as follows:

[0031]

[0032]

[0033] Among them, i represents the i-th feature sequence, represents the screening of the output vector of the L-th layer through the Bernoulli probability distribution; P represents the dropout probability; y(l) represents the output vector of the L-th layer; represents the result of the dot product operation between the probability mask and the output vector of the l-th layer; respectively represent the weight matrix and the bias vector of the (L + 1)-th layer; represents the input vector of the (L + 1)-th layer; f represents the sigmod activation function.

[0034] Step S11: The bidirectional long short-term memory module of the last layer uses return_sequences = False, indicating that h is output only at the last time step t , and the hidden state of the last time step is taken as a feature for subsequent processing.

[0035] Step S12: By converting the less-labeled text classification task into a text relevance estimation task and using the prompt learning model as a relevance metric, it well fits the pre-training task and can capture the interaction information between input text pairs to obtain higher prediction accuracy. The calculation process is as follows:

[0036]

[0037] where, the prompt learning model serves as the relevance metric score; θ represents the parameterized LSTM model; is the loss function of MetricPrompt, which is in the form of the cross-entropy loss between the probability distribution generated by label mapping and the true distribution. φ(·) represents the probability distribution over label classes, with the position corresponding to the true label of the input sample set to 1 and other positions set to 0, f cls (*, θ) represents mapping the output vocabulary probability distribution to a binomial distribution; d M respectively represent the sample text, label, training data, and a certain sample, where M represents prompt learning.

[0038] Step S13: The optimized prompt learning model is used as a text relevance metric to estimate the relevance between the test sample and each training sample, thereby completing the classification prediction.

[0039] The present invention adopts TF-IDF to evaluate the importance of words in a document. By constructing text vectors, key information is highlighted, thereby improving the accuracy of text classification. At the same time, TF-IDF tends to filter out common words and retain words with strong category discrimination ability, which helps the text classifier better identify texts of different categories. This method has significant advantages in text classification tasks and can improve the accuracy and efficiency of classification.

[0040] The bidirectional LSTM simultaneously learns the forward and backward dependencies of the sequence. When processing data at a specific time point, it can consider the context information before and after, allowing the model to more comprehensively understand the context of the input sequence. Using multiple layers of LSTM can enhance the learning ability and representation ability of the model. Each layer can build more advanced feature representations based on the features extracted by the previous layer. This hierarchical information processing method makes the bidirectional LSTM have higher accuracy in processing complex sequence modeling tasks. By combining the regularization technique dropout, the generalization performance is improved.

[0041] By converting the task of classifying text with few annotations into a task of estimating text relevance and using a prompt learning model as a relevance metric, it can capture the interaction information between input text pairs. This method enables the model to more accurately understand the semantic relationships between texts, thereby improving the prediction accuracy of text classification. Brief Description of the Drawings

[0042] Figure 1 It is a flowchart of text classification based on a bidirectional long short-term memory network.

[0043] Figure 2 It is a structural diagram of a bidirectional long short-term memory network architecture.

[0044] Figure 3 It is a schematic diagram of an LSTM memory cell.

[0045] Figure 4 It is a schematic diagram of the dropout mechanism, where Figure 4 (a) is a structural diagram of a standard fully understood network, Figure 4 (b) is a fully connected network after dropout. Detailed Implementation Manner

[0046] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] As Figure 1 shown, a text classification method based on bidirectional long short-term memory network prompt learning, the specific steps are as follows:

[0048] Step 1: Collect various types of texts, including 11 categories such as finance, real estate, home furnishing, education, technology, society, fashion, current politics, sports, games, and entertainment, and use the text D t as the training data. A sample is represented as d = (X d , y d ), where X d represents the sample text, and yd Indicates its label.

[0049] Step 2: Read the sample text data under each category for word segmentation, including data preprocessing operations such as removing punctuation marks, part-of-speech screening, deduplication, and removing stop words.

[0050] Step 3: Use TF-IDF to represent the text vector, that is, calculate the TF-IDF values of each word to form a vector. The dimension of this vector is the size of the vocabulary, and the value reflects the importance of the word in the document, and construct a batch of sample sequences.

[0051] Construct a word frequency list. The elements of the list are of dictionary type, that is, the list contains multiple dictionary elements. The structure of the dictionary is word: occurrence times. Each list stores the information of the occurrence times of all words in an article. For each word in the document, calculate its frequency of occurrence in the document. The calculation process is as follows:

[0052]

[0053] Among them, DF represents the word frequency, and d is the document containing the word.

[0054] Step Measure the importance of a certain word in the entire document set, divide the total number of documents by the logarithm of the number of documents containing the word. The calculation process is as follows:

[0055]

[0056] Among them, IDF represents the inverse document frequency, D is the total number of documents in the document set. To prevent the denominator from being zero, usually 1 is added to the number of documents containing the word t.

[0057] Multiply the TF value of each word by its IDF value to get the TF-IDF value of the word in the document. The calculation method is as follows:

[0058] TF~IDF(t,d,D)=TF(t,d)×IDF(t,D)

[0059] Among them, TF~IDF(t,d,D) represents the term frequency-inverse document frequency, and t, d, and D represent the word, the number of documents containing the word, and the total number of documents in the document set respectively.

[0060] Sort the words according to the TF-IDF values, and select the top X words with high TF-IDF values as the important keywords of the document, and construct a feature sequence.

[0061] Step 4: Construct a sample feature sequence, including batches of 11 samples. Each sample has N time steps, and the feature dimension of each time step is X. Under the setting of batch_first = True, the shape of the input data should be [11, N, X] as the input.

[0062] Step 5: As Figure 2 shown, construct a Bidirectional Long Short-Term Memory (Bi-LSTM) neural network. Through the selective input and selective forgetting mechanisms, obtain the key information in the sequence and forget the information irrelevant to the current prediction task.

[0063] Each batch has N sequences, and each sequence x i {i = 1, 2, 3, ..., N} consists of column vectors of a fixed length X to construct a feature sequence, that is, x i is the input for the sequence at each time step.

[0064] As Figure 3 shown, set half of the sequence length as the fixed number of neurons in the fully connected layer to meet the requirements of the subsequent layer or the output layer, and specify the number of LSTM layers. Multiple LSTM layers can capture more complex temporal dependencies.

[0065] (Bi-LSTM) The neural network builds two layers of bidirectional long short-term memory modules. Each layer is composed of a combination of a forward LSTM cell structure and a backward LSTM cell structure. At each time step, two LSTMs run simultaneously. One is forward to process the input sequence to obtain a series of forward hidden states; the other processes the sequence backward to obtain a series of backward hidden states. After processing, the forward and backward outputs are concatenated.

[0066] The feature sequence X has N sequences and each x i {i = 1, 2, 3, ..., N}. Each LSTM cell unit requires T time steps (i.e., self-loops T times), and x i is the input at each time step.

[0067] The output of each LSTM layer includes the hidden state (h t ) and the cell state (C t ) at the last time step. In the bidirectional LSTM, the shape of h t is (numlayers * num directions, batch size, hidden size), where numdirections is 2 (forward and backward).

[0068] In the BiLSTM, the input sequence is simultaneously passed to the forward and backward LSTMs, and the final output is the concatenation of the hidden states in both directions.

[0069] Step 6: The LSTM unit with a memory function includes an input gate, an output gate, a forget gate, and a memory cell block. The input gate determines the amount of information transmitted to the memory cell and is used to control the current input value x iHow much data is retained in C t to update the cell state C t ; the output gate determines how much information in the memory unit is transmitted to the current output; the forget gate controls the memory unit to determine the memory and forgetting of the memory unit, determining how much data from the memory unit at the previous moment will be passed to the present.

[0070] Set the number of hidden units to control the feature sequence. By extracting the sequence and core information, the input data is converted into high-level feature representations that contain important information and patterns in the data.

[0071] Receives an input vector at each time step i and updates its hidden state h t , and the calculation process can be as follows.

[0072] ht = f(Wh * [ht-1, xt] + bh)

[0073] where, W h and b h represent the weight matrix and bias term of the hidden layer respectively, f is the activation function, usually tanh, [h t-1 , x t represents the vector formed by concatenating the hidden state at the previous moment and the input vector at the current moment.

[0074] Step 7: The forget gate mechanism effectively improves the model's ability to handle long-term dependence problems by dynamically adjusting the content in the memory cells and only retaining the information useful for subsequent tasks.

[0075] The input of the forget gate includes the input (x t ) at the current time step t and the hidden state (h t-1 )) at the previous time step. The output is a vector between 0 and 1, representing the forgetting ratio on each dimension. The calculation process is as follows:

[0076] ft = σ(Wf * [ht-1, xt] + bf)

[0077] C t = f t * C t-1

[0078] where, f t is the output of the forget gate, representing the proportion of information retained in the memory cell; σ is the sigmoid function used to map the output value between 0 and 1; W f and b f are the weight matrix and bias vector of the forget gate respectively; [h t-1 , x tdenotes the vector formed by concatenating the hidden state of the previous time step and the input of the current time step, C t denotes the cell state corresponding to the next time step.

[0079] Step 8: The input gate mechanism plays an important role in controlling the inflow of new information and updating the memory cell state.

[0080] The input gate takes the hidden state h of the previous time step t-1 and the input of the current time step (x t ) as the common input. The Sigmoid function determines how much of the input information at the current time step should be added to the memory cell state. The Tanh function takes the input of the current time step and the hidden state of the previous time step as inputs and outputs a new candidate memory cell state. The calculation process is as follows:

[0081] It=σ(W I *[ht-1,xt]+b I )

[0082]

[0083] where, I i is the output of the input gate, representing the proportion of new information allowed to pass at the current time step; σ is the sigmoid function used to map the output value between 0 and 1; tanh represents the activation function; W I and b I represent the weight matrix and bias vector of the input gate respectively; W C and b C represent the weight matrix and bias vector of the current memory cell state respectively; [h t-1 ,x t denotes the vector formed by concatenating the hidden state of the previous time step and the input of the current time step.

[0084] The input gate processes the current memory cell state using the Tanh function and multiplies it by the Sigmoid output of the output gate to obtain the final hidden state output, which is passed to the next time step for output.

[0085] Step 9: The output gate controls the amount of information output from the memory cell. The LSTM network can process sequential data more effectively and generate accurate outputs.

[0086] The inputs of the output gate include: the input of the current time step (x t ), the hidden state of the previous time step (h t-1 ), and the current memory cell state h t) It indirectly affects the calculation of the output gate through other mechanisms and outputs a numerical vector between 0 and 1. This vector is multiplied by the current memory cell state to determine the amount of information in the final output:

[0087] ot = σ(Wo * [ht-1, xt] + bo)

[0088] ht = ot * tanh(Ct)

[0089] Among them, o t is the output of the output gate, representing the proportion of new information allowed to pass through at the current time step; σ is the sigmoid function used to map the output value between 0 and 1; tanh represents the activation function; W o and b o represent the weight matrix and bias vector of the output gate respectively; [h t-1 , x t represents the vector formed by concatenating the hidden state of the previous time step and the input of the current time step.

[0090] The calculation result of the output gate will be multiplied by the updated cell state C after being mapped by the activation function tanh t as the output result of the hidden state h t at the current moment.

[0091] Step 10: As Figure 4 shown, adopt the Dropout optimization strategy for the fully connected layer in the LSTM memory unit. By randomly removing some neurons and their connections during the training process, the dependence of the neural network on specific neurons is reduced, and the generalization ability of the model is improved.

[0092] Define the input as the hidden state h t-1 of the previous time step and the input (x t ) of the current time step as the common input, set the dropout rate, and control the disconnection ratio of the neurons in the linear transformation of the loop state. It needs to be adjusted through experiments and cross-validation to find the optimal Dropout ratio to obtain better performance and generalization ability.

[0093] Apply the dropout operation to the output of the fully connected layer to reduce the overfitting risk of the model. Dropout will randomly set the outputs of some neurons to 0, and these neurons will not be considered during forward propagation and backward propagation. Its calculation method is as follows:

[0094]

[0095] Among them, i represents the i-th feature sequence, Indicates that the output vector of the L-th layer is screened by the Bernoulli probability distribution; P represents the dropout probability; y (l) represents the output vector of the L-th layer; represents the result of the dot product operation between the probability mask and the output vector of the l-th layer; represent the weight matrix and bias vector of the (L + 1)-th layer respectively; represents the input vector of the (L + 1)-th layer; f represents the sigmod activation function.

[0096] The data of the input layer is calculated to obtain the output of the fully connected layer through matrix multiplication and bias addition, and the activation function is used to perform non-linear transformation on the output.

[0097] Refer to the dropout rate with a setting range of 0 - 0.5. Using the control variable method, keep the learning rate at 0.00001, the batch_size at 64, and the number of training epochs at 200, and select the optimal dropout rate.

[0098] Step 11: The bidirectional long short-term memory module of the last layer uses return_sequences = False, and takes the hidden state of the last time step as the feature for subsequent processing. Pair the few-labeled training data in pairs, and train the prompt learning model to estimate the relevance of the text pairs.

[0099] Step 12: By transforming the few-labeled text classification task into a text relevance prediction task and using the prompt learning model as the relevance metric, it well fits the pre-training task and can capture the interaction information between the input text pairs to obtain higher prediction accuracy.

[0100]

[0101] Among them, the prompt learning model serves as the relevance metric score; θ represents the parameterized LSTM model; is the loss function of MetricPrompt, and its form is the cross-entropy loss between the probability distribution generated by label mapping and the true distribution. φ(·) represents the probability distribution over the label classes. The position corresponding to the true label of the input sample is set to 1, and other positions are set to 0. f cls (*, θ) represents mapping the output vocabulary probability distribution to a binomial distribution; d M represents a certain sample text, the corresponding label, training data, and a certain sample set is represented by M represents prompt learning;

[0102] The optimized prompt learning method is used as a text relevance metric to estimate the relevance between the test sample and each training sample, thereby completing the classification prediction.

[0103] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of the present invention.

Claims

1. A text classification method based on bidirectional long short-term memory network prompt learning, characterized in that: The specific steps are as follows: Step 1: Construct multi-category text data and perform data cleaning; Step 2: Construct a batch sample sequence and calculate the word frequency-inverse frequency value to represent the text; Step 3: When building a bidirectional long short-term memory network, learn the forward and backward dependencies of the sequence, process the data at a specific time point, and output a vector that combines the previous and next context information; Step 4: Combine the regularization technology dropout to balance the risk of model overfitting caused by increasing the number of network layers; Step 5: Use the hint learning method to capture the interactive information between the input text pairs and calculate the relevance measure; Step 6: Use the correlation measure to estimate the correlation between the test sample and each training sample to complete the classification prediction.

2. A text classification method based on bidirectional long short-term memory network prompt learning according to claim 1, characterized in that: In step 1, various texts are collected and t Table training data, a sample is represented by d = (X d ,y d ), where X d represents the sample text, y d Indicates its label; Read sample text data under each category for word segmentation, including data preprocessing operations such as punctuation removal, part of speech screening, duplicate removal, and stop word removal; Use TF-IDF to represent text vectors, that is, calculate the TF-IDF value of each word to form a vector, build a batch sample sequence, and multiply the TF value of each word by its IDF value to get the TF-IDF value of the word in the document. The calculation process is as follows: TF~IDF(t,d,D)=TF(t,d)×IDF(t,D) Among them, DF represents word frequency, IDF represents inverse document frequency, TF~IDF(t,d,D) represents word frequency-inverse document frequency, t, d, and D represent the word, the document containing the word, and the total number of documents respectively.

3. A text classification method based on bidirectional long short-term memory network prompt learning according to claim 2, characterized in that: In step 2, construct a sample feature sequence. Each sample has N time steps. The feature dimension of each time step is X. The shape of the input data should be [11, N, X].

4. A text classification method based on bidirectional long short-term memory network prompt learning according to claim 3, characterized in that: In step 3, a bidirectional long short-term memory neural network is constructed to obtain key information in the sequence through selective input and selective forgetting mechanisms, and forget information that is irrelevant to the current prediction task; The LSTM unit with memory function includes input gate, output gate, forget gate and memory unit block. The input gate determines the amount of information transmitted to the memory unit and is used to control the current input value x. i Retained in C t The ratio of the cell state C t Updates; At each time step i, it receives an input vector and updates its hidden state h t , the calculation process is as follows: h t =f(W h *[h t-1 ,x t ]+b h ) Among them, W h is the weight matrix of the hidden layer, b h is the bias term of the hidden layer, f is the activation function tanh, [h t-1 ,x t ] represents the vector formed by concatenating the hidden state of the previous moment and the input vector of the current moment.

5. A text classification method based on bidirectional long short-term memory network prompt learning according to claim 4, characterized in that: In step 4, the forget gate mechanism dynamically adjusts the content in the memory cells to retain only the information that is useful for subsequent tasks. The specific calculation process is as follows: f t =σ(W f *[h t-1 ,x t ]+b f ) C t =f t *C t-1 Among them, f t represents the output of the forget gate, that is, the proportion of information retained in the memory cell; σ is the sigmoid function, which is used to map the output value between 0 and 1; W f is the weight matrix of the forget gate; b f is the bias vector of the forget gate; [h t-1 ,x t ] represents the vector formed by connecting the hidden state of the previous time step and the input of the current time step, C t Represents the cell state corresponding to the next time step; The input gate mechanism controls the inflow of new information and updates the state of memory cells. The calculation process is as follows: I t =σ(W I *[h t-1 ,x t ]+b I ) Among them, I i represents the output of the input gate, that is, the proportion of new information allowed to pass through at the current time step; σ is the sigmoid function, which is used to map the output value between 0 and 1; tanh represents the activation function; W I is the weight matrix of the input gate; b I is the bias vector of the input gate; W C The weight matrix representing the current state of the memory cell; b C The bias vector representing the current state of the memory cell; [h t-1 ,x t ] represents the vector formed by connecting the hidden state of the previous time step and the input of the current time step; The output gate controls the amount of information output from the memory cell. The LSTM network processes sequence data and generates accurate output. The calculation process is as follows: the t =σ(W o *[h t-1 ,x t ]+b o ) h t =o t *tanh(C t ) Among them, t is the output of the output gate, indicating the proportion of new information allowed to pass through at the current time step; σ is the sigmoid function, which is used to map the output value between 0 and 1; tanh represents the activation function; W o represents the weight matrix of the output gate; b o represents the bias vector of the output gate; [h t-1 ,x t ] represents the vector formed by connecting the hidden state of the previous time step and the input of the current time step; Step S10: The Dropout optimization strategy is used for the fully connected layer in the LSTM memory unit, by randomly removing some neurons and their connections during the training process. The calculation method is as follows: Among them, i represents the i-th feature sequence, indicates that the output vector of the Lth layer is filtered by the Bernoulli probability distribution; P indicates the discard probability; y (l) Represents the L-th layer output vector; Represents the dot product result of the probability mask and the output vector of the lth layer; represents the weight matrix of the L+1th layer; Represents the bias vector of the L+1th layer; represents the input vector of the L+1th layer; f represents the sigmoid activation function; The hidden state of the last time step of the bidirectional long short-term memory module of the last layer is taken as the feature for subsequent processing.

6. A text classification method based on bidirectional long short-term memory network prompt learning according to claim 5, characterized in that: In step 5, by converting the sparsely labeled text classification task into a text relevance estimation task, the hint learning model is used as the relevance metric to capture the interaction information between the input text pairs. The calculation process is as follows: in, The hint learning model is used as a relevance metric score, θ represents the LSTM model parameterized by, is the loss function of MetricPrompt, which is in the form of the cross entropy loss between the probability distribution generated by the label mapping and the true distribution. φ(·) represents the probability distribution over the label category. The position corresponding to the true label of the input sample is set to 1, while the other positions are set to 0. cls (*,θ) represents mapping the output vocabulary probability distribution to a binomial distribution, Represents sample text, Indicates the label, represents the training data, d M represents a sample, and M represents prompt learning.

7. A text classification method based on bidirectional long short-term memory network prompt learning according to claim 6, characterized in that: In step six, the optimized hint learning model is used as a text relevance metric to estimate the relevance between the test sample and each training sample, thereby completing the classification prediction.

Images