A classification prediction method driven by label information enhancement and strong negative samples

Abstract

This invention discloses a classification prediction method driven by label information enhancement and strong negative samples. The method comprises the following steps: first, constructing high-quality text by truncation of text and matching the content of a workbook; second, considering the informativeness of sentiment labels, using a vocabulary database and a professional vocabulary to enrich the sentiment label representation; third, using strong negative sampling to distinguish each text from strong negative samples; and finally, using a prediction loss function that incorporates sentiment label information and a contrastive loss function that incorporates strong negative sampling to ensure model accuracy. This method integrates text truncation, sentiment label information enhancement, and strong negative sampling to improve the accuracy of classification prediction.

Description

Technical Field

[0001] The present invention relates to text prediction tasks in the field of natural language, and specifically to a classification prediction method driven by label information enhancement and strong negative samples. Background Art

[0002] In recent years, classification prediction has become a key task in natural language processing. Its goal is to classify text data into different categories. Sentiment prediction, which infers emotional characteristics by analyzing individual behavior, language, and other relevant data, has garnered extensive attention and research. Sentiment prediction technology has important applications in a variety of fields, including dialogue systems and recommendation systems.

[0003] With the advancement of computer technology and natural language processing technology, sentiment prediction methods have gradually evolved from traditional questionnaire surveys to automated predictions based on text data. Text semantics-based sentiment prediction methods mainly focus on how to improve text semantic representation to better predict individual emotions.

[0004] Although text prediction methods based on text semantics have made significant progress with the development of natural language processing technology and deep learning models, they still face many challenges. The main issues are the following two: First, existing methods often focus solely on the semantic representation of text, ignoring the informativeness of sentiment labels. Second, existing text prediction models lack the ability to handle complex and edge cases, and their generalization capabilities are insufficient, making them unable to effectively distinguish between different sentiment labels. Summary of the Invention

[0005] In order to solve the problems existing in the existing methods, the present invention proposes a classification prediction method that enhances label information and drives strong negative samples, in order to accurately obtain the user's emotional characteristics, thereby improving the accuracy of text prediction.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0007] The characteristics of the classification prediction method of the present invention with label information enhancement and strong negative sample driving are that it is carried out according to the following steps:

[0008] Step 1: Define the user set U={u1,u2,…,u i ,…,u I}, where u i represents the i-th user, I is the number of users in the user set U;

[0009] Define the text set Set={S1,S2,…,S i ,…,S I}, where S i represents the i-th user u ithe text;

[0010] Define text ,in, Indicates S i The jth statement in , and , express The mth word in M ​​represents The number of words in S i The number of statements;

[0011] Define the problem set Q={q1,q2,…,q k ,…,q K}, where q k represents the kth exercise in the exercise set Q, and , Indicates q k The nth word in q k The number of words in , K is the number of exercises in the exercise set Q;

[0012] Define sentiment vocabulary V={v1,v2,…,v q …,v Q}, where v q is the qth sentiment word, Q is the number of sentiment words in V;

[0013] Define the sentiment dimension word set Y={ y1, y2,……, y p ,……, y P}, where y p is the pth sentiment dimension word, and P is the number of sentiment dimension words;

[0014] Let S i The emotional state of the emotional dimension word set Y is Y i ={ y1 i , y2 i ,……, y p i ,……, y P i}, where y p i For S i In the pth sentiment dimension word y p The emotional state on p i ∈{0,1};

[0015] Step 2: Build a classification prediction network based on label information enhancement and strong negative sample driving, including: text truncation module, label information enhancement module, and strong negative sampling module;

[0016] Step 2.1: The text truncation module performs i Process and get u i High-quality text ;

[0017] Step 2.2: The label information enhancement module processes the emotional dimension word set Y to obtain the emotional semantic information embedding ;

[0018] Step 2.3: The strong negative sampling module performs i Perform strong negative sampling to obtain S i In emotional state p i Attention weight set of strong negative sample set ;

[0019] Step 2.4: Use formula (9) to get In the pth emotional dimension y p The predicted value on :

[0020] (9)

[0021] In formula (9), W and b are two parameters of nonlinear transformation, σ is the sigmoid function, yes representation of;

[0022] Step 3: Construct a total loss function and train the classification prediction network to obtain the optimal classification prediction model;

[0023] Step 3.1: Use formula (10) to construct the prediction loss function L p :

[0024] (10)

[0025] Step 3.2: Use formula (11) to construct the contrast loss function L c :

[0026] (11)

[0027] In formula (11), yes The μth strong negative sample in the upper middle representation of;

[0028] Step 3.3: Use formula (12) to construct the total loss function L:

[0029] (12)

[0030] In formula (14), δ is a hyperparameter;

[0031] Step 3.4: Use the gradient descent method to train the classification prediction network and calculate the total loss function L to update the network parameters until the total loss function L converges, thereby obtaining the optimal classification prediction model after training, which is used to predict the user's emotional label.

[0032] The feature of the classification prediction method with label information enhancement and strong negative sample driving described in the present invention is that step 2.1 is performed as follows:

[0033] Step 2.1.1: Use formula (1) to get Characterization :

[0034] (1)

[0035] In formula (1), yes exist The frequency of occurrence in yes exist The number of yes word embeddings;

[0036] Step 2.1.2: Use formula (2) to get q k Characterization of vec k :

[0037] (2)

[0038] In formula (2), yes In q k The frequency of occurrence in yes In q k The number of yes word embeddings;

[0039] Step 2.1.3: Use formula (3) to get and q k Cosine similarity of :

[0040] (3)

[0041] Step 2.1.4: Use formula (4) to get Matching weight with Q :

[0042] (4)

[0043] In formula (4), Indicates q k The weight of

[0044] Step 2.1.5: Compose the top β sentences with the highest matching weights with the exercise set Q into high-quality text ;in, represents the εth high-quality sentence.

[0045] The step 2.2 is carried out as follows:

[0046] Step 2.2.1, obtain the pth sentiment dimension word y through the vocabulary database p Synonyms of syn p ;

[0047] Step 2.2.2: Use formula (6) to get the pth sentiment dimension word y p About the related word set rel of the sentiment vocabulary V p :

[0048] (6)

[0049] In formula (5), is the similarity function, τ is the similarity threshold;

[0050] Step 2.2.3: Use formula (6) to get the semantic information embedding of the sentiment dimension word set Y :

[0051] (6)

[0052] In formula (6), is the pth sentiment dimension word y p word embeddings, Synonym set syn p The embedding of the synonym w in rel p Related words Embedded, Represents vector concatenation.

[0053] The step 2.3 is carried out as follows:

[0054] Step 2.3.1. Use formula (7) to get S i In emotional state p i Strong negative sample set on ,in, Indicates S i In emotional statep i The rth strong negative sample on , R is the number of strong negative samples:

[0055] (7)

[0056] In formula (7), S t represents the tth user u t text, θ is the text similarity threshold, y p t Indicates S t In the pth sentiment dimension word y p emotional state;

[0057] Step 2.3.2: Use formula (8) to get The attention weight , thus obtaining S i In emotional state p i Attention weight set of strong negative sample set :

[0058] (8)

[0059] In formula (8), yes The representation of yes The μth strong negative sample in The representation of It's S i Characterization, exp() is the natural exponential function.

[0060] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute the classification prediction method, and the processor is configured to execute the program stored in the memory.

[0061] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the classification prediction method when the computer program is executed by a processor.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention aims to obtain accurate emotional characteristics of users and improves the accuracy of emotional classification prediction by adopting a classification prediction method that enhances label information and drives strong negative samples.

[0064] 2. The present invention constructs high-quality text by truncating text and matching the content of the exercise book, thereby solving the problem of low data processing efficiency in the prior art and enhancing the quality of text data.

[0065] 3. The present invention takes into account the informativeness of emotion tags and uses a vocabulary database and a professional vocabulary table to enrich the representation of emotion tags, overcoming the problem of ignoring the informativeness of emotion tags in the prior art and capturing the semantic differences of emotions more accurately.

[0066] 4. The present invention adopts strong negative sampling technology to distinguish each text from strong negative samples, and integrates the prediction loss combined with sentiment label information and the contrast loss function combined with strong negative sampling to improve the accuracy of model prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 4 is an overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0068] In this embodiment, a classification prediction method driven by label information enhancement and strong negative samples is to construct high-quality text using a text truncation method and a similarity screening method, enrich the emotional label representation with a vocabulary database and a professional vocabulary table, and simultaneously adopt a strong negative sampling technique to construct a strong negative sampling contrast loss function and combine it with a prediction loss containing emotional label information to train the model. Specifically, Figure 1 As shown, the method is carried out in the following steps:

[0069] Step 1: Define the user set U={u1,u2,…,u i ,…,u I}, where u i represents the i-th user, and I is the number of users in the user set U. In this example, Facebook data is used as an example. All "AUTHIDs" constitute the user set U, and the text data with the same "AUTHID" in the dataset are merged.

[0070] Define the text set Set={S1,S2,…,S i ,…,S I}, where S i represents the i-th user u i In this embodiment, the text set is obtained by performing data cleaning on the merged text data, selectively matching and removing special symbols and illegal characters through regular expressions, and removing stop words to ensure the accuracy and completeness of the data.

[0071] Define text ,in, Indicates S i The jth statement in , and , express The mth word in M ​​represents The number of words in S i In this embodiment, the sentence sequence of the text is the result of segmentation processing of the text according to the regular expression.

[0072] Define the problem set Q={q1,q2,…,q k ,…,q K}, where q k represents the kth exercise in the exercise set Q, and , Indicates q k The nth word in q k , K is the number of words in the exercise set Q; in this embodiment, the exercise set is the exercise content in the personality prediction scale.

[0073] Define sentiment vocabulary V={v1,v2,…,v q …,v Q}, where v q is the qth emotion word, Q is the number of emotion words in V; in this embodiment, the emotion vocabulary is all the words in NRC-Emotion-Lexicon.

[0074] Define the sentiment dimension word set Y={ y1, y2,……, y p ,……, y P}, where y p is the pth sentiment dimension word, and P is the number of sentiment dimension words. In this embodiment, the sentiment vocabulary is the sentiment vocabulary tags involved in the Facebook data.

[0075] Let S i The emotional state of the emotional dimension word set Y is Y i ={ y1 i , y2 i ,……, y p i ,……, y P i}, where y p i For S i In the pth sentiment dimension word y p The emotional state on p i∈{0,1}; In this embodiment, the sentiment vocabulary is the sentiment vocabulary label value involved in the Facebook data, the label value is 'y', indicating that the sentiment has this dimension, and the label value is 'n', indicating that the sentiment does not have this dimension; the sentiment state label sequence processes the label value, changes the label value 'y' to '1', and changes the label value 'n' to '0'.

[0076] Step 2: Build a classification prediction network based on label information enhancement and strong negative sample driving, including: text truncation module, label information enhancement module, and strong negative sampling module;

[0077] Step 2.1: Text truncation module i Process and get u i High-quality text ;

[0078] Step 2.1.1: Use formula (1) to get Characterization :

[0079] (1)

[0080] In formula (1), yes exist The frequency of occurrence in yes exist The number of yes word embeddings;

[0081] Step 2.1.2: Use formula (2) to get q k Characterization of vec k :

[0082] (2)

[0083] In formula (2), yes In q k The frequency of occurrence in yes In q k The number of yes word embeddings;

[0084] Step 2.1.3: Use formula (3) to get and q k Cosine similarity of :

[0085] (3)

[0086] Step 2.1.4: Use formula (4) to get Matching weight with Q :

[0087] (4)

[0088] In formula (4), Indicates q k The weight of

[0089] Step 2.1.5: Compose the top β sentences with the highest matching weights with the exercise set Q into high-quality text ;in, Represents the εth high-quality sentence; text with high matching weight values ​​can more accurately reflect the nuances and intensity of emotions.

[0090] Step 2.2: The label information enhancement module processes the emotional dimension word set Y to obtain the emotional semantic information embedding ;

[0091] Step 2.2.1, obtain the pth sentiment dimension word y through the vocabulary database p Synonyms of ;

[0092] Step 2.2.2: Use formula (6) to get the pth sentiment dimension word y p About the related word set of sentiment vocabulary V :

[0093] (6)

[0094] In formula (5), sim(,) is the similarity function, τ is the similarity threshold, and the threshold value can be set to 0.5;

[0095] Step 2.2.3: Use formula (6) to get the semantic information embedding of the sentiment dimension word set Y :

[0096] (6)

[0097] In formula (6), is the pth sentiment dimension word y p word embeddings, Synonym set syn p The embedding of the synonym w in rel p Related words Embedded, Represents vector concatenation;

[0098] Step 2.3, the strong negative sampling module is used to i Perform strong negative sampling to obtain S i In emotional state p i Attention weight set of strong negative sample set ;

[0099] Step 2.3.1. Use formula (7) to get S i In emotional state p i Strong negative sample set on ,in, Indicates S i In emotional state p i The rth strong negative sample on , R is the number of strong negative samples:

[0100] (7)

[0101] In formula (7), S t represents the tth user u t text, θ is the text similarity threshold, the similarity threshold can be set to 0.5, y p t Indicates S t In the pth sentiment dimension word y p emotional state;

[0102] Step 2.3.2: Use formula (8) to get The attention weight , and finally get S i In emotional state p i Attention weight set of strong negative sample set ,in, is the rth strong negative sample The attention weight is , and R is the number of strong negative samples:

[0103] (8)

[0104] In formula (8), yes The representation of yes The μth strong negative sample in the upper middle The representation of It's S i Characterization, exp() is the natural exponential function.

[0105] Step 2.4: Use formula (9) to get In the pth emotional dimension y pThe predicted value on :

[0106] (9)

[0107] In formula (9), W and b are two parameters of nonlinear transformation, σ is the sigmoid function, yes representation.

[0108] Step 3: Construct a total loss function and train the classification prediction network to obtain the optimal classification prediction model;

[0109] Step 3.1: Use formula (10) to construct the prediction loss function L p :

[0110] (10)

[0111] Step 3.2: Use formula (11) to construct the contrast loss function L c :

[0112] (11)

[0113] In formula (11), yes The μth strong negative sample in the upper middle representation.

[0114] Step 3.3: Use formula (12) to construct the total loss function L:

[0115] (12)

[0116] In formula (14), δ is a hyperparameter.

[0117] Step 3.4: Use the gradient descent method to train the classification prediction network and calculate the total loss function L to update the network parameters until the total loss function L converges, thereby obtaining the optimal classification prediction model after training, which is used to predict the user's emotional label.

[0118] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0119] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

Claims

1. A classification prediction method driven by label information enhancement and strong negative samples, characterized by: The steps are as follows: Step 1: Define the user set U={u1,u2,…,u i ,…,u I }, where u i represents the i-th user, I is the number of users in the user set U; Define the text set Set={S1,S2,…,S i ,…,S I }, where S i represents the i-th user u i the text; Define text ,in, Indicates S i The jth statement in , and , express The mth word in M ​​represents The number of words in S i The number of statements; Define the problem set Q={q1,q2,…,q k ,…,q K }, where q k represents the kth exercise in the exercise set Q, and , Indicates q k The nth word in q k The number of words in , K is the number of exercises in the exercise set Q; Define sentiment vocabulary V={v1,v2,…,v q …,v Q }, where v q is the qth sentiment word, Q is the number of sentiment words in V; Define the sentiment dimension word set Y={ y1, y2,……, y p ,……, y P }, where y p is the pth sentiment dimension word, and P is the number of sentiment dimension words; Let S i The emotional state of the emotional dimension word set Y is Y i ={ y1 i , y2 i ,……, y p i ,……, y P i }, where y p i For S i In the pth sentiment dimension word y p The emotional state on p i ∈{0,1}; Step 2: Build a classification prediction network based on label information enhancement and strong negative sample driving, including: text truncation module, label information enhancement module, and strong negative sampling module; Step 2.1: The text truncation module performs i Process and get u i High-quality text ; Step 2.2: The label information enhancement module processes the emotional dimension word set Y to obtain the emotional semantic information embedding ; Step 2.3: The strong negative sampling module performs i Perform strong negative sampling to obtain S i In emotional state p i Attention weight set of strong negative sample set ; Step 2.4: Use formula (9) to get In the pth emotional dimension y p The predicted value on : (9) In formula (9), W and b are two parameters of nonlinear transformation, σ is the sigmoid function, yes representation of; Step 3: Construct a total loss function and train the classification prediction network to obtain the optimal classification prediction model; Step 3.1: Use formula (10) to construct the prediction loss function L p : (10) Step 3.2: Use formula (11) to construct the contrast loss function L c : (11) In formula (11), yes The μth strong negative sample in the upper middle representation of; Step 3.3: Use formula (12) to construct the total loss function L: (12) In formula (14), δ is a hyperparameter; Step 3.4: Use the gradient descent method to train the classification prediction network and calculate the total loss function L to update the network parameters until the total loss function L converges, thereby obtaining the optimal classification prediction model after training, which is used to predict the user's emotional label.

2. The classification prediction method based on label information enhancement and strong negative sample driving according to claim 1, characterized in that: The step 2.1 is carried out as follows: Step 2.1.1: Use formula (1) to get Characterization : (1) In formula (1), yes exist The frequency of occurrence in yes exist The number of yes word embeddings; Step 2.1.2: Use formula (2) to get q k Characterization of vec k : (2) In formula (2), yes In q k The frequency of occurrence in yes In q k The number of yes word embeddings; Step 2.1.3: Use formula (3) to get and q k Cosine similarity of : (3) Step 2.1.4: Use formula (4) to get Matching weight with Q : (4) In formula (4), Indicates q k The weight of Step 2.1.5: Compose the top β sentences with the highest matching weights with the exercise set Q into high-quality text ;in, represents the εth high-quality sentence.

3. The classification prediction method based on label information enhancement and strong negative sample driving according to claim 2, characterized in that: The step 2.2 is carried out as follows: Step 2.2.1, obtain the pth sentiment dimension word y through the vocabulary database p Synonyms of syn p ; Step 2.2.2: Use formula (6) to get the pth sentiment dimension word y p About the related word set of sentiment vocabulary V : (6) In formula (5), sim(,) is the similarity function, τ is the similarity threshold; Step 2.2.3: Use formula (6) to get the semantic information embedding E of the sentiment dimension word set Y: (6) In formula (6), is the pth sentiment dimension word y p word embeddings, Synonym set syn p The embedding of the synonym w in rel p Related words Embedded, Represents vector concatenation.

4. The classification prediction method based on label information enhancement and strong negative sample driving according to claim 3 is characterized in that: The step 2.3 is carried out as follows: Step 2.3.

1. Use formula (7) to get S i In emotional state p i Strong negative sample set on ,in, Indicates S i In emotional state p i The rth strong negative sample on , R is the number of strong negative samples: (7) In formula (7), S t represents the tth user u t text, θ is the text similarity threshold, y p t Indicates S t In the pth sentiment dimension word y p emotional state; Step 2.3.2: Use formula (8) to get The attention weight , thus obtaining S i In emotional state p i Attention weight set of strong negative sample set : (8) In formula (8), yes The representation of yes The μth strong negative sample in The representation of It's S i Characterization, exp() is the natural exponential function.

5. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the classification prediction method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the classification prediction method according to any one of claims 1 to 4 are executed.

Images