Classification prediction method for label information enhancement and strong negative sample driving

Through the method of enhanced label information and strong negative sample-driven, the problem of ignoring the information and insufficient processing ability of emotional labels in the prior art is solved, the accuracy of emotional prediction and text data quality is improved, and more accurate emotional semantic differences are achieved. Capture and model differentiation capabilities.

CN119004174AActive Publication Date: 2024-11-22ANHUI UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411049070.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-22
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

The existing text prediction methods ignore the information of emotional labels in emotional prediction, and lack the ability to handle complex and marginal situations, so they cannot effectively distinguish the differences between different emotional labels.

Method used

Using the classification prediction method driven by tag information enhancement and strong negative sample, the network based on text truncation, tag information enhancement and strong negative sampling modules is built, and emotional dimension word collection and vocabulary database are used to enrich emotional label representation, and combined with strong negative sampling technology Construct a comparison loss function to improve the prediction accuracy of the model.

Benefits of technology

It improves the accuracy of emotion classification prediction, enhances text data quality, can capture emotional semantic differences more accurately, and improves the model's distinction ability through strong negative sampling technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004174A_ABST
    Figure CN119004174A_ABST
Patent Text Reader

Abstract

The invention discloses a label information enhancement and strong negative sample driven classification prediction method, which comprises the following steps of: firstly, constructing a high-quality text through text truncation and question set content matching; then, emotional label information is considered, and emotional label characterization is enriched by adopting a vocabulary database and a professional vocabulary; secondly, distinguishing each text from a strong negative sample by adopting a strong negative sampling technology; and finally, the accuracy of the model is ensured by utilizing prediction loss combined with emotional label information and a contrast loss function combined with strong negative sampling. According to the method, text truncation, emotion label information enhancement and strong negative sampling are integrated, and the accuracy of classification prediction is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to text prediction tasks in the field of natural language, and specifically to a classification prediction method driven by label information enhancement and strong negative samples. Background Art

[0002] In recent years, classification prediction is an important task in the field of natural language processing. Its goal is to classify text data into different categories. Among them, sentiment prediction has received extensive attention and research. By analyzing the behavior, language and other relevant data of individuals, their emotional characteristics can be inferred. Sentiment prediction technology has important application value in many fields such as dialogue systems and recommendation systems.

[0003] With the advancement of computer technology and natural language processing technology, sentiment prediction methods have gradually evolved from traditional questionnaire surveys to automated predictions based on text data. Sentiment prediction methods based on text semantics mainly focus on how to improve text semantic representation to better predict individual emotions.

[0004] Although with the development of natural language processing technology and deep learning models, text prediction methods based on text semantics have made significant progress, they still face many challenges. The main problems are as follows: First, existing methods often only focus on text semantic representation and ignore the informativeness of sentiment labels. Second, existing text prediction models are insufficient in processing complex and edge cases and generalization capabilities, and may not be able to effectively distinguish the differences between different sentiment labels. Summary of the invention

[0005] In order to solve the problems existing in the existing methods, the present invention proposes a classification prediction method driven by label information enhancement and strong negative samples, in order to accurately obtain the user's emotional characteristics, thereby improving the accuracy of text prediction.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0007] The characteristics of the classification prediction method of label information enhancement and strong negative sample driving of the present invention are that it is carried out according to the following steps:

[0008] Step 1: Define the user set U={u1,u2,…,u i ,…,u I}, where u i represents the i-th user, I is the number of users in the user set U;

[0009] Define a text set Set={S1,S2,…,S i ,…,S I}, where S i represents the i-th user u ithe text;

[0010] Defining text ,in, Indicates S i The jth statement in , and , express The mth word in The number of words in S i The number of statements;

[0011] Define the problem set Q = {q1,q2,…,q k ,…,q K}, where q k represents the kth exercise in the exercise set Q, and , Indicates q k The nth word in q k The number of words in , K is the number of exercises in the exercise set Q;

[0012] Define sentiment vocabulary V = {v1,v2,…,v q …,v Q}, where v q is the qth sentiment word, Q is the number of sentiment words in V;

[0013] Define the sentiment dimension word set Y={ y1, y2,……, y p ,……, y P}, where y p is the pth sentiment dimension word, and P is the number of sentiment dimension words;

[0014] Let S i The emotional state of the emotional dimension word set Y is Y i ={y1 i , y2 i ,……, y p i ,……, y P i}, where y p i For S i In the pth sentiment dimension word y p The emotional state on p i ∈{0,1};

[0015] Step 2: Construct a classification prediction network based on label information enhancement and strong negative sample driving, including: text truncation module, label information enhancement module, and strong negative sampling module;

[0016] Step 2.1: The text truncation module performs i Process and get u i High-quality text ;

[0017] Step 2.2: The label information enhancement module processes the sentiment dimension word set Y to obtain sentiment semantic information embedding ;

[0018] Step 2.3: The strong negative sampling module performs i Perform strong negative sampling to obtain S i In emotional state p i The attention weight set of the strong negative sample set ;

[0019] Step 2.4: Use formula (9) to get In the pth emotional dimension y p The predicted value on :

[0020] (9)

[0021] In formula (9), W and b are two parameters of nonlinear transformation, σ is the sigmoid function, yes Representation of;

[0022] Step 3: Construct a total loss function and train the classification prediction network to obtain the optimal classification prediction model;

[0023] Step 3.1: Use formula (10) to construct the prediction loss function L p :

[0024] (10)

[0025] Step 3.2: Use formula (11) to construct the contrast loss function L c :

[0026] (11)

[0027] In formula (11), yes The μ-th strong negative sample in the upper middle Representation of;

[0028] Step 3.3: Use formula (12) to construct the total loss function L:

[0029] (12)

[0030] In formula (14), δ is a hyperparameter;

[0031] Step 3.4: Use the gradient descent method to train the classification prediction network and calculate the total loss function L to update the network parameters until the total loss function L converges, thereby obtaining the optimal classification prediction model after training, which is used to predict the user's emotional label.

[0032] The feature of the classification prediction method of label information enhancement and strong negative sample driving described in the present invention is that step 2.1 is performed as follows:

[0033] Step 2.1.1: Use formula (1) to get Characterization :

[0034] (1)

[0035] In formula (1), yes exist The frequency of occurrence in yes exist The number of yes word embeddings;

[0036] Step 2.1.2: Use formula (2) to get q k Characterization of vec k :

[0037] (2)

[0038] In formula (2), yes In q k The frequency of occurrence in yes In q k The number of yes word embeddings;

[0039] Step 2.1.3: Use formula (3) to get and q k The cosine similarity of :

[0040] (3)

[0041] Step 2.1.4: Use formula (4) to get Matching weight with Q :

[0042] (4)

[0043] In formula (4), Indicates q k The weight of

[0044] Step 2.1.5: Compose the first β sentences with the highest matching weights with the exercise set Q into high-quality text ;in, represents the εth high-quality sentence.

[0045] The step 2.2 is performed as follows:

[0046] Step 2.2.1: Obtain the pth sentiment dimension word y through the vocabulary database p Synonyms of syn p ;

[0047] Step 2.2.2: Use formula (6) to get the pth sentiment dimension word y p About the related word set rel of the sentiment vocabulary V p :

[0048] (6)

[0049] In formula (5), is the similarity function, τ is the similarity threshold;

[0050] Step 2.2.3: Use formula (6) to get the semantic information embedding of the sentiment dimension word set Y :

[0051] (6)

[0052] In formula (6), is the pth sentiment dimension word y p word embedding, Synonym set p The embedding of the synonym w in , for rel p Related words Embedded, Represents vector concatenation.

[0053] The step 2.3 is carried out as follows:

[0054] Step 2.3.1: Use formula (7) to get S i In emotional state p i Strong negative sample set on ,in, Indicates S i In emotional statep i The rth strong negative sample on , R is the number of strong negative samples:

[0055] (7)

[0056] In formula (7), S t represents the tth user u t text, θ is the text similarity threshold, y p t Indicates S t In the pth sentiment dimension word y p emotional state;

[0057] Step 2.3.2: Use formula (8) to get The attention weight , thus obtaining S i In emotional state p i The attention weight set of the strong negative sample set :

[0058] (8)

[0059] In formula (8), yes The representation of yes The μth strong negative sample in The representation of YesS i Characterization, exp() is the natural exponential function.

[0060] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the classification prediction method, and the processor is configured to execute the program stored in the memory.

[0061] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the classification prediction method when the computer program is executed by a processor.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention aims to obtain accurate emotional characteristics of users and improves the accuracy of emotional classification prediction by adopting a classification prediction method driven by label information enhancement and strong negative samples.

[0064] 2. The present invention constructs high-quality text by truncating text and matching the content of the exercise set, thereby solving the problem of low data processing efficiency in the prior art and enhancing the quality of text data.

[0065] 3. The present invention takes into account the informativeness of emotion tags and uses a vocabulary database and a professional vocabulary table to enrich the representation of emotion tags, thus overcoming the problem of ignoring the informativeness of emotion tags in the prior art and capturing the semantic differences of emotions more accurately.

[0066] 4. The present invention adopts strong negative sampling technology to distinguish each text from strong negative samples, and integrates the prediction loss of sentiment label information and the contrast loss function combined with strong negative sampling to improve the accuracy of model prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is the overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0068] In this embodiment, a classification prediction method driven by label information enhancement and strong negative samples is to construct high-quality text using a text truncation method and a similarity screening method, enrich the sentiment label representation with a vocabulary database and a professional vocabulary table, and simultaneously use a strong negative sampling technique to construct a contrast loss function of strong negative sampling and combine it with a prediction loss containing sentiment label information to train a model. Specifically, Figure 1 As shown, the method is performed in the following steps:

[0069] Step 1: Define the user set U={u1,u2,…,u i ,…,u I}, where u i represents the i-th user, and I is the number of users in the user set U. In this embodiment, Facebook data is used as an example. All "AUTHIDs" form the user set U, and the text data with the same "AUTHID" in the data set are merged.

[0070] Define a text set Set={S1,S2,…,S i ,…,S I}, where S i represents the i-th user u i text; in this embodiment, the text set is obtained by performing data cleaning on the merged text data, selectively matching and removing special symbols and illegal characters through regular expressions, and removing stop words to ensure the accuracy and completeness of the data.

[0071] Defining text ,in, Indicates S i The jth statement in , and , express The mth word in The number of words in S i In this embodiment, the sentence sequence of the text is the result of segmenting the text according to the regular expression.

[0072] Define the problem set Q = {q1,q2,…,q k ,…,q K}, where q k represents the kth exercise in the exercise set Q, and , Indicates q k The nth word in q k , K is the number of words in the exercise set Q; in this embodiment, the exercise set is the exercise content in the personality prediction scale.

[0073] Define sentiment vocabulary V = {v1,v2,…,v q …,v Q}, where v q is the qth emotion word, Q is the number of emotion words in V; in this embodiment, the emotion word list is all words in NRC-Emotion-Lexicon.

[0074] Define the sentiment dimension word set Y={ y1, y2,……, y p ,……, y P}, where y p is the pth sentiment dimension word, and P is the number of sentiment dimension words; in this embodiment, the sentiment vocabulary is the sentiment vocabulary tags involved in the Facebook data.

[0075] Let S i The emotional state of the emotional dimension word set Y is Y i ={y1 i , y2 i ,……, y p i ,……, y P i}, where y p i For S i In the pth sentiment dimension word y p The emotional state on p i∈{0,1}; In this embodiment, the sentiment vocabulary is the sentiment vocabulary tag value involved in the Facebook data, the tag value is 'y', indicating the sentiment with this dimension, and the tag value is 'n', indicating the sentiment without this dimension; the sentiment state tag sequence processes the tag value, changes the tag value 'y' to '1', and changes the tag value 'n' to '0'.

[0076] Step 2: Construct a classification prediction network based on label information enhancement and strong negative sample driving, including: text truncation module, label information enhancement module, and strong negative sampling module;

[0077] Step 2.1: Text truncation module i Process and get u i High-quality text ;

[0078] Step 2.1.1: Use formula (1) to get Characterization :

[0079] (1)

[0080] In formula (1), yes exist The frequency of occurrence in yes exist The number of yes word embeddings;

[0081] Step 2.1.2: Use formula (2) to get q k Characterization of vec k :

[0082] (2)

[0083] In formula (2), yes In q k The frequency of occurrence in yes In q k The number of yes word embeddings;

[0084] Step 2.1.3: Use formula (3) to get and q k The cosine similarity of :

[0085] (3)

[0086] Step 2.1.4: Use formula (4) to get Matching weight with Q :

[0087] (4)

[0088] In formula (4), Indicates q k The weight of

[0089] Step 2.1.5: Compose the first β sentences with the highest matching weights with the exercise set Q into high-quality text ;in, represents the εth high-quality sentence; text with high matching weight values ​​can more accurately reflect the nuances and intensity of emotions.

[0090] Step 2.2: The label information enhancement module processes the sentiment dimension word set Y to obtain sentiment semantic information embedding ;

[0091] Step 2.2.1: Obtain the pth sentiment dimension word y through the vocabulary database p Synonyms of ;

[0092] Step 2.2.2: Use formula (6) to get the pth sentiment dimension word y p About the related word set of sentiment vocabulary V :

[0093] (6)

[0094] In formula (5), sim(,) is the similarity function, τ is the similarity threshold, and the threshold value can be set to 0.5;

[0095] Step 2.2.3: Use formula (6) to get the semantic information embedding of the sentiment dimension word set Y :

[0096] (6)

[0097] In formula (6), is the pth sentiment dimension word y p word embedding, Synonym set p The embedding of the synonym w in , for rel p Related words Embedded, Represents vector concatenation;

[0098] Step 2.3: Strong negative sampling module for S i Perform strong negative sampling to obtain S i In emotional state p i The attention weight set of the strong negative sample set ;

[0099] Step 2.3.1: Use formula (7) to get S i In emotional state p i Strong negative sample set on ,in, Indicates S i In emotional state p i The rth strong negative sample on , R is the number of strong negative samples:

[0100] (7)

[0101] In formula (7), S t represents the tth user u t text, θ is the text similarity threshold, the similarity threshold can be set to 0.5, y p t Indicates S t In the pth sentiment dimension word y p emotional state;

[0102] Step 2.3.2: Use formula (8) to get The attention weight , and finally get S i In emotional state p i The attention weight set of the strong negative sample set ,in, is the rth strong negative sample The attention weight is , and R is the number of strong negative samples:

[0103] (8)

[0104] In formula (8), yes The representation of yes The μ-th strong negative sample in the upper middle The representation of YesS i Characterization, exp() is the natural exponential function.

[0105] Step 2.4: Use formula (9) to get In the pth emotional dimension y pThe predicted value on :

[0106] (9)

[0107] In formula (9), W and b are two parameters of nonlinear transformation, σ is the sigmoid function, yes representation.

[0108] Step 3: Construct a total loss function and train the classification prediction network to obtain the optimal classification prediction model;

[0109] Step 3.1: Use formula (10) to construct the prediction loss function L p :

[0110] (10)

[0111] Step 3.2: Use formula (11) to construct the contrast loss function L c :

[0112] (11)

[0113] In formula (11), yes The μ-th strong negative sample in the upper middle representation.

[0114] Step 3.3: Use formula (12) to construct the total loss function L:

[0115] (12)

[0116] In formula (14), δ is a hyperparameter.

[0117] Step 3.4: Use the gradient descent method to train the classification prediction network and calculate the total loss function L to update the network parameters until the total loss function L converges, thereby obtaining the optimal classification prediction model after training, which is used to predict the user's emotional label.

[0118] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0119] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

Claims

1. A classification prediction method driven by label information enhancement and strong negative samples, characterized in that: The steps are as follows: Step 1: Define the user set U={u1,u2,…,u i ,…,u I }, where u i represents the i-th user, I is the number of users in the user set U; Define a text set Set={S1,S2,…,S i ,…,S I }, where S i represents the i-th user u i the text; Defining text ,in, Indicates S i The jth statement in , and , express The mth word in The number of words in S i The number of statements; Define the problem set Q = {q1,q2,…,q k ,…,q K }, where q k represents the kth exercise in the exercise set Q, and , Indicates q k The nth word in q k The number of words in , K is the number of exercises in the exercise set Q; Define sentiment vocabulary V = {v1,v2,…,v q …,v Q }, where v q is the qth sentiment word, Q is the number of sentiment words in V; Define the sentiment dimension word set Y={ y1, y2,……, y p ,……, y P }, where y p is the pth sentiment dimension word, and P is the number of sentiment dimension words; Let S i The emotional state of the emotional dimension word set Y is Y i ={ y1 i , y2 i ,……, y p i ,……, y P i }, where y p i For S i In the pth sentiment dimension word y p The emotional state on p i ∈{0,1}; Step 2: Construct a classification prediction network based on label information enhancement and strong negative sample driving, including: text truncation module, label information enhancement module, and strong negative sampling module; Step 2.1: The text truncation module performs i Process and get u i High-quality text ; Step 2.2: The label information enhancement module processes the sentiment dimension word set Y to obtain sentiment semantic information embedding ; Step 2.3: The strong negative sampling module performs i Perform strong negative sampling to obtain S i In emotional state p i The attention weight set of the strong negative sample set ; Step 2.4: Use formula (9) to get In the pth emotional dimension y p The predicted value on : (9) In formula (9), W and b are two parameters of nonlinear transformation, σ is the sigmoid function, yes Representation of; Step 3: Construct a total loss function and train the classification prediction network to obtain the optimal classification prediction model; Step 3.1: Use formula (10) to construct the prediction loss function L p : (10) Step 3.2: Use formula (11) to construct the contrast loss function L c : (11) In formula (11), yes The μ-th strong negative sample in the upper middle Representation of; Step 3.3: Use formula (12) to construct the total loss function L: (12) In formula (14), δ is a hyperparameter; Step 3.4: Use the gradient descent method to train the classification prediction network and calculate the total loss function L to update the network parameters until the total loss function L converges, thereby obtaining the optimal classification prediction model after training, which is used to predict the user's emotional label.

2. According to claim 1, a classification prediction method based on label information enhancement and strong negative sample driving is characterized in that: The step 2.1 is carried out as follows: Step 2.1.1: Use formula (1) to get Characterization : (1) In formula (1), yes exist The frequency of occurrence in yes exist The number of yes word embeddings; Step 2.1.2: Use formula (2) to get q k Characterization of vec k : (2) In formula (2), yes In q k The frequency of occurrence in yes In q k The number of yes word embeddings; Step 2.1.3: Use formula (3) to get and q k Cosine similarity of : (3) Step 2.1.4: Use formula (4) to get Matching weight with Q : (4) In formula (4), Indicates q k The weight of Step 2.1.5: Compose the first β sentences with the highest matching weights with the exercise set Q into high-quality text ;in, represents the εth high-quality sentence.

3. The classification prediction method based on label information enhancement and strong negative sample driving according to claim 2 is characterized in that: The step 2.2 is carried out as follows: Step 2.2.1: Obtain the pth sentiment dimension word y through the vocabulary database p Synonyms of syn p ; Step 2.2.2: Use formula (6) to get the pth sentiment dimension word y p About the related word set of sentiment vocabulary V : (6) In formula (5), sim(,) is the similarity function, τ is the similarity threshold; Step 2.2.3: Use formula (6) to get the semantic information embedding E of the sentiment dimension word set Y: (6) In formula (6), is the pth sentiment dimension word y p word embedding, Synonym set p The embedding of the synonym w in , for rel p Related words The embedding Represents vector concatenation.

4. The classification prediction method based on label information enhancement and strong negative sample driving according to claim 3 is characterized in that: The step 2.3 is carried out as follows: Step 2.3.1: Use formula (7) to get S i In emotional state p i Strong negative sample set on ,in, Indicates S i In emotional state p i The rth strong negative sample on , R is the number of strong negative samples: (7) In formula (7), S t represents the tth user u t text, θ is the text similarity threshold, y p t Indicates S t In the pth sentiment dimension word y p emotional state; Step 2.3.2: Use formula (8) to get The attention weight , thus obtaining S i In emotional state p i The attention weight set of the strong negative sample set : (8) In formula (8), yes The representation of yes The μth strong negative sample in The representation of YesS i Characterization, exp() is the natural exponential function.

5. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the classification prediction method described in any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the classification prediction method according to any one of claims 1 to 4 are executed.

Citation Information

Patent Citations

  • Text classification model optimization method and device

    CN114416989A

  • Multi-modal sentiment analysis method, system and equipment based on multivariate loss function and medium

    CN116701996A

  • Method for Text Classification and Feature Selection Using Class Vectors and the System Thereof

    US20180357531A1

  • Emotion classification method and apparatus, electronic device, and storage medium

    WO2022141861A1

  • Text sentiment analysis method and apparatus, device and storage medium

    WO2023173537A1