Behavior polarity classification algorithm based on CLCA-CGCN

By integrating context attention and the comparative learning strategy of CGCN model, the problem of behavior polarity classification in complex legal texts is solved, the classification accuracy and efficiency are improved, information compression and noise problems are alleviated, and more reasonable behavior polarity is provided. judge.

CN118916737AActive Publication Date: 2024-11-08UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410907710.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-08
Estimated Expiration
2044-07-08

AI Technical Summary

Technical Problem

The existing technology is difficult to effectively classify behavioral polarity in complex legal texts, especially when dealing with long texts, complex context dependencies and ambiguity abstractions, and lack of high-quality data and standardized classification standards, resulting in classification accuracy and efficiency. low.

Method used

Using the behavioral polarity classification model of fusion context attention (CLCA) and Core Graph Convolutional Network (CGCN), enhanced word embedding representation through prompt word splicing, improved context attention mechanism, random jump connection and contrast learning strategy Learn and graph structure feature to improve the model's ability to classify behavioral polarity.

Benefits of technology

It significantly improves the classification accuracy and efficiency of complex legal text data, alleviates information compression and noise problems, enhances the stability and diversity of the model, and provides more reasonable judgment on behavioral polarity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916737A_ABST
    Figure CN118916737A_ABST
Patent Text Reader

Abstract

In order to cope with the challenge of behavior polarity classification in legal texts, an innovative algorithm model is introduced, and a behavior polarity classification model of contextual attention (CLCA) contrast learning (CLCN) and a behavior polarity classification model of a CGCN (Core Graph Configuration Network (CGCN) are fused. The model is specially designed to be used for a behavior polarity classification task, and the classification accuracy and efficiency of complex legal text data are improved by combining context information and a contrast learning strategy. Aiming at fuzziness and abstractness of behavior polarity classification in the legal field, the invention provides a comparative learning algorithm fusing label information, gives consideration to a behavior polarity classification task with labels and a comparative learning task, and improves the polarity classification effect of behaviors under different case category labels. For the context dependence problem, a special context attention mechanism is introduced, the model is guided to reasonably aggregate and distinguish existing behaviors, and the characterization capacity of behavior words embedded in polarity information and the understanding capacity of the model for context content are enhanced. According to the method, the CGCN network is provided, the defects of the GCN network in the anti-noise aspect and the feature representation capability degradation phenomenon are relieved based on the sampling thought and the transverse expansion of the model, and the generalization and the anti-noise capability of behavior polarity classification are improved. Experiments on a public text classification data set and a legal data set show that by using the model provided by the invention, the effect of behavior polarity classification can be effectively improved, certain polarity classification universality is achieved, and the model relatively conforms to behavior polarity classification requirements in the legal field.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention introduces an innovative algorithm model that integrates Contrastive Learning with Context Attention (CLCA) and CGCN (Core Graph Convolutional Network, CGCN) behavioral polarity classification model. The invention is specifically designed for behavioral polarity classification tasks, and improves the classification accuracy and efficiency of complex legal text data by combining contextual information and contrastive learning strategies. Background Art

[0002] Another key point in extracting judgment elements is the classification of behavior polarity. In law, behavior polarity usually refers to the evaluation of the nature or value of a behavior within the legal framework, that is, whether the behavior is positive (legal, legitimate or beneficial) or negative (illegal, inappropriate or harmful). This classification is based on legal norms, moral standards and social values, and is used to guide people's behavior, evaluate case facts and make judicial judgments. The determination of behavior polarity involves a comprehensive consideration of the behavior itself and its consequences, motivations, environment and other aspects. In actual legal practice, due to the influence of various complex factors, the polarity evaluation of the same behavior may be controversial, especially in the legal margins or emerging social phenomena. Therefore, judges, lawyers and other legal professionals need to conduct careful analysis and judgment based on laws, precedents, principles and specific case circumstances.

[0003] The polarity classification standard for an action is not very clear at present. Differences in regions, human factors, judicial practices, etc. often lead to different polarity judgment results for similar actions. In general, legal texts or events are usually highly structural and semantically complex. General polarity classification methods cannot distinguish the differences in polarity of actions under different legal provisions in this context, especially similar actions with different consequences. In addition, since legal texts are generally long and complex, there may be certain correlations between different actions. It is necessary to introduce a contextual attention mechanism to target this feature, so that the model can focus on descriptions related to the behavior and capture the correlation between the past and the present, which is more conducive to the classification of behavior.

[0004] The goal of this invention is to analyze the extracted behaviors and distinguish the behaviors that affect the judgment results in a legal sense. The difficulties faced are mainly the following:

[0005] 1. Professional terminology and complex expressions. Specifically, there are many detailed descriptions of causal relationships, clauses, situations, etc. in legal texts, including many nested and subordinate clauses, which bring more difficulties to the extraction of behaviors and thus affect the judgment of the polarity of behaviors.

[0006] 2. Context dependency. The behavior in the legal text is closely related to the context and has temporal characteristics. It is often necessary to combine the case background and context relationship of the current behavior to make a reasonable judgment on the polarity of the behavior. Therefore, it is necessary to deal with complex context dependencies and integrate the current case background information.

[0007] 3. The ambiguity and abstractness of legal texts. The interpretation of legal texts is somewhat abstract, and different experts have different interpretations of a behavior. This abstractness makes it impossible to establish a suitable standard for the behavior polarity category, and the classification basis of behavior polarity lacks objective standards. In addition, the definition of behavior polarity in different cases may be inconsistent, and it is difficult to have a unified standard for judging behavior polarity.

[0008] 4. Lack of long texts and effective data. Legal texts are often long texts containing a large amount of text descriptions. Processing long texts requires the model to have the ability to capture long-distance text dependencies and to process these dependencies in parallel relatively efficiently. Another difficult problem is that high-quality standard data is difficult to obtain. There is a serious class imbalance problem in the data, and the behaviors in a large amount of data are homogenized. For example, most of the negative behaviors in divorce cases are concentrated in a few high-frequency behaviors, which is not conducive to model training. Summary of the invention

[0009] In order to meet the challenge of behavioral polarity classification in legal texts, this paper introduces an innovative algorithm model that integrates the behavioral polarity classification model of Contrastive Learning with Context Attention (CLCA) and CGCN (Core Graph Convolutional Network, CGCN). This model is specially designed for behavioral polarity classification tasks, and improves the classification accuracy and efficiency of complex legal text data by combining contextual information and contrastive learning strategies. This paper aims to design a lightweight non-convolutional visual detection network, and the technical solution adopted is:

[0010] Step 1: The overall algorithm uses the sentence pair input method of the PLM model to make the model pay more attention to the description related to the target label. Using this feature, a method of splicing the original text with prompt words is designed to guide the model to pay attention to the part related to the current behavior. Combined with the special attention mechanism proposed in this invention, the word embedding representation of the key elements of the behavior can be directly completed, and the information related to the behavior polarity in the word embedding is enhanced.

[0011] Step 2: When designing the contextual attention module, the PLM model is often used to encode the input sequence by weighting and obtaining new results based on its relevance, which only considers positive information and ignores negative information. In some situations where positive and negative information need to be clearly distinguished, its word embedding representation has defects, especially in the problem of behavior polarity classification, where the word embedding representation of behavior needs to integrate relevant polarity information. Based on this problem, this module proposes an improved contextual attention, which considers both the polarity of attention and the contextual representation problem.

[0012] Step 3: There are two key tasks in designing the CGCN module: sampling and segmenting the behavior graph; adding random jump connections. The purpose of sampling the graph is to reduce the size of the gradient when the model aggregates at a single node. When the number of aggregated nodes decreases, the problem of information compression will also be alleviated, and it can also help the model learn the subspace characteristics of the behavior graph.

[0013] Step 4: When designing the contrastive learning module, the purpose is to help the model distinguish the polarity of behaviors that are difficult to judge as an auxiliary task. The basic principle is to compare and analyze the output results of the behavior polarity classification through the similarity information of the label and the behavior, so as to help the model learn the similarity characteristics between behaviors and the similarity information of the behavior polarity categories. An improved strategy is also proposed, which performs a dropout operation on the embedded representation within the batch to obtain the disturbed positive samples, increase the contrast samples within a single batch, and improve the versatility and diversity of the model of the present invention.

[0014] Step 5: When designing the training loss function, for contrastive learning tasks, the training goal is to make the distance between positive examples closer and the distance between negative examples farther. The cross entropy function cannot adapt well to the training goal. KL divergence is a way to compare the similarity between two distributions.

[0015] Step 6: Connect the above modules, and the overall model adopts the loss function proposed in step 5. The data set is input into the algorithm to obtain the result. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 : Algorithm flow chart of CLCA-CGCN.

[0017] Figure 2 For: Contextual attention module diagram.

[0018] Figure 3 For: Behavior diagram.

[0019] Figure 4 : Schematic diagram of CGCN.

[0020] Figure 5 Figure 1: Schematic diagram of comparative learning module. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings.

[0022] Figure 1 It is the algorithm flow chart of the CLCA-CGCN model of the present invention.

[0023] In the above, we have put forward the main problems in the behavior polarity classification: identification of key elements of behavior, dependence on context and case background, and unclear polarity classification standards. The following is a solution based on the CLCA-CGCN model.

[0024] Generally speaking, the basis for determining whether a behavior is positive or negative is generally the words with clear polarity information in the behavior elements. When the amount of data is relatively large, it is difficult to manually mark the keywords in the behavior. At this time, some special methods are needed to guide the model to pay attention to the keywords in the behavior. The sentence pair input method of the PLM model can make the model pay more attention to the description related to the target label. Using this feature, a method of splicing the original text with prompt words is designed to guide the model to pay attention to the part related to the current behavior. Combined with the special attention mechanism proposed in this invention, the word embedding representation of the key elements of the behavior can be directly completed, and the information related to the polarity of the behavior in the word embedding is enhanced.

[0025] The description of a behavior consists of multiple words. How to reasonably aggregate behavior features from the description is the key to behavior polarity classification. CGCN is used to aggregate the elements of the behavior to obtain the behavior representation vector, which is then concatenated with the sentence embedding and input into the classifier for calculation to obtain the behavior polarity classification result.

[0026] The difficulty of behavior polarity classification comes from different case categories, human factors, unconventional terms, etc. In this regard, supervised contrastive learning is introduced to distinguish the categories of behavior. In the contrastive learning task, the classification model is guided to pay attention to the case category to which the behavior category belongs, and the polarity of behavior under different case categories and the differences between behaviors are compared to improve the effect of behavior polarity classification.

[0027] Contextual attention module design When using the PLM model to encode the input sequence, it often only weights and obtains new results based on its relevance, only considering positive information and ignoring negative information. In some situations where it is necessary to clearly distinguish between positive and negative information, its word embedding representation has defects, especially in the behavior polarity classification problem, the behavior word embedding representation needs to integrate relevant polarity information. Based on this problem, this section proposes an improved contextual attention that considers both the polarity of attention and the contextual representation problem. The contextual attention model in this section is as follows Figure 2 shown.

[0028] The core idea of ​​the contextual attention of the present invention is that the inputs are not equivalent. Attention learns not the similarity information of the inputs, but the combination of addition, deletion and subtraction of the inputs. The calculation process mainly includes deleting the matrix and the linear combination matrix, and adding some biases to make the result more reasonable.

[0029] The contextual attention module mainly includes two parts: global attention calculation and contextual attention calculation.

[0030] The first is the global attention calculation part. The global attention module is mainly responsible for calculating the linear combination part in the polarity, and its result is the similarity measure between each input element. global The calculation is shown in formula (1).

[0031]

[0032] Q∈R n×d , K∈R n×d The input map is obtained by equal size, n is the input sequence length, and d is the integer division result of the word embedding dimension and the number of attention heads.

[0033] Next, we calculate the context (polarity) attention module, and the input Q c ∈R n×d , K c ∈R n×d The input sequence is mapped to the same size, where n is the sequence length and d is the integer divisibility of the word embedding dimension and the number of attention heads. context The calculation of is shown in formula (2)(3).

[0034]

[0035]

[0036] Function f represents the negative value of the L1 norm between sequence elements, which is used to evaluate the non-correlation score. X represents the input sequence, and W is the corresponding mapping parameter. The result of the context attention calculation is between 0 and 1, which is mainly used to control whether the influence of the corresponding attention should be deleted and not calculated.

[0037] When performing the final attention calculation, the global attention is treated as zero mean. The purpose of this step is to make the final calculation result of the global attention close to -1 and 1, indicating the positive and negative results of polarity.

[0038] A global =A global -Mean(A global ) (4)

[0039] Based on this, the final contextual attention score is calculated as shown in formula (5).

[0040] A=(tanh(A global )+λsoftmax(A global ))⊙A context (5)

[0041] λ is a hyperparameter that controls the bias ratio, and ⊙ represents the element-wise multiplication operation. Formula (5) contains two parts of attention results. One is the contextual attention result, and the output value is biased towards -1, 0, and 1. Its value includes operations of addition, subtraction, and deletion, indicating positive, negative, and irrelevant operations in polarity. The second attention result is the global attention bias term, which is used to control the bias of similarity. Because attention is not a strict addition or subtraction operation, adding some offsets to fit the prior information is more conducive to the calculation of polarity.

[0042] After obtaining the attention weights, use V∈R from the sequence mapping n×d As the value, the weighted attention output result of a single head is obtained, as shown in formula (6).

[0043]

[0044] represents the output of the ith attention head. Next, the output of each attention head is concatenated to obtain the final output of the contextual attention, which is restored to the original size of the word embedding input, and then the final output is obtained through linear mapping and regularization. W represents the linear mapping parameter, and Norm represents regularization.

[0045] X context =Norm(X context +X context W) (7)

[0046] After obtaining the word embedding with contextual polarity information, the behavior is subjected to polarity classification. However, the behavior is a description containing multiple words, and how to represent this word has a decisive influence on the polarity classification result of the behavior.

[0047] The behavior itself conforms to the characteristics of the graph structure. It can be represented as a graph structure with strong aggregation centered on the predicate, such as Figure 3 shown.

[0048] exist Figure 3In the graph, all points will be aggregated at the node "repayment". There is a very serious information compression problem when using GCN to calculate the behavior. Within a limited vector dimension, using a large amount of information for aggregation will cause the feature representation of the vector at the node to tend to the average value in the classification space. In the training of the model, this feature will also cause the gradient to be too large and the training to be slow. The original intention of using GCN for graph classification is destroyed. Based on this problem, the CGCN network is proposed to alleviate this problem.

[0049] The CGCN network has two key tasks: sampling and segmenting the behavior graph; adding random jump connections. The purpose of sampling the graph is to reduce the size of the gradient when the model aggregates at a single node. When the number of aggregated nodes decreases, the problem of information compression will also be alleviated. It can also help the model learn the subspace characteristics of the behavior graph. The CGCN model is as follows Figure 4 shown.

[0050] First, the graph sampling task is performed. According to the core predicate of the graph, the original graph is sampled to obtain several sampling head results. Then, according to a certain ratio, jump connections are randomly added to each sampling head. The purpose of this step is to improve the stability of the model when there are certain errors in the graph input. Because GCN is relatively weak in processing data with errors in the graph input, it is necessary to actively add some noise to make the model less sensitive, thereby improving the stability of GCN.

[0051] For a single sampling head, the input word embedding is first mapped to an integer value of the word embedding size and the number of sampling heads according to the number of sampling heads, as shown in formula (8).

[0052] H=MLP(X)(8)

[0053] H∈R n×d , X∈R n×h , d represents the word embedding size of the sampling head, h represents the word embedding size of the input, and the number of sampling heads is the ratio of h to d.

[0054] Then the output result of the sampling head is calculated, as shown in formula (9).

[0055] H (i+1) = dropout(f(D -0.5 (A+I)D -0.5 )H (i) W i ) (9)

[0056] H (i) ∈R n×d Represents the output result of the i-th layer, D∈R n×n Represents the degree matrix of the graph, A∈R n×n Represents the adjacency matrix of the graph, Wi is the weight parameter of the i-th layer, and f represents the activation function. Then the results of each sampling head are concatenated to obtain the output H∈R of CGCN n×h .

[0057] H = concat(H (1) , …, H (k) ) (10)

[0058] After obtaining the CGCN calculation results, the predicate items in the output value are concatenated with the CLS word embedding and input into the classification head for calculation to obtain the final behavior classification results.

[0059] outputs = softmax((h CLs ⊙h v )W) (11)

[0060] W∈R 2h×L is the classification head parameter, L is the number of categories, h∈R h is the word embedding, ⊙ represents the vector concatenation operation, and the maximum index value after softmax calculation is the polarity of the behavior under the corresponding category.

[0061] CGCN can better alleviate the information compression problem when classifying behavior graphs. It takes predicates as the core and samples and calculates the polarity results of behavior elements. CGCN also has relatively good results when facing noise problems. Its sampling and random noise characteristics can enhance the model's ability to distinguish wrong behavior graphs and improve the judgment results of behavior polarity.

[0062] The contrastive learning module is designed as an auxiliary task to help the model distinguish the polarity of behaviors that are difficult to judge. Its basic principle is to compare and analyze the output results of behavior polarity classification through the similarity information of labels and behaviors, so as to help the model learn the similarity features between behaviors and the similarity information of behavior polarity categories.

[0063] In the word embedding encoding process of PLM, if texts with different semantics are similar on the surface, their embedding representations are often highly similar. The existence of this similarity may cause the classifier to be unable to accurately distinguish these texts, thereby reducing the classification effect. As shown in Table 1, the BERT model is prone to making mistakes when processing such similar texts. In order to enhance the ability of PLM in similar word embedding representation, some studies have adopted adversarial training methods. Gradient-based word embedding perturbation technology is introduced in the training process, so that the word embedding representation of the model has better stability under certain noise. High-quality adversarial text samples are used to train the model to identify the similarity between input texts and improve the model's word embedding representation ability on similarity.

[0064] Table 1 Different semantically similar texts

[0065]

[0066] In Table 1, when learning the representation of sentences, the PLM model often recognizes the negative expression "not slow" used in the sentence as "slow", because they are similar from the perspective of BERT, and thus obtains a prediction value of non-dangerous driving. On the other hand, for more complex questions, double negations and other forms, the model often recognizes them as incorrect prediction values. The reason for this problem is that models such as BERT do not consider the correlation between different words when calculating attention. The theoretical basis is that different input words are independent of each other. Therefore, in some semantic representations, the BERT model often ignores some negative prefixes and only uses the core words as its semantic information, resulting in BERT considering the semantics to be the same when the sentence descriptions are similar but the semantics are different, thus obtaining wrong information.

[0067] From the above analysis, we can see that the model recognizes similar texts with different semantics as texts with the same semantics, and the model ignores the semantic differences. The idea of ​​the present invention to solve this problem is to introduce a contrastive learning auxiliary task that integrates label information to enhance the model's ability to distinguish semantic differences and perceive the correlation between different categories.

[0068] The contrastive learning module mainly includes the calculation of behavior similarity and label similarity. The calculation idea is as follows Figure 5 shown.

[0069] The basic principle of contrastive learning is to bring the embedding representations of similar texts closer together and push away the embedding representations of dissimilar texts, which is conducive to the model learning the features between different semantics and improving the effect of downstream tasks. In contrastive learning, how to construct positive and negative sample pairs is the key to improving the semantic sensitivity of the model. In the model of the present invention, samples of the same category in a batch are regarded as positive samples, and samples of different categories are regarded as negative samples. However, there is a problem that some batches may all be samples of the same category, lacking corresponding negative samples or, on the contrary, lacking positive samples. Therefore, the present invention proposes an improved strategy, which performs a dropout operation on the embedding representations in the batch to obtain disturbed positive samples, increase the contrast samples in a single batch, and improve the versatility and diversity of the model of the present invention.

[0070] Assuming there is an input embedding representation x, a linear calculation is first performed on the embedding representation before contrastive learning calculation, as shown in formula (12). The MLP layer can improve the effect of downstream contrastive learning embedding representation.

[0071] h=MLP(x) (12)

[0072] h represents the embedding representation used to calculate the similarity. Next, the similarity between every two samples is calculated for all samples in the batch using formula (13).

[0073]

[0074] || ||2 represents the calculation of the embedded 2-norm, and ξ represents a minimum value to ensure that the calculation does not overflow.

[0075] Assuming that the number of samples in the original batch is n, and after adding samples through dropout, the number of samples becomes 2n, based on formula (13), we can get a similarity matrix S∈R 2n×2n , as shown in formula (14).

[0076]

[0077] In S, it is unnecessary to calculate the similarity with itself, and it will interfere with the contrastive learning task. Therefore, the self-similarity value is set to a negative number.

[0078] sim(h i ,h i )=-∞ (15)

[0079] Contrastive learning can make samples of the same category closer and samples of different categories as far apart as possible, effectively improving the classification ability of the model. However, if only the same and different categories are considered without considering the specific relationship between categories, the model may treat all different samples equally and ignore the different degrees of correlation that may exist between them. This approach limits the model's meticulousness in distinguishing different heterogeneous samples and also weakens the model's ability to handle complex classification tasks, especially in the case of behavioral classification in the legal field. Harmful behavior is generally judged based on a certain case category, such as marriage, contract, etc. It is unreasonable to ignore the case category and directly judge the polarity of the behavior.

[0080] To alleviate this problem, in addition to similarity calculation, the present invention also introduces the feature of label distance to quantify and utilize the correlation between categories. The core idea of ​​this method is that the similarity or correlation between different categories can be represented by the "distance" between them. Categories with similar concepts or attributes are considered to be closer, while categories with greater differences are farther. This label distance matrix allows the model to consider not only whether samples belong to the same category, but also the specific relationship between the categories of samples when performing contrastive learning.

[0081] Specifically, this method completes the construction of the label distance matrix based on the label matrix. The label distance can be calculated by formula (16).

[0082]

[0083] L∈R N×N represents the label matrix, N represents the number of labels, y i represents the one-hot label representation of the i-th sample, d ij Represents the label distance between the i-th sample and the j-th sample.

[0084] The construction of label matrix can be divided into two methods: static method and dynamic method. The static method directly obtains the label matrix through manual design, while the dynamic method is based on the confusion matrix. The construction process is shown in Algorithm 1.

[0085]

[0086] The dynamic label matrix obtains the confusion matrix through the output and label of the model, and then calculates the distance representation of the label matrix L based on the confusion matrix. The update of L should be performed after the model begins to stabilize, otherwise the update of the label matrix is ​​useless.

[0087] After obtaining the label matrix L, the label distance matrix can be calculated as shown in Formula 17.

[0088]

[0089] LD∈R 2m×2m is the label distance matrix, m is the number of original batch samples, d ij is the distance between label i and label j in the label matrix, y i is the one-hot representation of the label of sample i.

[0090] Finally, the score of the contrastive learning auxiliary task of this batch of data is calculated according to formula (18).

[0091] CL=S⊙LD (18)

[0092] ⊙ represents the dot product of matrix elements. CL is the score result of the contrastive learning task.

[0093] The model of the present invention mainly includes two outputs in the training stage: classification output and contrastive learning output. Therefore, the loss mainly includes two items, as shown in Formula 19.

[0094] L=λL CL +(1-λ)L C (19)

[0095] λ is a hyperparameter, L CL is the contrastive learning loss, L C is the classification loss.

[0096] For the polarity classification task, cross entropy is used as the loss function, as shown in formula (20).

[0097]

[0098] Where m is the number of samples, n is the number of labels, and p is the number of ij is the score of sample i under label j, y ij is the label of sample i at label j.

[0099] For contrastive learning tasks, the training goal is to make the distance between positive examples closer and the distance between negative examples farther. The cross entropy function cannot adapt well to the training goal. KL divergence is a way to compare the similarity between two distributions. The calculation is shown in formula (21).

[0100]

[0101] p i and q i Represents two distributions at i, when L KL The smaller it is, the more similar the two distributions are.

[0102] Based on this property of KL divergence, the loss function of the contrastive learning task is shown in formula (22).

[0103]

[0104] z ij and q ij The calculation of is shown in formula (24) and formula (23).

[0105]

[0106] 2m represents the number of samples, CL represents the score output of the contrastive learning task, and d ij Represents the label distance between label i and label j.

Claims

1. A behavioral polarity classification algorithm based on CLCA-CGCN, characterized by The following steps are involved: Step 1: The overall algorithm uses the sentence pair input method of the PLM model to make the model pay more attention to the description related to the target label. Using this feature, a method of splicing the original text with prompt words is designed to guide the model to pay attention to the part related to the current behavior. Combined with the special attention mechanism proposed in this invention, the word embedding representation of the key elements of the behavior can be directly completed, and the information related to the behavior polarity in the word embedding is enhanced. Step 2: When designing the contextual attention module, the PLM model is often used to encode the input sequence by weighting and obtaining new results based on its relevance, which only considers positive information and ignores negative information. In some situations where positive and negative information need to be clearly distinguished, its word embedding representation has defects, especially in the problem of behavior polarity classification, where the word embedding representation of behavior needs to integrate relevant polarity information. Based on this problem, this module proposes an improved contextual attention, which considers both the polarity of attention and the contextual representation problem. Step 3: There are two key tasks in designing the CGCN module: sampling and segmenting the behavior graph; adding random jump connections. The purpose of sampling the graph is to reduce the size of the gradient when the model aggregates at a single node. When the number of aggregated nodes decreases, the problem of information compression will also be alleviated, and it can also help the model learn the subspace characteristics of the behavior graph. Step 4: When designing the contrastive learning module, the purpose is to help the model distinguish the polarity of behaviors that are difficult to judge as an auxiliary task. The basic principle is to compare and analyze the output results of behavior polarity classification through the similarity information of labels and behaviors, so as to help the model learn the similarity characteristics between behaviors and the similarity information of behavior polarity categories. An improved strategy is also proposed, which performs a dropout operation on the embedded representation within the batch to obtain the disturbed positive samples, increase the comparison samples within a single batch, and improve the versatility and diversity of the model of the present invention. Step 5: When designing the training loss function, for contrastive learning tasks, the training goal is to make the distance between positive examples closer and the distance between negative examples farther. The cross entropy function cannot adapt well to the training goal. KL divergence is a way to compare the similarity between two distributions. Step 6: Connect the above modules, and the overall model adopts the loss function proposed in step 5. The data set is input into the algorithm to obtain the result.

2. The method according to claim 1, characterized in that: The contextual attention module in step 2 adopts a special attention mechanism to directly complete the word embedding representation of the key elements of the behavior, and the information related to the behavior polarity in the word embedding is enhanced as a solution.

3. The method according to claim 1, characterized in that: The module in step 3 adopts sampling segmentation of the behavior graph; adding random skip connections.

4. The method according to claim 1, characterized in that: In step 4, the output results of the behavior polarity classification are compared and analyzed through the similarity information of labels and behaviors, which helps the model learn the similarity features between behaviors and the similarity information of behavior polarity categories.

5. The method according to claim 1, characterized in that: The cross entropy function in step 5 is not well adapted to the training objective. KL divergence is a way to compare the similarity between two distributions.

Citation Information

Patent Citations

  • Chinese attribute level sentiment analysis method, system and equipment based on multi-task learning and medium

    CN116029305A

  • Graph attention web text classification model fusing ternary loss and label embedding

    CN116089618A

  • Methods and apparatuses for training service model and determining text classification category

    US11216620B1