A hierarchical multi-task individuality detection method based on a large language model

Abstract

The application provides a layered multi-task personality detection method based on a large language model. The method first encodes each historical post of a user independently by using a large language model to retain fine-grained personality clues. Secondly, a gating layered context encoder is designed to avoid semantic oversmoothing problems in long text processing through adaptive fusion of local and global information. Finally, a "decoupling-interaction" reasoning paradigm is introduced. Four dimensions of the Myers-Briggs Type Indicator are decoupled through dimensional attention pooling, and the structured dependency relationship between the dimensions is explicitly expressed by using a label graph convolution network. In addition, a class-balanced weighted loss, R-Drop consistency regularization and supervised contrastive learning are combined for joint optimization, which significantly improves the generalization ability and prediction performance of the model in the personality detection task.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a hierarchical multi-task personality detection method based on a large language model. Background Technology

[0002] Personality analysis aims to automatically infer a user's personality traits, such as the Myers-Briggs type index, by analyzing their posts on social media. Current technologies can be broadly categorized into three phases: early statistical methods based on psycholinguistic lexicons (such as LIWC) and traditional machine learning models (such as SVM); subsequent methods based on deep learning (such as CNN and RNN) and graph neural networks, attempting to capture deep semantics and discontinuous semantic dependencies in text; and the recently emerging Large Language Model (LLM) approach. However, existing methods still have the following shortcomings: First, semantic smoothing is problematic. Current mainstream methods, especially those based on LLM, typically concatenate a user's lengthy posting history into a single long document for overall encoding. This approach leads to sparse but crucial personality cues being overwhelmed by frequent, noisy daily content, resulting in overly homogenized user representations and weakening the reliability of personality inferences.

[0003] Second, there is insufficient modeling of structural dependencies. Myers-Briggs type index prediction is a multi-dimensional task that requires joint inference of four binary classification dimensions. Each dimension is supported by partially different linguistic evidence, but there are statistical correlations between the dimensions. Most existing methods either train independent classifiers for each dimension or rely on implicit parameter sharing in multi-task settings, lacking explicit modeling of inter-dimensional dependencies and failing to effectively utilize complementary signals across dimensions. Summary of the Invention

[0004] The purpose of this invention is to provide a hierarchical multi-task personality detection method based on a large language model, which improves the accuracy, robustness and interpretability of personality detection through fine-grained semantic modeling and structured dimensional reasoning.

[0005] To achieve the above objectives, the present invention provides the following solution: A hierarchical multi-task personality detection method based on a large language model includes the following steps: Retrieve the posting sequence based on the user's posting history; The posting sequence is independently encoded using a pre-trained large language model to obtain the post embedding sequence; Context awareness is obtained by associating and fusing context features in the embedded sequence of posts using a gated hierarchical context encoder. Parallel attention pooling is applied to context-aware data to obtain dimension-specific representations; The final prediction result is obtained by performing dimensional reasoning and information aggregation on dimensional-specific representations through a specific network.

[0006] Optionally, the posting sequence is independently encoded using a pre-trained large language model to obtain the post embedding sequence, including: The hidden states of posting sequences are extracted using a pre-trained large language model; Perform layer-dimensional mean pooling on the hidden states to obtain the first sequence; The first sequence is masked by attention masking and mean pooling to obtain the second sequence, and the second sequence is then integrated into the post embedding sequence.

[0007] Optionally, contextual feature association and feature fusion are performed on the post embedding sequence using a gated hierarchical context encoder to obtain context-aware features, including: Global context features are obtained by capturing the temporal dependencies and contextual evolution of post embedding sequences through the transformer of a gated hierarchical context encoder. The fusion weights of global context features and post embedding sequences are calculated using a gating mechanism; Based on fusion weights, the global context features are fused with the post embedding sequence through the gating aggregation layer of the gating hierarchical context encoder to obtain context awareness.

[0008] Optionally, parallel attention pooling is performed on the context-aware representation to obtain a dimension-specific representation, including: Context awareness is decoupled into Myers-Briggs type indicator dimensions through parallel attention pooling layers; Myers-Briggs type indicator dimensions include: I / E, S / N, T / F, and J / P; Attention weights for the Myers-Briggs type index dimension are calculated using a feedforward network. The Myers-Briggs type index dimensions are weighted and summed based on attention weights to obtain a dimension-specific representation.

[0009] Optionally, dimensional reasoning and information aggregation are performed on dimension-specific representations using a specific network to obtain the final prediction result, including: Stack dimension-specific representations into an input matrix; The statistical correlation between the dimensions of the input matrix is ​​captured by the learnable matrix to obtain the updated dimension representation; the diagonal elements of the learnable matrix are used to retain information of its own dimension, and the off-diagonal elements of the learnable matrix are used to learn the information propagation strength between different dimensions. The classification head aggregates information from the updated dimensional representation to obtain the final prediction result.

[0010] Optionally, a specific network can be optimized using a joint loss function, which includes: class-balanced weighted cross-entropy loss, consistency regularization loss, and supervised contrastive learning loss.

[0011] A hierarchical multi-task personality detection system based on a large language model includes: The data acquisition module is used to obtain the posting sequence based on the user's posting history. The feature extraction module is used to independently encode the posting sequence using a pre-trained large language model to obtain the post embedding sequence. The hierarchical context encoding module is used to perform context feature association and feature fusion on the post embedding sequence through a gated hierarchical context encoder to obtain context awareness. The dimension decoupling module is used to perform parallel attention pooling on context-aware data to obtain dimension-specific representations. The graph interaction reasoning module is used to perform dimensional reasoning and information aggregation on dimensional-specific representations through a specific network to obtain the final prediction result.

[0012] According to specific embodiments provided by the present invention, the following technical effects are disclosed: The hierarchical multi-task personality detection method based on a large language model provided by the present invention includes: obtaining a posting sequence based on the user's posting history; independently encoding the posting sequence using a pre-trained large language model to obtain a post embedding sequence; performing context feature association and feature fusion on the post embedding sequence using a gated hierarchical context encoder to obtain context awareness; performing parallel attention pooling on the context awareness to obtain a dimension-specific representation; and performing dimension inference and information aggregation on the dimension-specific representation using a specific network to obtain the final prediction result. This method, through independent encoding, adaptive fusion of local and global information, and feature decoupling, preserves fine-grained personality cues, avoids the semantic oversmoothing problem in long text processing, and explicitly defines the structured dependencies between dimensions, thereby improving the collaborative effect of multi-task learning. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of the hierarchical multi-task personality detection method based on a large language model according to the present invention; Figure 2 This is a comparison diagram of the individual detection principle of this invention and existing detection methods; Figure 3 This is a schematic diagram of the overall framework architecture of an embodiment of the present invention; Figure 4 This is a visual diagram illustrating the dimensional attention weights in an embodiment of the present invention; Figure 5 This is a heatmap of the adjacency matrix learned by the label graph convolutional network in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] like Figure 1 and Figure 2 As shown, a hierarchical multi-task personality detection method based on a large language model includes the following steps: Step 100: Obtain the posting sequence based on the user's posting history; Step 200: Independently encode the posting sequence using a pre-trained large language model to obtain the post embedding sequence; Step 300: Perform context feature association and feature fusion on the post embedding sequence using a gated hierarchical context encoder to obtain context-aware data; Step 400: Perform parallel attention pooling on the context-aware representation to obtain a dimension-specific representation; Step 500: Perform dimensional reasoning and information aggregation on the dimension-specific representation through a specific network to obtain the final prediction result.

[0018] In the specific implementation process, step 100 applies to each user. Collect its posting history sorted by time. ,in This refers to the number of posts sampled (in this embodiment, the Kaggle dataset is used). =50, taking from the Pandora dataset =500).

[0019] In the specific implementation process, step 200 employs an independent post-by-post encoding strategy to avoid semantic oversmoothing caused by long text concatenation. Specifically, a pre-trained large language model, Meta-Llama-3-8B-Instruct, is used as the frozen feature extractor for each post. Input it into the LLM and extract the last... =5 layers of hidden state ,in For sequence length, =4096 represents the hidden layer dimension. Subsequently, mean pooling is performed across the layer dimensions to obtain a unified word-level sequence representation. The expression is: Then, perform mask mean pooling along the sequence dimension to compress and obtain a fixed-dimensional post representation. The expression is: ,in This is an attention mask used to exclude filler terms. Ultimately, each user is represented as a sequence of post embeddings. .

[0020] like Figure 3 As shown, in the specific implementation process, step 300 models the post embedding sequence using a gated hierarchical context encoder. The gated hierarchical context encoder includes a transformer layer and a gated aggregation layer. Specifically, the post embedding sequence is first... Input a transformer (using Transformer Encoder in this example) to capture temporal dependencies and contextual evolution between posts, and output global contextual features. Next, the original local features... With global context features The data are fed into the gated aggregation layer, and the fusion weights are calculated through a learnable gating mechanism. ,in For the Sigmoid function, Given learnable parameters, the resulting context-aware representation is: This allows the model to dynamically balance local details and global context based on the content of posts, effectively alleviating the problem of secondary semantic oversmoothing caused by deep networks.

[0021] In the specific implementation process, step 400 achieves feature decoupling through dimensional attention pooling in order to perceive from a shared context. Each dimension's unique evidence is extracted, and context awareness is decoupled into Myers-Briggs type indicator dimensions through four parallel attention pooling layers; the Myers-Briggs type indicator dimensions include four dimensions: I / E, S / N, T / F, and J / P. For example... Figure 4 As shown, for the first Each dimension, specifically represented The calculation formula is: Among them, attention weight Through a dimension The relevant feedforward network calculation is used to measure the first... One post on inference dimensions The importance of this approach is that each dimension can "retrieve" the most relevant information from the post history, reducing feature entanglement between dimensions.

[0022] In the specific implementation process, step 500 uses a label graph convolutional network to achieve structured dimensional interactive reasoning. Specifically, as follows: Figure 5 As shown, the four-dimensional specific representations obtained in step 400 are stacked into a matrix. Take a graph convolutional network as input and output the updated dimensionality representation. The expression is: ,in It is a learnable adjacency matrix, where the diagonal elements are used to preserve information about its own dimension, and the off-diagonal elements are used to learn information from other dimensions. To dimension The information propagation intensity is determined to explicitly capture the statistical correlation between dimensions (e.g., the potential association between N and P dimensions). Finally, a classification head (using a combination of linear layers and Softmax in this embodiment) is used to obtain the final prediction for each dimension. .

[0023] Furthermore, to further improve the model's generalization ability and robustness, this embodiment uses a joint loss function to optimize the label map convolutional network. The function expression is as follows: ,in It is a class-balanced weighted cross-entropy loss, which addresses the class imbalance problem in individual datasets by assigning higher weights to a minority of classes. It is the R-Drop consistency regularization loss, which improves the stability of the model by minimizing the difference in output distribution for the same input under two different Dropouts; It is a supervised contrastive learning loss that learns more discriminative user representations by bringing samples with the same composite label closer together in the representation space (in this embodiment, this is manifested as combining four binary labels into a 16-category label) and pushing samples with different labels further apart.

[0024] This invention also provides a hierarchical multi-task personality detection system based on a large language model, comprising: The data acquisition module is used to obtain the posting sequence based on the user's posting history. The feature extraction module is used to independently encode the posting sequence using a pre-trained large language model to obtain the post embedding sequence. The hierarchical context encoding module is used to perform context feature association and feature fusion on the post embedding sequence through a gated hierarchical context encoder to obtain context awareness. The dimension decoupling module is used to perform parallel attention pooling on context-aware data to obtain dimension-specific representations. The graph interaction reasoning module is used to perform dimensional reasoning and information aggregation on dimensional-specific representations through a specific network to obtain the final prediction result.

[0025] The beneficial effects of this invention are as follows: 1) In terms of feature extraction, the traditional long text splicing strategy was abandoned, and LLM was used to encode each post one by one, which effectively alleviated the semantic oversmoothing phenomenon and preserved the fine-grained clues that are crucial for personality detection. 2) In terms of sequence modeling, a gated hierarchical context encoder was designed. By fusing local and global information through an adaptive gating mechanism, the secondary oversmoothing that may be introduced by deep networks is avoided, thereby improving the accuracy of user modeling. 3) In terms of multi-task reasoning, a “decoupling-interaction” paradigm is proposed. First, dimensional attention pooling is used to separate dimensional-specific features (decoupling). Then, the structured dependencies between dimensions are explicitly modeled through a label graph convolutional network (interaction), thereby making full use of shared and unique information in multi-task learning. 4) In terms of model optimization, the model combines class-balanced weighted loss, R-Drop consistency regularization and supervised contrastive learning to comprehensively improve the model's generalization ability and robustness from three dimensions: data imbalance, model stability and representation discriminability.

[0026] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0027] Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A hierarchical multi-task personality detection method based on a large language model, characterized in that, Includes the following steps: Retrieve the posting sequence based on the user's posting history; The posting sequence is independently encoded using a pre-trained large language model to obtain the post embedding sequence; Context awareness is obtained by performing context feature association and feature fusion on the embedded sequence of posts using a gated hierarchical context encoder. Parallel attention pooling is applied to the context-aware representation to obtain a dimension-specific representation; The final prediction result is obtained by performing dimensional reasoning and information aggregation on the dimension-specific representation through a specific network.

2. The hierarchical multi-task personality detection method based on a large language model according to claim 1, characterized in that, The posting sequence is independently encoded using a pre-trained large language model to obtain a post embedding sequence, including: The hidden states of the posting sequence are extracted using the pre-trained large language model; The hidden states are subjected to layer-dimensional mean pooling to obtain the first sequence; The first sequence is masked by attention masking and mean pooling to obtain the second sequence, and the second sequence is then integrated into the post embedding sequence.

3. The hierarchical multi-task personality detection method based on a large language model according to claim 1, characterized in that, The post embedding sequence is subjected to context feature association and feature fusion using a gated hierarchical context encoder to obtain context-awareness, including: The transducer of the gated hierarchical context encoder captures the temporal dependency and contextual evolution of the post embedding sequence to obtain global contextual features; The fusion weights of the global context features and the post embedding sequence are calculated using a gating mechanism. Based on the fusion weights, the global context features and the post embedding sequence are fused through the gating aggregation layer of the gating hierarchical context encoder to obtain the context awareness.

4. The hierarchical multi-task personality detection method based on a large language model according to claim 1, characterized in that, Parallel attention pooling is performed on the context-aware representation to obtain a dimension-specific representation, including: The context awareness is decoupled into Myers-Briggs type index dimensions through parallel attention pooling layers; the Myers-Briggs type index dimensions include: I / E, S / N, T / F, and J / P; The attention weights of the Myers-Briggs type index dimensions are calculated using a feedforward network. The Myers-Briggs type index dimensions are weighted and summed according to the attention weights to obtain the dimension-specific representation.

5. The hierarchical multi-task personality detection method based on a large language model according to claim 1, characterized in that, By performing dimensional reasoning and information aggregation on the dimension-specific representation through a specific network, the final prediction result is obtained, including: Stack the dimension-specific representations into an input matrix; The statistical correlation between the dimensions of the input matrix is ​​captured by a learnable matrix to obtain an updated dimension representation; the diagonal elements of the learnable matrix are used to retain information about its own dimension, and the off-diagonal elements of the learnable matrix are used to learn the information propagation strength between different dimensions. The final prediction result is obtained by aggregating information from the updated dimension representation using a classification head.

6. The hierarchical multi-task personality detection method based on a large language model according to claim 1, characterized in that, The specific network is optimized using a joint loss function, which includes: class-balanced weighted cross-entropy loss, consistency regularization loss, and supervised contrastive learning loss.

7. A hierarchical multi-task personality detection system based on a large language model, characterized in that, include: The data acquisition module is used to obtain the posting sequence based on the user's posting history. The feature extraction module is used to independently encode the posting sequence using a pre-trained large language model to obtain the post embedding sequence; The hierarchical context encoding module is used to perform context feature association and feature fusion on the post embedding sequence through a gated hierarchical context encoder to obtain context awareness; The dimension decoupling module is used to perform parallel attention pooling on the context awareness to obtain a dimension-specific representation; The graph interaction reasoning module is used to perform dimensional reasoning and information aggregation on the dimension-specific representation through a specific network to obtain the final prediction result.

Images