Large five personality automatic identification method, apparatus and device, and storage medium

Abstract

The invention relates to the technical field of personality recognition, and discloses a large five personality automatic recognition method and device, equipment and a storage medium. The method comprises the steps that user personality vocabularies extracted from a personality dictionary are introduced, a large five-personality automatic recognition model fusing multi-modal information is designed and achieved, the personality dictionary, post texts of users and image features are fully utilized by the model, and the performance of the model on five-personality dimensions can be accurately predicted. Experimental results show that the average accuracy of the model in a big five-personality prediction task reaches 80% or above, and the accuracy and effect of big five-personality prediction are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of personality recognition technology, and in particular to a method, device, equipment and storage medium for automatic Big Five personality recognition. Background Art

[0002] Personality is generally defined as the sum of an individual's internal tendencies and psychological characteristics in terms of behavior, mannerisms, thinking, and emotions. It reflects a person's cognitive abilities, behavioral motivations, and attitudes throughout their life. Personality traits have a significant impact on our lives, often playing a significant role in personal well-being, the outcomes of social systems, interpersonal relationships, and personal choices. Therefore, the ability to measure individual personality has extremely important practical applications.

[0003] The Big Five personality theory, also known as the Five-Factor Model, was proposed by American psychologists Paul Costa and Robert MacRae. The theory categorizes personality traits into five dimensions: neuroticism, extraversion, openness to experience, agreeableness, and conscientiousness. Neuroticism involves the ability to regulate emotions and cope with stress. Those high in neuroticism experience greater mood swings, while those low in neuroticism are calmer and more stable. Extraversion involves sociability and extroversion. Those high in extraversion are talkative, sociable, and confident. Openness involves curiosity, imagination, and creativity. Those high in openness are open to new knowledge and experiences. Agreeableness involves cooperation and trustworthiness. Those high in agreeableness are easygoing and cooperative, while those low in agreeableness are more aloof and uncooperative. Conscientiousness involves goal setting, planning, and self-discipline. Those high in conscientiousness are organized, reliable, and self-disciplined, while those low in conscientiousness are more spontaneous and impulsive.

[0004] Generally speaking, everyone possesses a unique set of psychological traits, with personality traits varying across every dimension. Directly analyzing these unique traits and determining personality tendencies is often difficult. In traditional psychology, personality traits are typically measured using fixed personality inventories. Participants answer basic questions related to each personality dimension, and the questionnaire assesses the responses and calculates a baseline score for each dimension. While this method is relatively simple, it is time-consuming and labor-intensive, and its results are easily influenced by subjective factors such as the participant's current mental state and emotions. Consequently, many researchers have begun focusing on automated personality assessment, aiming to design an efficient and simple automated personality assessment method.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a method, device, equipment and storage medium for automatic recognition of the Big Five personality, aiming to solve the problem that existing personality recognition methods are time-consuming and labor-intensive and are easily affected by the current psychological state and subjective emotional factors of the test subject, resulting in inaccurate recognition.

[0007] The first aspect of the present invention provides a method for automatic identification of Big Five personality, comprising: obtaining a user's post avatar, original post text data, and image data within the post; pre-building a personality dictionary; using the personality dictionary to extract personality vocabulary from all the user's original post text data, and constructing a reorganized sentence containing the user's personalized features based on the personality vocabulary; building an initial Big Five personality automatic identification model, the initial Big Five personality automatic identification model comprising a Bert module, a ResNet module with a fully connected layer removed, a fusion module, and a classification module, the Bert module being used to receive the original post text data and the reorganized sentence and output a text feature representation, the ResNet module being used to receive the post avatar and the image data within the post and output an image feature representation, and the fusion module being used to The text feature representation and the image feature representation are fused and a comprehensive feature representation including text and image is output; the classification module is used to receive the comprehensive feature representation and output the Big Five personality category; a personality score is calculated based on a personality dictionary, and the Big Five personality of each user is labeled according to the personality score, and the Big Five personality category labels are divided into low category and high category; the initial Big Five personality automatic recognition model is trained based on the original post text data, reorganized sentences, post avatars and image data of the labeled user to obtain a trained Big Five personality automatic recognition model; the original post text data, reorganized sentences, post avatars and image data of the user to be identified are input into the trained Big Five personality automatic recognition model, and the user's Big Five personality category is output.

[0008] Optionally, in a first implementation method of the first aspect of the present invention, obtaining the user's post avatar, original post text data, and image data in the post includes the steps of: pre-setting a data structure to be crawled, the data structure including the post ID, posting time, data source, post type, post content, whether the post has pictures, picture storage location, whether the post has videos, video storage location, video category, whether the post has an avatar, avatar storage location, and the first 10 comments on the post; simulating real users, by continuously sending requests to the social platform, continuously crawling the user social data of the social platform according to a preset data structure to obtain initial post data; pre-processing the initial post data to obtain the user's post avatar, original post text data, and image data in the post.

[0009] Optionally, in a second implementation of the first aspect of the present invention, a personality dictionary is pre-constructed, including the steps of: obtaining seed words of personality vocabulary from classic scales, research literature and personality dictionary thesaurus, and establishing a seed word dictionary after deduplication; annotating the dimensions, aspects, word positivity, whether it is a language dimension and whether it is a behavioral dimension of the personality vocabulary in the seed word dictionary to form an initial dictionary, wherein the initial dictionary is classified into a total of 5 dimensions, and each dimension is divided into 6 aspects, among which the behavioral dimension is defined as whether the vocabulary can reflect the behavioral characteristics of an individual, and the language dimension is defined as whether the vocabulary can reflect the language characteristics of an individual; word positivity is defined as whether the vocabulary has a positive emotional color, a negative emotional color or a neutral emotional color when describing a person's characteristics; the results of the standard Big Five personality questionnaire are compared with the conformity of the dimensions and aspects of the personality vocabulary in the initial dictionary to verify the reliability of the dimension and aspect annotations of the personality vocabulary in the initial dictionary, thereby constructing a personality dictionary.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the personality dictionary is used to extract personality vocabulary from all original post text data of the user, and a reorganized sentence containing the user's personalized characteristics is constructed based on the personality vocabulary, including the steps of: based on the personality dictionary, extracting the personality vocabulary of each personality dimension used in all original post text data of each user, and calculating the frequency of use of the user's personality vocabulary under each personality dimension; based on the personality vocabulary and the frequency of use of the personality vocabulary, with the help of Pormote technology, a vocabulary-driven personalized information enhancement method is adopted to construct the personality vocabulary into a reorganized sentence containing the user's personalized characteristics.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, the Bert module is used to receive original post text data and reorganized sentences and output text feature representation, including the steps of: performing word segmentation on the original post text data, converting it into a corresponding vocabulary index, and adding special tags to obtain preprocessed text data; performing format conversion processing on the preprocessed text data to obtain pre-input post text data, and the format conversion processing includes input ID, tag attention mask and tag type ID; inputting the pre-input post text data into the Bert module, and the Bert module first converts the pre-input post text data into a fixed-length vector representation, and the vector representation is processed by multiple encoding layers using a self-attention mechanism to output a context-related vector representation of each word; inputting the reorganized sentence into the Bert module, and the Bert module is used to capture the implicit information and complex language patterns of the reorganized sentence, and output a language feature vector representation, and the text feature representation is composed of the context-related vector representation and the language feature vector representation.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, the fusion module is used to fuse the text feature representation and the image feature representation and output a comprehensive feature representation containing text and image, including the steps of: inputting the text feature representation and the image feature representation into the fusion module, mapping the feature representations of the two modalities to the same feature space so as to compare them at the same semantic level; within the fusion module, calculating the correlation score between the image feature representation and each lexical feature in the text feature representation; assigning a weight to each word in the text according to the correlation score, and performing weighted summation on the text feature representation according to the weight to obtain a weighted text feature representation; fusing the weighted text feature representation with the image feature representation, outputting a comprehensive feature representation containing text and image, and using it as the final feature representation for personality prediction.

[0013] Optionally, in a seventh implementation of the first aspect of the present invention, a personality score is calculated based on a personality dictionary, and a Big Five personality is labeled for each user based on the personality score, wherein the category labels of the Big Five personality are divided into low and high categories, including the steps of: matching the user's original post text data with the personality dictionary, and counting the number of times the user's vocabulary appears in each personality dimension; calculating the average number of sentence personality vocabulary of the user in each personality dimension based on the number of sentences contained in the user's original post text data and the number of times the user's vocabulary appears in each personality dimension; counting the total number of vocabulary in each dimension in the personality dictionary, and calculating the proportion of vocabulary in each dimension in the total vocabulary; calculating a personality proportion score based on the average number of sentence personality vocabulary in each personality dimension and the proportion of vocabulary in the dimension in the total vocabulary; normalizing the obtained personality proportion score to obtain the user's personality score in each personality dimension, comparing the personality score with the classification threshold of the corresponding personality dimension, and dividing each personality dimension of the user into low and high categories based on the comparison result, thereby labeling each user's Big Five personality.

[0014] The second aspect of the present invention provides a Big Five personality automatic recognition device, comprising: an acquisition module for acquiring a user's post avatar, original post text data, and image data within the post; a dictionary construction module for pre-building a personality dictionary; a sentence construction module for extracting personality vocabulary from all the user's original post text data using the personality dictionary, and constructing a reorganized sentence containing the user's personalized features based on the personality vocabulary; a model construction module for constructing an initial Big Five personality automatic recognition model, the initial Big Five personality automatic recognition model comprising a Bert module, a ResNet module with the fully connected layer removed, a fusion module, and a classification module, the Bert module being used to receive the original post text data and the reorganized sentence and output a text feature representation, the ResNet module being used to receive the post avatar and the image data within the post and output an image feature representation, and the The fusion module is used to fuse the text feature representation and the image feature representation and output a comprehensive feature representation containing text and image; the classification module is used to receive the comprehensive feature representation and output the Big Five personality category; the labeling module is used to calculate the personality score based on the personality dictionary and label the Big Five personality of each user according to the personality score, and the Big Five personality category labels are divided into low category and high category; the training module is used to train the initial Big Five personality automatic recognition model based on the original post text data, reorganized sentences, post avatars and image data in the posts of the labeled users to obtain a trained Big Five personality automatic recognition model; the output module is used to input the original post text data, reorganized sentences, post avatars and image data in the posts of the user to be identified into the trained Big Five personality automatic recognition model and output the user's Big Five personality category.

[0015] A third aspect of the present invention provides a Big Five personality automatic recognition device, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor calls the computer-readable instructions in the memory to enable the Big Five personality automatic recognition device to perform each step of the Big Five personality automatic recognition method described above.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having computer-readable instructions stored therein. When the computer-readable instructions are executed on a computer, the computer is caused to execute the steps of the above-described method for automatic identification of the Big Five personality.

[0017] Beneficial Effects: This paper designs and implements a multimodal Big Five personality automatic recognition model by incorporating user personality vocabulary extracted from a personality dictionary. This model leverages the personality dictionary, along with user text and image features, to accurately predict their performance across the five personality dimensions. Experimental results show that the model achieves an average accuracy of over 80% in Big Five personality prediction tasks, significantly improving the accuracy and effectiveness of Big Five personality prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the Big Five personality automatic recognition method provided by an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the Big Five personality automatic recognition method provided by an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the Big Five personality automatic recognition device provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the Big Five personality automatic recognition device provided by the present invention. DETAILED DESCRIPTION

[0022] The embodiments of the present invention provide a method, apparatus, device and storage medium for automatic identification of the Big Five personality. The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0023] See also Figure 1 , Figure 1 The flowchart of the method for automatic identification of Big Five personality provided by the present invention is shown in the figure, which includes the following steps:

[0024] S10, obtaining the user's post avatar, original post text data, and image data within the post;

[0025] Specifically, this embodiment uses a crawler method to collect data and pre-sets the data structure to be crawled. The data structure includes the post ID, posting time, data source, post type, post content, whether the post contains pictures, picture storage location, whether the post contains videos, video storage location, video category, whether the post contains avatars, avatar storage location, and the first 10 comments on the post. Post types include original posts and forwarded posts. As an example, a programming language (such as Python) can be used to define a class (or dictionary structure) that contains all the data fields to be crawled. An instance of this class is created in the crawler program to store the data of each post.

[0026] This embodiment continuously sends requests to the social platform, and continuously crawls the social data of users of the social platform according to the preset data structure to obtain the initial post data; as an example, the API documentation of the target social platform is studied to understand how to send requests, receive responses, and how to handle paging and authentication; according to the API requirements, the parameters required for the request (such as user ID, post ID, paging parameters, etc.) and header information (such as authentication token, User-Agent, etc.) are set; an HTTP library (such as Python's requests library) is used to send a GET or POST request to the social platform API; response data is received and parsed, usually in JSON format; a loop structure is used to continuously send requests and parse responses until the stop condition is met (such as a sufficient number of posts are crawled or the API limit is reached); the crawled data is stored in a previously defined instantiated object, or saved to a storage medium such as a file or database. This embodiment simulates real user behavior, reduces the risk of being banned by the social platform, and uses an automated crawling process to improve the efficiency of data collection.

[0027] The initial post data is preprocessed to obtain the user's post avatar, original post text data, and in-post image data. For example, Selenium technology is used to simulate real users by continuously sending requests to social platforms such as Twitter, WeChat, and Sina to continuously crawl social data of platform users. The crawled data is cleaned and invalid and abnormal data is deleted during the crawling process.

[0028] Data preprocessing techniques typically convert unstructured data into a structured format, reducing document size and facilitating feature selection. It's a crucial step before model training. Generally speaking, the natural language users use on social media is informal, often containing spelling errors and inappropriate hashtags. Specifically, the text data we crawled this time included English, Traditional Chinese, Thai, Spanish, and other languages. Therefore, before feeding this natural language into model training, we needed to preprocess it. In this project, the preprocessing steps included: converting all text in different languages to Simplified Chinese; removing stop characters and non-Simplified Chinese characters; removing HTML, URLs, and other content from the text; removing the # symbol, but retaining the text associated with the tag; removing special symbols such as @, &, and *; removing all hashtags; and removing line breaks and invalid whitespace.

[0029] In deep learning tasks, the balance of data is crucial to the performance of the model. Unbalanced sample distribution in the data set may cause the model to favor the majority samples, thereby affecting the prediction accuracy of the minority samples. This imbalance is particularly evident in classification tasks, where the model may overestimate the accuracy of the majority class and ignore the performance of the minority class. Therefore, the problem of data balance needs to be effectively addressed before model training to ensure that the model can learn and predict each category fairly. In social media, since each user posts at different frequencies, the number of their posts is also different within a certain period of time. The number of user posts we crawled this time is more than 1,000, and the least is only dozens. Given that the distribution of personality dimensions of each person is inconsistent, this will cause our sample data to be extremely unbalanced. To solve this problem, this embodiment uses data sampling technology to balance the data set. For users with a large number of samples, we used the undersampling method to randomly extract 100 data from their overall sample to form the user's personality sample; for users with a small number of samples, we used the oversampling method to increase the number of minority samples and duplicated the existing minority class samples so that their number also reached 100.

[0030] Finally, more than 35,000 pieces of data published by 96 users were collected. This embodiment effectively captures and pre-processes the social data of social platform users, providing strong support for subsequent data analysis and mining.

[0031] S20, pre-construct personality dictionary;

[0032] This embodiment first obtains seed words of personality vocabulary from classic scales, research literature and personality dictionary, and establishes a seed word dictionary after deduplication; then the personality vocabulary in the seed word dictionary is labeled with dimensions, aspects, word positiveness and negativeness, whether it is a language dimension and whether it is a behavior dimension to form an initial dictionary. The initial dictionary is classified into a total of 5 dimensions, each of which is divided into 6 aspects. Extraversion includes 6 aspects: enthusiasm, sociability, self-affirmation, activity, excitement pursuit and positive emotions; Agreeableness includes 6 aspects: trust, honesty, altruism, obedience, modesty and warmth; Openness includes 6 aspects: fantasy, art, emotion, action, concept and speculation; Responsibility includes 6 aspects: competence , organization, responsibility, ambition, self-discipline and prudence; neuroticism includes anxiety, depression, anger and hostility, self-consciousness, impulsiveness and vulnerability; the behavioral dimension is defined as whether the vocabulary can reflect the behavioral characteristics of an individual, and the language dimension is defined as whether the vocabulary can reflect the language characteristics of an individual; the positivity and negativity of a vocabulary is defined as whether the vocabulary has positive, negative or neutral emotional colors when describing a person's characteristics; finally, the results of the standard Big Five personality questionnaire are compared with the conformity of the dimensions and aspects of the personality vocabulary in the initial dictionary to verify the reliability of the dimension and aspect annotations of the personality vocabulary in the initial dictionary, thereby constructing a personality dictionary.

[0033] During the comparison process, the results of the subject's Big Five personality dimensions or aspects (gold standard) are compared with the dimensions or aspects of the vocabulary scored by the subject to see if they are consistent (for example, among subjects with medium or high Big Five personality agreeableness, if their average score for positive words also marked as agreeable in the initial personality dictionary is three points or above, or the average score for negative words is below three points, it is considered consistent). The comparison results will be used to calculate the hit rate of personality vocabulary annotation using the confusion matrix method and the rank equality method (that is, the degree of consistency between the respondent's Big Five personality dimensions and the dimensions of the vocabulary annotation; the higher the consistency, the higher the hit rate).

[0034] S30: extracting personality words from all original post text data of the user using the personality dictionary, and constructing a reorganized sentence containing the user's personalized characteristics based on the personality words.

[0035] In this embodiment, based on the personality dictionary, personality vocabulary for each personality dimension used in all original post text data of each user is extracted, and the frequency of use of the personality vocabulary for each personality dimension is calculated. Based on the personality vocabulary and its frequency of use, Pormote technology is used to construct the personality vocabulary into restructured sentences that contain the user's personalized characteristics using a vocabulary-driven personalized information enhancement method. Specifically, users' language usage preferences are closely related to the Big Five personality dimensions. Traditional features extracted through the introduction of dictionaries can provide additional information to the model, thereby improving predictive capabilities. However, these features are often independent and lack contextual connections, which may limit the model's understanding of complex language patterns. In light of this, this embodiment designs a method for extracting personalized features of users based on their language behavior using a personality dictionary. This embodiment extracts personality vocabulary from each user's original post text data for each personality dimension and calculates the frequency of use of personality vocabulary for each dimension, i.e., the proportion of the number of times a personality vocabulary appears in each dimension relative to the total vocabulary. Because frequency reflects a user's preference or tendency toward specific personality traits and helps to gain a deeper understanding of their personality, this personality vocabulary usage information is then used, using a vocabulary-driven personalized information enhancement method with the help of promote technology, to construct sentences that contain the user's personalized features. The Pormote technology in this embodiment is a vocabulary-based personalized information recommendation or enhancement method that leverages the semantics and context of vocabulary to generate personalized content. Based on the frequency of personality vocabulary use, this embodiment constructs a representation for each user that includes feature vectors for each personality dimension. Pormote technology then uses this user feature vector to select vocabulary from the vocabulary library that matches the user's personality, constructing a reconstructed sentence that contains the user's personalized features. Based on this, these reconstructed sentences can then be further learned using the BERT module to capture implicit information and complex language patterns. This approach not only improves the model's sensitivity to personalized language behavior, but also enhances its understanding of user characteristics, thereby achieving higher accuracy and robustness in predicting the Big Five personality dimensions.

[0036] S40. Construct an initial Big Five personality automatic recognition model, which includes a Bert module, a ResNet module without a fully connected layer, a fusion module, and a classification module. The Bert module is used to receive original post text data and reorganized sentences and output a text feature representation. The ResNet module is used to receive post avatars and image data within the post and output an image feature representation. The fusion module is used to fuse the text feature representation and the image feature representation and output a comprehensive feature representation containing text and image. The classification module is used to receive the comprehensive feature representation and output a Big Five personality category.

[0037] Specifically, if Figure 2 As shown, the initial Big Five personality automatic recognition model constructed in this embodiment includes a Bert module, a ResNet module with the fully connected layer removed, a fusion module, and a classification module. The Bert module is a pre-trained language model based on Transformer. It processes the input text through a self-attention mechanism to capture semantic information in the text and establish global dependencies between different words. This embodiment uses the Bert module to perform two different operations on the user's post text. On the one hand, it directly encodes the post text and extracts features from it to train the model. On the other hand, it uses a personality dictionary to extract personality vocabulary from all the user's documents to generate personalized features based on the user's language behavior to assist in model training.

[0038] In the process of encoding the post text, this embodiment first performs word segmentation on the original post text data, converts it into a corresponding vocabulary index, and adds special tags [CLS] (used to represent the summary of the entire sentence) and [SEP] (used to separate different sentences) to obtain preprocessed text data; before entering the BERT module, the preprocessed text data needs to be converted into an input format acceptable to BERT, including input ID (the index of each word in the BERT vocabulary), attention mask (marking which words need attention calculation and which are filler words), and tag type ID (used to distinguish different parts of multiple sentences); the pre-input post text data is input into the BERT module, and the BERT module converts the pre-input post text data into a fixed-length vector representation. The vector representation is processed by multiple encoding layers using a self-attention mechanism and outputs a context-related vector representation (contextual embedding) of each word; in BERT, the first position in the hidden state of the last layer is usually a special tag [CLS] for classification tasks, which is regarded as a summary representation of the entire text. We use the hidden state corresponding to this tag as the global feature representation of the text and pass it to downstream modules for further processing and fusion with other modal features. This enables the model to capture complex information in the text about the user's psychological state, emotional tendencies, and other complex information, providing strong support for the prediction of personality traits.

[0039] Since the features extracted by traditional dictionaries are often independent and lack contextual associations, the model's understanding of complex language patterns may be limited. Based on this, this embodiment first extracts personality vocabulary from all the user's original post text data based on the personality dictionary, and constructs a reorganized sentence containing the user's personalized features based on the personality vocabulary. The reorganized sentence is then input into the Bert module. The Bert module is used to capture the implicit information and complex language patterns of the reorganized sentence and output a language feature vector representation. The text feature representation is composed of the context-related vector representation and the language feature vector representation. This embodiment uses the Bert module to further learn these reorganized sentences to capture the implicit information and complex language patterns therein. This method not only improves the model's sensitivity to personalized language behavior, but also enhances its understanding of user characteristics, thereby achieving higher accuracy and robustness in the prediction of the Big Five personality dimensions.

[0040] Since the user's profile picture and the content of the shared images may reflect their interests, values or lifestyles, they can provide valuable clues for personality prediction. Therefore, this embodiment uses a pre-trained ResNet50 module to process the user's post profile picture and post image data. ResNet50 is a classic convolutional neural network (CNN) that has been pre-trained on large-scale image datasets (such as ImageNet) and has good feature extraction capabilities. ResNet50 consists of multiple convolutional layers, pooling layers and residual blocks. The residual blocks can effectively solve the gradient vanishing problem in deep networks, so that it can still maintain good feature learning capabilities in deep networks. To make visual features more suitable for fusion with features from other modalities (such as text), this embodiment removes the last fully connected layer (i.e., the classification layer) of ResNet50. This means that the ResNet50 module does not directly classify the image, but instead outputs an embedded representation of the image. This embedded representation is high-dimensional and retains rich visual information, including multiple clues such as the user's emotional state, aesthetic preferences, and personality traits. These clues are passed to downstream modules as visual feature outputs for combination with other modal information, thereby providing assistance for final personality prediction.

[0041] In this embodiment, after obtaining the text feature representation and the image feature representation, the text feature representation and the image feature representation are input into the fusion module, and the feature representations of the two modalities are mapped to the same feature space so that they can be compared at the same semantic level. In this step, a fully connected layer (FC) or a transformation matrix is used to map the feature representations of text and image to the same dimension and feature space. This can be achieved by training a multi-layer perceptron (MLP), in which the input is the feature vector of text and image, and the output is the mapped feature vector. The advantage of this is that the features of text and image can be compared and fused on the same dimension, which improves the generalization ability of the model because the mapping process can learn the potential relationship between different modalities.

[0042] Then, within the fusion module, the correlation score between the image feature representation and each word feature in the text feature representation is calculated. In this step, the dot product or cosine similarity between the image feature vector and each word feature vector in the text is calculated to form a correlation matrix; then a self-attention mechanism (such as the multi-head attention in the Transformer) is used to more finely calculate the correlation score, taking into account the contextual relationship between different words. The correlation score can reflect the degree of association between the image and each word in the text, which is helpful for subsequent weighted processing. The attention mechanism can capture the contextual information within the text and improve the model's understanding of complex texts.

[0043] Based on the relevance score, a weight is assigned to each word in the text, and the text feature representation is weighted and summed according to the weight to obtain a weighted text feature representation. This weighted summation step can highlight the features of words that are highly relevant to the image and reduce the influence of irrelevant words, making the text feature representation more focused on content related to the image, thereby improving the model's performance in personality prediction tasks.

[0044] The weighted text feature representation and the image feature representation are fused together to output a comprehensive feature representation encompassing both text and image, which serves as the final feature representation for personality prediction. In this step, the weighted text feature representation and image feature representation are concatenated or weighted summed to form a comprehensive feature representation. This comprehensive feature representation serves as input to the personality prediction model, and the classification module outputs the personality prediction result. Through these specific implementation steps, we can achieve the fusion of text and image feature representations and achieve better performance in personality prediction tasks. Each step has its own unique benefits, and together they contribute to improved model performance.

[0045] S50, calculating a personality score based on the personality dictionary, and labeling each user's Big Five personality according to the personality score, wherein the Big Five personality category labels are divided into low category and high category;

[0046] In this embodiment, the user's original post text data is matched with a personality dictionary, and the number of occurrences of the user's vocabulary on each personality dimension is counted. Based on the number of sentences contained in the user's original post text data and the number of occurrences of the user's vocabulary on each personality dimension, the average number of sentence personality vocabulary on each personality dimension is calculated, using the following formula: average sentence personality vocabulary = number of personality dimension vocabulary occurrences / number of text sentences. The total number of vocabulary on each dimension in the personality dictionary is counted, and the proportion of vocabulary on each dimension in the total vocabulary is calculated. Based on the average number of sentence personality vocabulary on each personality dimension and the proportion of vocabulary on that dimension in the total vocabulary, a personality proportion score is calculated, using the following formula: personality proportion score = average sentence personality vocabulary × personality dimension vocabulary proportion. The obtained personality proportion score is normalized to a score between 0 and 1, using the following formula: normalized personality score = personality proportion score / maximum possible score for that dimension (usually 1 or the maximum value of the vocabulary proportion for that dimension). The personality score of the user on each personality dimension is then compared with the classification threshold of the corresponding personality dimension. Based on the comparison result, each personality dimension of the user is classified into low and high categories, thereby labeling each user's Big Five personality. For example, each character is classified on each dimension according to the set classification threshold (such as conscientiousness, openness, agreeableness, and neuroticism are scored as high when the score is above 40%, and extraversion is scored as high when the score is above 60%.

[0047] This embodiment uses a personality dictionary-based approach to obtain the Big Five personality labels of the 96 target persons. The results are shown in Table 1.

[0048] Table 1 Personality dimension classification labels

[0049] Rigor Openness Extraversion Agreeableness neuroticism high 58 37 57 38 38 Low 38 59 39 58 58

[0050] S60: Training the initial Big Five personality automatic recognition model based on the tagged user's original post text data, reorganized sentences, post avatars, and post image data to obtain a trained Big Five personality automatic recognition model.

[0051] Specifically, AdamW, as an improved optimization algorithm, not only retains the efficient learning ability of Adam, but also effectively suppresses overfitting by adding weight decay terms, thereby improving the generalization performance of the model. In this embodiment, AdamW is used as the optimizer of the initial Big Five personality automatic recognition model, and its initial learning rate is set to 2e -5, and set the weight decay to 0.02. Based on the optimizer, this embodiment introduces a cosine annealing learning rate scheduler, and the initial learning rate of the scheduler is 2e -5 , the minimum learning rate is 1e -5 , and warm up within the first epoch of training. This scheduling strategy helps stabilize model learning in the early stages of training and gradually reduces the learning rate as training progresses to ensure smooth convergence of the model.

[0052] In terms of loss function, considering that the Big Five personality labels have category differences in some dimensions, this embodiment adopts a variant of the cross-entropy loss function (Focal Loss). During model training, Focal Loss assigns higher weights to samples that are difficult to classify while reducing the contribution of samples that are easy to classify to the loss. It is usually used to solve the problem of category imbalance. The formula of Focal Loss is as follows:

[0053] F_loss = α × (1-pt) γ ×BCE_loss#(1)

[0054] Here, pt is the probability output by the model, that is, the model's confidence in the prediction of the current sample. α is used to adjust the weight imbalance between positive and negative samples. By setting an appropriate α, the model can pay more attention to minority class samples. γ is used to reduce the weight of easy-to-classify samples, allowing the model to focus more on difficult-to-classify samples. When γ is large, the focal loss for easy-to-classify samples becomes very small. BCE_loss is the cross entropy loss.

[0055] For model performance evaluation, we selected accuracy and F1 score as the main evaluation indicators. Accuracy reflects the overall prediction effect of the model, while F1 score can better balance precision and recall in the case of class imbalance, and is particularly suitable for the evaluation of minority class samples. Accuracy is shown in Formula 2. , where TP (True Positives) is the number of samples correctly predicted as positive, TN (True Negatives) is the number of samples correctly predicted as negative, FP (False Positives) is the number of samples incorrectly predicted as positive, and FN (False Negatives) is the number of samples incorrectly predicted as negative.

[0056] The calculation method of F1 score is shown in formula 3.

[0057] Precision refers to the proportion of samples that are actually positive among the samples predicted by the model to be positive. The calculation formula is:

[0058] Recall (recall rate) indicates the proportion of samples that are correctly predicted as positive among all samples that are actually positive. The calculation formula is:

[0059] In order to better verify the performance of the model, this embodiment divides the user's original post text data, reorganized sentences, post avatars, and image data in the post into training sets, validation sets, and test sets in a ratio of 8:1:1 to the number of users, where all samples of each user are in the same data set. We set the training batch size (Batch Size) to 8, ensuring that while ensuring a sufficient amount of training samples, each training can fully utilize computing resources. The entire training process was carried out in a GPU4090 environment and lasted for 10 epochs to ensure that the model fully learned on the training set and was able to obtain good generalization ability, and finally obtained a trained Big Five personality automatic recognition model.

[0060] S70: Input the original post text data, reorganized sentences, post avatar, and image data of the user to be identified into the trained Big Five personality automatic recognition model, and output the user's Big Five personality category.

[0061] In this embodiment, after obtaining a trained Big Five personality automatic recognition model, the original post text data of the user to be identified is processed in the same manner as the training data to obtain a reconstructed sentence. The original post text data, reconstructed sentence, post avatar, and image data within the post are then input into the trained Big Five personality automatic recognition model to output the user's Big Five personality category. Our model achieved excellent results in Big Five personality prediction, with accuracy rates exceeding 75% for each personality dimension and an average accuracy rate of 84.68%. The specific results are shown in Table 2.

[0062] Table 2 Prediction results of the Big Five personality dimensions

[0063] Rigor Openness Extraversion Agreeableness neuroticism average Accuracy 0.7679 0.7513 0.9509 0.7730 0.9362 0.8468 F1 0.7758 0.8085 0.9358 0.7418 0.9141 0.8358

[0064] As can be seen from the results in the table above, the Accuracy and F1 values for each personality dimension are relatively close. This indicates that our trained Big Five personality automatic recognition model performs relatively evenly across the categories of each personality dimension and has excellent recognition capabilities for all five personality dimensions.

[0065] This paper designs and implements a multimodal Big Five personality automatic recognition model by incorporating user personality vocabulary extracted from a personality dictionary. This model leverages the personality dictionary, along with user text and image features, to accurately predict their performance across the five personality dimensions. Experimental results show that the model achieves an average accuracy of over 80% in Big Five personality prediction tasks, significantly improving the accuracy and effectiveness of Big Five personality prediction.

[0066] The above describes the Big Five personality automatic recognition method in the embodiment of the present invention. The following describes the Big Five personality automatic recognition device in the embodiment of the present invention. Figure 3 In one embodiment of the present invention, an automatic Big Five personality recognition device includes:

[0067] Acquisition module 10, for acquiring the user's post avatar, original post text data and image data in the post;

[0068] A dictionary building module 20 is used to pre-build a personality dictionary;

[0069] A sentence construction module 30 is used to extract personality words from all original post text data of the user using the personality dictionary, and construct a reorganized sentence containing the user's personalized characteristics based on the personality words;

[0070] A model construction module 40 is configured to construct an initial Big Five personality automatic recognition model, the initial Big Five personality automatic recognition model comprising a Bert module, a ResNet module with the fully connected layer removed, a fusion module, and a classification module. The Bert module is configured to receive original post text data and reorganized sentences and output a text feature representation. The ResNet module is configured to receive post avatars and image data within a post and output an image feature representation. The fusion module is configured to fuse the text feature representation and the image feature representation and output a comprehensive feature representation comprising text and image. The classification module is configured to receive the comprehensive feature representation and output a Big Five personality category.

[0071] A labeling module 50 is used to calculate a personality score based on the personality dictionary and label each user's Big Five personality according to the personality score, where the Big Five personality category labels are divided into low and high categories;

[0072] A training module 60 is configured to train the initial Big Five personality automatic recognition model based on the original post text data, reorganized sentences, post avatars, and image data of the tagged user to obtain a trained Big Five personality automatic recognition model;

[0073] The output module 70 is used to input the original post text data, reorganized sentences, post avatar and image data of the user to be identified into the trained Big Five personality automatic recognition model and output the user's Big Five personality category.

[0074] Based on the same idea as the method in the above embodiment, the device provided in the present application can implement the method in the above embodiment. For the convenience of explanation, the structural diagram of the device embodiment only shows the parts related to the embodiment of the present application. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer modules than shown in the figure, or a combination of certain modules, or different module arrangements.

[0075] Figure 3 The Big Five personality automatic recognition device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The Big Five personality automatic recognition device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0076] Figure 4 : This is a schematic diagram of the structure of a Big Five personality automatic recognition device provided by an embodiment of the present invention. The Big Five personality automatic recognition device 100 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 11 (for example, one or more processors) and memory 12, and one or more storage media 13 (for example, one or more mass storage devices) for storing application programs 133 or data 132. The memory 12 and storage medium 13 can be temporary storage or permanent storage. The program stored in the storage medium 13 may include one or more modules (not shown in the figure), each of which may include a series of instruction operations in the Big Five personality automatic recognition device 100. Furthermore, the processor 11 can be configured to communicate with the storage medium 13 to execute a series of instruction operations in the storage medium 13 on the Big Five personality automatic recognition device 100.

[0077] The Big Five personality automatic recognition device 100 may also include one or more power supplies 14, one or more wired or wireless network interfaces 15, one or more input and output interfaces 16, and / or one or more operating systems 131, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 4 The device structure shown does not constitute a limitation on the Big Five personality automatic recognition device 100 , and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0078] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to execute the steps of the Big Five personality automatic identification method.

[0079] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0080] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0081] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic identification of Big Five personality, characterized by: Including steps: Get the user's post avatar, original post text data, and image data within the post; Pre-built personality dictionary; Extracting personality words from all original post text data of the user using the personality dictionary, and constructing a reorganized sentence containing the user's personalized characteristics based on the personality words; Constructing an initial Big Five personality automatic recognition model, the initial Big Five personality automatic recognition model includes a Bert module, a ResNet module with the fully connected layer removed, a fusion module, and a classification module, the Bert module is used to receive original post text data and reorganized sentences and output text feature representations, the ResNet module is used to receive post avatars and image data within the post and output image feature representations, the fusion module is used to fuse the text feature representations and the image feature representations and output a comprehensive feature representation containing text and images; the classification module is used to receive the comprehensive feature representations and output the Big Five personality categories; Calculating a personality score based on a personality dictionary, and labeling each user's Big Five personality according to the personality score, wherein the Big Five personality category labels are divided into low and high categories; Training the initial Big Five personality automatic recognition model based on the tagged user's original post text data, reorganized sentences, post avatars, and post image data to obtain a trained Big Five personality automatic recognition model; The original post text data, reorganized sentences, post avatars and image data of the user to be identified are input into the trained Big Five personality automatic recognition model, and the user's Big Five personality category is output.

2. The Big Five personality automatic recognition method according to claim 1, characterized in that: Obtaining the user's post avatar, original post text data, and image data within the post includes the following steps: Pre-set the data structure to be crawled, which includes the post ID, posting time, data source, post type, post content, whether the post has pictures, picture storage location, whether the post has videos, video storage location, video category, whether the post has avatars, avatar storage location, and the first 10 comments on the post; Simulate real users and continuously send requests to the social platform to continuously crawl the user social data of the social platform according to the preset data structure to obtain the initial post data; The initial post data is pre-processed to obtain the user's post avatar, original post text data, and image data within the post.

3. The Big Five personality automatic recognition method according to claim 1, characterized in that: Pre-build personality dictionary, including the following steps: Obtain seed words for personality vocabulary from classic scales, research literature, and personality dictionary lexicon, and establish a seed word dictionary after removing duplicates; The personality words in the seed word dictionary are labeled with respect to their dimensions, aspects, word positivity, whether they are language dimensions, and whether they are behavioral dimensions to form an initial dictionary. The initial dictionary is classified into five dimensions in total, and each dimension is divided into six aspects. The behavioral dimension is defined as whether the word can reflect the behavioral characteristics of an individual, and the language dimension is defined as whether the word can reflect the language characteristics of an individual. The word positivity is defined as whether the word has a positive, negative, or neutral emotional color when describing a person's characteristics. The results of the standard Big Five personality questionnaire were compared with the degree of conformity of the dimensions and aspects of the personality vocabulary in the initial dictionary to verify the reliability of the dimensions and aspects of the personality vocabulary in the initial dictionary, thereby constructing a personality dictionary.

4. The Big Five personality automatic recognition method according to claim 1, characterized in that: The personality dictionary is used to extract personality words from all original post text data of the user, and a reorganized sentence containing the user's personalized characteristics is constructed based on the personality words, including the steps of: Based on the personality dictionary, extract the personality vocabulary of each personality dimension used in all original post text data of each user, and calculate the frequency of use of the personality vocabulary of the user under each personality dimension; According to the personality vocabulary and the usage frequency of the personality vocabulary, with the help of Pormote technology, a vocabulary-driven personalized information enhancement method is adopted to construct the personality vocabulary into a reorganized sentence containing the user's personalized characteristics.

5. The Big Five personality automatic recognition method according to claim 1, characterized in that: The Bert module is used to receive original post text data and reorganized sentences and output text feature representation, including the following steps: Segmenting the original post text data, converting it into corresponding vocabulary indexes, and adding special tags to obtain preprocessed text data; Performing format conversion processing on the pre-processed text data to obtain pre-input post text data, the format conversion processing including inputting ID, marking attention mask and marking type ID; Inputting the pre-input post text data into the Bert module, the Bert module first converts the pre-input post text data into a fixed-length vector representation, the vector representation is processed through multiple encoding layers using a self-attention mechanism, and outputs a context-related vector representation of each word; The reorganized sentence is input into the Bert module, which is used to capture the implicit information and complex language patterns of the reorganized sentence and output a language feature vector representation. The text feature representation is composed of the context-related vector representation and the language feature vector representation.

6. The Big Five personality automatic recognition method according to claim 1, characterized in that: The fusion module is used to fuse the text feature representation and the image feature representation and output a comprehensive feature representation containing text and image, including the steps of: Inputting the text feature representation and the image feature representation into a fusion module, mapping the feature representations of the two modalities into the same feature space so as to enable comparison at the same semantic level; Within the fusion module, calculating a correlation score between the image feature representation and each lexical feature in the text feature representation; Assigning a weight to each word in the text according to the relevance score, and performing weighted summation on the text feature representation according to the weight to obtain a weighted text feature representation; The weighted text feature representation and the image feature representation are fused to output a comprehensive feature representation containing text and image, which is used as the final feature representation for personality prediction.

7. The Big Five personality automatic recognition method according to claim 1, characterized in that: Calculating a personality score based on a personality dictionary and labeling each user's Big Five personality according to the personality score, wherein the Big Five personality category labels are divided into low category and high category, including the steps of: Match the user's original post text data with the personality dictionary and count the number of times the user's vocabulary appears on each personality dimension; According to the number of sentences contained in the user's original post text data and the number of times the user's vocabulary appears on each personality dimension, the average number of sentence personality vocabulary of the user on each personality dimension is calculated; Count the total number of words in each dimension of the personality dictionary and calculate the proportion of words in each dimension in the total vocabulary; The personality proportion score is calculated based on the average number of personality words in each personality dimension and the proportion of the vocabulary in the total vocabulary of that dimension. The obtained personality proportion scores are normalized to obtain the user's personality score on each personality dimension. The personality score is compared with the classification threshold of the corresponding personality dimension. Based on the comparison results, the user's personality dimensions are divided into low and high categories to achieve labeling of each user's Big Five personality.

8. A Big Five personality automatic recognition device, characterized by: include: The acquisition module is used to obtain the user's post avatar, original post text data and image data in the post; Dictionary building module, used to pre-build personality dictionaries; A sentence construction module, configured to extract personality words from all original post text data of the user using the personality dictionary, and construct a reorganized sentence containing the user's personalized characteristics based on the personality words; A model construction module is used to construct an initial Big Five personality automatic recognition model, the initial Big Five personality automatic recognition model comprising a Bert module, a ResNet module with the fully connected layer removed, a fusion module, and a classification module. The Bert module is used to receive original post text data and reorganized sentences and output a text feature representation. The ResNet module is used to receive post avatars and image data within the post and output an image feature representation. The fusion module is used to fuse the text feature representation and the image feature representation and output a comprehensive feature representation containing text and image. The classification module is used to receive the comprehensive feature representation and output a Big Five personality category. A labeling module is used to calculate a personality score based on a personality dictionary and label each user's Big Five personality according to the personality score, where the Big Five personality category labels are divided into low and high categories; A training module is used to train the initial Big Five personality automatic recognition model based on the original post text data, reorganized sentences, post avatars and image data of the tagged users to obtain a trained Big Five personality automatic recognition model; The output module is used to input the original post text data, reorganized sentences, post avatars and image data of the user to be identified into the trained Big Five personality automatic recognition model and output the user's Big Five personality category.

9. A Big Five personality automatic recognition device, characterized in that: comprising a memory and at least one processor, wherein the memory has computer-readable instructions stored therein; The at least one processor calls the computer-readable instructions in the memory to execute each step of the Big Five personality automatic identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer-readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the steps of the method for automatic Big Five personality recognition according to any one of claims 1 to 7 are implemented.

Images