A personality prediction method and social portrait generation method based on text data

Abstract

The present invention provides a personality prediction method and social portrait generation method based on text data, including collecting and cleaning post data published by users to generate post data, obtaining all post data of users to generate user data; LDA model identifies user data to obtain topic data and keyword data corresponding to the topic; MBTI model determines the mapping relationship between the four dimensions of MBTI based on keyword data to determine the four dimensional features of the user's MBTI, and predicts the user's personality characteristics based on the four dimensional features. The present invention can predict the personality characteristics of users based on the topics and keywords contained in the texts published by users, and then generate user portraits based on the personality characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of user portrait generation, and in particular to a personality prediction method and a social portrait generation method based on text data. Background Art

[0002] Latent Dirichlet Allocation (LDA) is a statistical model that is suitable for discovering implicit topic structures in document collections or corpora. LDA assumes that each document is a mixture of multiple topics, and each topic is defined by a probability distribution of a series of words, which makes it very suitable for processing text data and can automatically identify the potential topics behind documents without pre-specification. By identifying the distribution of topics in documents, LDA reveals deep connections between documents, helping to understand which topics co-occur and how they are related to each other in different documents.

[0003] MBTI (Myers-Briggs Type Indicator) is a personality classification tool based on Swiss psychologist Carl Jung's psychological type theory and further developed by American psychologist Isabel Briggs Myers and her mother, Katherine Cook Briggs. MBTI assesses an individual's personality type through four dimensions.

[0004] User portrait refers to a labeled user model abstracted from user attributes, user preferences, living habits, user behavior and other information. In layman's terms, it means labeling users, and labels are highly refined feature identifiers obtained through analysis of user information. Labeling can be used to describe users using some highly generalized and easy-to-understand features, which makes it easier for people to understand users and facilitates computer processing.

[0005] User portraits not only help companies better understand their target customers, but also support more effective market segmentation, product design, and personalized service provision. At the same time, user portraits can also be used in the field of public opinion to predict the public opinion reaction of specific groups with similar user portraits and issue early warnings. The personality characteristics of user portraits refer to the user's psychological and behavioral tendencies inferred from the user's behavior, preferences, interactions and other data. These characteristics are usually abstract, but can be reflected through the user's specific behavior and data.

[0006] With the popularity of social media, the acquisition and analysis of personal information has become increasingly important. MBTI personality type, as a popular psychological classification method, has been widely used in career counseling, team building and other fields. However, there is currently a lack of a system and method for applying MBTI personality type to social portrait analysis. Summary of the invention

[0007] In view of this, the problem to be solved by the present invention is to provide a personality prediction method and a social portrait generation method based on text data, which can predict the personality characteristics of users according to the topics and keywords contained in the texts posted by users, and then generate user portraits based on the personality characteristics.

[0008] To solve the above technical problems, the technical solution adopted by the present invention is: an MBTI prediction method based on text data, including collecting post data published by users and cleaning and processing to generate post data, obtaining all post data of users to generate user data;

[0009] The LDA model identifies user data to obtain topic data and keyword data corresponding to the topic;

[0010] The MBTI model determines the mapping relationship between the keyword data and the four dimensions of MBTI to determine the four dimensional characteristics of the user's MBTI. The NERIS model predicts the user's personality characteristics based on the four dimensional characteristics.

[0011] Furthermore, the cleaning process includes: post segmentation processing, removing meaningless words, extracting the stems of the posts based on the subject, predicate, object, verb, attributive, adverbial, complement and central word, deleting posts with abnormal stem descriptions or incomplete stem descriptions, and arranging the words in the posts in a specific format to generate post data.

[0012] Furthermore, each of the post data contains several topics.

[0013] Furthermore, the personality characteristics generation process includes: obtaining user behavior characteristics and four dimensional characteristics, and the NERIS model predicts the user's personality characteristics based on the user behavior data and the four dimensional characteristics;

[0014] The behavior characteristics include the posting time, posting frequency and posting reply frequency.

[0015] Furthermore, the MBTI model construction method is: obtaining the word multinomial distribution of personality words respectively Multinomial distribution of keywords The distribution of the subject's personality judgments ρ and the data of several individuals;

[0016] Get the personality characteristics bf_u of individual u and the corresponding topic distribution θu and a number of social posts to determine the topic of each social post zu,s ;

[0017] A social post s contains a number of words n. The judgment factor of word n ​​for the topic personality word is determined based on the personality judgment distribution ρ. xu,s,n , if the judgment factor xu,s,n= 0, if word n ​​is a personality word, then Sampling words corresponding to personality traits from the multinomial distribution of parameters wu,sn If the judgment factor xu,s,n = 1, if word n ​​is a topic word, then words corresponding to personality traits are sampled from a multinomial distribution with φzu.s as a parameter. wu,sn .

[0018] Furthermore, the social portrait includes a user portrait, and the user portrait generation method includes: obtaining the user's topic data including the type and quantity of topics to determine the user's preference for different topics, and generating the user portrait based on the user's topic preference and personality characteristics.

[0019] Furthermore, the social portrait includes a user group portrait, which obtains the topic preferences of several users, clusters the users according to the topic preferences to generate a user group, and obtains the personality characteristics of the users in the user group to generate a user group portrait.

[0020] The advantages and positive effects of the present invention are:

[0021] (1) The LDA model can automatically extract topic features from text data, avoiding the tedious process of manually designing features in traditional methods and improving the efficiency and accuracy of feature extraction. The LDA model can capture the deep semantic information in text data, reveal the potential structure and meaning behind the text through topic distribution, and improve the accuracy and reliability of prediction.

[0022] (2) Due to the adoption of the LDA topic model, the user feature prediction model of the present invention has strong interpretability, making it easier for relevant personnel and departments to understand and trust the prediction results of the model.

[0023] (3) Through the MBTI model, the correspondence between the text's topic features and keyword features and the user's personality traits is taken into account to improve the prediction accuracy of personality traits; by adding the correspondence between user behavior data, social network structure and personality traits to the MBTI model, the accuracy and stability of the prediction of user personality traits are further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0025] Figure 1 It is an overall flow chart of a personality prediction method based on text data of the present invention;

[0026] Figure 2It is the training flow chart of the MBTI model in a personality prediction method based on text data of the present invention. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used herein in the description of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0029] The present invention provides a personality prediction method and a social portrait generation method based on text data, as Figure 1 shown, including collecting the post data published by users and cleaning and processing it to generate post data, and obtaining all the post data of users to generate user data.

[0030] User data includes the text published or commented by users on social media. An embodiment of the present application is: for #a certain event# on Weibo, all the posts published by users under this event are obtained from the social media platform through the API interface.

[0031] The post data is the text of the posts published by users. Due to the high randomness of users' publishing or commenting on posts, some post texts contain unstructured descriptions or meaningless descriptions. Unstructured descriptions or meaningless descriptions are likely to lead to unclear semantic descriptions and affect the topic judgment of the LDA model.

[0032] To remove the influence of unstructured content on the subsequent judgment results, the post data is cleaned and processed. The process includes: performing word segmentation on the posts and removing meaningless conjunctions, extracting the stems of the posts based on the subject, predicate, object, verb, attributive, adverbial, complement, and headword, deleting the posts with abnormal or incomplete stem descriptions, and arranging the sentences in the posts in a specific format to generate post data.

[0033] The LDA model identifies the user data to obtain the topic data and the keyword data corresponding to the topic.

[0034] LDA assumes that each post is a mixture of multiple topics. Based on the LDA model, post data is identified to determine the topics and keywords contained in each post. Assume that the identification result of a post is: the post contains topics 1 and 2. Topic 1 corresponds to keyword 11 with a weight of 0.2, keyword 12 with a weight of 0.3, keyword 13 with a weight of 0.1, and keyword 14 with a weight of 0.5; Topic 2 corresponds to keyword 21 with a weight of 0.5, keyword 22 with a weight of 0.25, and keyword 23 with a weight of 0.25.

[0035] The weight indicates the frequency of occurrence of a keyword in a post. The sum of the weights of all keywords in each topic is 1, and: if a post contains a topic, it must contain all the keywords corresponding to the topic, and the frequency of occurrence is also corresponding.

[0036] The MBTI model determines the mapping relationship between the keyword data and the four dimensions of MBTI to determine the four dimensional characteristics of the user's MBTI and predicts the user's personality characteristics based on the four dimensional characteristics.

[0037] The four dimensions of MBTI include the source of motivation, the way of receiving information, the way of decision-making, and the attitude towards uncertainty. The NERIS model evaluates the individual's personality type through the four dimensions, and: when the four dimensions of the user are determined, the user's personality can be predicted.

[0038] Sources of drive: Extraversion (E) and Introversion (I). This dimension describes how individuals gain energy and motivation. Extraverts tend to gain energy from interacting with others, while introverts prefer to gain energy from their own inner world and reflection.

[0039] How to receive information: Sensing (S) and Intuition (N). This dimension involves how individuals perceive and receive information. Sensing people focus on practical experience and specific details, while intuitive people focus more on abstract concepts and future possibilities.

[0040] Decision-making style: Thinking (T) and Feeling (F). This dimension examines the individual's preferences when making decisions. Thinking people tend to rely on logic and objective analysis, while Feeling people place more emphasis on personal values ​​and care for others.

[0041] Attitude towards uncertainty: Judging (J) and Perceiving (P). This dimension describes how individuals deal with uncertainty in the external world and the future. Judging people prefer a planned, organized lifestyle, while Perceiving people are more adaptable to flexibility and openness to change.

[0042] The prediction process of the MBTI model is: obtain the keyword data of the subject, which includes personality words and subject words. Based on the correspondence between personality words and dimensional features, and the correspondence between subject words and dimensional features, use the same method to determine the correspondence between the keyword data and dimensional features of all the user's subjects, jointly determine the dimensional features of the user's MBTI, and determine the user's personality features based on the user's four dimensional features.

[0043] An embodiment of the present application is: when determining the source of driving force based on the keywords of the subject, the number of keywords corresponding to extroversion (E) is greater than the number of keywords corresponding to introversion (I), and the source of the user's driving force is determined to be extroverted.

[0044] The training process of the MBTI model includes: In order to be able to build a relatively prepared personality matrix, it is necessary to find a suitable data set. Select the MBTI professional data set of personality trait vectors. The data in the professional data set corresponds to 16 personalities, including many personality-based words and sentences corresponding to the personality, describing the person's preferences and behaviors. The data set contains more than 8,600 data, each of which represents an individual, with 2 columns of data: Type column (the four-dimensional characteristic letters of the individual's MBTI code), posts column (recording part of each of the 50 things recently posted by the individual).

[0045] First, we need to clean the posts part of each row of data and delete useless URL information, stop words and other useless information. Secondly, we use scikit-learn's TfidfVectorizer to fit all the data in the posts column and generate a global TF-IDF matrix. Finally, TfidfVectorizer generates a corresponding vector for each row of data.

[0046] MBTI model training mainly draws on the idea of ​​Twitter-LDA model. The model makes the following assumption: when creating a social post, users do not directly select a random topic from multiple topics, but first make a judgment to choose whether to use a topic word or a personality word. The meaning of personality words is driven by the user's personality factors. Therefore, when using the LDA model to identify posts, not only can the topic data be obtained through the model, but also the distribution model of personality words corresponding to different personality traits can be obtained through the labeled user personality data.

[0047] like Figure 2 As shown in the figure, the MBTI model is a supervised probabilistic generation model. When a large number of individual parameters are used for training, the individual MBTI dimension characteristics need to be provided for training. The generation process of the probabilistic model of the MBTI topic model is as follows:

[0048] 1. Get the multinomial distribution of personality words

[0049] 2. Get the multinomial distribution of words in the topic

[0050] 3. Obtain the distribution ρ of the subject personality judgment.

[0051] 4. For each individual u=1,2,3,…,U;

[0052] 1) Select the dimensional feature bf_u of individual u;

[0053] 2) Obtain the topic distribution corresponding to the dimensional features of individual u θu

[0054] 3) For each social post s=1,2,3,…,Nu corresponding to the dimension feature of individual u

[0055] A. From θu Get the topic of social posts for the parameter zu,s

[0056] B. For each word n=1,2,3,…, Nus

[0057] a) Determine the judgment factor of word n ​​for the topic personality word based on the personality judgment distribution ρ xu,s,n

[0058] b) If the judgment factor xu,s,n = 0, if word n ​​is a personality word, then Sampling from a multinomial distribution of parameters wu,sn , wu,sn It corresponds to the personality bf_u; if the judgment factor xu,s,n = 1, if word n ​​is a topic word, then it is sampled from a multinomial distribution with φzu.s as parameter. wu,sn , wu,sn It corresponds to the character bf_u.

[0059] Through a large amount of individual parameter training, the correspondence between personality words and themes and personality traits is determined so that the MBTI model can determine the user's personality traits based on keyword data.

[0060] The MBTI model construction method includes: obtaining individual behavior data and dimension characteristics bf_u, the behavior data includes several behavior characteristics, and determining the corresponding relationship between behavior characteristics and personality characteristics through a large amount of data training;

[0061] Behavioral characteristics include the time when users post posts, the frequency of posting posts, and the frequency of replying to posts. Determining the user's personality characteristics based on the user's behavioral characteristics and keyword data can improve the accuracy of predicting the user's personality characteristics.

[0062] A social portrait generation method includes a social portrait including a user portrait, and the user portrait generation method includes: obtaining user's topic data including topic types and quantities to determine the user's preference for different topics, and generating a user portrait based on the user's topic preferences and personality characteristics. The topic data includes personality characteristics and preferences for different topics, and the higher the frequency of the same topic in the topic data, the more preferred the topic is.

[0063] Social portraits include user group portraits, obtaining the topic preferences of several users, clustering users based on the topic preferences to generate user groups, and obtaining the personality characteristics of users in the user group to generate user group portraits.

[0064] The embodiments of the present invention are described in detail above, but the contents described are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of this patent.

Claims

1. An MBTI prediction method based on text data, characterized in that: Including collecting and cleaning the post data published by users to generate post data, and obtaining all post data of users to generate user data; The LDA model identifies user data to obtain topic data and keyword data corresponding to the topic; The MBTI model determines the mapping relationship between the keyword data and the four dimensions of MBTI to determine the four dimensional characteristics of the user's MBTI and predicts the user's personality characteristics based on the four dimensional characteristics.

2. The MBTI prediction method based on text data according to claim 1, characterized in that: The cleaning process includes: post segmentation processing, removing meaningless words, extracting the stems of posts based on the subject, predicate, object, verb, attributive, adverbial, complement and central word, deleting posts with abnormal stem descriptions or incomplete stem descriptions, and arranging the words in the posts in a specific format to generate post data.

3. The MBTI prediction method based on text data according to claim 1, characterized in that: Each of the post data includes a number of topics.

4. The MBTI prediction method based on text data according to claim 1, characterized in that: The MBTI model construction method: obtaining the word multinomial distribution of personality words respectively Multinomial distribution of keywords The distribution of the subject's personality judgments ρ and the data of several individuals; Get the personality characteristics bf_u of individual u and the corresponding topic distribution θu and a number of social posts to determine the topic of each social post zu,s ; A social post s contains a number of words n. The judgment factor of word n ​​for the topic personality word is determined based on the personality judgment distribution ρ. xu,s,n , if the judgment factor xu,s,n = 0, if word n ​​is a personality word, then Sampling words corresponding to personality traits from the multinomial distribution of parameters wu,sn If the judgment factor xu,s,n = 1, if word n ​​is a topic word, then words corresponding to personality traits are sampled from a multinomial distribution with φzu.s as a parameter. wu,sn .

5. The MBTI prediction method based on text data according to claim 4, characterized in that: The MBTI model construction method includes: obtaining individual behavior data and personality characteristics bf_u, wherein the behavior data includes a number of behavior characteristics, and determining the corresponding relationship between the behavior characteristics and the personality characteristics; The behavior characteristics include the posting period, posting frequency and posting reply frequency.

6. A method for generating a social portrait, according to the MBTI prediction method based on text data according to any one of claims 1 to 5, characterized in that: The social portrait includes a user portrait, and the user portrait generation method includes: obtaining the user's topic data including the type and quantity of topics to determine the user's preference for different topics, and generating the user portrait based on the user's topic preference and personality characteristics.

7. A social portrait generation method according to claim 6, characterized in that: Social portraits include user group portraits, obtaining the topic preferences of several users, clustering users based on the topic preferences to generate user groups, and obtaining the personality characteristics of users in the user group to generate user group portraits.

Images