A processing method for the detection model of depressed users on social media

Abstract

The present application provides a processing method for a detection model of depressed users on social media, which is used to extract user behavior features related to depressed users by combining text features and topic features. The model trained thereby has significantly improved pertinence for depressed users and has higher detection accuracy. The method includes: obtaining UGC on a social media platform; extracting text feature information from UGC; extracting topic feature information from UGC; fusing topic feature information with text feature information to obtain fused feature information; extracting user behavior feature information from the fused feature information; using user behavior feature information as sample data, training an initialized BiLSTM model to identify depressed users in input data.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a processing method for a detection model of depressed users on social media. Background Art

[0002] Mental illness has become a serious problem for people all over the world, and depression, a more serious mental health problem than any other mental illness, has become the leading cause of disability worldwide. In the early stages of depression, people with depression usually do not actively seek help from psychiatrists, so early detection of depression is a daunting challenge.

[0003] The rapid development of social media has enabled users to express their feelings, opinions and ideas on social platforms in their own way, which has led to social media data gradually becoming an effective tool for monitoring various public health issues such as depression. In recent years, the method of using social media data to detect depressed users has attracted widespread attention from natural language processing (NLP) researchers. These studies use user generated content (UGC) on social media (such as Weibo or Twitter) to detect depressed users and achieve good performance.

[0004] The depressed user detection task for social media platforms focuses on modeling user characteristics along the timeline, and uses the collected text features, visual features, and behavioral features to build a multimodal system to identify users with depressive tendencies. This type of depressed user detection algorithm can automatically detect a large number of potential depression patients in social media, which can help prevent major casualties in the future and provide medical assistance to these potential patients.

[0005] During the research process of existing related technologies, the inventors found that the detection effect of existing depression user detection technology is unstable, which means that the existing depression user detection model selects features with low correlation or ignores certain important features during processing, resulting in low detection accuracy of the final model. Summary of the invention

[0006] The present application provides a processing method for a detection model of depressed users on social media, which is used to extract user behavior features related to depressed users by combining text features and topic features. The model trained in this way has significantly improved targeting for depressed users and has higher detection accuracy.

[0007] In a first aspect, the present application provides a method for processing a detection model of depressed users on social media, the method comprising:

[0008] Capture UGC on social media platforms;

[0009] Extract text feature information from UGC;

[0010] Extract topic feature information from UGC;

[0011] Fusing the topic feature information with the text feature information to obtain fused feature information;

[0012] Extracting user behavior feature information from fused feature information;

[0013] Using user behavior feature information as sample data, the initialized BiLSTM model is trained to identify depressed users in the input data.

[0014] In combination with the first aspect of the present application, in a first possible implementation manner of the first aspect of the present application, extracting text feature information in UGC includes:

[0015] Get contextual embeddings of short text in UGC using the XLNet model X , as text feature information.

[0016] In combination with the first possible implementation of the first aspect of the present application, in the second possible implementation of the first aspect of the present application, the XLNet model is specifically a model based on the transformer-XL architecture and combined with permutation language modeling. The XLNet model arranges and combines each token to generate all possible word orders. During the training process of the XLNet model, the transformer-XL architecture first stores the hidden state generated by the previous segment, and then uses the cached hidden state as context when calculating the next segment.

[0017] In combination with the first aspect of the present application, in a third possible implementation manner of the first aspect of the present application, extracting topic feature information in UGC includes:

[0018] The topic feature information in UGC is extracted through the Conditional Random Field regularizedTopic Model (CRFTM). During the extraction process, CRFTM merges the short texts in UGC into pseudo documents with longer text length, and then extracts the topic feature information from the pseudo documents by combining word vectors and word-related knowledge.

[0019] In combination with the first aspect of the present application, in a fourth possible implementation manner of the first aspect of the present application, the CRFTM includes the following contents in the process of extracting topic feature information:

[0020] The distance between different short texts in UGC is calculated by embedding-based distance metric, and the short texts are aggregated into pseudo-long documents with longer length using K-center point clustering algorithm, where each pseudo-long document aggregates the same number of short texts.

[0021] Sample a topic distribution θ~Dir(α) for the entire corpus, and sample a word distribution φ for each topic k k ~Dir(β), α represents the Dirichlet prior hyperparameter of the document topic density, β represents the Dirichlet prior hyperparameter of the topic word density, for each word n ​​in each pseudo-long document j, sample each word The topic assignment z for each word j Calculated by the following formula:

[0022]

[0023] Among them, r j,n Representation word t j,n Contextual words, r j represents the set of context-related words, Ψ represents the potential function considering the influence of semantic relevance, N represents the number of words in j, in CRFTM, the collapsed Gibbs sampling method is used to infer the posterior distribution of topic labels, and the word t in the pseudo-long document j j ,n's theme z j,n Calculated by the following formula:

[0024]

[0025] in, Indicates that when word t is excluded from topic k or pseudo document j j,n , the number of times the word is assigned to topic k, V represents the dimension of the vocabulary, and the topic distribution θ and topic word-distribution φ of the document are calculated by the following formulas:

[0026]

[0027]

[0028] Among them, K represents the number of topics.

[0029] In combination with the first aspect of the present application, in a fifth possible implementation manner of the first aspect of the present application, the subject feature information and the text feature information are feature-fused to obtain fused feature information, including:

[0030] The subject feature information R C and text feature information R X Perform feature connection operation to obtain fused feature information R text ,

[0031] In combination with the first aspect of the present application, in a sixth possible implementation manner of the first aspect of the present application, extracting user behavior feature information from the fused feature information includes:

[0032] In the fusion feature information, starting from the perspective of all users involved, emotional behavior feature information, user personal feature information and psychological related word information are extracted to obtain user behavior feature information.

[0033] In a second aspect, the present application provides a processing device for a detection model of depressed users on social media, the device comprising:

[0034] An acquisition unit for acquiring UGC on social media platforms;

[0035] An extraction unit, used to extract text feature information from UGC;

[0036] An extraction unit, used to extract topic feature information from UGC;

[0037] A fusion unit, used for fusing the topic feature information with the text feature information to obtain fused feature information;

[0038] The extraction unit is further used to extract user behavior feature information from the fused feature information;

[0039] The training unit is used to train the initialized BiLSTM model to identify depressed users in the input data using user behavior feature information as sample data.

[0040] In conjunction with the second aspect of the present application, in a first possible implementation manner of the second aspect of the present application, the extraction unit is specifically used to:

[0041] Get contextual embeddings of short text in UGC using the XLNet model X , as text feature information.

[0042] In combination with the first possible implementation of the second aspect of the present application, in the second possible implementation of the second aspect of the present application, the XLNet model is specifically a model based on the transformer-XL architecture and combined with permutation language modeling. The XLNet model arranges and combines each token to generate all possible word orders. During the training process of the XLNet model, the transformer-XL architecture first stores the hidden state generated by the previous segment, and then uses the cached hidden state as context when calculating the next segment.

[0043] In combination with the second aspect of the present application, in a third possible implementation manner of the second aspect of the present application, the extraction unit is specifically configured to:

[0044] The topic feature information in UGC is extracted through CRFTM. During the extraction process, CRFTM merges the short texts in UGC into pseudo documents with longer text length, and then extracts the topic feature information from the pseudo documents through the combination of word vectors and word-related knowledge.

[0045] In combination with the second aspect of the present application, in a fourth possible implementation manner of the second aspect of the present application, the CRFTM includes the following contents in the process of extracting topic feature information:

[0046] The distance between different short texts in UGC is calculated by embedding-based distance metric, and the short texts are aggregated into pseudo-long documents with longer length using K-center point clustering algorithm, where each pseudo-long document aggregates the same number of short texts.

[0047] Sample a topic distribution θ~Dir(α) for the entire corpus, and sample a word distribution φ for each topic k k ~Dir(β), α represents the Dirichlet prior hyperparameter of the document topic density, β represents the Dirichlet prior hyperparameter of the topic word density, for each word n ​​in each pseudo-long document j, sample each word The topic assignment zj for each word is calculated as follows:

[0048]

[0049] Among them, r j,n Representation word t j,n Contextual words, r j represents the set of context-related words, Ψ represents the potential function considering the influence of semantic relevance, N represents the number of words in j, in CRFTM, the collapsed Gibbs sampling method is used to infer the posterior distribution of topic labels, and the word t in the pseudo-long document j j,n Theme of zj ,n Calculated by the following formula:

[0050]

[0051] in, Indicates that when word t is excluded from topic k or pseudo document j j ,n, the number of times the word is assigned to topic k, V represents the dimension of the vocabulary, and the topic distribution θ and topic word-distribution φ of the document are calculated by the following formulas:

[0052]

[0053]

[0054] Among them, K represents the number of topics.

[0055] In combination with the second aspect of the present application, in a fifth possible implementation manner of the second aspect of the present application, the fusion unit is specifically configured to:

[0056] The subject feature information R C and text feature information R X Perform feature connection operation to obtain fused feature information R text ,

[0057] In combination with the second aspect of the present application, in a sixth possible implementation manner of the second aspect of the present application, the extraction unit is specifically used to:

[0058] In the fusion feature information, starting from the perspective of all users involved, emotional behavior feature information, user personal feature information and psychological related word information are extracted to obtain user behavior feature information.

[0059] In a third aspect, the present application provides a processing device, including a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method provided in the first aspect of the present application or any possible implementation method of the first aspect of the present application is executed.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the method provided by the first aspect of the present application or any possible implementation of the first aspect of the present application.

[0061] From the above content, it can be concluded that the present application has the following beneficial effects:

[0062] For depressed users on social media platforms, after obtaining UGC on the social media platform, this application extracts its text feature information on the one hand, and its topic feature information on the other hand, and then fuses the features of the two to obtain fused feature information, and extracts user behavior feature information from it. In this way, the user behavior feature information is used as sample data to train the initialized BiLSTM model to identify depressed users in the input data. In the training framework of this model, the topic features are fused with the text features, which greatly expands the text semantics, and then quantifies the user behavior related to the depressed users. Training the model in this way can significantly reduce the training cost of the model and significantly improve the detection accuracy of the model. The trained model has significantly improved pertinence for depressed users and has higher detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 A flowchart of a processing method for a detection model of depressed users on social media in this application;

[0065] Figure 2 A schematic diagram of the architecture of the feature fusion framework of this application;

[0066] Figure 3 A schematic diagram of a joint learning framework for this application;

[0067] Figure 4 A schematic diagram showing a comparison of the detection effects of various detection models in this application;

[0068] Figure 5 A structural schematic diagram of a processing device for a detection model of depressed users on social media in this application;

[0069] Figure 6 A schematic diagram of the structure of the processing equipment of this application. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0071] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0072] The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present application.

[0073] Before introducing the processing method of the detection model of depressed users on social media provided by this application, the background content involved in this application is first introduced.

[0074] The processing method, device and computer-readable storage medium of the detection model of depressed users on social media provided in the present application can be applied to a processing device to extract user behavior features related to depressed users by combining text features and topic features. The model trained in this way has significantly improved targeting for depressed users and has higher detection accuracy.

[0075] The processing method of the detection model of depressed users on social media mentioned in this application can be executed by a processing device of the detection model of depressed users on social media, or a server, physical host or user equipment (UE) and other different types of processing devices integrated with the processing device of the detection model of depressed users on social media. Among them, the processing device of the detection model of depressed users on social media can be implemented in hardware or software, and the UE can be a terminal device such as a smart phone, tablet computer, laptop computer, desktop computer or personal digital assistant (PDA), and the processing device can be set in the form of a device cluster.

[0076] Next, we will introduce the processing method of the detection model of depressed users on social media provided by this application.

[0077] First, see Figure 1 , Figure 1 A flowchart of a method for processing a detection model of depressed users on social media provided by the present application is shown. The method for processing a detection model of depressed users on social media provided by the present application may specifically include the following steps S101 to S106:

[0078] Step S101, obtaining UGC on the social media platform;

[0079] It is understandable that the social media platform involved in this application can be any type of social media platform, on which users can express their own ideas through text, sound, pictures or videos. Of course, if it involves sound, pictures or videos, considering that this application is processed from the perspective of text, it can also involve extracting the corresponding text from the content of the sound, picture or video. The extraction technology involved is usually existing technology, and this application will not be specifically introduced here. Of course, in practical applications, this application can also introduce self-created extraction technology to extract the corresponding text content from the content of sound, picture or video to form a processable text.

[0080] For example, social media platforms can be different types of platforms such as Weibo and Tik Tok. Users can publish new information (including forwarding information) in the form of their own updates, or they can publish relevant information in the comment section of other information to express their own ideas.

[0081] The extracted data are referred to as UGC in this application, i.e. user-generated content, and the specific content is the content that users make public on social media platforms.

[0082] Step S102, extracting text feature information in UGC;

[0083] In this application, for UGC, compared with the prior art of directly extracting text features related to depressed users, this application configures a feature fusion architecture to more accurately and completely extract features related to depressed users.

[0084] Specifically, the present application can extract text features related to depressed users from UGC to form text feature information. It can be understood that the text feature information is similar to the text feature information collected and processed by the training model of the prior art, and will not be described in detail here.

[0085] It should be noted that the text feature information extracted in this application is configured as a basic information to provide data support for subsequent feature fusion processing.

[0086] Step S103, extracting topic feature information in UGC;

[0087] It can be understood that, for the detection of depressed users, this application specially configures the extraction processing of subject feature information to provide good data support for the subsequent feature fusion processing.

[0088] Among them, the extraction and processing of the topic feature information is to process the topic features related to depressed users and their user behaviors in the process of preparing sample data, so that the text content related to depressed users and their user behaviors can be accurately locked and tracked based on the topic features, and the degree of depression can be determined therefrom.

[0089] Step S104, fusing the topic feature information with the text feature information to obtain fused feature information;

[0090] After obtaining the subject feature information and text feature information of both aspects, they can be fused to form new feature information that retains the feature contents of both, which is recorded as fused feature information in this application.

[0091] Step S105, extracting user behavior feature information from the fused feature information;

[0092] It can be understood that the fused feature information obtained by processing in the present application not only includes the content of basic text features, but also includes the content of topic features, and has richer text semantics. At this time, more accurate user behavior features related to depressed users can be extracted from its rich content to form user behavior feature information.

[0093] Step S106, using the user behavior feature information as sample data, training the initialized BiLSTM model to identify depressed users in the input data.

[0094] In this way, after obtaining user behavior feature information that is highly matched with depressed users, it can be used as sample data. After labeling the information of the corresponding sample depressed users, it can be put into model training to train the user depression detection model.

[0095] from Figure 1 It can be seen from the illustrated embodiment that, for depressed users on social media platforms, after obtaining UGC on the social media platform, the present application extracts, on the one hand, its text feature information, and on the other hand, its topic feature information, and then fuses the two features to obtain fused feature information, and extracts user behavior feature information therefrom. In this way, the user behavior feature information is used as sample data to train the initialized BiLSTM model to identify depressed users in the input data. In the training framework of this model, the topic features are fused with the text features, which greatly expands the text semantics, and then the user behavior related to the depressed users is quantified. Training the model in this way can significantly reduce the training cost of the model and significantly improve the detection accuracy of the model. The trained model has significantly improved pertinence for depressed users and has higher detection accuracy.

[0096] Continue to the above Figure 1 Each step of the illustrated embodiment and possible implementation methods thereof in practical applications are described in detail.

[0097] As a specific implementation scheme, in the process of extracting text feature information from UGC, this application can be implemented through the following scheme:

[0098] Get contextual embeddings of short text in UGC using the XLNet model X , as text feature information.

[0099] It can be understood that the context embedding provided by the XLNet model can easily solve the polysemy phenomenon where words have multiple meanings, thus achieving the purpose of accurately extracting text features.

[0100] In contrast, in the prior art, traditional methods such as glove or word2vec use fixed word vectors to represent each word and ignore their context, which makes it difficult to solve the ambiguity problem of polysemous words.

[0101] In addition, based on the XLNet model, the present application also configures a more specific optimization application solution as another practical implementation method. In actual applications, the XLNet model of the present application can be specifically a model based on the transformer-XL architecture and combined with permutation language modeling. The XLNet model arranges and combines each token to generate all possible word orders. During the training process of the XLNet model, the transformer-XL architecture first stores the hidden state generated by the previous segment, and then uses the cached hidden state as context when calculating the next segment.

[0102] To be specific, in recent years, transformer-based pre-trained language models have achieved success in various fields of NLP. Transformer uses the self-attention mechanism to perform attention calculations on the current word and other words in the sentence respectively, so as to obtain better word representation. The context representation generated by the Bidirectional Encoder Representations from Transformers (BERT) model based on the transformer structure has been successfully applied to many NLP tasks, such as text classification and opinion mining.

[0103] However, during the training of the BERT model, the prediction of mask characters is performed in parallel, which may lead to incorrect predictions. This is because the values ​​of mask characters are dependent on each other, and parallel prediction cannot take advantage of this dependency. For example, for the text "She wrote a[MASK]with a[MASK]", the probability of filling in [MASK] with "letter" and "pencil" is higher than that with "letter" and "bicycle". When predicting the second [MASK], the BERT model will ignore the words of the first [MASK], which will lead to incorrect prediction results.

[0104] In order to solve this problem, this application can also introduce the transformer-XL architecture when applying the XLNet model, and combine it with permutation language modeling, and arrange and combine each token to generate all possible word orders. Based on the transformer structure, the transformer-XL architecture applies a new position encoding and segmented recursive method. During the training process, transformer-XL first stores the hidden state generated by the previous segment. Then, when calculating the next segment, the transformer-XL architecture uses the cached hidden state as context. Similar to traditional language models, the permutation language modeling mechanism predicts the next character by given context in the pre-training stage. Unlike the BERT model, this mechanism does not predict characters in sequential order, but in random order. Therefore, permutation language modeling does not require the symbol [MASK], nor will it cause inconsistencies in the two stages of pre-training and fine-tuning, and can achieve high-precision text feature extraction effects.

[0105] Back to the extraction and processing of topic features, this application also provides a specific implementation scheme. As another practical implementation method, this application can extract topic features through the following content when extracting topic feature information from UGC:

[0106] The topic feature information in UGC is extracted through CRFTM. During the extraction process, CRFTM merges the short texts in UGC into pseudo documents with longer text length, and then extracts the topic feature information from the pseudo documents through the combination of word vectors and word-related knowledge.

[0107] Specifically, CRFTM, or conditional random field regularized topic model, can be understood as a model for finding the conditional probability distribution of another set of output random variables under a given set of input random variables. Its characteristic is that it assumes that the output random variables constitute a Markov random field. Conditional random fields can be used for different prediction problems. For natural language processing, it is mainly linear conditional random fields. At this time, the problem becomes a discriminant model for predicting the output sequence from the input sequence, in the form of a log-linear model. The learning method is maximum likelihood estimation or regularized maximum likelihood estimation. Conditional random fields are similar to hidden Markov fields, corresponding to three basic problems: probability calculation problem, learning problem and prediction problem.

[0108] In the process of extracting topic feature information using CRFTM, the present application can also specifically configure a corresponding topic feature information extraction strategy, that is, merging the short texts in UGC into pseudo documents with longer text length, and then extracting topic feature information from the pseudo documents through the combination of word vectors and word-related knowledge.

[0109] It is important to understand that automatically extracting topic knowledge from a large collection of short texts has been widely used in NLP tasks such as context analysis, user interest analysis, and text classification. Traditional topic feature extraction methods, such as Latent Dirichlet Allocation (LDA) and Dynamic Topic Model (DTM), can automatically mine latent topic structures from long documents such as scientific papers or news articles. These topic feature extraction methods use word co-occurrence patterns to reveal the probability distribution of topic words and document topics.

[0110] However, the sparsity problem of short texts on social media platforms seriously affects the topic knowledge mining performance of these models. In order to solve the sparsity problem, this application uses CRFTM to mine topic features from social media short texts. CRFTM first merges short texts to construct longer pseudo-long documents, and then uses semantic associations between words to improve the coherence of topic features, thereby achieving the technical effect of extracting topic features with high precision.

[0111] Specifically, in the process of extracting topic feature information, CRFTM can also include the following contents:

[0112] The distance between different short texts in UGC is calculated using an embedding-based distance metric (this distance metric can identify semantically related words in two short texts, such as sadness and sorrow. If the two short texts are close in distance, they may belong to the same topic tag), and the short texts are aggregated into longer pseudo-long documents using the K-center point clustering algorithm. In order to further alleviate sparsity, the CRFTM of this application adds a constraint to the clustering algorithm, that is, the number of short texts aggregated in each pseudo-long document is the same.

[0113] Next, CRFTM samples a topic distribution θ~Dir(α) for the entire corpus and a word distribution φ for each topic k k ~Dir(β), α represents the Dirichlet prior hyperparameter of the document topic density, β represents the Dirichlet prior hyperparameter of the topic word density, for each word n ​​in each pseudo-long document j, sample each word The topic assignment z for each word j It can be calculated by the following formula:

[0114]

[0115] Among them, r j,n Representation word t j,n Contextual words, r jrepresents the set of context-related words, Ψ represents the potential function considering the influence of semantic relevance, N represents the number of words in j, in CRFTM, the collapsed Gibbs sampling method is used to infer the posterior distribution of topic labels, and the word t in the pseudo-long document j j,n Theme j,n It can be calculated by the following formula:

[0116]

[0117] in, Indicates that when word t is excluded from topic k or pseudo document j j,n When , the word is assigned to the topic k. V represents the dimension of the vocabulary. The topic distribution θ and topic word distribution φ of the document can be calculated as follows:

[0118]

[0119]

[0120] Among them, K represents the number of topics.

[0121] Taking the above XLNet model and CRFTM as examples, see Figure 2 A schematic diagram of the architecture of the feature fusion framework of the present application is shown. The processing involved in step S104 of the present application, that is, fusing the subject feature information with the text feature information to obtain the fused feature information, may include:

[0122] The subject feature information R C and text feature information R X Perform feature connection operation to obtain fused feature information R text ,

[0123] It can be understood that in the feature fusion architecture proposed in this application, the XLNet model is used to obtain the context embedding R of the input short text. X After training the entire corpus, CRFTM is used to calculate the topic feature distribution of the document of the input short text, denoted as R C , CRFTM can obtain the topic feature distribution of all short text documents by only training once on the entire dataset in advance, which will not increase the training time of the model. Since the reasoning of the topic model and the forward propagation of the context representation are decoupled, this is easy to achieve in the feature fusion architecture model. C Contextual representation of short text R X Combined with the above, we can get the final text feature representation R text :

[0124]

[0125] in, Represents the feature concatenation operator.

[0126] Under this setting, the feature connection operation can completely and efficiently preserve the topic features and the feature content of text adjustment, and its data representation is also convenient for subsequent data use.

[0127] In the process of extracting user behavior feature information from the fused feature information, this application also provides a specific implementation method, namely:

[0128] In the fusion feature information, starting from the perspective of all users involved, emotional behavior feature information, user personal feature information and psychological related word feature information are extracted to obtain user behavior feature information.

[0129] Specifically, when extracting user behavior features related to depression, this application specifically extracts three user-level features: emotions, user information, and psychological related words. On social media platforms, there are various features to represent the overall behavior of users. Unlike extracting user behavior features from a single information source on a social media platform (for example, extracting user-level features from a single post), this application obtains user behavior features from all information sources on the social media platform from a holistic perspective (for example, extracting user-level features from all posts) to achieve higher feature extraction accuracy, where:

[0130] For emotional behavior features, taking emoticons as an example, this application believes that since users are accustomed to expressing their emotional states with simple symbols, emoticons have become a common communication tool for users on social media platforms, so distinguishing positive and negative emotions is very helpful for understanding the semantics of short texts. To this end, this application first calculates the number of negative, positive, and neutral emoticons that appear in each short text, and then adds the number of emoticons for each user to obtain the number of positive, neutral, and negative emoticons at the user level. In addition, the Valence Aware Dictionary and sEntiment Reasoner (VADER) can also be used to extract other emotional behavior features of users.

[0131] For user personal characteristic information, this application can specifically obtain user personal characteristics such as gender, age, number of friends and number of followers from the user account, which can reflect the unique information of each user. In addition, this application can also collect the number of tweets and reposts of each user. For each user, this application regards gender as a binary vector and discretizes other characteristics into unique hot vectors in five intervals. The time distribution of users posting tweets every day will be counted and represented by a 24-dimensional vector.

[0132] The feature information of psychological related words can be realized through LIWC. LIWC is an analysis tool that can count the psychological related words in the text and can be used to analyze the user's psychological state from the text. Many studies have shown that when LIWC is applied to the text content of social media platforms, it is correlated with depressive symptoms. This application specifically uses 8 LIWC features and analyzes all texts of each user.

[0133] In addition, for the BiLSTM model used in this application, it can use a joint learning mechanism to fuse multiple features to detect depressed users. The joint learning framework can refer to Figure 3 A framework diagram of the joint learning framework of the present application is shown.

[0134] Specifically, a recurrent neural network (RNN) is a neural network model that sequentially links hidden layer nodes and can dynamically extract sequence features. Although native RNNs can transmit semantic relationships between words, it is difficult to capture long-distance semantic information. Using the forget gate f, output gate o, and input gate i mechanism, the Long Short-Term Memory (LSTM) model can effectively capture long-distance connections between user features. However, the unidirectional LSTM only captures the forward connection of features, while this application uses the BiLSTM model to obtain the forward and backward relationships between user-level features. The hidden layer of the BiLSTM model has a backward output and a forward output, which can be defined as:

[0135]

[0136]

[0137]

[0138] Among them, h t Represents the hidden layer state of the LSTM model at the current moment, and They represent the backward and forward hidden layer outputs respectively, and x represents the user-level features. The calculation process of the BiLSTM model is as follows:

[0139] f t =σ(W f ·[h t-1 ,x t ])

[0140] i t =σ(W i ·[h t-1 ,x t ])

[0141] o t =σ(W o ·[h t-1 ,x t ])

[0142]

[0143]

[0144] h t =o t ·tanh(m t ),

[0145] Among them, σ is the sigmoid activation function, W represents the parameters of the model, and after encoding the features, they are cascaded and input into the final softmax layer for classification.

[0146] In order to further understand the above content of the present application, the present application can also be explained with the help of the following set of examples.

[0147] Specifically, the detection model for depressed users on social media trained and constructed in the present application can be recorded as a Topic-enriched Depression Detection Model (TDDM), and its advancement can be verified by experimental comparison with a baseline model.

[0148] For example, the benchmark models may include a basic CNN model, a BiLSTM model, a BERT model, an ALBERT model, and an XLNet model.

[0149] This application uses commonly used evaluation indicators for detecting depressed users on social media, including precision, recall, and F1 value, to analyze the experimental results.

[0150] The data set used in this application is a large-scale manually annotated real Twitter data, which contains 2153 depressed users and 2200 normal users. 80% of the data are randomly selected from the data set for training, and the rest of the data are used as a test set.

[0151] This application runs 1000 Gibbs samplings and sets the Dirichlet prior parameters to: α=0.5, β=0.01.

[0152] For the CNN model and BiLSTM model, this application uses 300-dimensional open source word2vec word vectors, and the dropout probability is set to 0.2. These word vectors will construct a vector matrix to convert the words of the input short text into the corresponding word vector representation.

[0153] For BERT and XLNet, this application uses the pre-trained 12-layer BERT-Base and XLNet-Base structures. The present invention uses Gelu as the activation function, uses the Adam optimizer, and the learning rate is set to 5e-5.

[0154] For the ALBERT model, we use albert-base-v2 for experiments, where the hidden dimension and attention heads are set to 768 and 12, respectively.

[0155] The experimental results are as follows Figure 4 As shown in a schematic diagram comparing the detection effects of various detection models of the present application, the TDDM constructed in the present application is superior to the other five benchmark models in three evaluation indicators, thereby verifying the advanced nature of the present application.

[0156] The above is an introduction to the processing method of the detection model of depressed users on social media provided by the present application. In order to facilitate better implementation of the processing method of the detection model of depressed users on social media provided by the present application, the present application also provides a processing device for the detection model of depressed users on social media from the perspective of functional modules.

[0157] See also Figure 5 , Figure 5 This is a schematic diagram of a structure of a processing device for a detection model of depressed users on social media in this application. In this application, the processing device 500 for a detection model of depressed users on social media may specifically include the following structure:

[0158] An acquisition unit 501 is used to acquire UGC on a social media platform;

[0159] An extraction unit 502, used to extract text feature information in UGC;

[0160] An extraction unit 503, used to extract topic feature information in UGC;

[0161] A fusion unit 504 is used to fuse the topic feature information with the text feature information to obtain fused feature information;

[0162] The extraction unit 502 is further used to extract user behavior feature information from the fused feature information;

[0163] The training unit 505 is used to train the initialized BiLSTM model to identify depressed users in the input data using the user behavior feature information as sample data.

[0164] In conjunction with the second aspect of the present application, in a first possible implementation manner of the second aspect of the present application, the extraction unit 502 is specifically configured to:

[0165] Get contextual embeddings of short text in UGC using the XLNet model X , as text feature information.

[0166] In combination with the first possible implementation of the second aspect of the present application, in the second possible implementation of the second aspect of the present application, the XLNet model is specifically a model based on the transformer-XL architecture and combined with permutation language modeling. The XLNet model arranges and combines each token to generate all possible word orders. During the training process of the XLNet model, the transformer-XL architecture first stores the hidden state generated by the previous segment, and then uses the cached hidden state as context when calculating the next segment.

[0167] In combination with the second aspect of the present application, in a third possible implementation manner of the second aspect of the present application, the extraction unit 503 is specifically configured to:

[0168] The topic feature information in UGC is extracted through CRFTM. During the extraction process, CRFTM merges the short texts in UGC into pseudo documents with longer text length, and then extracts the topic feature information from the pseudo documents through the combination of word vectors and word-related knowledge.

[0169] In combination with the second aspect of the present application, in a fourth possible implementation manner of the second aspect of the present application, the CRFTM includes the following contents in the process of extracting topic feature information:

[0170] The distance between different short texts in UGC is calculated by embedding-based distance metric, and the short texts are aggregated into pseudo-long documents with longer length using K-center point clustering algorithm, where each pseudo-long document aggregates the same number of short texts.

[0171] Sample a topic distribution θ~Dir(α) for the entire corpus, and sample a word distribution φ for each topic k k ~Dir(β), α represents the Dirichlet prior hyperparameter of the document topic density, β represents the Dirichlet prior hyperparameter of the topic word density, for each word n ​​in each pseudo-long document j, sample each word The topic assignment z for each word j Calculated by the following formula:

[0172]

[0173] Among them, r j,n Representation word t j,n Contextual words, r jrepresents the set of context-related words, Ψ represents the potential function considering the influence of semantic relevance, N represents the number of words in j, in CRFTM, the collapsed Gibbs sampling method is used to infer the posterior distribution of topic labels, and the word t in the pseudo-long document j j,n Theme of j,n Calculated by the following formula:

[0174]

[0175] in, Indicates that when word t is excluded from topic k or pseudo document j j,n , the number of times the word is assigned to topic k, V represents the dimension of the vocabulary, and the topic distribution θ and topic word-distribution φ of the document are calculated by the following formulas:

[0176]

[0177]

[0178] Among them, K represents the number of topics.

[0179] In combination with the second aspect of the present application, in a fifth possible implementation manner of the second aspect of the present application, the fusion unit 504 is specifically configured to:

[0180] The subject feature information R C and text feature information R X Perform feature connection operation to obtain fused feature information R text ,

[0181] In combination with the second aspect of the present application, in a sixth possible implementation manner of the second aspect of the present application, the extraction unit 502 is specifically configured to:

[0182] In the fusion feature information, starting from the perspective of all users involved, emotional behavior feature information, user personal feature information and psychological related word information are extracted to obtain user behavior feature information.

[0183] This application also provides a processing device from the perspective of hardware structure, see Figure 6 , Figure 6 601, a memory 602, and an input / output device 603. The processor 601 is used to execute the computer program stored in the memory 602 to implement the following Figure 1 The steps of the processing method of the detection model of depressed users on social media in the corresponding embodiment; or, the processor 601 is used to execute the computer program stored in the memory 602 to implement the following Figure 5Corresponding to the functions of each unit in the embodiment, the memory 602 is used to store the processor 601 executing the above Figure 1 The computer program required for the processing method of the detection model of depressed users on social media in the corresponding embodiment.

[0184] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 602, and executed by the processor 601 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0185] The processing device may include, but is not limited to, a processor 601, a memory 602, and an input / output device 603. Those skilled in the art will appreciate that the illustration is merely an example of a processing device and does not constitute a limitation on the processing device, and may include more or fewer components than shown in the illustration, or a combination of certain components, or different components. For example, the processing device may also include a network access device, a bus, etc., and the processor 601, the memory 602, the input / output device 603, etc. are connected via a bus.

[0186] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the processing device, and uses various interfaces and lines to connect various parts of the entire device.

[0187] The memory 602 can be used to store computer programs and / or modules. The processor 601 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 602 and calling the data stored in the memory 602. The memory 602 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the processing device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0188] When the processor 601 is used to execute the computer program stored in the memory 602, the following functions can be implemented:

[0189] Capture UGC on social media platforms;

[0190] Extract text feature information from UGC;

[0191] Extract topic feature information from UGC;

[0192] Fusing the topic feature information with the text feature information to obtain fused feature information;

[0193] Extracting user behavior feature information from fused feature information;

[0194] Using user behavior feature information as sample data, the initialized BiLSTM model is trained to identify the user's depression level in the input data.

[0195] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the processing device, processing equipment and corresponding units of the detection model of depressed users on social media described above can refer to the following. Figure 1 The description of the processing method of the detection model of depressed users on social media in the corresponding embodiment will not be repeated here in detail.

[0196] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0197] To this end, the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the present application as follows: Figure 1 The steps of the processing method of the detection model of depressed users on social media in the corresponding embodiment, the specific operations can be referred to as follows Figure 1 The description of the processing method of the detection model of depressed users on social media in the corresponding embodiment will not be repeated here.

[0198] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0199] Due to the instructions stored in the computer-readable storage medium, the present application can be executed. Figure 1 The steps of the processing method of the detection model of depressed users on social media in the corresponding embodiment, therefore, the present application can be implemented as follows Figure 1 The beneficial effects that can be achieved by the processing method of the detection model of depressed users on social media in the corresponding embodiment are detailed in the previous description and will not be repeated here.

[0200] The above is a detailed introduction to the processing method, apparatus, processing device and computer-readable storage medium of the detection model of depressed users on social media provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A processing method for a detection model of depressed users on social media, characterized in that: The method comprises: Capture user-generated content (UGC) on social media platforms; Extracting text feature information from the UGC; Extracting topic feature information from the UGC; Fusing the topic feature information with the text feature information to obtain fused feature information; Extracting user behavior feature information from the fused feature information; Using the user behavior feature information as sample data, training the initialized BiLSTM model to identify depressed users in the input data; The extracting the topic feature information in the UGC includes: The topic feature information in the UGC is extracted by a conditional random field regularized subject model CRFTM. During the extraction process, the CRFTM merges the short texts in the UGC into pseudo documents with longer text lengths, and then extracts the topic feature information from the pseudo documents by combining word vectors and word-related knowledge; The CRFTM includes the following contents in the process of extracting the topic feature information: The distance between different short texts in the UGC is calculated by using an embedding-based distance metric, and the short texts are aggregated into pseudo long documents with longer lengths using a K-center point clustering algorithm, wherein each pseudo long document has the same number of aggregated short texts. Sample a topic distribution for the entire corpus , for each topic Sampling a word distribution , The Dirichlet prior hyperparameter representing the document topic density, The Dirichlet prior hyperparameter representing the density of the topic words, for each of the pseudo long documents Each word in , sample each word , the topic assignment for each word Calculated by the following formula: ， in, Expressive words Contextual words, represents a set of context-related words, represents the potential energy function considering the influence of semantic relevance, express The number of words in the CRFTM, the collapsed Gibbs sampling method is used to infer the posterior distribution of the topic label, the pseudo long document Chinese words Theme Calculated by the following formula: ， in, Indicates that when from the topic or pseudo document Exclude words The word is assigned to the topic The number of times, Represents the dimension of the vocabulary, the topic distribution of the document and keyword-distribution Calculated by the following formulas: ， ， in, Indicates the number of topics.

2. The method according to claim 1, characterized in that The extracting text feature information from the UGC includes: Use the XLNet model to obtain the contextual embedding of the short text in the UGC , as the text feature information.

3. The method according to claim 2, characterized in that The XLNet model is specifically a model based on the transformer-XL architecture combined with permutation language modeling. The XLNet model permutes and combines each token to generate all possible word orders. During the training process of the XLNet model, the transformer-XL architecture first stores the hidden state generated by the previous segment, and then uses the cached hidden state as context when calculating the next segment.

4. The method according to claim 1, characterized in that: The step of fusing the topic feature information with the text feature information to obtain fused feature information includes: The subject feature information With the text feature information Perform feature connection operation to obtain the fusion feature information , .

5. The method according to claim 1, characterized in that Extracting user behavior feature information from the fused feature information includes: In the fused feature information, emotional behavior feature information, user personal feature information and psychology-related word information are extracted from the perspective of all users involved to obtain the user behavior feature information.

6. A processing device for a detection model of depressed users on social media, characterized in that: The device comprises: An acquisition unit, used for acquiring user-generated content (UGC) on social media platforms; An extraction unit, used to extract text feature information from the UGC; An extraction unit, used to extract topic feature information from the UGC; A fusion unit, used for fusing the topic feature information with the text feature information to obtain fused feature information; The extraction unit is further used to extract user behavior feature information from the fused feature information; A training unit, used to train an initialized BiLSTM model to identify depressed users in input data using the user behavior feature information as sample data; The extraction unit is specifically used for: The topic feature information in the UGC is extracted by a conditional random field regularized subject model CRFTM. During the extraction process, the CRFTM merges the short texts in the UGC into pseudo documents with longer text lengths, and then extracts the topic feature information from the pseudo documents by combining word vectors and word-related knowledge; The CRFTM includes the following contents in the process of extracting the topic feature information: The distance between different short texts in the UGC is calculated by using an embedding-based distance metric, and the short texts are aggregated into pseudo long documents with longer lengths using a K-center point clustering algorithm, wherein each pseudo long document has the same number of aggregated short texts. Sample a topic distribution for the entire corpus , for each topic Sampling a word distribution , The Dirichlet prior hyperparameter representing the document topic density, The Dirichlet prior hyperparameter representing the density of the topic words, for each of the pseudo long documents Each word in , sample each word , the topic assignment for each word Calculated by the following formula: ， in, Expressive words Contextual words, represents a set of context-related words, represents the potential energy function considering the influence of semantic relevance, express The number of words in the CRFTM, the collapsed Gibbs sampling method is used to infer the posterior distribution of the topic label, the pseudo long document Chinese words Theme Calculated by the following formula: ， in, Indicates that when from the topic or pseudo document Exclude words The word is assigned to the topic The number of times, Represents the dimension of the vocabulary, the topic distribution of the document and keyword-distribution Calculated by the following formulas: ， ， in, Indicates the number of topics.

7. A processing device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor executes the method according to any one of claims 1 to 5 when calling the computer program in the memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 5.

Images