Depression detection method based on image-text social media analysis

Abstract

The invention discloses a depression detection method based on image-text social media analysis, relates to the technical field of automatic detection of depression, and solves the technical problem in the prior art that whether a user is a depression patient or not is detected by analyzing character and picture data published by the user on a social media platform. Specifically, the method comprises four steps of data capture, post screening, image-text representation and fusion classification. Generally speaking, the method comprises the following steps: firstly, capturing a posted document or posted post of a user in a certain time period from a social media platform, including characters and picture data in the posted document; then, performing depression related symptom prediction on the posted document, and screening out the most related K posts; then, respectively carrying out text and picture representation on the screened posts; and finally, fusing the characterization vectors of the two modes for depression detection.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic detection of depression, and in particular to a depression detection method based on graphic and text social media analysis. Background Art

[0002] Depression is a mental illness with low mood and loss of interest as its core symptoms. The key to reducing the harm of depression lies in early diagnosis and treatment. The traditional doctor's interview diagnosis method is inefficient, costly, and late in detection. Therefore, automatic depression detection technology and methods have important research and practical value. Depression detection based on social media analysis refers to the analysis of the behavior and content expression of social media users to distinguish between depressed patients and healthy people. It has the advantages of a wide audience, automation, low cost, and early warning.

[0003] With the popularity of social media platforms and the widespread use of smart devices, people are becoming more and more accustomed to sharing personal emotions and information through platforms such as WeChat, Weibo, and Twitter. At the same time, as mental health issues become more prominent, the public's attention to mental health has gradually increased, and discussions on mental illnesses such as depression have become increasingly active on social media. This provides a data basis for depression detection based on social media analysis, attracting the attention of more and more researchers.

[0004] Although depression detection based on social media analysis has a good data and theoretical basis, it faces great technical challenges. On the one hand, depression has significant heterogeneity, and individual depression symptoms vary greatly, so automatic detection of depression itself is a very difficult task; on the other hand, on social media, especially on non-professional depression community social platforms, depression-related information is scarce and often mixed with a large amount of irrelevant information, which makes it difficult to extract depression features. At the same time, it is impossible to conduct timely detection and intervention for undiagnosed depression patients through social media analysis.

[0005] In view of the above technical defects, a solution is now proposed. Summary of the invention

[0006] The purpose of the present invention is to solve the above-mentioned problems and to propose a depression detection method based on text and image social media analysis. By analyzing the text and image data posted by the user on the social media platform, it is detected whether the user is a depressed patient. The task faces the challenges that depression has significant heterogeneity and the depression features of social media data are sparse.

[0007] In view of the above challenges, in order to achieve reliable depression detection, the present invention proposes a depression detection method for social media users based on text and images, which includes four steps: data capture, post screening, text and image representation, and fusion classification. In summary, the method first captures the posts or articles posted by users in a certain period of time from the social media platform, including the text and image data in the posts; then, depression-related symptoms are predicted for the posts, and the most relevant K posts are screened out; next, the screened posts are represented by text and images respectively; finally, the representation vectors of the two modalities are fused for depression detection.

[0008] The purpose of the present invention can be achieved through the following technical solutions:

[0009] A depression detection method based on text-graphic social media analysis. The depression detection method is divided into the following four steps:

[0010] Step 1: Data Capture

[0011] Capture posts from users on social media platforms within a set time period, including text and image data in the posts;

[0012] Step 2: Post Filtering

[0013] Build a deep neural network model to filter out the K most relevant posts about depression from a large number of posts published by users;

[0014] Step 3: Graphical representation

[0015] Represent the text and pictures in the filtered posts respectively to obtain a one-dimensional text representation vector and a one-dimensional picture representation vector;

[0016] Step 4: Fusion Classification

[0017] In the fusion classification stage, the representation vectors of text and image are fused, and a fully connected network FC is constructed followed by a Softmax activation function. The fused user image and text representation vectors are classified to detect depressed users.

[0018] As a preferred implementation of the present invention, the specific steps of step 2 post screening are as follows:

[0019] In the sub-process of constructing the dataset, a large number of user posts are captured from the depression section of the social media platform, and after data cleaning and labeling, a labeled dataset is formed; data cleaning refers to deleting posts that do not meet the requirements manually or automatically; data labeling includes two parts: user labeling and post symptom labeling. Among them, user labeling refers to labeling users as depressed patients or healthy people by checking the content of the posts published by users; post symptom labeling refers to manually reading the text content of each post to label the symptoms in the depression-related symptom set, indicating whether each symptom appears. The depression-related symptom set is drawn up by psychiatrists, or with reference to the diagnostic criteria for mental illness or professional books.

[0020] As a preferred embodiment of the present invention, the constructed data set is used for model training. The model input is the text content of the post. After the text content is characterized by the Bert / Albert / Roberta text pre-training model, it passes through the linear layer and the Sigmoid non-linear layer to output the probabilities of various depressive symptoms. The above probability values ​​are summed to obtain the correlation between the post and depression.

[0021] As a preferred embodiment of the present invention, suppose that a post p consists of n tokens, that is, p = {t 1 , t 2 ,…,t n}, after encoding by the text pre-training model, the representation vector p of the post is obtained e :

[0022] p e =Encoder [cls] (p)

[0023] In the above formula, Encoder [cls] (p) indicates that the input text p is encoded using the Bert, Albert or Roberta text pre-training model, and the feature vector corresponding to the [CLS] tag is extracted.

[0024] Let M be the number of depression-related symptoms. The linear layer maps the representation vector of the post to an M-dimensional vector, which is passed through the Sigmoid function to output the symptom probability vector q:

[0025] q=Sigmoid(Linear(p e ))

[0026] In the above formula, Linear(x) represents a linear layer (fully connected layer), which projects the input feature vector x into the M-dimensional space to generate an unnormalized output vector; Sigmoid(x) represents the output vector of vector x after the Sigmoid function, q is an M-dimensional vector, and q = {q 1,q 2 ,…,q M}, where q i , i∈[1,M] represents the probability that the post embodies the i-th symptom.

[0027] As a preferred embodiment of the present invention, during the offline training stage of the model, the parameters of the pre-trained model of the text are frozen, and only the parameters of the linear layer of the model are trained. The training goal is to make the symptom probability output by the Sigmoid layer of the model close to the symptoms annotated in the post.

[0028] As a preferred implementation of the present invention, online post screening can use the trained model to perform online high-risk post screening; let K be the preset number of target posts, and use the model to calculate the relevance of each post; let p be a certain post, q represent the symptom probability vector output by the Sigmoid layer of the model, q i represents the probability that the post embodies the i-th depressive symptom, and the relevance of the post r q It can be calculated by the following formula:

[0029]

[0030] In the above formula, r q It represents the sum of the probabilities of all symptoms in the set of depression-related symptoms reflected in the post. The larger the value, the richer the depression characteristics contained in the post, and the higher the correlation with depression. Therefore, post screening is to screen out the K most relevant posts for each user;

[0031] As a preferred embodiment of the present invention, the specific process of step three is as follows:

[0032] In the image and text representation stage, the text content of K posts is extracted to form a text sequence consisting of K texts. Each text is represented by the Bert / Albert / Roberta text pre-training model and then output as a vector. Let T j The text p of the jth post j The output vector after the text pre-training model, where j∈[1,K]; that is:

[0033] T j =Encoder [cls] (p j ), j∈[1,k]

[0034] As before, Encoder [cls] (x) indicates that the input text x is encoded using the Bert, Albert or Roberta text pre-training model, and the feature vector corresponding to the [CLS] tag is extracted.

[0035] Considering that the post text contains rich semantic information and has strong relevance, in order to make the model focus on the most important information to improve the detection accuracy, a Transformer encoder is constructed to encode the vector of the text sequence represented by the pre-trained model; let T′ j Represents text p j The representation vector output from the Transformer encoder:

[0036] T′ j = Transformer(T j ), j∈[1,K]

[0037] In the above formula, Transformer (T j ) represents the text sequence T input by the encoder of the Transformer model j For encoding, the Transformer model mentioned above is a deep learning model architecture based on the self-attention mechanism proposed by Google researchers.

[0038] As a preferred implementation of the present invention, for image data, all images are extracted from the screened K posts to form an image sequence, and h is set as the total number of images; for any image, the image is firstly resized and pixel normalized for preprocessing, and then the image is represented by a pre-trained VGG or CLIP image processing model, and a one-dimensional vector is output after a Flatten operation; finally, all the one-dimensional vectors output from the image sequence are averaged to obtain the final image feature representation V, where V is a one-dimensional vector;

[0039] The representation of the text sequence T' j , j∈[1,K] and the representation V of the image sequence are concatenated together to form the user's graphic representation U.

[0040] As a preferred embodiment of the present invention, the process of step 4 is as follows:

[0041] During the offline training stage of the model, the pre-trained model parameters of text and images in the image and text representation and fusion classification models will be frozen, and only the parameters of the Transformer encoder and the fully connected network FC will be trained. The training data can use the dataset constructed in post screening and input the user's image and text data, so that the results of the model fusion classification are close to the user labels.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The present invention combines the text content and pictures in the user's post to perform fusion classification detection, thereby improving the accuracy of detection through information complementarity.

[0044] (2) The present invention mainly solves the problem of sparse depression information in social media data. To this end, the present invention introduces prior knowledge of depression-related symptoms and annotates social media data at two levels: user-level and symptom-level annotation.

[0045] User-level annotation refers to the annotation of healthy people and depressed patients on social media users; post symptom annotation refers to manually annotating whether each symptom in the depression-related symptom set appears in each post; dual annotation at the user level and symptom level enables the trained model to identify depression symptoms reflected in a single post;

[0046] On the one hand, it can filter out the posts most relevant to depression from a large number of irrelevant user posts, avoid the interference of irrelevant and redundant posts, and improve the accuracy of user depression detection;

[0047] On the other hand, the user's depression state can be inferred in real time through the process of identifying depression symptoms in the posts, achieving early depression detection and real-time depression monitoring, and avoiding missing out on timely intervention when the user's depression is not diagnosed in the early stages;

[0048] This method can effectively and timely provide early warning and intervention, improving the identification and treatment of early depression.

[0049] (3) Considering the high cost of dataset annotation and the limited size of datasets, the present invention widely uses pre-trained models to perform feature encoding on the text and images posted by users in post screening and image and text representation, and uses pre-trained models to ensure robustness and generalization.

[0050] (4) In the image-text representation sub-process, considering that the text content posted by users contains rich semantic information and has strong correlation, the present invention adopts a Transformer encoder based on the self-attention mechanism to encode the representation vector of the text sequence, so that the model focuses on important information, eliminates the interference of redundant information and noise, and improves the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0052] Figure 1 Schematic diagram of the process of the depression detection method in the present invention;

[0053] Figure 2 A schematic diagram of the offline construction process of the post screening model in the present invention;

[0054] Figure 3 A schematic diagram of the post screening model structure in the present invention;

[0055] Figure 4 It is a schematic diagram of the image and text representation and fusion classification process in the present invention. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] Social media platforms, especially non-professional depression communities with a large user base (such as WeChat Moments), have data that is significantly sparse in depression information. However, existing research mainly focuses on users in professional depression communities, such as Sina Weibo's Depression Chat. In professional depression communities, the pictures and texts posted by users are highly correlated with depression, and the correlation with depression is high. However, this method is limited to users in professional depression communities and is not applicable to the large number of undiagnosed depressed users, which makes it impossible to identify depressed users early and delays the early treatment of patients.

[0059] In order to expand the application scope of depression detection for social media users, this method studies depression user detection on more general social media platforms, such as WeChat Moments, which have a very large number of users. However, the new challenge is that the topics discussed by users on general platforms are open, and information related to depression is sparse.

[0060] Existing technologies have little research on "anti-sparse depression feature extraction methods", so they are only applicable to depression identification of users in depression communities. In contrast, the present invention focuses on solving the problem of sparse depression information in social media data. The present invention introduces prior knowledge of depression-related symptoms, annotates social media data at both the user level and the symptom level, and enables the model to identify depression symptoms through training, thereby filtering out the posts most relevant to depression from a large number of irrelevant user posts, achieving an anti-sparse effect, improving the accuracy of depression detection, and having a wider range of applications; please refer to Figure 1As shown in FIG, a depression detection method based on text-image social media analysis is used to distinguish between depression patients and healthy people by analyzing the text and images posted by users on social media platforms; Figure 1 As shown, this method is divided into the following four steps:

[0061] Step 1: Data Capture

[0062] Capture the posts (or articles) of users within a certain period of time from social media platforms, including text and image data in the posts;

[0063] Step 2: Post Filtering

[0064] Considering that the posting frequency of different users varies greatly, and the users' expressions on social media platforms are rich in content, only a small amount of information is related to depressive symptoms, and depressive features are sparse and difficult to extract, the present invention constructs a deep neural network model to screen out "high-risk posts" related to depression; the model needs to be built and trained offline so that it can be used for online post screening.

[0065] (1) Offline model construction

[0066] The construction process of the above-mentioned "high-risk posts" screening model is as follows: Figure 2 As shown, it mainly includes three steps: data set construction, model training and model preservation. Among them, in the sub-process of constructing the data set, this method captures a large number of user posts (including text and pictures) from the depression area in the social media platform (such as Sina Weibo Depression Talk, Reddit Depression Community), and after data cleaning and annotation, constitutes an annotated data set. Data cleaning refers to deleting posts that do not meet the requirements manually or automatically. Data annotation includes two parts: user annotation and post symptom annotation. User annotation refers to the annotation of users as depressed patients or healthy people by checking the content of the posts published by users. Post symptom annotation refers to manually reading the text content of each post and annotating whether each symptom in the depression-related symptom set appears, which is used to determine whether it reflects the relevant symptoms of depression. The depression-related symptom set can be formulated by a psychiatrist or with reference to professional books on the diagnosis of mental illness (such as the Diagnostic and Statistical Manual of Mental Disorders and the Diagnostic Criteria of Chinese Mental Disorders). Table 1 is an example of a depression-related symptom set formulated with reference to the Diagnostic and Statistical Manual of Mental Disorders (5th Edition) (DSM-5).

[0067] Table 1 Depression-related symptom set (example)

[0068] Serial number Symptom description 1 Depressed mood, feeling sad, empty, or hopeless 2 A noticeable decrease in interest or pleasure 3 Weight loss or increase in appetite 4 Insomnia or excessive sleeping 5 Psychomotor agitation or retardation 6 Fatigue or lack of energy 7 Feelings of worthlessness and inappropriate guilt 8 Decreased thinking ability, difficulty concentrating, or indecision 9 Thoughts of death or suicide 10 Social and occupational impairment 11 Unattributable physiological effects or medical illness

[0069] The constructed data set is used for model training. The model structure is as follows Figure 3 as shown; Figure 3 As shown in the figure, the model input is the text content of the post. After the text content is represented by the Bert / Albert / Roberta text pre-training model, it passes through the linear layer and the Sigmoid nonlinear layer to output the probability of various depressive symptoms. The above probability values ​​are summed to obtain the correlation between the post and depression. The specific description is as follows:

[0070] Suppose post p consists of n tokens, that is, p = {t 1 , t 2 ,…,t n}, after encoding by the text pre-training model, the representation vector p of the post is obtained e :

[0071] p e =Encoder [cls] (p)

[0072] In the above formula, Encoder [cls] (p) indicates that the input text p is encoded using the Bert, Albert or Roberta text pre-training model, and the feature vector corresponding to the [CLS] tag is extracted.

[0073] Let M be the number of depression-related symptoms. The linear layer maps the representation vector of the post to an M-dimensional vector, which is passed through the Sigmoid function to output the symptom probability vector q:

[0074] q=Sigmoid(Linear(p e ))

[0075] In the above formula, Linear(x) represents a linear layer (fully connected layer), which projects the input feature vector x into the M-dimensional space to generate an unnormalized output vector; Sigmoid(x) represents the output vector of vector x after the Sigmoid function, q is an M-dimensional vector, and q = {q 1 ,q 2 , …, q M}, where qi,i∈[1,M] represents the probability that the post embodies the i-th symptom.

[0076] During the offline training phase of the model, the parameters of the pre-trained model of the text are frozen, and only the parameters of the linear layer of the model are trained. The training goal is to make the symptom probability output by the Sigmoid layer of the model close to the symptoms annotated in the post.

[0077] (2) Online post screening

[0078] The trained model can be used to screen online high-risk posts; let K be the preset target number of posts, and the present invention adopts Figure 3The model shown calculates the relevance of each post. As mentioned above, let p be a post, q represent the symptom probability vector output by the Sigmoid layer of the model, and the relevance of the post is calculated as follows:

[0079] In the above formula, r q represents the sum of the probabilities of all symptoms in the set of depression-related symptoms reflected in the post, q i represents the probability that the post embodies the i-th depressive symptom;

[0080] Post screening is to screen out the K most relevant posts for each user, where the highest relevance is represented by the maximum sum of all symptom probabilities;

[0081] Step 3: Graphical representation

[0082] The present invention adopts the above method to represent the text and pictures in the screened posts respectively, and obtains a one-dimensional text representation vector and a one-dimensional picture representation vector, such as Figure 4 shown.

[0083] In the image and text representation stage, the present invention first extracts the text content in K posts to form a text sequence consisting of K texts. Each text is represented by a text pre-training model such as Bert / Albert / Roberta and then output as a vector. Let T j The text p of the jth post j The output vector after the text pre-training model, where j∈[1,K]; that is:

[0084] T j =Encoder [cls] (p j ), j∈[1,K]

[0085] As before, Encoder [cls] (x) indicates that the input text x is encoded using the Bert, Albert or Roberta text pre-training model, and the feature vector corresponding to the [CLS] tag is extracted;

[0086] Considering that the post text contains rich semantic information and has strong correlation, in order to make the model focus on the most important information to improve the detection accuracy, the present invention constructs a Transformer encoder to encode the vector of the text sequence represented by the pre-trained model. Let T′ j Represents text p j The representation vector output from the Transformer encoder:

[0087] T′ j = Transformer(Tj ), j∈[1,K]

[0088] In the above formula, Transformer (T j ) represents the text sequence T input by the encoder of the Transformer model j Encoding, the Transformer model mentioned above is a deep learning model architecture based on the self-attention mechanism proposed by Google researchers;

[0089] For image data, the present invention extracts all images from the K screened posts to form an image sequence, and assumes h is the total number of images. For any image, firstly, the image is preprocessed by resizing and pixel normalization, and then the image is characterized by a pre-trained image processing model such as VGG or CLIP, and a one-dimensional vector is output after a Flatten operation (the dimension is related to the image pre-training model); finally, all (h) one-dimensional vectors output from the image sequence are averaged to obtain the final image feature representation V, where V is a one-dimensional vector.

[0090] The representation of the text sequence T′ j , j∈[1,K] and the representation V of the image sequence are concatenated together to form the user's graphic representation U.

[0091] Step 4: Fusion Classification

[0092] like Figure 4 As shown, in the fusion classification stage, the present invention constructs a fully connected network FC (followed by a Softmax activation function) to classify the fused user image and text representations. The input dimension of the fully connected network is the dimension of the representation vector U, and the output is two-dimensional.

[0093] During the offline training phase of the model, Figure 4 In the picture-text representation and fusion classification models shown, the pre-trained model parameters of text and images will be frozen, and only the parameters of the Transformer encoder and the fully connected network FC will be trained. The training data can use the dataset constructed in post screening and input the user's picture and text data, so that the result of the model fusion classification is close to the user label (depression or health).

[0094] The above formulas are obtained by collecting a large amount of data and performing software simulation, and a formula close to the actual value is selected. The coefficients in the formula are set by technicians in this field according to actual conditions;

[0095] When the present invention is in use, it detects whether a user is a depressed patient by analyzing the text and picture data posted by the user on the social media platform. The task faces the challenge that depression has significant heterogeneity and the depression features of social media data are sparse.

[0096] In view of the above challenges, in order to achieve reliable depression detection, the present invention proposes a depression detection method for social media users based on text and images, which includes four steps: data capture, post screening, text and image representation, and fusion classification. In summary, the method first captures the user's posts (or articles) within a certain period of time from the social media platform, including the text and image data in the posts; then, depression-related symptoms are predicted for the posts, and the most relevant K (preset parameters) posts are screened out; next, the screened posts are represented by text and images respectively; finally, the representation vectors of the two modalities are fused for depression detection.

[0097] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A depression detection method based on graphic and text social media analysis, characterized in that: The depression detection method is divided into the following four steps: Step 1: Data Capture Capture posts from users on social media platforms within a set time period, including text and image data in the posts; Step 2: Post Filtering Build a deep neural network model to filter out the K most relevant posts about depression from a large number of posts published by users; Step 3: Graphical representation Represent the text and pictures in the filtered posts respectively to obtain a one-dimensional text representation vector and a one-dimensional picture representation vector; Step 4: Fusion Classification In the fusion classification stage, the representation vectors of text and image are fused, and a fully connected network FC is constructed followed by a Softmax activation function. The fused user image and text representation vectors are classified to detect depressed users.

2. A depression detection method based on graphic and text social media analysis according to claim 1, characterized in that: Step 2: Post screening: In the sub-process of constructing the dataset, a large number of user posts are captured from the depression section of the social media platform, and after data cleaning and labeling, a labeled dataset is formed; data cleaning refers to deleting posts that do not meet the requirements manually or automatically; data labeling includes two parts: user labeling and post symptom labeling. User labeling refers to labeling users as depressed patients or healthy people by checking the content of their posts; post symptom labeling refers to manually reading the text content of each post to label the symptoms in the depression-related symptom set, indicating whether each symptom appears; the depression-related symptom set is drawn up by psychiatrists, or is drawn up with reference to the diagnostic standards for mental illness or professional books.

3. A depression detection method based on graphic and text social media analysis according to claim 2, characterized in that: The constructed dataset is used for model training. The model input is the text content of the post. After being represented by the Bert / Albert / Roberta text pre-training model, the text content passes through the linear layer and the Sigmoid non-linear layer to output the probabilities of various depressive symptoms. The above probability values ​​are summed to obtain the correlation between the post and depression.

4. The depression detection method based on graphic and text social media analysis according to claim 3 is characterized in that Post p consists of n tokens, that is, p = t1, t2, ..., t n }, after encoding by the text pre-training model, the representation vector p of the post is obtained e : p e =Encoder [cls] (p) In the above formula, Encoder [cls] (p) indicates that the input text p is encoded using the Bert, Albert or Roberta text pre-training model, and the feature vector corresponding to the [CLS] tag is extracted; Let M be the number of depression-related symptoms. The linear layer maps the representation vector of the post to an M-dimensional vector, which is passed through the Sigmoid function to output the symptom probability vector q: q=Sigmoid(Linear(p e )) In the above formula, Linear(x) represents a linear layer (fully connected layer), which projects the input feature vector x into the M-dimensional space to generate an unnormalized output vector; Sigmoid(x) represents the output vector of vector x after the Sigmoid function, q is an M-dimensional vector, and q = {q1, q2, …, q M }, where q i , i∈[1,M] represents the probability that the post embodies the i-th symptom.

5. A depression detection method based on graphic and text social media analysis according to claim 4, characterized in that: During the offline training phase of the model, the parameters of the pre-trained model of the text are frozen, and only the parameters of the linear layer of the model are trained. The training goal is to make the symptom probability output by the Sigmoid layer of the model close to the symptoms annotated in the post.

6. A depression detection method based on graphic and text social media analysis according to claim 5, characterized in that: Online post screening can use the trained model to screen online high-risk posts; let K be the preset number of target posts, and use the model to calculate the relevance of each post; let p be a post, q represent the symptom probability vector output by the Sigmoid layer of the model, and the relevance of the post is calculated by the following formula: In the above formula, r q represents the sum of the probabilities of all symptoms in the set of depression-related symptoms reflected in the post, q i represents the probability that the post embodies the i-th depressive symptom; Post screening is to filter out the K most relevant posts for each user.

7. The depression detection method based on graphic and text social media analysis according to claim 1, characterized in that: The specific process of step three is as follows: In the image and text representation stage, the text content of K posts is extracted to form a text sequence consisting of K texts. Each text is represented by the Bert / Albert / Roberta text pre-training model and then output as a vector. Let T j The text p of the jth post j The output vector after the text pre-training model, where j∈[1,K]; that is: T j =Encoder [cls] (p j ),j∈[1,K] Considering that the post text contains rich semantic information and has strong relevance, in order to make the model focus on the most important information to improve the detection accuracy, a Transformer encoder is constructed to encode the vector of the text sequence represented by the pre-trained model; let T′ j Represents text p j The representation vector output from the Transformer encoder: T′ j = Transformer(T j ), j∈[1,K], where Transformer(T j ) represents the text sequence T input by the encoder of the Transformer model j to encode.

8. The depression detection method based on graphic and text social media analysis according to claim 7, characterized in that: For image data, all images are extracted from the K screened posts to form an image sequence, where h is the total number of images. For any image, the image is first resized and pixel normalized for preprocessing, and then the pre-trained VGG or CLIP image processing model is used to represent the image, and a one-dimensional vector is output after the Flatten operation. Finally, all the one-dimensional vectors output from the image sequence are averaged to obtain the final image feature representation V, where V is a one-dimensional vector. The representation of the text sequence T′ j , j∈[1,K] and the representation V of the image sequence are concatenated together to form the user's graphic representation U.

9. The depression detection method based on graphic and text social media analysis according to claim 1, characterized in that: Step 4 process is as follows: During the offline training stage of the model, the pre-trained model parameters of text and images in the image and text representation and fusion classification models will be frozen, and only the parameters of the Transformer encoder and the fully connected network FC will be trained. The training data can use the dataset constructed in post screening and input the user's image and text data, so that the results of the model fusion classification are close to the user labels.

Images