Multimodal social network depression detection method based on hypergraph transformer

Abstract

The application discloses to the technical field of natural language processing and graph neural network, specifically a multi-modal social network depression detection method based on hypergraph Transformer, and comprises the following specific steps: preprocessing text data and picture data, for the text data, extracting text features by using a pre-training language model, and combining a theme analysis method to construct a text hypergraph, so as to fully capture semantic correlations in the text data; for the image data, constructing an image hypergraph by calculating feature representation correlations, introducing a visual hypergraph convolution network, further extracting image features, and strengthening the feature representation capability of the hypergraph. The method of the application constructs a hypergraph model capable of reflecting the complexity of multi-modal data according to the characteristics of social network depression detection, fully utilizes multi-modal data such as text and images of the social network, and comprehensively mines complex correlations between cross modalities.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and graph neural networks, and particularly relates to a multi-modal social network depression detection method based on hypergraph Transformer. BACKGROUND

[0002] With the rapid development of social media and mobile Internet, people's online social activities are becoming more and more frequent. People express their emotional state and life state on social networks by publishing text, pictures, videos and other content. In recent years, researchers have found that in-depth analysis of these social network data can effectively discover some potential mental health problems, providing important clues for timely intervention and diagnosis.

[0003] In particular, in the diagnosis of depression, existing clinical methods often rely on face-to-face questionnaire surveys and professional assessments, which are low in efficiency. The digital diagnosis method based on social network can more continuously and actively monitor the emotional changes of users and timely discover abnormal conditions. This not only greatly improves the diagnosis efficiency, but also reduces the medical cost of patients, and shows great application potential in the field of mental health.

[0004] However, the existing depression detection methods based on social network data mostly rely only on single modal text or image data, and cannot fully mine the complex correlations between cross-modal. At the same time, these methods also ignore the graph structure characteristics of social network data itself, and cannot effectively model the social connection relationship between users. This limits the further improvement of detection performance to some extent.

[0005] Therefore, the present application provides a multi-modal social network depression detection method based on hypergraph Transformer, which can combine the graph structure characteristics of social network data to accurately identify the depression symptoms of users, and provide a more effective auxiliary diagnosis means for the field of mental health. This not only helps to reduce the social burden of depression, but also provides protection for the mental health of the general public. SUMMARY

[0006] In view of the above and / or existing problems in the multi-modal social network depression detection method based on hypergraph Transformer, the present application is proposed.

[0007] Therefore, the purpose of the present application is to provide a multi-modal social network depression detection method based on hypergraph Transformer, which can solve the above-mentioned existing problems.

[0008] To solve the above technical problems, according to one aspect of the present application, the present application provides the following technical scheme:

[0009] The method comprises the following specific steps:

[0010] S1: preprocessing text data and picture data, for text data, extracting text features by using a pre-trained language model, and constructing a text hypergraph by combining a topic analysis method to fully capture semantic associations in the text data;

[0011] S2: for image data, constructing an image hypergraph by calculating feature representation correlations, introducing a visual hypergraph convolution network to further extract image features, and enhancing the feature representation capability of the hypergraph;

[0012] S3: calculating the feature similarity of text entities and image entities to construct a graph-text relationship, and constructing a comprehensive graph-text hypergraph representation of the user level hypergraph by using an adaptive hypergraph fusion algorithm;

[0013] S4: introducing a hypergraph Transformer, calculating the features of each node, enhancing the hypergraph feature representation, and providing a basis for subsequent classification tasks;

[0014] S5: combining the hypergraph feature representation to perform depression detection tasks, and realizing accurate identification of depression social network patients.

[0015] As a preferred scheme of the method for detecting depression in a multi-modal social network based on a hypergraph Transformer, the pre-trained language model in S1 is BERT, which has strong understanding ability for text and can extract features from short text data in a social network.

[0016] As a preferred scheme of the method for detecting depression in a multi-modal social network based on a hypergraph Transformer, the topic analysis method in S1 is a document topic generation model Latent Dirichlet Allocation, which is used to identify the hidden topic information in short text due to the lack of training data.

[0017] As a preferred scheme of the method for detecting depression in a multi-modal social network based on a hypergraph Transformer, the hypergraph in S1, S2 and S3 is defined as G=(ν,ε), where ν=(v1,...,v n ) represents the set of nodes in the graph, ε=(e1,...,e m ) represents the set of hyperedges, and the topological result of the hypergraph is represented by the association matrix , which is defined as:

[0018]

[0019] As a preferred scheme of the multi-modal social network depression detection method based on hypergraph Transformer, wherein the specific process of S1 is:

[0020] S11: subject analysis is performed on the preprocessed text data to obtain the relationship between the text and the entity to construct the subject hyperedge of the text hypergraph, which is defined as:

[0021]

[0022] wherein, represents a node, v i belongs to the subject hyperedge

[0023] S12: after the text is extracted by the preprocessed model BERT, the text hypergraph is constructed, and the node features of the text hypergraph are constructed, which are defined as:

[0024]

[0025] wherein, represents a node, v i belongs to the article hyperedge

[0026] As a preferred scheme of the multi-modal social network depression detection method based on hypergraph Transformer, wherein the specific process of S2 is:

[0027] S21: the image is divided into multiple image pieces, a multi-layer convolution algorithm is introduced, and image embedding is extracted as the feature representation x l of the image, and l is the number of convolution layers;

[0028] S22: based on the local features extracted by the convolution kernel, in order to construct the relationship in the image space, the correlation of the image piece features in the image space is calculated to give the node weight in each hyperedge, and the formula is:

[0029]

[0030] wherein, W Γ , W λ , W μ are parameters for constructing linear embedding, is the linear embedding obtained by 1x1 convolution of the image feature representation x l , N represents the number of nodes, is the diagonal matrix representing the distance of the nodes obtained by global average pooling, and D represents the dimension of the feature, The evaluation node obtained by (MxM) convolution is the weight matrix of its hyperedge, and M represents the number of hyperedges;

[0031] S23: the global relationship of the image pieces represented by Γ(x l ), λ(x l ), μ(x l ) in space is combined to construct an image hypergraph, and the formula is:

[0032] H=Γ(x)λ(x)Γ T (x)μ(x)

[0033] S24: in order to reduce the noise of constructing a hypergraph, a hard connection filter is constructed to reduce the noise in model training by deleting weak connections, and the definition is:

[0034] H new =x≥c,x∈H。

[0035] As a preferred scheme of the multi-modal social network depression detection method based on hypergraph Transformer, wherein the specific process of S3 is:

[0036] S31: the feature similarity of text entities and image entities is calculated to construct the relationship between image entities and text entities, for each entity, the entity with a similarity score greater than λ is selected as the object to construct the relationship, the feature similarity is calculated, and the formula is:

[0037] sim(o i ,e j )=X text (e j ) T X img (o i )

[0038] Wherein, sim(o i ,e j ) represents the feature similarity of the jth text entity e j and the ith image entity o i , X text (e j ) and X img (o i ) represent the features of the text entity e j and the image entity o i , respectively;

[0039] S32: for different categories of hypergraphs, an adaptive hypergraph fusion algorithm is introduced to construct a graph-text hypergraph, and the formula is:

[0040]

[0041] where w k is the trainable parameter shared by all hyper-edges in the hypergraph k, sigmoid(·) is the standardization function, M k is the number of hyper-edges in the hypergraph k, the copy(a, b) function returns a vector of size b, whose values are filled by copying b times, and the vector is the weight of the i-th hyper-edge in the hypergraph k, || is the concatenation operation, and H is the hypergraph adjacency matrix obtained by splicing all hypergraph adjacency matrices;

[0042] S33: In combination with the adaptive hypergraph fusion algorithm, the text hypergraph and the image hypergraph are subjected to hypergraph fusion convolution to enhance the hypergraph node feature expression, and the formula is as follows:

[0043]

[0044] where D v and D e are the diagonal matrices of vertex degree and edge degree, respectively, and Θ is a learnable parameter.

[0045] As a preferred scheme of the hypergraph Transformer-based multi-modal social network depression detection method, the specific process of S4 is as follows:

[0046] S41: A node-level hypergraph attention mechanism is introduced to update the hyper-edge feature by aggregating all nodes on the hyper-edge e j , and to highlight more important nodes, and the formula is as follows:

[0047]

[0048]

[0049] where σ(·) is a nonlinear activation function, W1 is a trainable weight matrix, and α jk represents the attention coefficient of node v j in the hyper-edge e k , and a1 is a trainable weight coefficient for measuring the importance of the node;

[0050] S42: A hyper-edge-level attention mechanism is introduced to update the node feature by aggregating the hyper-edges connected to the node, and to highlight more important hyper-edges, and the formula is as follows:

[0051]

[0052] where W2 is a trainable weight matrix, and β ij represents the attention coefficient of node v i to the hyper-edge e j ​The attention coefficient of the attention coefficient, a2 is the trainable weight coefficient of the super-edge importance, and || is the connection operation;

[0053] S43: In combination with the multi-head attention mechanism concept, the supergraph attention is extended to multi-head supergraph attention, and the formula is:

[0054]

[0055] Where, HA(·) represents the double-layer supergraph attention mechanism of S41, S42, σ(·) is an activation function, W O is a multi-head attention weight coefficient, and || is a connection operation;

[0056] S44: On the basis of multi-head supergraph attention, combined with the concept of Transformer, supergraph Transformer is introduced to strengthen the supergraph node feature representation and relieve feature over-smoothing, and the formula is:

[0057]

[0058] Where, LN(·) is a hierarchical normalization function, and FFN(·) is a feed-forward propagation network.

[0059] As a preferred scheme of the supergraph Transformer-based multi-modal social network depression detection method described in the application, wherein: the specific process of S5 is:

[0060] S51: For all nodes, the final node feature can be obtained by using supergraph Transformer, combined with node mask, and the final supergraph feature z representation is obtained by weighted average summing of all nodes, and provided to softmax to output the predicted label The formula is:

[0061]

[0062] Where, W3 is a trainable weight matrix mapped to the output space, and b is a bias number;

[0063] S52: In combination with the predicted label A binary cross-entropy loss function is used to calculate the training loss, and the formula is:

[0064]

[0065] Where, N is the number of training samples.

[0066] Compared with the prior art:

[0067] 1.The method of the present application is characterized by social network depression detection, which constructs a hypergraph model that can reflect the complexity of multi-modal data, fully utilizes multi-modal data such as social network text and image, and comprehensively mines the complex correlation between cross-modal data.

[0068] 2.The method of the present application models social network data as a hypergraph structure, effectively capturing the connection relationship between nodes of different multi-modal types, and laying a foundation for subsequent feature extraction and classification tasks.

[0069] 3.The method of the present application introduces a hypergraph Transformer module, dynamically calculates node weights, further enhances the richness and expressiveness of hypergraph feature representation, and improves the performance of depression detection tasks. BRIEF DESCRIPTION OF DRAWINGS

[0070] Fig. 1 The flowchart of the present application;

[0071] Fig. 2 The overall architecture of the model of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0073] The present application provides a multi-modal social network depression detection method based on hypergraph Transformer, please refer to Figs. 1-2 , including the following specific steps:

[0074] S1: Preprocess the text data and picture data, for the text data, use a pre-trained language model to extract text features, and combine a topic analysis method to construct a text hypergraph, which fully captures the semantic association in the text data; wherein the pre-trained language model is BERT (Bidirectional Encoder Representations from Transformers), which uses the powerful understanding ability of pre-trained BERT to extract features from social network short text data; the topic analysis method is a document topic generation model Latent Dirichlet Allocation used to identify the hidden topic information in short text due to the lack of training data for short text;

[0075] The specific process of S1 is as follows:

[0076] S11: Perform topic analysis on the preprocessed text data to obtain the relationship between the text and the entity, so as to construct the topic hyperedge of the text hypergraph, which is defined as:

[0077]

[0078] wherein, denotes a node, v i belongs to the topic hyperedge

[0079] S12: After feature extraction of the text combined with the pre-processing model BERT, the text hypergraph is constructed, and the node features of the text hypergraph are constructed, which are defined as:

[0080]

[0081] wherein, denotes a node, v i belongs to the article hyperedge

[0082] S2: For image data, the image hypergraph is constructed by calculating the correlation of feature representation, and the visual hypergraph convolution network is introduced to further extract image features and strengthen the feature representation ability of the hypergraph;

[0083] The specific process of S2 is as follows:

[0084] S21: The image is divided into multiple image patches, and a multi-layer convolution algorithm is introduced to extract image embedding as the feature representation x l of the image, and l is the number of convolution layers;

[0085] S22: Based on the local features extracted by the convolution kernel, in order to construct the relationship in the image space, the correlation of the image patch features in the image space is calculated to give the node weight in each hyperedge, and the formula is:

[0086]

[0087] wherein, W Γ , W λ , W μ are parameters for constructing linear embedding, is the linear embedding obtained by 1x1 convolution (linear transformation) of the image feature representation x l , N represents the number of nodes, is a diagonal matrix representing the distance between nodes obtained by global average pooling, D represents the dimension of the feature, is a weight matrix for evaluating the nodes in the hyperedge obtained by (MxM) convolution, and M represents the number of hyperedges;

[0088] S23: Combined with the global relationship of the image patches in the space represented by Γ(x l ), λ(x l ), μ(x l ), the image hypergraph is constructed, and the formula is:

[0089] H=Γ(x)λ(x)ΓT (x)μ(x)

[0090] S24: In order to reduce the noise of constructing the hypergraph, a hard connection filter is constructed to reduce the noise during model training by removing weak connections. It is defined as:

[0091] H new =x≥c,x∈H;

[0092] S3: Calculate the feature similarity between text entities and image entities to construct the image-text relationship, and construct a comprehensive image-text hypergraph representation of the user-level hypergraph through an adaptive hypergraph fusion algorithm;

[0093] The specific process of S3 is:

[0094] S31: Calculate the feature similarity between the text entity and the image entity to build a relationship between the image entity and the text entity. For each entity, select the entity with a similarity score greater than λ as the object to build the relationship and calculate the feature similarity. The formula is:

[0095] sim(o i ,e j )=X text (e j ) T X img (o i )

[0096] Among them, sim(o i ,e j ) represents the jth text entity e j and the i-th image entity o i The feature similarity of X text (e j ) and X img (o i ) represent text entities e j and image entity o i characteristics;

[0097] S32: For different types of hypergraphs, an adaptive hypergraph fusion algorithm is introduced to construct a graph-text hypergraph. The formula is:

[0098]

[0099] Among them, w k is a trainable parameter shared by all hyperedges in hypergraph k, sigmoid(·) is a normalization function, and M k is the number of hyperedges in hypergraph k, the copy(a,b) function returns a vector of size b, whose value is filled by copying b times, the vector middle is the weight of the i-th hyperedge in hypergraph k, || is the concatenation operation, and H is the hypergraph adjacency matrix obtained by splicing all hypergraph adjacency matrices;

[0100] S33: Combined with the adaptive hypergraph fusion algorithm, the text hypergraph and image hypergraph are subjected to hypergraph fusion convolution to enhance the hypergraph node feature expression, and the formula is:

[0101]

[0102] where D v and D e are the diagonal matrices of vertex degree and edge degree, and Θ is a learnable parameter;

[0103] S4: Introducing hypergraph Transformer, calculating the feature of each node, strengthening the hypergraph feature representation, and providing basis for subsequent classification tasks;

[0104] The specific process of S4 is:

[0105] S41: Introducing node-level hypergraph attention mechanism, updating the hyperedge feature by aggregating all nodes on the hyperedge e j , and highlighting more important nodes, and the formula is:

[0106]

[0107] where σ(·) is a nonlinear activation function, W1 is a trainable weight matrix, and α jk represents the attention coefficient of node v j in hyperedge e k , and a1 is a trainable weight coefficient for measuring the importance of nodes;

[0108] S42: Introducing hyperedge-level attention mechanism, updating the node feature by aggregating the connected hyperedges of the node, and highlighting more important hyperedges, and the formula is:

[0109]

[0110] where W2 is a trainable weight matrix, β ij represents the attention coefficient of node v i to hyperedge e j , and a2 is a trainable weight coefficient for measuring the importance of hyperedges, and || is the concatenation operation;

[0111] S43: Combined with the concept of multi-head attention mechanism, the hypergraph attention is extended to multi-head hypergraph attention (MHHA, Multi-Head Hypergraph Attention), and the formula is:

[0112]

[0113] wherein, HA(·) represents the double-layer supergraph attention mechanism of S41, S42, σ(·) is an activation function, W O is a multi-head attention weight coefficient, and || is a connection operation;

[0114] S44: On the basis of multi-head supergraph attention, the concept of Transformer is combined to introduce supergraph Transformer, strengthen supergraph node feature representation, and relieve feature over-smoothing, and the formula is:

[0115]

[0116] wherein, LN(·) is a hierarchical normalization function, and FFN(·) is a feed-forward propagation network;

[0117] S5: Combined with supergraph feature representation, a depression detection task is performed to realize accurate identification of a depression social network patient;

[0118] The specific process of S5 is as follows:

[0119] S51: For all nodes, the final node feature can be obtained by using supergraph Transformer, combined with node mask, and the final supergraph feature z representation is obtained by weighted average summation of all nodes, and is provided to softmax to output a predicted label The formula is:

[0120]

[0121] wherein, W3 is a trainable weight matrix mapped to an output space, and b is a bias number;

[0122] S52: Combined with the predicted label A binary cross-entropy loss function is used to calculate the training loss, and the formula is:

[0123]

[0124] wherein, N is the number of training samples;

[0125] wherein, the supergraph in S1, S2 and S3 is defined as G=(ν,ε), wherein, ν=(v1,...,v n ) represents a set of nodes in the graph, ε=(e1,...,e m ) represents a set of superedges, and the topological result of the supergraph is represented by an association matrix , which is defined as:

[0126]

[0127] The application provides a multi-modal social network depression detection method based on a hypergraph Transformer, aiming to construct a multi-modal hypergraph, extract multi-modal node features, and utilize a hypergraph Transformer mechanism to realize accurate detection of social network depression users.

[0128] The application has the beneficial effect that the multi-modal data such as text and images in the social network can be effectively utilized, the complex correlation between cross modalities is comprehensively mined, the correlation is fully captured through the structure of the hypergraph, the hypergraph Transformer mechanism is introduced, the richness and expressiveness of the hypergraph feature representation are further enhanced, the accuracy and efficiency of the social network depression detection are improved, and an innovative solution is provided for the auxiliary diagnosis in the field of mental health.

[0129] Although the application has been described with reference to the embodiments above, various modifications can be made thereto and equivalents can be substituted for elements thereof without departing from the scope of the application. In particular, features of the disclosed embodiments can be combined together in any manner, provided that there is no structural conflict. Since these combinations have not been exhaustively described in the specification, they are not described herein only for the purpose of saving space and resources. Therefore, the application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for detecting depression in a multi-modal social network based on hypergraph Transformer, characterized in that, The specific steps include the following: S1: Preprocessing the text data and image data, for the text data, extracting text features using a pre-trained language model, and constructing a text hypergraph using a topic analysis method to fully capture the semantic associations in the text data, the specific process being: S11: Topic analysis of preprocessed text data to obtain the relationship between text and entities to construct the topic hyperedge of the text hypergraph, defined as: wherein, representing node v i whether it belongs to the topic superedge S12: After feature extraction of the text using the pre-trained language model BERT, construct the order hyperedge of the text hypergraph and the node features of the text hypergraph, defined as: wherein, represents a node v i whether it belongs to a syntactic hyperedge S2: For image data, construct an image hypergraph by calculating feature representation correlations, introduce a visual hypergraph convolution network to further extract image features, and strengthen the feature representation capability of the hypergraph; S3: Calculate the feature similarity between text entities and image entities to construct a graph-text relationship, and construct a comprehensive graph-text hypergraph representation of the user level hypergraph through an adaptive hypergraph fusion algorithm; S4: Introduce a hypergraph Transformer to calculate the features of each node, strengthen the hypergraph feature representation, and provide a basis for subsequent classification tasks, the specific process being: S41: Introduce node-level hypergraph attention mechanism, update hyperedge features by aggregating all nodes on hyperedge e j , and highlight more important nodes, whose formula is: where σ(·) is a nonlinear activation function, W1is a trainable weight matrix, and a1is a trainable weight coefficient that measures the importance of node v jk denotes the hyperedge e j v k Note that the attention coefficient a1is a trainable weight coefficient that measures the importance of node v S42: Introduce a hyperedge-level attention mechanism to update the node features by aggregating the connected hyperedges of the node, and highlight more important hyperedges, the formula being: where W2is a trainable weight matrix, β ij denotes the node v i the attention coefficient of the hyperedge e j , a2is a trainable weight coefficient that measures the importance of the hyperedge, and || is the concatenation operation; S43: Combine the concept of multi-head attention mechanism to extend the hypergraph attention to multi-head hypergraph attention, the formula being: wherein, HA(·) represents the double-layer supergraph attention mechanism of S41, S42, σ(·) is an activation function, W O is a multi-head attention weight coefficient, and || is a connection operation. S44: Based on multi-head hypergraph attention, combine the concept of Transformer to introduce a hypergraph Transformer to strengthen the hypergraph node feature representation and alleviate feature over-smoothing, the formula being: Where LN(·) is a hierarchical normalization function, and FFN(·) is a feed-forward propagation network; S5: Combine the hypergraph feature representation to perform depression detection tasks, achieving accurate identification of depression social network patients.

2. The hypergraph Transformer-based multi-modal social network depression detection method of claim 1, wherein, The topic analysis method in S1 is to use the document topic generation model Latent Dirichlet Allocation to identify the latent topic information in short text due to the lack of training data for short text.

3. The hypergraph Transformer-based multi-modal social network depression detection method of claim 1, wherein, The definition of hypergraph in the S1, the S2 and the S3 is G=(ν,ε), wherein ν=(v1,...,v n ) represents the set of nodes in the graph, ε=(e1,...,e m ) represents the set of hyperedges, and the topological result of the hypergraph is represented by an incidence matrix , which is defined as:

4. The hypergraph Transformer-based multi-modal social network depression detection method of claim 1, wherein, The specific process of S2 is: S21: divide the image into multiple image pieces, introduce a multi-layer convolution algorithm, and extract image embedding as a feature representation x of the image l , l is the number of convolution layers; S22: Based on the local features extracted by the convolution kernel, to construct the relationship in the image space, the correlation of the image patch features in the image space is calculated to give the node weight in each hyperedge, the formula being: where W Γ , W λ , and W μ are parameters for constructing linear embeddings, respectively, is a linear embedding of the image feature representation x l obtained by 1x1 convolution, and N denotes the number of nodes, is a diagonal matrix representing node distances, and D denotes the dimension of the feature, is a weight matrix of the evaluation node in its hyperedge obtained by MxM convolution, and M denotes the number of hyperedges; S23: Combine Γ(x l ), λ(x l ), μ(x l ) to represent the global relationship of the image slices in space, and construct an image hypergraph, whose formula is: H = Γ(x l )λ(x l )Γ T (x l )μ(x l ) S24: To reduce the noise in constructing the hypergraph, a hard connection filter is constructed to reduce the noise in model training by deleting weak connections, defined as: H new = x ≥ c, x ∈ H.

5. The hypergraph Transformer-based multi-modal social network depression detection method of claim 1, wherein, The specific process of S3 is: S31: Calculate the feature similarity between text entities and image entities to construct the relationship between image entities and text entities, for each entity, select the entity with a similarity score greater than λ as the object to construct the relationship, calculate the feature similarity, the formula being: sim(o i ,e j ) = X text (e j ) T X img (o i ) wherein sim(o i , e j ) represents the feature similarity of the jth text entity e j and the ith image entity o i , X text (e j ) and X img (o i ) represent the features of the text entity e j and the image entity o i , respectively. S32: For hypergraphs of different categories, introduce an adaptive hypergraph fusion algorithm to construct a graph-text hypergraph, the formula being: Among them, w k is a trainable parameter shared by all hyperedges in hypergraph k, sigmoid(·) is a normalization function, and M k is the number of hyperedges in hypergraph k, the copy(a,b) function returns a vector of size b, whose value is filled by copying b times, the vector middle is the weight of the i-th hyperedge in hypergraph k, || is the connection operation, and H is the hypergraph incidence matrix obtained by concatenating all hypergraph incidence matrices; S33: Combine the adaptive hypergraph fusion algorithm to perform hypergraph fusion convolution on the text hypergraph and the image hypergraph to enhance the hypergraph node feature representation, the formula being: where D v and D e are diagonal matrices of vertex and edge degrees, respectively, and Θ are learnable parameters.

6. The hypergraph Transformer-based multi-modal social network depression detection method of claim 1, wherein, The specific process of S5 is: S51: For all nodes, the final node features can be obtained by using the hypergraph Transformer, combined with the node mask, and the final hypergraph feature z representation is obtained by weighted average summation of all nodes and provided to the softmax to output the predicted label The formula is: Wherein, W3 is a trainable weight matrix mapped to the output space, and b is a bias number; S52: Combine predicted labels A binary cross-entropy loss function is used to calculate the training loss, which is given by: Wherein, N is the number of training samples.

Images