Multimodal social network depression detection method based on hypergraph Transform
By constructing multimodal hypergraphs and introducing hypergraph Transformer, the problem that single modal data and graph structure characteristics in the existing technology are ignored, and accurate detection of users with depression on social networks is achieved, and detection accuracy is improved. Efficiency provides auxiliary diagnostic solutions for mental health.
Patent Information
- Application Number
- CN202410949121.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-16
AI Technical Summary
The existing depression detection methods based on social network data mainly rely on single modal data, and cannot fully explore complex relationships between modes, and ignore the graph structure characteristics of social network data, limiting the improvement of detection performance.
Using a multimodal social network depression detection method based on hypergraph Transformer, the text hypergraph and image hypergraph are constructed by preprocessing text and image data, the feature similarity is calculated, and the adaptive hypergraph fusion algorithm and hypergraph are introduced. Transformer, enhances feature representation ability and realizes depression detection.
Effectively utilize multimodal data in social networks to mine cross-modal associations, improve the accuracy and efficiency of depression detection, and provide more effective auxiliary diagnostic methods for the field of mental health.
Smart Images

Figure CN118919056A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing and graph neural network, and specifically to a multimodal social network depression detection method based on a hypergraph Transformer. Background Art
[0002] With the rapid development of social media and mobile Internet, people's online social activities are becoming more and more frequent. People express their emotional state and life status on social networks by posting text, pictures, videos and other content. In recent years, researchers have found that in-depth analysis of these social network data can effectively discover some potential mental health problems and provide important clues for timely intervention and diagnosis.
[0003] Especially in the diagnosis of depression, existing clinical methods often rely on face-to-face questionnaires and professional assessments, which are inefficient. Digital diagnostic methods based on social networks can more continuously and proactively monitor users' emotional changes and detect abnormalities in a timely manner. This not only greatly improves diagnostic efficiency, but also reduces patients' medical costs, showing great application potential in the field of mental health.
[0004] However, most existing depression detection methods based on social network data rely only on single-modality text or image data and cannot fully explore complex cross-modal associations. At the same time, these methods also ignore the graph structure characteristics of social network data itself and cannot effectively model the social connection relationship between users. This, to a certain extent, limits the further improvement of detection performance.
[0005] Therefore, the invention of a multimodal social network depression detection method based on hypergraph transformer can combine the graph structure characteristics of social network data to accurately identify users' depressive symptoms and provide more effective auxiliary diagnosis methods for the field of mental health. This will not only help reduce the social burden of depression, but also protect the mental health of the general public. Summary of the invention
[0006] In view of the above-mentioned problems and / or existing problems in the multimodal social network depression detection method based on hypergraph Transformer, the present invention is proposed.
[0007] Therefore, the purpose of the present invention is to provide a multimodal social network depression detection method based on hypergraph Transformer, which can solve the above-mentioned existing problems.
[0008] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:
[0009] The multimodal social network depression detection method based on hypergraph Transformer includes the following specific steps:
[0010] S1: Preprocess text data and image data. For text data, use the pre-trained language model to extract text features, and build a text hypergraph in combination with the topic analysis method to fully capture the semantic associations in the text data.
[0011] S2: For image data, we construct an image hypergraph by calculating the correlation of feature representations, introduce a visual hypergraph convolutional network, further extract image features, and enhance the feature representation capability of the hypergraph;
[0012] S3: Calculate the feature similarity between text entities and image entities to construct the image-text relationship, and construct a comprehensive image-text hypergraph representation of the user-level hypergraph through an adaptive hypergraph fusion algorithm;
[0013] S4: Introduce the hypergraph transformer to calculate the features of each node, strengthen the hypergraph feature representation, and provide a basis for subsequent classification tasks;
[0014] S5: Combining hypergraph feature representation with depression detection tasks, we achieved accurate identification of patients with depression in social networks.
[0015] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, the pre-trained language model in S1 is BERT, and the pre-trained BERT's powerful text understanding ability is used to extract features from social network short text data.
[0016] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, wherein: the topic analysis method in S1 is to use the document topic generation model Latent Dirichlet Allocation to identify the hidden topic information in the short text due to the lack of training data for the short text.
[0017] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, wherein: the hypergraph in S1, S2 and S3 is defined as G = (ν, ε), wherein ν = (v1, ..., v n ) represents the set of nodes in the graph, ε=(e1,...,e m ) represents the set of hyperedges, and the topological result of the hypergraph is represented by the incidence matrix It is defined as:
[0018]
[0019] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, the specific process of S1 is:
[0020] S11: Perform topic analysis on the preprocessed text data to obtain the relationship between text and entities to construct the topic hyperedge of the text hypergraph, which is defined as:
[0021]
[0022] in, represents a node, v i Subject Superedge
[0023] S12: After extracting features from the text using the preprocessing model BERT, the word order hyperedges and node features of the text hypergraph are constructed, which are defined as:
[0024]
[0025] in, represents a node, v i Belong to the article super edge
[0026] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, the specific process of S2 is:
[0027] S21: Divide the image into multiple image slices, introduce a multi-layer convolution algorithm, and extract image embedding as the feature representation of the image x l , l is the number of convolutional layers;
[0028] S22: Based on the local features extracted by the convolution kernel, in order to construct the relationship in the image space, the correlation of the image patch features in the image space is calculated to give the node weight in each hyperedge. The formula is:
[0029]
[0030] Among them, W Γ , W λ , W μ are the parameters for constructing linear embedding, is the image feature representation x l The linear embedding obtained by 1x1 convolution, N represents the number of nodes, is the diagonal matrix representing node distance obtained by global average pooling, D represents the dimension of the feature, is the weight matrix of the evaluation node on its hyperedge obtained by (MxM) convolution, where M represents the number of hyperedges;
[0031] S23: Combined with Γ(x l ),λ(x l ), μ(x l ) represents the global relationship of image slices in space and constructs an image hypergraph, whose formula is:
[0032] H=Γ(x)λ(x)Γ T (x)μ(x)
[0033] S24: In order to reduce the noise of constructing the hypergraph, a hard connection filter is constructed to reduce the noise during model training by removing weak connections, which is defined as:
[0034] H new =x≥c,x∈H.
[0035] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, the specific process of S3 is:
[0036] S31: Calculate the feature similarity between the text entity and the image entity to build the relationship between the image entity and the text entity. For each entity, select the entity with a similarity score greater than λ as the object to build the relationship and calculate the feature similarity. The formula is:
[0037] sim(o i ,e j )=X text (e j ) T X img (o i )
[0038] Among them, sim(o i ,e j ) represents the jth text entity e j and the i-th image entity o i The feature similarity of X text (e j ) and X img (o i ) represent text entities e j and image entity o i Features;
[0039] S32: For different types of hypergraphs, an adaptive hypergraph fusion algorithm is introduced to construct a graph-text hypergraph. The formula is:
[0040]
[0041] Among them, w k is a trainable parameter shared by all hyperedges in hypergraph k, sigmoid(·) is a normalization function, and M k is the number of hyperedges in hypergraph k. The copy(a,b) function returns a vector of size b, whose value is filled by copying b times. middle is the weight of the ith hyperedge in hypergraph k, || is the connection operation, and H is the hypergraph association matrix obtained by concatenating all hypergraph association matrices;
[0042] S33: Combined with the adaptive hypergraph fusion algorithm, hypergraph fusion convolution is performed on the text hypergraph and image hypergraph to enhance the hypergraph node feature expression. The formula is:
[0043]
[0044] Among them, D v and D e are the diagonal matrices of vertex degree and edge degree respectively, and Θ is a learnable parameter.
[0045] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, the specific process of S4 is:
[0046] S41: Introducing the node-level hypergraph attention mechanism, by aggregating hyperedges j All nodes on , to update the hyperedge features and highlight more important nodes, the formula is:
[0047]
[0048]
[0049] Among them, σ(·) is a nonlinear activation function, W1 is a trainable weight matrix, α jk Represents the hyperedge e j Middle node v k Attention coefficient, a1 is a trainable weight coefficient that measures the importance of a node;
[0050] S42: Introduce a hyperedge-level attention mechanism to update node features by aggregating hyperedges connected to nodes and highlight more important hyperedges. The formula is:
[0051]
[0052] Among them, W2 is the trainable weight matrix, β ij Represents node v i For the hyperedge e jThe attention coefficient, a2 is the trainable weight coefficient that measures the importance of the hyperedge, and || is the connection operation;
[0053] S43: Combining the concept of multi-head attention mechanism, the hypergraph attention is expanded to multi-head hypergraph attention, and its formula is:
[0054]
[0055] Among them, HA(·) represents the double-layer hypergraph attention mechanism of S41 and S42, σ(·) is the activation function, and W O is the multi-head attention weight coefficient, || is the connection operation;
[0056] S44: Based on the multi-head hypergraph attention, combined with the concept of Transformer, the hypergraph transformer is introduced to strengthen the hypergraph node feature representation and alleviate feature over-smoothing. The formula is:
[0057]
[0058] Among them, LN(·) is the hierarchical normalization function, and FFN(·) is the feedforward propagation network.
[0059] As a preferred solution of the multimodal social network depression detection method based on hypergraph Transformer described in the present invention, the specific process of S5 is:
[0060] S51: For all nodes, the final node features can be obtained by using the hypergraph transformer. Combined with the node mask, the final hypergraph feature z representation is obtained by weighted average summation of all nodes and provided to softmax to output the predicted label. The formula is:
[0061]
[0062] Where W3 is the trainable weight matrix mapped to the output space, and b is the number of deviations;
[0063] S52: Combined predicted labels The binary cross entropy loss function is used to calculate the training loss, and its formula is:
[0064]
[0065] Among them, N is the number of training samples.
[0066] Compared with existing technologies:
[0067] 1. The method of the present invention aims at the characteristics of depression detection in social networks, constructs a hypergraph model that can reflect the complexity of multimodal data, makes full use of multimodal data such as text and images in social networks, and comprehensively mines complex cross-modal associations.
[0068] 2. The method of the present invention models social network data as a hypergraph structure, effectively capturing the connection relationship between multimodal nodes, and laying the foundation for subsequent feature extraction and classification tasks.
[0069] 3. The method of the present invention introduces a hypergraph Transformer module, which further enhances the richness and expressiveness of the hypergraph feature representation by dynamically calculating node weights, thereby improving the performance of the depression detection task. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] Figure 1 is a flow chart of the present invention;
[0071] Figure 2 This is the overall architecture diagram of the model of the present invention. DETAILED DESCRIPTION
[0072] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0073] The present invention provides a multimodal social network depression detection method based on hypergraph Transformer, please refer to Figure 1-Figure 2 , including the following specific steps:
[0074] S1: Preprocess text data and image data. For text data, use the pre-trained language model to extract text features, and build a text hypergraph in combination with the topic analysis method to fully capture the semantic associations in the text data. The pre-trained language model is BERT (Bidirectional Encoder Representations from Transformers). The pre-trained BERT has a strong ability to understand text and is used to extract features from short text data on social networks. The topic analysis method uses the document topic generation model Latent Dirichlet Allocation to identify the topic information hidden in short texts due to the lack of training data for short texts.
[0075] The specific process of S1 is:
[0076] S11: Perform topic analysis on the preprocessed text data to obtain the relationship between text and entities to construct the topic hyperedge of the text hypergraph, which is defined as:
[0077]
[0078] in, represents a node, v i Subject Superedge
[0079] S12: After extracting features from the text using the preprocessing model BERT, the word order hyperedges and node features of the text hypergraph are constructed, which are defined as:
[0080]
[0081] in, represents a node, v i Belong to the article super edge
[0082] S2: For image data, we construct an image hypergraph by calculating the correlation of feature representations, introduce a visual hypergraph convolutional network, further extract image features, and enhance the feature representation capability of the hypergraph;
[0083] The specific process of S2 is:
[0084] S21: Divide the image into multiple image slices, introduce a multi-layer convolution algorithm, and extract image embedding as the feature representation of the image x l , l is the number of convolutional layers;
[0085] S22: Based on the local features extracted by the convolution kernel, in order to construct the relationship in the image space, the correlation of the image patch features in the image space is calculated to give the node weight in each hyperedge. The formula is:
[0086]
[0087] Among them, W Γ , W λ , W μ are the parameters for constructing linear embedding, is the image feature representation x l The linear embedding obtained after 1x1 convolution (linear transformation), N represents the number of nodes, is the diagonal matrix representing node distance obtained by global average pooling, D represents the dimension of the feature, is the weight matrix of the evaluation node on its hyperedge obtained by (MxM) convolution, where M represents the number of hyperedges;
[0088] S23: Combined with Γ(x l ),λ(x l ), μ(x l ) represents the global relationship of image slices in space and constructs an image hypergraph, whose formula is:
[0089] H=Γ(x)λ(x)ΓT (x)μ(x)
[0090] S24: In order to reduce the noise of constructing the hypergraph, a hard connection filter is constructed to reduce the noise during model training by removing weak connections, which is defined as:
[0091] H new =x≥c,x∈H;
[0092] S3: Calculate the feature similarity between text entities and image entities to construct the image-text relationship, and construct a comprehensive image-text hypergraph representation of the user-level hypergraph through an adaptive hypergraph fusion algorithm;
[0093] The specific process of S3 is:
[0094] S31: Calculate the feature similarity between the text entity and the image entity to build the relationship between the image entity and the text entity. For each entity, select the entity with a similarity score greater than λ as the object to build the relationship and calculate the feature similarity. The formula is:
[0095] sim(o i ,e j )=X text (e j ) T X img (o i )
[0096] Among them, sim(o i ,e j ) represents the jth text entity e j and the i-th image entity o i The feature similarity of X text (e j ) and X img (o i ) represent text entities e j and image entity o i Features;
[0097] S32: For different types of hypergraphs, an adaptive hypergraph fusion algorithm is introduced to construct a graph-text hypergraph. The formula is:
[0098]
[0099] Among them, w k is a trainable parameter shared by all hyperedges in hypergraph k, sigmoid(·) is a normalization function, and M k is the number of hyperedges in hypergraph k. The copy(a,b) function returns a vector of size b, whose value is filled by copying b times. middle is the weight of the ith hyperedge in hypergraph k, || is the connection operation, and H is the hypergraph association matrix obtained by concatenating all hypergraph association matrices;
[0100] S33: Combined with the adaptive hypergraph fusion algorithm, hypergraph fusion convolution is performed on the text hypergraph and image hypergraph to enhance the hypergraph node feature expression. The formula is:
[0101]
[0102] Among them, D v and D e are the diagonal matrices of vertex degree and edge degree respectively, Θ is a learnable parameter;
[0103] S4: Introduce the hypergraph transformer to calculate the features of each node, strengthen the hypergraph feature representation, and provide a basis for subsequent classification tasks;
[0104] The specific process of S4 is:
[0105] S41: Introducing the node-level hypergraph attention mechanism, by aggregating hyperedges j All nodes on , to update the hyperedge features and highlight more important nodes, the formula is:
[0106]
[0107] Among them, σ(·) is a nonlinear activation function, W1 is a trainable weight matrix, α jk Represents the hyperedge e j Middle node v k Attention coefficient, a1 is a trainable weight coefficient that measures the importance of a node;
[0108] S42: Introduce a hyperedge-level attention mechanism to update node features by aggregating hyperedges connected to nodes and highlight more important hyperedges. The formula is:
[0109]
[0110] Among them, W2 is the trainable weight matrix, β ij Represents node v i For the hyperedge e j The attention coefficient, a2 is the trainable weight coefficient that measures the importance of the hyperedge, and || is the connection operation;
[0111] S43: Combining the concept of multi-head attention mechanism, the hypergraph attention is extended to multi-head hypergraph attention (MHHA, Multi-HeadHypergraphAttention), and its formula is:
[0112]
[0113] Among them, HA(·) represents the double-layer hypergraph attention mechanism of S41 and S42, σ(·) is the activation function, and W O is the multi-head attention weight coefficient, || is the connection operation;
[0114] S44: Based on the multi-head hypergraph attention, combined with the concept of Transformer, the hypergraph transformer is introduced to strengthen the hypergraph node feature representation and alleviate feature over-smoothing. The formula is:
[0115]
[0116] Among them, LN(·) is the layer normalization function, FFN(·) is the feedforward propagation network;
[0117] S5: Combined with hypergraph feature representation, depression detection task was carried out to achieve accurate identification of patients with depression in social networks;
[0118] The specific process of S5 is as follows:
[0119] S51: For all nodes, the final node features can be obtained by using the hypergraph transformer. Combined with the node mask, the final hypergraph feature z representation is obtained by weighted average summation of all nodes and provided to softmax to output the predicted label. The formula is:
[0120]
[0121] Where W3 is the trainable weight matrix mapped to the output space, and b is the number of deviations;
[0122] S52: Combined predicted labels The binary cross entropy loss function is used to calculate the training loss, and its formula is:
[0123]
[0124] Where N is the number of training samples;
[0125] The hypergraphs in S1, S2, and S3 are defined as G = (ν, ε), where ν = (v1, ..., v n ) represents the set of nodes in the graph, ε=(e1,...,e m ) represents the set of hyperedges, and the topological result of the hypergraph is represented by the incidence matrix It is defined as:
[0126]
[0127] This paper proposes a multimodal social network depression detection method based on hypergraph Transformer, aiming to achieve accurate detection of depressed users on social networks by constructing a multimodal hypergraph, extracting multimodal node features, and utilizing the hypergraph Transformer mechanism.
[0128] The beneficial effect of the present invention is that it can effectively utilize multimodal data such as text and images in social networks, comprehensively mine complex cross-modal associations, and fully capture associations through the structure of a hypergraph. The introduction of the hypergraph Transformer mechanism further enhances the richness and expressiveness of the hypergraph feature representation, improves the accuracy and efficiency of depression detection in social networks, and provides an innovative solution for auxiliary diagnosis in the field of mental health.
[0129] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A multimodal social network depression detection method based on hypergraph Transformer, characterized by: The specific steps are as follows: S1: Preprocess text data and image data. For text data, use the pre-trained language model to extract text features, and build a text hypergraph in combination with the topic analysis method to fully capture the semantic associations in the text data. S2: For image data, we construct an image hypergraph by calculating the correlation of feature representations, introduce a visual hypergraph convolutional network, further extract image features, and enhance the feature representation capability of the hypergraph; S3: Calculate the feature similarity between text entities and image entities to construct the image-text relationship, and construct a comprehensive image-text hypergraph representation of the user-level hypergraph through an adaptive hypergraph fusion algorithm; S4: Introduce the hypergraph transformer to calculate the features of each node, strengthen the hypergraph feature representation, and provide a basis for subsequent classification tasks; S5: Combining hypergraph feature representation with depression detection tasks, we achieved accurate identification of patients with depression in social networks.
2. According to claim 1, the method for detecting depression in a multimodal social network based on a hypergraph transformer is characterized in that: The pre-trained language model in S1 is BERT, which uses the powerful text understanding ability of pre-trained BERT to extract features from short text data on social networks.
3. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The topic analysis method in S1 is to use the document topic generation model Latent Dirichlet Allocation to identify the topic information hidden in the short text due to the lack of training data for the short text.
4. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The hypergraph in S1, S2 and S3 is defined as G = (ν, ε), where ν = (v1, ..., v n ) represents the set of nodes in the graph, ε=(e1,...,e m ) represents the set of hyperedges, and the topological result of the hypergraph is represented by the incidence matrix It is defined as:
5. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The specific process of S1 is as follows: S11: Perform topic analysis on the preprocessed text data to obtain the relationship between text and entities to construct the topic hyperedge of the text hypergraph, which is defined as: in, represents a node, v i Subject Superedge S12: After extracting features from the text using the preprocessing model BERT, the word order hyperedges and node features of the text hypergraph are constructed, which are defined as: in, represents a node, v i Belong to the article super edge 6. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The specific process of S2 is: S21: Divide the image into multiple image slices, introduce a multi-layer convolution algorithm, and extract image embedding as the feature representation of the image x l , l is the number of convolutional layers; S22: Based on the local features extracted by the convolution kernel, in order to construct the relationship in the image space, the correlation of the image patch features in the image space is calculated to give the node weight in each hyperedge. The formula is: Among them, W Γ , W λ , W μ are the parameters for constructing linear embedding, is the image feature representation x l The linear embedding obtained by 1x1 convolution, N represents the number of nodes, is the diagonal matrix representing node distance obtained by global average pooling, D represents the dimension of the feature, is the weight matrix of the evaluation node on its hyperedge obtained by (MxM) convolution, where M represents the number of hyperedges; S23: Combined with Γ(x l ),λ(x l ), μ(x l ) represents the global relationship of image slices in space and constructs an image hypergraph, whose formula is: H=Γ(x)λ(x)Γ T (x)μ(x) S24: In order to reduce the noise of constructing the hypergraph, a hard connection filter is constructed to reduce the noise during model training by removing weak connections, which is defined as: H new =x≥c,x∈H。 7. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The specific process of S3 is as follows: S31: Calculate the feature similarity between the text entity and the image entity to build the relationship between the image entity and the text entity. For each entity, select the entity with a similarity score greater than λ as the object to build the relationship and calculate the feature similarity. The formula is: yes(the i ,and j )=X text (and j ) T X img (the i ) Among them, sim(o i ,e j ) represents the jth text entity e j and the i-th image entity o i The feature similarity of X text (e j ) and X img (o i ) represent text entities e j and image entity o i Features; S32: For different types of hypergraphs, an adaptive hypergraph fusion algorithm is introduced to construct a graph-text hypergraph. The formula is: Among them, w k is a trainable parameter shared by all hyperedges in hypergraph k, sigmoid(·) is a normalization function, and M k is the number of hyperedges in hypergraph k. The copy(a,b) function returns a vector of size b, whose value is filled by copying b times. middle is the weight of the ith hyperedge in hypergraph k, || is the connection operation, and H is the hypergraph association matrix obtained by concatenating all hypergraph association matrices; S33: Combined with the adaptive hypergraph fusion algorithm, hypergraph fusion convolution is performed on the text hypergraph and image hypergraph to enhance the hypergraph node feature expression. The formula is: Among them, D v and D e are the diagonal matrices of vertex degree and edge degree respectively, and Θ is a learnable parameter.
8. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The specific process of S4 is as follows: S41: Introducing the node-level hypergraph attention mechanism, by aggregating hyperedges j All nodes on , to update the hyperedge features and highlight more important nodes, the formula is: Among them, σ(·) is a nonlinear activation function, W1 is a trainable weight matrix, α jk Represents the hyperedge e j Middle node v k Attention coefficient, a1 is a trainable weight coefficient that measures the importance of a node; S42: Introduce a hyperedge-level attention mechanism to update node features by aggregating hyperedges connected to nodes and highlight more important hyperedges. The formula is: Among them, W2 is the trainable weight matrix, β ij Represents node v i For the hyperedge e j The attention coefficient, a2 is the trainable weight coefficient that measures the importance of the hyperedge, and || is the connection operation; S43: Combining the concept of multi-head attention mechanism, the hypergraph attention is expanded to multi-head hypergraph attention, and its formula is: Among them, HA(·) represents the double-layer hypergraph attention mechanism of S41 and S42, σ(·) is the activation function, and W O is the multi-head attention weight coefficient, || is the connection operation; S44: Based on the multi-head hypergraph attention, combined with the concept of Transformer, the hypergraph transformer is introduced to strengthen the hypergraph node feature representation and alleviate feature over-smoothing. The formula is: Among them, LN(·) is the hierarchical normalization function, and FFN(·) is the feedforward propagation network.
9. The method for detecting depression in a multimodal social network based on a hypergraph transformer according to claim 1, characterized in that: The specific process of S5 is as follows: S51: For all nodes, the final node features can be obtained by using the hypergraph transformer. Combined with the node mask, the final hypergraph feature z representation is obtained by weighted average summation of all nodes and provided to softmax to output the predicted label. The formula is: Where W3 is the trainable weight matrix mapped to the output space, and b is the number of deviations; S52: Combined predicted labels The binary cross entropy loss function is used to calculate the training loss, and its formula is: Among them, N is the number of training samples.
Citation Information
Patent Citations
Data classification method and device based on multi-head self-attention hypergraph neural network
CN115130544A
Multimodal depression detection system and device based on hypergraph neural network, and medium
CN116130089A
Multimodal multi-label classification method and system based on hypergraph
CN116956128A
Coronary heart disease multi-modal data feature extraction method based on knowledge graph
CN117151215A
Alzheimer's disease early diagnosis method based on adaptive hypergraph representation
CN117995415A
Cited By
API misuse detection method based on hypergraph neural network
CN119597522A
Attitude-guided figure image synthesis method based on converter
CN120298521A
A pose-guided character image synthesis method based on transformer
CN120298521B
Tranform-based hypergraph association privacy protection method
CN120449212A
Community abnormal behavior identification method based on multi-source data
CN122223346A
