Self-coding graph convolutional network clustering method based on cross attention fusion
By introducing a cross-attention fusion mechanism and self-supervising module in the deep clustering method, combining the content autoencoder and graph convolution autoencoder, the problem of ignoring the paper relationship in the existing technology is solved, and the clustering effect is significantly improved and robustness.
Patent Information
- Application Number
- CN202510005531.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2025-02-07
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing deep clustering methods often ignore relationships between papers, resulting in poor clustering effects, especially when dealing with complex higher-order structural relationships.
A self-coded graph convolution network clustering method based on cross attention fusion is proposed. By constructing a data relationship graph, combining the content self-encoder module and the graph convolution self-encoder module, the cross attention fusion mechanism is used to fuse content information and relationship information. , and introduce self-supervised module to optimize clustering results.
Effectively utilize the higher-order relationships between papers, improve the clustering effect, and significantly improve the accuracy and robustness of clustering, especially when dealing with complex academic paper relationships.
Smart Images
Figure CN119397024A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer machine learning, and in particular to a self-encoding graph convolutional network clustering method based on cross-attention fusion. Background Art
[0002] With the explosive growth of academic journals and conferences, clustering of papers has become one of the important tasks in the field of clustering. Deep clustering methods often extract discriminative information from single data and obtain satisfactory clustering results. However, they usually ignore the relationship between data. Fortunately, graph convolutional networks can handle this relationship and open up new research directions for deep clustering.
[0003] With the development of paper collection and analysis technology, people can not only collect paper information, but also obtain or establish the relationship between papers in the form of graphs. Based on these original relationship graphs, we need to effectively mine the potential relationships between papers. As we all know, the edges in the graph represent direct relationships, also known as first-order structural relationships. However, the relationship between academic papers in the real world is complex. There are still many indirect relationships, namely high-order structural relationships. In other words, two nodes are not directly connected in the graph, but they have many common neighbors. We believe that these two nodes have a high-order structural relationship. In order to improve the clustering effect of deep clustering methods, high-order relationships must be utilized. As an important branch of deep learning methods, graph convolutional networks (GCNs) can mine this potential high-order relationship between data. In addition, GCN encodes this graph structure and node content at the same time, which has received little attention in deep clustering tasks.
[0004] Clustering is an essential topic in the field of data mining. It divides a collection of data objects into multiple clusters consisting of similar objects. Accurate clustering of academic papers has gradually become a widely concerned proposition. Inspired by the powerful feature extraction capabilities of deep convolutional networks, many deep learning-based clustering methods have been proposed in recent years, which have made significant progress in clustering research. However, these deep clustering methods only focus on the content of the papers and usually ignore the relationships between papers, that is, structural information. Therefore, how to embed the relationships between data into deep clustering and improve the clustering effect has become a thorny problem. Summary of the invention
[0005] The purpose of the present invention is to provide a self-encoding graph convolutional network clustering method based on cross-attention fusion to solve the problems raised in the above background technology.
[0006] To achieve the above object, the present invention provides the following technical solutions: The autoencoder graph convolutional network clustering method based on cross-attention fusion has the following specific steps: Step 1: Build a data relationship diagram Before proposing the model, we first construct the necessary graph of the original data. Given a set of N samples and the i-th sample Data ,We use the commonly used K nearest neighbors (KNN) to construct the corresponding graph and display its structural information.
[0007] For image data, we use the heat kernel method to calculate the similarity between samples as follows: (1) Where t represents the time parameter.
[0008] For natural language data, the inner product method is selected to measure the similarity between samples, as shown below: (2) Then, based on the above similarities between all samples, we select the K highest relevant neighbors of each sample and connect them; thus, we can obtain Figure A.
[0009] Step 2: Build a network model 1. Content Autoencoder Module (CAE) It is well known that deep convolutional networks can effectively extract key features from complex data. Autoencoders can reconstruct samples and reduce lost information during the learning process, which is naturally suitable for unsupervised learning. In order to extract content information from the data, we first trained an autoencoder module based on a deep convolutional network, called the content autoencoder module (CAE).
[0010] We will l The input of the layer is represented as H l-1 , then its output H l It can be obtained by the following formula: (3) Among them l The activation function of the layer can be selected according to the actual application, such as ReLU or Sigmoid. U l and B l Respectively represent the first l The weights and biases of the layers. In addition, the first layer input of CAE is the raw data X ,Right now H 0 = X The last layer of output reconstructs the original data, i.e. = Hl , the final loss function of CAE can be defined as: (4) 2. Cross-Attention Fusion Module like Figure 1 As shown in Figure 1, CAE extracts content information from the data, while GAE exploits the correspondence between the data. How to fuse these two types of information for clustering tasks is a key issue. The cross-attention fusion mechanism has global learning capabilities and good parallelism, which can further highlight the key information in the fused representation while suppressing useless noise. Therefore, we adopt the cross-attention fusion mechanism to integrate the content information learned by CAE and the data relationship learned by GAE in a multi-level adaptive manner, namely the so-called cross-attention fusion module.
[0011] We define the cross-attention fusion mechanism as: (5) Where query Q=W q x, key K=W k x and value V=W v x has the same input x=Concat(H1;G l ), and they can be regarded as the mapping of the fused data x in different spaces. H1 is the output of the lth layer in CAE, and G1 is the output of the corresponding layer in GAE. Concat() represents the connection operation between vectors. To generalize the cross-attention fusion mechanism, we first calculate the query q a With key k b The similarity between ab , (6) where q a and k b denote the a-th and b-th vectors in Q and K respectively.
[0012] Then, we ab Perform softmax normalization to obtain the associated weights ab as follows: (7) Finally, the cross attention fusion mechanism R = (r1; r2; r3;; r Datt ) can be written as: (8) in D att is the dimensionality of the fused representation R.
[0013] In order to further perceive different aspects of the data, a multi-head mechanism is also introduced, which contains multiple parallel cross-attention fusion modules. Specifically, we repeatedly project the query Q, key K, and value V to obtain M parallel cross-attention modules. Each cross-attention fusion module is regarded as a head, and each head has a different weight matrix.
[0014] (9) We concatenate the outputs of all M heads and multiply them by the weight matrix , to obtain the final cross-attention fusion representation, (10) 3. Graph Convolutional Autoencoder Module The previous cross-attention mechanism module incorporates the content information H from CAE l Relationship with data from GAE l Combined together to output the fused representation R in different layers l Then, our GAE in this R l Spectral graph convolution is performed on G to learn high-order discriminative information based on the adjacency matrix A. Finally, the intermediate layer G L / 2 Used for clustering.
[0015] The convolution operation in each GAE layer can be expressed as follows: (11) in is an approximate graph convolution filter, is the degree matrix of A, With the identity matrix I and the adjacency matrix A, we use = A + I to ensure self-loops in each node.
[0016] In addition, U1 represents the weight of the lth layer, and G l is the output of the lth layer of GAE.
[0017] (12) After this multi-layer learning, our GAE encoder encodes both the original relation A and the content X into a useful representation G L / 2 In order to retain more information, we set the graph reconstruction and content reconstruction errors as the loss function of GAE.
[0018] i) Graph reconstruction loss We choose a simple inner product operation to reconstruct the relationship between samples: (13) Among them G Lis the output of the last GAE layer, Instead, it is the reconstructed adjacency matrix. The loss of graph reconstruction can be defined as: (14) ii) Content reconstruction loss In addition to the relationship between data, we still restrict the GAE module to preserve sufficient content information, which is very different from formula (14); therefore, we creatively define its loss function as: (15) 4. Self-supervision module Since it is difficult to judge whether the representation learned during the optimization process is optimal for clustering, we need to give an optimization goal for clustering. To solve this problem, we first L / 2 Perform K-means on the cluster to obtain a set of initial cluster centers , where C is the number of clusters. These cluster centers roughly guide G L / 2 The optimization direction of G L / 2 is the so-called self-supervised module.
[0019] We use Student’s t distribution to calculate the intermediate layer representation H L / 2 The similarity with the cluster center c is as follows: (16) where h i Yes H L / 2 The i-th sample of q ic Measures the probability that the i-th sample is assigned to the c-th cluster, so Q=[q ic ] is the overall soft allocation distribution.
[0020] In addition, the choice of target distribution directly determines the clustering quality. We believe that high confidence distributions in Q are reliable and can be used as target distributions. We hold up a picture to emphasize the role of high confidence distributions, (17) in, is the soft cluster frequency. The distributions of Q and P should be close to each other as shown below: (18) Similarly, it is easy to express G L / 2 Construct a soft assignment distribution G, and then we can use the target distribution P to supervise the distribution G, (19) Now, the optimization objectives of GAE and CAE are unified into the distribution P, which makes the learned representations more suitable for clustering tasks.
[0021] 5. Overall loss function The overall objective loss function of the Cross-Attention Fusion-based Enhanced Graph Convolutional Network (CaEGCN) can be summarized as: (20) After optimizing the above objective function, the best representation G is obtained L / 2 Then, we L / 2 Perform the softmax operation on to obtain the final clustering result, that is, .
[0022] Compared with the prior art, the beneficial effects of the present invention are: the cross-attention fusion module in the present invention innovatively connects the content autoencoder module (CAE) related to a single data, the graph convolutional autoencoder module (GAE) related to the relationship between the data in a layer-by-layer manner, which also highlights the discriminative information used for clustering tasks. Note that this module fuses two heterogeneous data. The CAE module is a commonly used autoencoder network, which supplements the content information of the GAE module. In the GAE module, we propose two novel loss functions, which are used to reconstruct the relationship between content and data, respectively. The intermediate layer representation of GAE is used for clustering. Finally, the self-supervision module constrains the distribution of the intermediate layer representations of CAE and GAE to be consistent. Experimental results on different types of datasets demonstrate the superiority and robustness of the CaEGCN we proposed. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 This is a diagram of the architecture of the enhanced graph convolutional network based on cross-attention fusion in the present invention. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] The autoencoder graph convolutional network clustering method based on cross-attention fusion has the following specific steps: Step 1: Build a data relationship diagram Before proposing the model, we first construct the necessary graph of the original data. Given a set of N samples and the i-th sample Data ,We use the commonly used K nearest neighbors (KNN) to construct the corresponding graph and display its structural information.
[0026] For image data, we use the heat kernel method to calculate the similarity between samples as follows: (1) Where t represents the time parameter.
[0027] For natural language data, the inner product method is selected to measure the similarity between samples, as shown below: (2) Then, based on the above similarities between all samples, we select the K highest relevant neighbors of each sample and connect them; thus, we can obtain Figure A.
[0028] Step 2: Build a network model 1. Content Autoencoder Module (CAE) It is well known that deep convolutional networks can effectively extract key features from complex data. Autoencoders can reconstruct samples and reduce lost information during the learning process, which is naturally suitable for unsupervised learning. In order to extract content information from the data, we first trained an autoencoder module based on a deep convolutional network, called the content autoencoder module (CAE).
[0029] We will l The input of the layer is represented as H l-1 , then its output H l It can be obtained by the following formula: (3) Among them l The activation function of the layer can be selected according to the actual application, such as ReLU or Sigmoid. U l and B l Respectively represent the first l The weights and biases of the layers. In addition, the first layer input of CAE is the raw data X ,Right now H 0 = X The last layer of output reconstructs the original data, i.e. = H l , the final loss function of CAE can be defined as: (4) 2. Cross-Attention Fusion Module like Figure 1As shown in Figure 1, CAE extracts content information from the data, while GAE exploits the correspondence between the data. How to fuse these two types of information for clustering tasks is a key issue. The cross-attention fusion mechanism has global learning capabilities and good parallelism, which can further highlight the key information in the fused representation while suppressing useless noise. Therefore, we adopt the cross-attention fusion mechanism to integrate the content information learned by CAE and the data relationship learned by GAE in a multi-level adaptive manner, namely the so-called cross-attention fusion module.
[0030] We define the cross-attention fusion mechanism as: (5) Where query Q=W q x, key K=W k x and value V=W v x has the same input x=Concat(H1;G l ), and they can be regarded as the mapping of the fused data x in different spaces. H1 is the output of the lth layer in CAE, and G1 is the output of the corresponding layer in GAE. Concat() represents the connection operation between vectors. To generalize the cross-attention fusion mechanism, we first calculate the query q a With key k b The similarity between ab , (6) where q a and k b denote the a-th and b-th vectors in Q and K respectively.
[0031] Then, we ab Perform softmax normalization to obtain the associated weights ab as follows: (7) Finally, the cross attention fusion mechanism R = (r1; r2; r3;; r Datt ) can be written as: (8) in D att is the dimensionality of the fused representation R.
[0032] In order to further perceive different aspects of the data, a multi-head mechanism is also introduced, which contains multiple parallel cross-attention fusion modules. Specifically, we repeatedly project the query Q, key K, and value V to obtain M parallel cross-attention modules. Each cross-attention fusion module is regarded as a head, and each head has a different weight matrix.
[0033] (9) We concatenate the outputs of all M heads and multiply them by the weight matrix , to obtain the final cross-attention fusion representation, (10) 3. Graph Convolutional Autoencoder Module The previous cross-attention mechanism module incorporates the content information H from CAE l Relationship with data from GAE l Combined together to output the fused representation R in different layers l Then, our GAE in this R l Spectral graph convolution is performed on G to learn high-order discriminative information based on the adjacency matrix A. Finally, the intermediate layer G L / 2 Used for clustering.
[0034] The convolution operation in each GAE layer can be expressed as follows: (11) in is an approximate graph convolution filter, is the degree matrix of A, With the identity matrix I and the adjacency matrix A, we use = A + I to ensure self-loops in each node.
[0035] In addition, U1 represents the weight of the lth layer, and G l is the output of the lth layer of GAE.
[0036] (12) After this multi-layer learning, our GAE encoder encodes both the original relation A and the content X into a useful representation G L / 2 In order to retain more information, we set the graph reconstruction and content reconstruction errors as the loss function of GAE.
[0037] iii) Graph reconstruction loss We choose a simple inner product operation to reconstruct the relationship between samples: (13) Among them G L is the output of the last GAE layer, Instead, it is the reconstructed adjacency matrix. The loss of graph reconstruction can be defined as: (14) iv) Content reconstruction loss In addition to the relationship between data, we still restrict the GAE module to preserve sufficient content information, which is very different from formula (14); therefore, we creatively define its loss function as: (15) 4. Self-supervision module Since it is difficult to judge whether the representation learned during the optimization process is optimal for clustering, we need to give an optimization goal for clustering. To solve this problem, we first L / 2 Perform K-means on the cluster to obtain a set of initial cluster centers , where C is the number of clusters. These cluster centers roughly guide G L / 2 The optimization direction of G L / 2 is the so-called self-supervised module.
[0038] We use Student’s t distribution to calculate the intermediate layer representation H L / 2 The similarity with the cluster center c is as follows: (16) where h i Yes H L / 2 The i-th sample of q ic Measures the probability that the i-th sample is assigned to the c-th cluster, so Q=[q ic ] is the overall soft allocation distribution.
[0039] In addition, the choice of target distribution directly determines the clustering quality. We believe that high confidence distributions in Q are reliable and can be used as target distributions. We hold up a picture to emphasize the role of high confidence distributions, (17) in, is the soft cluster frequency. The distributions of Q and P should be close to each other as shown below: (18) Similarly, it is easy to express G L / 2 Construct a soft assignment distribution G, and then we can use the target distribution P to supervise the distribution G, (19) Now, the optimization objectives of GAE and CAE are unified into the distribution P, which makes the learned representations more suitable for clustering tasks.
[0040] 5. Overall loss function The overall objective loss function of the Cross-Attention Fusion-based Enhanced Graph Convolutional Network (CaEGCN) can be summarized as: (20) After optimizing the above objective function, the best representation G is obtained L / 2 Then, we L / 2 Perform the softmax operation on to obtain the final clustering result, that is, .
[0041] The present invention has experimentally verified the above method and achieved obvious results, as described below: 1. Dataset ACM contains 3025 papers divided into 3 main categories, and the keywords of each paper are used as its features.
[0042] DBLP is an author network dataset collected from the DBLP website, which includes 4057 authors in 4 categories. The keywords of each author are considered as features.
[0043] Citeseer is a citation network dataset, which consists of paper features and citation connections between papers. The dataset contains 3327 papers in 6 categories.
[0044] HHAR consists of 10,299 sensor records collected from smartphones and smartwatches, classified into 6 categories.
[0045] USPS contains 9298 grayscale images of 10 handwritten digits, each image is 16*16 in size.
[0046] 2. Evaluation indicators In order to evaluate all methods from multiple aspects, we selected four popular clustering evaluation metrics, including accuracy (ACC), normalized mutual information (NMI), average rand index (ARI), and macro F1 score (F1). For all metrics, the higher the score, the better the cluster quality.
[0047] 3. Implementation details We set the dimensions of the CAE and GAE modules to input-500-10-cluster-500-500-input, where input is the dimension of the original data and cluster is the number of cluster categories. In the multi-head cross-attention fusion module, the number of heads is set to 8. In the self-supervised module, the number of Kmeans iterations is set to 1000 to initialize the cluster centers. Finally, we initialize the model parameters using the Xavier method and set the initial learning rate to 0:001.
[0048] 4. Comparison method K-means is a basic clustering algorithm.
[0049] AE performs K-means on the learned low-dimensional representations from a deep autoencoder network.
[0050] IEDC adds clustering-oriented loss and reconstruction loss to the deep autoencoder network.
[0051] VGAE introduces GCN and graph reconstruction loss to build a graph autoencoder network.
[0052] SDCN uses GCN modules to enhance the data representation learned by the autoencoder network, which is an important baseline.
[0053] Table 1: Experimental results Clustering results for all five datasets. We mark the best and second best results in bold and underline.
[0054] In summary, for the purpose of paper clustering, we constructed an autoencoder module, the so-called GAE module, on top of graph convolution (GCN). However, it turns out that GCN tends to over-smooth when the number of layers becomes large. To overcome this shortcoming, we introduced a commonly used autoencoder network to supplement the data content information to GAE, which is similar to the effect of residual networks. Multiple layers are usually stacked in a deep network, and each layer captures different latent features of the data. In order to combine the high-order relations of the data (in GAE) with the latent details of the corresponding content (in the autoencoder) layer by layer, we proposed a cross-attention fusion mechanism that highlights the discriminative information for the clustering task. Different from the traditional attention mechanism, our cross-attention fusion mechanism fuses two kinds of heterogeneous information, namely regular data and irregular graphics.
[0055] In this paper, we propose a cross-attention based deep clustering framework, called Cross-Attention Fusion Based Enhanced Graph Convolutional Network (CaEGCN), which contains four main modules: The cross-attention fusion module innovatively connects the content autoencoder module (CAE) related to a single data and the graph convolutional autoencoder module (GAE) related to the relationship between data in a layer-by-layer manner, which also highlights the discriminative information for clustering tasks. Note that this module fuses two kinds of heterogeneous data. The CAE module is a commonly used autoencoder network, which supplements the content information of the GAE module. In the GAE module, we propose two novel loss functions, which are used to reconstruct the content and the relationship between data, respectively. The intermediate layer representation of GAE is used for clustering. Finally, the self-supervision module constrains the distribution of the intermediate layer representations of CAE and GAE to be consistent. Experimental results on different types of datasets demonstrate the superiority and robustness of our proposed CaEGCN.
[0056] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates. In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. The autoencoder graph convolutional network clustering method based on cross-attention fusion is characterized by: The steps include: Step 1: Build a data relationship diagram: First, construct the necessary graph of the original data. Given a set of N samples and the i-th sample Data , use K nearest neighbor KNN to construct the corresponding graph and display its structural information; For image data, the heat kernel method is used to calculate the similarity between samples as follows: (1) Where t represents the time parameter; For natural language data, the inner product method is selected to measure the similarity between samples, as shown below: (2) Then, based on the above similarity between all samples, the K highest relevant neighbors of each sample are selected and connected; Step 2: Build a network model: (1) Content Autoencoder Module CAE First, we train an autoencoder module based on a deep convolutional network, called the content autoencoder module CAE; The first l The input of the layer is represented as H l-1 , then its output H l Obtained by the following formula: (3) U l and B l Respectively represent the first l The weights and biases of the layers. In addition, the first layer input of the content autoencoder module CAE is the original data X ,Right now H 0 = X， The last layer output reconstructs the original data, i.e. = H l , the final loss function of the content autoencoder module CAE is defined as: （4） (2) Cross-attention fusion module Adopting the cross-attention fusion mechanism, the content information learned by the content autoencoder module CAE and the data relationship learned by the graph convolutional autoencoder module GAE are integrated in a multi-level adaptive manner, namely the cross-attention fusion module; The cross-attention fusion mechanism is defined as: （5） Where query Q=W q x, key K=W k x and value V=W v x has the same input x=Concat(H1;G l ), and regard them as the mapping of the fused data x in different spaces. H1 is the output of the lth layer in the content autoencoder module CAE, and G1 is the output of the corresponding layer in the graph convolutional autoencoder module GAE. Concat() represents the connection operation between vectors. First, the query q is calculated. a With key k b The similarity between ab , （6） where q a and k b denote the a-th and b-th vectors in Q and K respectively; Then, for the above s ab Perform softmax normalization to obtain the associated weights ab as follows: （7） Finally, the cross attention fusion mechanism R = (r1; r2; r3;; r Datt ) is written as: （8） in D att is the dimension of the fused representation R; A multi-head mechanism is introduced to further perceive different aspects of the data. The multi-head mechanism contains multiple parallel cross-attention fusion modules; the query Q, key K and value V are repeatedly projected to obtain M parallel cross-attention modules. Each cross-attention fusion module is regarded as a head, and each head has a different weight matrix: （9） Connect the outputs of all M heads and multiply them by the weight matrix , to obtain the final cross-attention fusion representation: （10） (3) Graph Convolutional Autoencoder Module The cross attention mechanism module takes the content information H from the content autoencoder module CAE and l The relationship G with the data from the graph convolutional autoencoder module GAE l Combined together to output the fused representation R in different layers l ; Then, the graph convolutional autoencoder module GAE is used in this type of R l Spectral graph convolution is performed on the layer to learn high-order discriminative information based on the adjacency matrix A. Finally, the intermediate layer G L / 2 Used for clustering; The convolution operation in each GAE layer of the graph convolutional autoencoder module is expressed as follows: （11） in is an approximate graph convolution filter, is the degree matrix of A, With the identity matrix I and the adjacency matrix A, use = A + I ensures self-loops in each node; In addition, U1 represents the weight of the lth layer, and G l is the output of the l-th layer graph convolutional autoencoder module GAE: （12） After multiple layers of learning, the content autoencoder module CAE encoder encodes both the original relation A and the content X into a useful representation G L / 2 , the graph reconstruction and content reconstruction errors are set as the loss function of the content autoencoder module CAE to retain more information; i) Graph reconstruction loss Reconstruct the relationship between samples: （13） Among them G L is the output of the last GAE layer of the graph convolutional autoencoder module, is the reconstructed adjacency matrix, and the loss of graph reconstruction is defined as: （14） ii) Content reconstruction loss The GAE loss function of the graph convolutional autoencoder module is defined as: （15） (4) Self-supervision module First, through H L / 2 Perform K-means on the cluster to obtain a set of initial cluster centers , where C is the number of clusters, and these cluster centers guide G L / 2 The optimization direction of G L / 2 is the self-supervision module; Use Student’s t distribution to calculate the intermediate layer representation H L / 2 The similarity with the cluster center c is as follows: （16） where h i Yes H L / 2 The i-th sample of ic Measures the probability that the i-th sample is assigned to the c-th cluster, so Q=[q ic ] is the overall soft allocation distribution; Hold up the picture to emphasize the role of high-confidence distribution: （17） in, is the soft cluster frequency, the distributions of Q and P should be close to each other as shown below: （18） G L / 2 Construct a soft assignment distribution G, and then use the target distribution P to supervise the distribution G, （19） The optimization objectives of the graph convolutional autoencoder module GAE and the content autoencoder module CAE are unified into the distribution P, making the learned representation more suitable for clustering tasks; (5) Overall loss function The overall objective loss function of the enhanced graph convolutional network based on cross-attention fusion is summarized as: （20） After optimizing the above objective function, the best representation G is obtained L / 2 , then, in G L / 2 Perform the softmax operation on to obtain the final clustering result, that is, .