Method and system for classifying authors in a heterogeneous academic network based on large language model perception

Abstract

The application discloses a method and system for classifying authors in a heterogeneous academic network based on a large language model perception, and the method comprises the following steps: firstly, processing heterogeneous academic network data to construct a heterogeneous graph, wherein the heterogeneous graph comprises text attributes and graph structure information of nodes; generating semantic embeddings of nodes and relationships by using a large language model, and integrating node and relationship information within a multi-hop range by using a Transformer to obtain deep node embedding vectors; subsequently, designing a contrastive learning loss function in a pre-training stage to optimize embedding representation quality to capture complex semantic relationships between nodes, and inputting the deep node embeddings into a classification model to predict the categories of the author nodes; and finally, optimizing model parameters by using a cross-entropy classification loss function to complete the classification task of the author nodes in the academic network and improve classification accuracy. The application can be applied to scientific research resource management, academic search optimization and related field data analysis.

Description

Technical Field

[0001] This invention belongs to the field of large language models and graph neural networks, and relates to a method and system for classifying authors in heterogeneous academic networks based on large language model perception. Background Technology

[0002] With the continuous growth of scientific research activities, the amount of information in academic networks is increasing exponentially. Academic networks are typically composed of different types of entities such as papers, authors, and institutions, with nodes connected by various edges such as citations and collaborations, forming complex, massive, and heterogeneous graph networks. These graphs not only contain rich structural information but also carry a large amount of textual content, such as paper titles, abstracts, keywords, authors' research interests, and institutions' research directions. How to efficiently manage and utilize this complex heterogeneous data has become a crucial problem that urgently needs to be solved.

[0003] Node classification, a fundamental task in graph mining, aims to automatically infer the category labels of other nodes using the label information of some known nodes. In academic networks, node classification technology has broad application prospects, such as automatically identifying the research direction of papers, analyzing researchers' professional fields, and classifying the technical characteristics of research institutions. Accurate node classification can not only improve the accuracy and efficiency of academic searches and enhance the organization of scientific research resources, but also provide key technical support for recommendation systems, scientific data analysis, and scientific research management. However, academic networks are highly heterogeneous, characterized by diverse node types, rich edge types, complex relationships between nodes, and a large amount of diverse and semantically rich textual information. Traditional graph learning methods often face challenges such as limited expressive power and insufficient generalization performance when dealing with this type of data. To address these challenges, this patent focuses on author node classification in heterogeneous academic networks, proposing a novel classification method that fully utilizes structural information and textual semantics, improving classification performance while maintaining computational efficiency.

[0004] In recent years, Large Language Models (LLMs) have made groundbreaking progress in the field of natural language processing, demonstrating outstanding text understanding and semantic representation capabilities. This has brought new opportunities to graph learning tasks, especially heterogeneous graph modeling containing large amounts of textual information. By introducing LLMs, the textual attribute information carried by nodes in the graph can be mined more deeply, significantly improving the semantic richness and accuracy of node representations. However, effectively integrating LLMs with heterogeneous graph structures still faces many challenges. On the one hand, LLMs themselves are highly dependent on computational resources, resulting in relatively low inference efficiency; on the other hand, how to fully preserve the multi-type relationships and topological information in the graph structure while utilizing their semantic understanding capabilities is also a key technical problem that urgently needs to be solved. These challenges are particularly prominent in scenarios such as academic networks that contain massive amounts of textual data and complex structural relationships. Therefore, there is an urgent need for an efficient heterogeneous graph learning method for LLMs that can fully leverage the semantic advantages of language models while balancing the efficiency of graph structure modeling and inference performance. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method and system for classifying authors in heterogeneous academic networks based on a large language model.

[0006] This invention takes into account the structural characteristics of heterogeneous academic networks, models the data of heterogeneous academic networks as heterogeneous graphs, and, based on the deep understanding of the text and complex semantic relationships of the nodes in the heterogeneous graphs by large language models, completes the classification of author nodes in heterogeneous academic networks by pre-training and fine-tuning on Transformer.

[0007] The technical solution adopted in this invention is:

[0008] It includes the following steps:

[0009] S1. Abstract the citation network of a certain research field into a heterogeneous graph, and define the heterogeneous graph and the meta-paths contained in the heterogeneous graph respectively;

[0010] S2. Based on the defined heterogeneous graph, a large language model is used to generate node embeddings according to the text information of the nodes or the information of the neighboring nodes. Then, based on the relational path hints between nodes in the heterogeneous graph, relational embeddings are generated through the large language model to obtain the semantic representation of nodes and relations.

[0011] S3. Semantic representation of nodes and their relationships: First, mean pooling and fusion operations are used to generate embedding representations of different types of neighbors of the target node within a multi-hop range. Then, a multi-layer Transformer structure is used to perform feature interaction and integration on the embeddings of multiple types of nodes within the same hop range, obtaining the node representation for each hop. Next, another set of Transformers is used to further process the node representations with different numbers of hops, achieving the fusion of multi-hop semantic information. Finally, the fusion results of each hop are weighted and converged based on an attention mechanism to generate a deep-level node embedding of the target node.

[0012] S4. In the pre-training phase, a contrastive learning loss is designed based on the deep representation of the target nodes to improve the embedding quality. By sampling directly connected node pairs in the graph as positive samples and unconnected node pairs as negative samples, a weighted dot product similarity of node pairs is defined. After Sigmoid normalization, the similarity of positive samples is maximized and the similarity of negative samples is minimized, thereby optimizing the deep node representation and enhancing the model's ability to capture heterogeneous structures and semantic relationships.

[0013] S5. After pre-training, the model is used for author node classification to predict the author's research field. Nodes are divided into training, validation, and test sets. Classification is achieved by fine-tuning the deep node representation and optimizing the cross-entropy classification loss. The final classification is output by multiple linear layers and Softmax, with each node predicting the probability of its corresponding class.

[0014] Furthermore, the specific process of S1 is as follows:

[0015] Abstracting the citation network of a research field into a heterogeneous graph, we define a heterogeneous graph as follows: in: ε is the set of nodes; ε is the set of edges; It is the collection of text attributes of a node; It is a collection of node types, including author nodes, paper nodes, institution nodes, and keyword nodes; It is a set of edge types, including relationships between nodes, including "authors wrote papers", "papers cited papers", "authors belong to institutions", and "paper-related keywords". Each node... The type is determined by the mapping function This indicates that the type of each edge e∈ε is determined by the mapping function. Representation. A meta-path is a path on a heterogeneous graph that connects edges of different types. Definition. Represents the meta-path, R l This represents the edge of type l in the metapath.

[0016] Furthermore, the specific process of S2 is as follows:

[0017] S21. Obtain the node and relation embeddings perceived by the large language model; after obtaining the heterogeneous graph, use the large language model to obtain the node and relation embeddings for the nodes and relations in the heterogeneous graph.

[0018] S22, Node Embedding Generation:

[0019] For nodes in an academic network, embeddings are generated based on the richness of their textual information. For nodes with rich textual information, paper nodes and keyword nodes are extracted separately and input into a large language model to generate semantic embeddings to represent the core semantic features of the nodes. Given any text-rich node *s*, its node embedding *u*... s =LLM(x s ), where x s The text attribute information of node s is taken as input, and LLM() is the large language model computation function.

[0020]

[0021] Where l = 1, ..., L llm L represents the l-th layer of the Transformer encoder. llm The value is 32. The initial input is...

[0022] FFN(U)=ReLU(Linear(U)W1+b1)W2+b2

[0023]

[0024] Where MHA(·) represents a multi-head self-attention mechanism, n is the number of attention heads, W1, W2, b1, b2 are learnable weights and biases, and W j Q W j K and W j V These are parameter matrices used for querying, key projection, and value projection, respectively.

[0025] For nodes with limited text information, including author and institution nodes, the semantic embeddings of the target node are generated by aggregating the embeddings of neighboring nodes using mean pooling, leveraging the text information of neighboring nodes. This compensates for the lack of text information and ensures that each node has a reasonable semantic representation. Given any node *s* with limited text information, its node embeddings...

[0026] middle, For all neighboring nodes of node s.

[0027] S22, Relation Embedding Generation:

[0028] In academic networks, the relationships between nodes often possess complex semantics. For example, "paper-citation-paper" represents a citation relationship between papers, and "author-writing-paper" represents a writing relationship between an author and a paper. To capture these semantic relationships, a meta-path is used to design relationship hints. Specific relationship paths are input into a large language model to generate semantic embeddings of the relationships. For different relationships, the hint content includes path type and its proportion information. Relation embeddings are generated through the large language model, effectively capturing the complex semantic associations between nodes. Specifically, for any node s, its i-th hop node t is used to obtain the relationship hint RelationPrompt(s,t). i The maximum number of jumps is 2.

[0029] For one-hop relationships: Given a paper [PH] and another paper [PH], predict the probability of a relationship between them based on the path "paper-paper" with a path percentage of [PA]. Given a paper [PH] and an author [PH], predict the probability of a relationship between them based on the path "paper-author" with a path percentage of [PA]. Given a paper [PH] and a keyword [PH], predict the probability of a relationship between them based on the path "paper-keyword" with a path percentage of [PA]. Given an author [PH] and a paper [PH], predict the probability of a relationship between them based on the path "author-paper" with a path percentage of [PA]. Given an author [PH] and an institution [PH], predict the probability of a relationship between them based on the path "author-institution" with a path percentage of [PA]. Given a keyword [PH] and a paper [PH], predict the probability of a relationship between them based on the path "keyword-paper" with a path percentage of [PA]. Given an institution [PH] and an author [PH], predict the probability that there is a relationship between them based on the path "institution-author" with path percentage [PA].

[0030] For two-hop relationships: Given one paper [PH] and another paper [PH], based on the path "paper-paper-paper", its path percentage is: [PA], and for "paper-author-paper", its path percentage is:

[0031] Given a paper [PH] and an author [PH], based on the path "paper-paper-author", with a path percentage of [PA], predict the probability of a relationship between them. Given a paper [PH] and a keyword [PH], based on the path "paper-paper-keyword", with a path percentage of [PA], predict the probability of a relationship between them.

[0032] [PA], predict the probability of a relationship between them. Given a paper [PH] and an institution [PH], based on the path "paper-author-institution", with a path percentage of [PA], predict the probability of a relationship between them. Given an author [PH] and a paper [PH], based on the path "author-paper-paper", with a path percentage of [PA], predict the probability of a relationship between them. Given an author [PH] and another author [PH], based on the path "author-paper-author", with a path percentage of [PA], predict the probability of a relationship between them.

[0033] Given an author [PH] and a keyword [PH], based on the path "author-paper-keyword", with a path percentage of [PA], predict the probability of a relationship between them. Given a keyword [PH] and a paper [PH], based on the path "keyword-paper-paper", with a path percentage of [PA], predict the probability of a relationship between them.

[0034] [PA], predict the probability of a relationship between them. Given a keyword [PH] and another keyword [PH], based on the path "keyword-paper-keyword", with a path percentage of [PA], predict the probability of a relationship between them. Given a keyword [PH] and an author [PH], based on the path "keyword-paper-author", with a path percentage of [PA], predict the probability of a relationship between them. Given an institution [PH] and another institution [PH], based on the path "institution-author-institution", with a path percentage of [PA], predict the probability of a relationship between them. Given an institution [PH] and a paper [PH], based on the path "institution-author-paper", with a path percentage of [PA], predict the probability of a relationship between them.

[0035] Specifically, for RelationPrompt(s,t) i [PH] is a placeholder, representing the node embeddings of nodes s and t, respectively. [PA] is a placeholder, representing the proportion of each path within hop i to all paths.

[0036] Furthermore, the relational hints are input into a large language model to obtain the relational embeddings r. i (s,t)=LLM(RelationPrompt(s,t) i ).

[0037] Furthermore, the specific process of S3 is as follows:

[0038] S31, Compute the embedding vector of deep nodes in the graph;

[0039] After obtaining the node and relation embeddings, for the complex structure of the heterogeneous graph, the Transformer is used to integrate the semantic information between the node and its multi-hop neighbors to generate a more expressive deep node embedding.

[0040] S32, Multi-hop Embedding Generation:

[0041] First, obtain the multi-hop node embeddings. For the target node s, the embedding within the i-th hop range is:

[0042]

[0043] in, Indicates that the type is τ within the range of the i-th jump. k Node embedding. The target node s is of type τ within the range of i hops. k The set of neighboring nodes. t It is the embedding of each neighbor node t. Mean-Pooling(·) is the mean pooling.

[0044] Subsequently, node embedding and relationship embedding are used for fusion:

[0045]

[0046] in: It is the target node s and the aggregated nodes within the range of the i-th hop. Relational embedding. Fusion(·) is a simple multilayer perceptron that embeds relationships by... and The connection vectors are processed to generate multi-hop embeddings.

[0047] S33. Calculate the single-hop node representation.

[0048] For the embedding set of the target node s within the i-hop range By processing through multiple layers of Transformers, a mixed-type embedding can be obtained.

[0049]

[0050] Where l = 1, ..., L t L represents the l-th layer of the Transformer encoder. t The value is 2, and the initial input is 2.

[0051] For mixed embeddings of target node types, an attention mechanism is used to integrate them and generate the final i-hop node representation.

[0052]

[0053] S34. Obtain deep node representations

[0054] For the embedding set within the multi-hop range of the target node s Multi-hop hybrid embeddings are obtained through multi-layer Transformer processing.

[0055]

[0056] Where l = 1, ..., L h L represents the l-th layer of the Transformer encoder. h The value is 2, and the initial input is 2.

[0057] For the multi-hop hybrid embedding of the target node, an attention mechanism is used to integrate them and generate a deep node representation z. s :

[0058]

[0059] Where W is a learnable parameter and K is the total number of hops.

[0060] Furthermore, the specific process for calculating the pre-training loss in step S4 is as follows:

[0061] The goal of the pre-training phase is to further enhance the expressive power of node embeddings. By designing a pre-training loss function based on contrastive learning, the model can effectively capture structural information and semantic relationships in heterogeneous academic networks. Positive and negative samples are defined, and the quality of embeddings is improved by maximizing the similarity between positive samples while minimizing the similarity between negative samples. Positive samples are directly connected node pairs sampled from the graph structure. For example, an author node and the nodes of the papers it has written are positive samples. Negative samples are randomly sampled node pairs without direct connections or with weak semantic relationships. For example, nodes from papers in different fields can be used as negative samples.

[0062] For the deep node representation z of two nodes s and t... s and zt Their similarity is calculated by dot product and then normalized as follows:

[0063] sim(s,t)=σ(z s W τ(s )·z t W τ(t) )

[0064] Where: σ(·) is the Sigmoid function, used to normalize the similarity values ​​to [0,1]. W τ(s) and W τ(t) It is a learnable weight matrix associated with node types τ(s) and τ(t) to handle embedding differences between heterogeneous nodes.

[0065] The pre-training loss is defined as follows:

[0066]

[0067] Where Pos is the set of positive samples, and Neg is the set of negative samples.

[0068] Furthermore, the author node classification task described in step S5 includes:

[0069] After pre-training the model, it was applied to an author node classification task. The goal of this task is to predict the author's research category (e.g., database, machine learning, natural language processing). All nodes in the heterogeneous academic network dataset were partitioned, with each category divided into training, validation, and test sets in a 100:100:remaining ratio. By fine-tuning the pre-trained model and optimizing the author node prediction loss function, the model efficiently achieves author node classification.

[0070] Multiple linear layers are used to obtain the final prediction result. The input is a deep node representation z. s The output is a C-dimensional score, where C is the number of categories. In the heterogeneous academic network author classification model, C=3. The probability score of the node is obtained after softmax. The author node prediction loss function is:

[0071]

[0072] Among them, L cls It is the cross-entropy classification loss function. s It is the actual label of the node. These are the model's predicted values.

[0073] The model's performance was evaluated using the F1 score, and the model parameters with the best average performance were selected to obtain the final academic network author node prediction model. The table below shows the performance of the heterogeneous academic network author classification method (Ours) based on large language model perception on the academic network dataset DBLP.

[0074]

[0075] A second aspect of the present invention relates to an author classification system for heterogeneous academic networks based on a large language model, characterized in that it includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the author classification method for heterogeneous academic networks based on a large language model of the present invention.

[0076] The advantages of this invention are: it models heterogeneous academic networks as heterogeneous graph data, combining the textual features of nodes with the complex structural characteristics of heterogeneous graphs, thus fully utilizing the semantic information of nodes and the structural information of the network. Secondly, it employs a pre-training and fine-tuning strategy based on large language model perception, significantly improving the model's classification accuracy and generalization ability through optimization of node embedding and relation embedding. Addressing the challenges of diverse node types and complex edge types in academic networks, it designs a relational hint mechanism based on meta-paths and a multi-hop embedding generation method, effectively capturing deep semantic relationships between nodes. To meet the computational challenges of large-scale heterogeneous graph data, it utilizes Transformer for embedding fusion and adapts to the heterogeneity between nodes through an attention mechanism, further enhancing the model's representational ability. In terms of model evaluation, the F1 score is used as a key indicator, combined with multiple experimental verifications, to comprehensively evaluate the model's performance in the author classification task. Finally, this model can efficiently and accurately complete the classification task of author nodes in academic networks, providing important technical support for fields such as academic search, scientific research resource organization, and recommendation systems. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the model architecture of the present invention; Detailed Implementation

[0078] The technical solution of the present invention will be clearly and completely explained and described below.

[0079] Example 1

[0080] See attached document Figure 1 This embodiment describes a heterogeneous academic network author classification method based on a large language model. It includes the following steps: S1, abstracting the citation network of a research field into a heterogeneous graph, and defining the heterogeneous graph and the meta-paths contained within it. The specific process is as follows:

[0081] Abstracting the citation network of a research field into a heterogeneous graph, we define a heterogeneous graph as follows: in: ε is the set of nodes; ε is the set of edges; It is the collection of text attributes of a node; It is a collection of node types, including author nodes, paper nodes, institution nodes, and keyword nodes; It is a set of edge types, including relationships between nodes, including "authors wrote papers", "papers cited papers", "authors belong to institutions", and "paper-related keywords". Each node... The type is determined by the mapping function This indicates that the type of each edge e∈ε is determined by the mapping function. Representation. A meta-path is a path on a heterogeneous graph that connects edges of different types. Definition. Represents the meta-path, R l This represents the edge of type l in the metapath.

[0082] S2. Based on the defined heterogeneous graph, a large language model is used to generate node embeddings according to the textual information of the nodes or the information of their neighboring nodes. Then, based on the relational path hints between nodes in the heterogeneous graph, relational embeddings are generated through the large language model, thereby obtaining the semantic representation of nodes and relations. The specific process is as follows:

[0083] S21. Obtain the node and relation embeddings perceived by the large language model; after obtaining the heterogeneous graph, use the large language model to obtain the node and relation embeddings for the nodes and relations in the heterogeneous graph.

[0084] S22, Node Embedding Generation:

[0085] For nodes in an academic network, embeddings are generated based on the richness of their textual information. For nodes with rich textual information, paper nodes and keyword nodes are extracted separately. For example, for paper nodes and keyword nodes, their titles, abstracts (paper nodes), and term names (keyword nodes) are extracted respectively, and these are input into a large language model to generate semantic embeddings to represent the core semantic features of the nodes. Given any text-rich node *s*, its node embedding *u*... s =LLM(x s ), where x s The text attribute information of node s is taken as input, and LLM() is the large language model computation function.

[0086]

[0087] Where l = 1, ..., Lllm L represents the l-th layer of the Transformer encoder. llm The value is 32. The initial input is...

[0088] FFN(U)=ReLU(Linear(U)W1+b1)W2+b2

[0089]

[0090] Where MHA(·) represents a multi-head self-attention mechanism, n is the number of attention heads, W1, W2, b1, b2 are learnable weights and biases, and W j Q W j K and W j V These are parameter matrices used for querying, key projection, and value projection, respectively.

[0091] For nodes with limited text information (including author and institution nodes), the semantic embeddings of the target node are generated by aggregating the embeddings of neighboring nodes using mean pooling, leveraging the text information of their neighbors. This compensates for the lack of text information and ensures that each node has a reasonable semantic representation. Given any node *s* with limited text information, its node embedding... middle, For all neighboring nodes of node s.

[0092] S22, Relation Embedding Generation:

[0093] In academic networks, the relationships between nodes often possess complex semantics. For example, "paper-citation-paper" represents a citation relationship between papers, and "author-writing-paper" represents a writing relationship between an author and a paper. To capture these semantic relationships, meta-path design is used to create relationship hints. Specific relationship paths are input into a large language model to generate semantic embeddings of the relationships. For instance, for the relationships "paper→citation→paper" and "paper→author→paper," the hints include path type and its proportion information. Relation embeddings are generated through the large language model, effectively capturing the complex semantic associations between nodes. Specifically, for any node s, its i-th hop node t is used to obtain the relationship hint RelationPrompt(s,t). i The maximum number of jumps is 2.

[0094] For one-hop relationships: Given a paper [PH] and another paper [PH], predict the probability of a relationship between them based on the path "paper-paper" (path percentage: [PA]). Given a paper [PH] and an author [PH], predict the probability of a relationship between them based on the path "paper-author" (path percentage: [PA]). Given a paper [PH] and a keyword [PH], predict the probability of a relationship between them based on the path "paper-keyword" (path percentage: [PA]). Given an author [PH] and a paper [PH], predict the probability of a relationship between them based on the path "author-paper" (path percentage: [PA]). Given an author [PH] and an institution [PH], predict the probability of a relationship between them based on the path "author-institution" (path percentage: [PA]). Given a keyword [PH] and a paper [PH], predict the probability of a relationship between them based on the path "keyword-paper" (path percentage: [PA]). Given an institution [PH] and an author [PH], predict the probability of a relationship between them based on the path "institution-author" (path percentage: [PA]).

[0095] For two-hop relationships: Given a paper [PH] and another paper [PH], predict the probability of a relationship between them based on the paths "paper-paper-paper" (path percentage: [PA]), "paper-author-paper" (path percentage: [PA]), and "paper-keyword-paper" (path percentage: [PA]). Given a paper [PH] and an author [PH], predict the probability of a relationship between them based on the path "paper-paper-author" (path percentage: [PA]). Given a paper [PH] and a keyword [PH], predict the probability of a relationship between them based on the path "paper-paper-keyword" (path percentage: [PA]). Given a paper [PH] and an institution [PH], predict the probability of a relationship between them based on the path "paper-author-institution" (path percentage: [PA]). Given an author [PH] and a paper [PH], predict the probability of a relationship between them based on the path "author-paper-paper" (path percentage: [PA]). Given one author [PH] and another author [PH], based on the paths "author-paper-author" (path percentage: [PA]) and "author-institution-author" (path percentage: [PA]),

[0096] [PA]) predicts the probability of a correlation between them. Given an author [PH] and a keyword

[0097] [PH], predict the probability of a relationship between them based on the path "author-paper-keyword" (path percentage: [PA]). Given a keyword [PH] and a paper [PH], predict the probability of a relationship between them based on the path "keyword-paper-paper" (path percentage: [PA]). Given a keyword [PH] and another keyword [PH], predict the probability of a relationship between them based on the path "keyword-paper-keyword" (path percentage: [PA]). Given a keyword [PH] and an author [PH], predict the probability of a relationship between them based on the path "keyword-paper-author" (path percentage: [PA]). Given an institution [PH] and another institution [PH], predict the probability of a relationship between them based on the path "institution-author-institution" (path percentage: [PA]). Given an institution [PH] and a paper [PH], predict the probability of a relationship between them based on the path "institution-author-paper" (path percentage: [PA]).

[0098] Specifically, for RelationPrompt(s,t) i [PH] is a placeholder, representing the node embeddings of nodes s and t, respectively. [PA] is a placeholder, representing the proportion of each path within hop i to all paths.

[0099] Furthermore, the relational hints are input into a large language model to obtain the relational embeddings r. i (s,t)=LLM(RelationPrompt(s,t) i ).

[0100] S3. Semantic representation of nodes and their relationships: First, mean pooling and fusion operations are used to generate embedding representations of different types of neighbors of the target node within a multi-hop range. Then, a multi-layer Transformer structure is used to perform feature interaction and integration on the embeddings of multiple types of nodes within the same hop range, obtaining the node representation for each hop. Next, another set of Transformers is used to further process the node representations with different numbers of hops, achieving the fusion of multi-hop semantic information. Finally, the fusion results of each hop are weighted and converged based on an attention mechanism to generate a deep-level node embedding of the target node. The specific process is as follows:

[0101] S31, Compute the embedding vector of deep nodes in the graph;

[0102] After obtaining the node and relation embeddings, for the complex structure of the heterogeneous graph, the Transformer is used to integrate the semantic information between the node and its multi-hop neighbors to generate a more expressive deep node embedding.

[0103] S32, Multi-hop Embedding Generation:

[0104] First, obtain the multi-hop node embeddings. For the target node s, the embedding within the i-th hop range is:

[0105]

[0106] in, Indicates that the type is τ within the range of the i-th jump. k Node embedding. The target node s is of type τ within the range of i hops. k The set of neighboring nodes. t It is the embedding of each neighbor node t. Mean-Pooling(·) is the mean pooling.

[0107] Subsequently, node embedding and relationship embedding are used for fusion:

[0108]

[0109] in: It is the target node s and the aggregated nodes within the range of the i-th hop. Relational embedding.

[0110] Fusion(·) is a simple multilayer perceptron that, through the analysis of... and The connection vectors are processed to generate multi-hop embeddings.

[0111] S33. Calculate the representation of a single-hop node;

[0112] For the embedding set of the target node s within the i-hop range By processing through multiple layers of Transformers, a mixed-type embedding can be obtained.

[0113]

[0114]

[0115] Where l = 1, ..., L t L represents the l-th layer of the Transformer encoder. t The value is 2, and the initial input is 2.

[0116] For mixed embeddings of target node types, an attention mechanism is used to integrate them and generate the final i-hop node representation.

[0117]

[0118] S34. Obtain the deep-level node representation;

[0119] For the embedding set within the multi-hop range of the target node s Multi-hop hybrid embeddings are obtained through multi-layer Transformer processing.

[0120]

[0121] Where l = 1, ..., L h L represents the l-th layer of the Transformer encoder. h The value is 2, and the initial input is 2.

[0122] For the multi-hop hybrid embedding of the target node, an attention mechanism is used to integrate them and generate a deep node representation z. s :

[0123]

[0124] Where W is a learnable parameter and K is the total number of hops.

[0125] S4. In the pre-training phase, based on the deep representation of the target nodes, a contrastive learning loss is designed to improve embedding quality. Directly connected node pairs in the sampling graph are used as positive samples, and unconnected node pairs are used as negative samples. A weighted dot product similarity of node pairs is defined, and after sigmoid normalization, the similarity of positive samples is maximized and the similarity of negative samples is minimized, thereby optimizing the deep node representation and enhancing the model's ability to capture heterogeneous structures and semantic relationships. The specific process is as follows:

[0126] The goal of the pre-training phase is to further enhance the expressive power of node embeddings. By designing a pre-training loss function based on contrastive learning, the model can effectively capture structural information and semantic relationships in heterogeneous academic networks. Positive and negative samples are defined, and the quality of embeddings is improved by maximizing the similarity between positive samples while minimizing the similarity between negative samples. Positive samples are directly connected node pairs sampled from the graph structure. For example, an author node and the nodes of the papers it has written are positive samples. Negative samples are randomly sampled node pairs without direct connections or with weak semantic relationships. For example, nodes from papers in different fields can be used as negative samples.

[0127] For the deep node representation z of two nodes s and t... s and z t Their similarity is calculated by dot product and then normalized as follows:

[0128] sim(s,t)=σ(z s W τ(s) ·z t Wτ(t) )

[0129] Where: σ(·) is the Sigmoid function, used to normalize the similarity values ​​to [0,1]. W τ(s) and W τ(t) It is a learnable weight matrix associated with node types τ(s) and τ(t) to handle embedding differences between heterogeneous nodes.

[0130] The pre-training loss is defined as follows:

[0131]

[0132] Where Pos is the set of positive samples, and Neg is the set of negative samples.

[0133] S5. After pre-training, the model is used for author node classification to predict the author's research field. Nodes are divided into training, validation, and test sets. Classification is achieved by fine-tuning the deep node representation and optimizing the cross-entropy classification loss. The final classification is output by multiple linear layers and Softmax, with each node predicting the probability of its corresponding class. The specific process is as follows:

[0134] After pre-training the model, it was applied to an author node classification task. The goal of this task is to predict the author's research category (e.g., database, machine learning, natural language processing). All nodes in the heterogeneous academic network dataset were partitioned, with each category divided into training, validation, and test sets in a 100:100:remaining ratio. By fine-tuning the pre-trained model and optimizing the author node prediction loss function, the model efficiently achieves author node classification.

[0135] Multiple linear layers are used to obtain the final prediction result. The input is a deep node representation z. s The output is a C-dimensional score, where C is the number of categories. In the heterogeneous academic network author classification model, C=3. The probability score of the node is obtained after softmax. The author node prediction loss function is:

[0136]

[0137] Among them, L cls It is the cross-entropy classification loss function. s It is the actual label of the node. These are the model's predicted values.

[0138] The model's performance was evaluated using the F1 score, and the model parameters with the best average performance were selected to obtain the final academic network author node prediction model. The table below shows the performance of the heterogeneous academic network author classification method (Ours) based on large language model perception on the academic network dataset DBLP.

[0139]

[0140] Example 2

[0141] This embodiment relates to a heterogeneous academic network author classification system based on large language model perception. It is characterized by including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they implement the heterogeneous academic network author classification method based on large language model perception of Embodiment 1.

[0142] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A heterogeneous academic network author classification method based on large language model perception, characterized in that It includes the following steps: S1. Abstract the citation network of a certain research field into a heterogeneous graph, and define the heterogeneous graph and the meta-paths contained in the heterogeneous graph respectively; S2. Based on the defined heterogeneous graph, use a large language model to generate node embeddings according to the text information of the nodes or the information of the neighboring nodes. Then, based on the relational path hints between nodes in the heterogeneous graph, relational embeddings are generated through a large language model to obtain semantic representations of nodes and relations; S3. Semantic representation of nodes and their relationships: First, through mean pooling and fusion operations, embedding representations of different types of neighbors of the target node within a multi-hop range are generated. Subsequently, a multi-layer Transformer structure is used to perform feature interaction and integration on the embeddings of multiple types within the same hop range to obtain the node representation for each hop. Next, another set of Transformers is used to further process the node representations with different numbers of hops, achieving the fusion of multi-hop semantic information. Finally, the fusion results of each hop are weighted and converged based on an attention mechanism to generate the deep node embedding of the target node. Specifically, this includes: S31, Computational graph deep node embedding vector; After obtaining the node and relation embeddings, for the complex structure of the heterogeneous graph, the Transformer is used to integrate the semantic information between the node and its multi-hop neighbors to generate a more expressive deep node embedding. S32, Multi-hop Embedding Generation: First, obtain the multi-hop node embedding; for the target node... In the The embedding within the jump range is: in, Indicates the first Type within the jump range Node embedding; The target node exist Type within the jump range The set of neighboring nodes; It is each neighbor node Embedding; It is mean pooling; Subsequently, node embedding and relationship embedding are used for fusion: in: The target node With the Aggregation nodes within jump range Relational embedding; It is a simple multilayer perceptron, which, through the analysis of... and The connection vectors are processed to generate multi-hop embeddings; S33. Calculate the representation of a single-hop node; For the target node exist Embedded sets within jump range By processing through multiple layers of Transformers, a mixed-type embedding can be obtained. : , in, This represents the first... layer, The value is 2, and the initial input is 2. ; For mixed embeddings of target node types, an attention mechanism is used to integrate them and generate the final result. Jump node representation : S34. Obtain the deep-level node representation; For the target node Embedded sets within multi-hop range Multi-hop hybrid embeddings are obtained through multi-layer Transformer processing. : in, Represents the first... of the Transformer encoder layer, The value is 2, and the initial input is 2. ; For multi-hop hybrid embeddings of target nodes, an attention mechanism is used to integrate them and generate deep node representations. : in, These are learnable parameters; It is the total number of hops; S4. In the pre-training stage, based on the deep representation of the target node, a contrastive learning loss is designed to improve the embedding quality. By sampling directly connected node pairs in the graph as positive samples and unconnected node pairs as negative samples, a weighted dot product similarity of node pairs is defined. After Sigmoid normalization, the similarity of positive samples is maximized and the similarity of negative samples is minimized, thereby optimizing the deep node representation and enhancing the model's ability to capture heterogeneous structures and semantic relationships. S5. After pre-training, the model is used for author node classification tasks to predict the research fields of authors. The nodes are divided into training, validation and test sets. The classification is completed by fine-tuning the deep node representation and optimizing the cross-entropy classification loss. The final classification is output by multiple linear layers and Softmax, and each node predicts the probability of the corresponding class.

2. The method for classifying authors in heterogeneous academic networks based on large language model perception as described in claim 1, characterized in that: The specific process of S1 is as follows: Abstracting the citation network of a research field into a heterogeneous graph, we define a heterogeneous graph as follows: ,in: It is a set of nodes; It is a set of edges; It is the collection of text attributes of a node; It is a collection of node types, including author nodes, paper nodes, institution nodes, and keyword nodes; It is a set of edge types, including relationships between nodes, including "authors wrote papers", "papers cited papers", "authors belong to institutions", and "paper-related keywords"; each node The type is determined by the mapping function This indicates that each edge The type is determined by the mapping function Representation; a meta-path is a path connecting edges of different types on a heterogeneous graph; definition Indicates the meta-path. This represents the edge of type l in the metapath. .

3. The method for classifying authors in heterogeneous academic networks based on large language model perception as described in claim 2, characterized in that: The specific process of S2 is as follows: S21. Obtain the node and relation embeddings perceived by the large language model; after obtaining the heterogeneous graph, use the large language model to obtain the node and relation embeddings for the nodes and relations in the heterogeneous graph. S22, Node Embedding Generation: For nodes in an academic network, embeddings are generated based on the richness of their textual information. For nodes with rich textual information, paper nodes and keyword nodes are extracted separately and input into a large language model to generate semantic embeddings, which are used to represent the core semantic features of the nodes. Given any text-rich node Its node embedding ;in, For nodes Text attribute information, This is a computation function for large language models; for LLM(), its input is text attribute information. ; , in, This represents the first... layer, The value is 32; the initial input is ; in, This indicates a multi-head self-attention mechanism. It's about the number of heads to focus on. These are learnable weights and biases. and These are the parameter matrices used for querying, key-value projection, and other operations, respectively. For nodes with limited text information, including author and institution nodes, the text information of neighboring nodes is used to aggregate the embeddings of neighboring nodes through mean pooling to generate the semantic embedding of the target node. This compensates for the lack of text information and ensures that each node has a reasonable semantic representation. Given any node with limited text information... Its node embedding middle, For nodes All neighboring nodes; S22, Relation Embedding Generation: To capture semantic relationships in academic networks, a meta-path design is used to provide relation hints. Specific relation paths are input into a large language model to generate semantic embeddings of the relationships. For different relationships, hint content is designed, including path type and its proportion information. Relation embeddings are then generated through the large language model, effectively capturing complex semantic associations between nodes. Specifically, for any node... Regarding its first Jump Node To obtain relationship hints The maximum number of jumps is 2. For one-hop relationships: Given a paper [PH] and another paper [PH], predict the probability of a relationship between them based on the path "paper-paper" with a path percentage of [PA]; Given a paper [PH] and an author [PH], predict the probability of a relationship between them based on the path "paper-author" with a path percentage of [PA]; Given a paper [PH] and a keyword [PH], predict the probability of a relationship between them based on the path "paper-keyword" with a path percentage of [PA]; Given an author [PH] and a paper [PH], based on the path "... Given an author-paper path with a percentage of [PA], predict the likelihood of a connection between them; given an author [PH] and an institution [PH], predict the likelihood of a connection between them based on the path "author-institution" with a percentage of [PA]; given a keyword [PH] and a paper [PH], predict the likelihood of a connection between them based on the path "keyword-paper" with a percentage of [PA]; given an institution [PH] and an author [PH], predict the likelihood of a connection between them based on the path "institution-author" with a percentage of [PA]. For two-hop relationships: Given one paper [PH] and another paper [PH], based on the path "paper-paper-paper", its path percentage is [PA], and for "paper-author-paper", its path percentage is [PA]. Given the path "paper-keyword-paper", with a path percentage of [PA], predict the probability of a relationship between them; given a paper [PH] and an author [PH], based on the path "paper-paper-author", with a path percentage of [PA], predict the probability of a relationship between them; given a paper [PH] and a keyword [PH], based on the path "paper-paper-keyword", with a path percentage of [PA], predict the probability of a relationship between them; given a paper [PH] and an institution [PH], based on the path "paper-author-institution", with a path percentage of [PA], predict the probability of a relationship between them; given an author [PH] and a paper [PH], based on the path "author-paper-paper", with a path percentage of [PA], predict the probability of a relationship between them; given an author [PH] and another author [PH], based on the path "author-paper-author", with a path percentage of [PA], and "author-institution-author", with a path percentage of [PA], predict the probability of a relationship between them; given a... Given an author [PH] and a keyword [PH], based on the path "author-paper-keyword" with a path percentage of [PA], predict the probability of a relationship between them; given a keyword [PH] and a paper [PH], based on the path "keyword-paper-paper" with a path percentage of [PA], predict the probability of a relationship between them; given a keyword [PH] and another keyword [PH], based on the path "keyword-paper-keyword" with a path percentage of [PA], predict the probability of a relationship between them; given a keyword [PH] and an author [PH], based on the path "keyword-paper-author" with a path percentage of [PA], predict the probability of a relationship between them; given an institution [PH] and another institution [PH], based on the path "institution-author-institution" with a path percentage of [PA], predict the probability of a relationship between them; given an institution [PH] and a paper [PH], based on the path "institution-author-paper" with a path percentage of [PA], predict the probability of a relationship between them. Among them, for [PH] is a placeholder for each node. and The node is embedded; [PA] is a placeholder, corresponding to The proportion of each path within a jump to all paths; Furthermore, relational hints are input into a large language model to obtain relational embeddings. .

4. The method for classifying authors in heterogeneous academic networks based on large language model perception as described in claim 3, characterized in that: Step S4 includes calculating the pre-training loss, the specific process of which is as follows: The goal of the pre-training phase is to further improve the expressive power of node embeddings. By designing a pre-training loss function based on contrastive learning, the model can effectively capture structural information and semantic relationships in heterogeneous academic networks. Positive and negative samples are defined, and the quality of embeddings is improved by maximizing the similarity between positive samples and minimizing the similarity between negative samples. Positive samples are node pairs that are directly connected by sampling the graph structure, while negative samples are node pairs that are randomly sampled without direct connections or with weak semantic relationships. For two nodes and Deep node representation and Their similarity is calculated by dot product and then normalized as follows: in: It's the Sigmoid function, used to normalize similarity values ​​to... and Is related to node type and The relevant learnable weight matrix is ​​used to handle embedding differences between heterogeneous nodes; The pre-training loss is defined as follows: in, It is a set of positive samples; It is a set of negative samples.

5. The method for classifying authors in heterogeneous academic networks based on large language model perception as described in claim 4, characterized in that: The author node classification task in step S5 includes: After completing the pre-trained model, the model was applied to the author node classification task. The goal of this task is to predict the research category of the authors. All nodes in the heterogeneous academic network dataset are divided into training, validation and test sets according to the ratio of 100, 100 and the remainder. By fine-tuning the pre-trained model and optimizing the author node prediction loss function, the model can efficiently classify author nodes. Multiple linear layers are used to obtain the final prediction result; the input is a deep node representation. The output is a C-dimensional score, where C is the number of categories. In the heterogeneous academic network author classification model, C = 3. The probability score of the node is obtained after softmax. The author node prediction loss function is: in, It is the cross-entropy classification loss function; It is the actual label of the node; These are the model's predicted values.

6. A heterogeneous academic network author classification system based on large language model perception, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the heterogeneous academic network author classification method based on large language model awareness as described in any one of claims 1-5.

Images