Hypergraph-based citation recommendation method and electronic device

Abstract

The present invention discloses a citation recommendation method and electronic device based on a hypergraph, wherein the method comprises: generating an initial embedding vector for the text content of an existing paper; clustering the subvectors of all papers according to intent to generate an intent prototype vector; constructing a citation hypergraph to reflect the relationship between citing papers and cited papers, adjusting the weights of the paper nodes in the hypergraph information propagation, and performing micro-intention decoupling; classifying the citation intent in macro-semantics to perform macro-intention decoupling; calculating the relevance score between the manuscript and the existing paper, sorting the manuscript from high to low according to the relevance score, and generating a list of recommended papers. In terms of recommendation accuracy, the present invention can more accurately understand the semantic intent between citing and cited papers, significantly improving the accuracy of recommendation results; effectively modeling the topological characteristics of the paper co-citation network, making the recommendation results more consistent with the citation pattern of specific intent.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing for document analysis, and in particular to a hypergraph-based citation recommendation method and electronic equipment. Background Art

[0002] Looking at the current state of citation recommendation research, most existing citation recommendation systems focus primarily on semantic similarity matching or utilize structured information such as authors and citations for recommendations, but rarely characterize the task from the perspective of citation intent. In reality, scholars' reference needs often vary in different contexts. If the actual intent of citation behavior cannot be effectively captured and utilized, the recommendation results may deviate from the user's actual needs.

[0003] Existing citation research based on graph neural networks is mostly based on traditional graphs, which makes it difficult to fully capture the complex multi-dimensional relationships in citation recommendation scenarios. Hypergraphs are a more general and flexible graph structure representation that can naturally describe complex high-order relationships. However, there is currently little research on applying hypergraph neural networks to citation recommendation tasks.

[0004] How to effectively design hypergraph neural network models, reasonably define hypergraph data structures, learn potential hypergraph representations, etc., as well as how to model citation intentions based on the professionalism and complexity of academic scenarios, and automatically and efficiently capture and utilize the true intentions of citations are issues that need to be studied. Summary of the Invention

[0005] To solve the above problems, the present invention proposes a hypergraph-based citation recommendation method and electronic device, which aims to capture the implicit multiple citation intentions and high-order associations in academic citation collections, thereby improving the accuracy and relevance of citation recommendations.

[0006] In one aspect, the present invention provides a hypergraph-based citation recommendation method, comprising the steps of constructing a model and recommending citations, wherein the constructing the model comprises:

[0007] S1. Collect existing papers to obtain the target paper set, preprocess the text content of each paper, and generate the initial embedding vector;

[0008] S2. Decompose the initial embedding vector of each paper into several sub-vectors under potential citation intentions, namely, intention embedding vectors; cluster all the sub-vectors of the paper according to the intention to generate the intention prototype vector;

[0009] S3. Construct a citation hypergraph to reflect the relationship between citing papers and cited papers. The hypergraph includes co-citation hyperedges and intention hyperedges. Adjust the weights of paper nodes in the hypergraph information propagation to perform micro-intention decoupling. Classify citation intentions based on macro-semantics to perform macro-intention decoupling.

[0010] The citation recommendation step includes:

[0011] S4. Process the new manuscript using the model to generate an intent embedding vector, calculate the relevance score between the manuscript and the existing papers, sort them from high to low according to the relevance score, and generate a list of recommended papers.

[0012] Furthermore, the preprocessing of the text content of each paper includes processing the text content of each paper using a pre-trained language model or traditional natural language processing technology, and the text content includes at least one of the title, abstract, keywords, and author.

[0013] Furthermore, decomposing the embedding vector of each paper into several sub-vectors under potential citation intentions specifically includes slicing the embedding vector according to the citation intention, dividing it into K independent sub-vectors, each sub-vector corresponding to the embedding vector of the paper under the corresponding citation intention.

[0014] Furthermore, the citation hypergraph includes a node set and a hyperedge set, each node in the node set represents a cited paper, and the hyperedge set includes co-citation hyperedges and intention hyperedges, the co-citation hyperedges connect all papers cited by the same paper, and the intention hyperedges connect all papers under the same intention.

[0015] Furthermore, it also includes filtering sub-hyperedges that are not related to the intent. Specifically, when the cosine value between the paper and a certain intent is greater than the set threshold, there is a hyperedge between the two, otherwise it does not exist.

[0016] Furthermore, the adjustment of the weight of the paper's node in the hypergraph information propagation and the micro-intention decoupling specifically include deploying a hypergraph attention network in each subgraph of specific intent, adaptively adjusting the weight of the node in the hypergraph information propagation, updating the intention embedding of the hyperedge by aggregating node features related to the hyperedge, and updating the intention embedding of the node by integrating the hyperedge information associated with the node; finally, each node obtains a decoupled embedding for different reference intentions.

[0017] The macro-semantic classification of reference intent and the macro-intention decoupling specifically include extracting features that are helpful for intent classification from the decoupled embedding through a multi-layer perceptron, and then calculating the predicted probability of each reference intent.

[0018] The calculation of the relevance score between the manuscript and the existing paper in step S4 is specifically as follows:

[0019]

[0020] in Indicates manuscript Embedding on intent k, Indicates existing papers Embedding on intent k, Comprehensively evaluate manuscripts by integrating different perspectives of intent Compared with existing papers correlation.

[0021] As a preferred embodiment, the present invention also includes a model optimization step, specifically, adding the loss function of the citation recommendation step to the loss function of the macro-intention decoupling step to obtain the loss function of the intent-aware citation recommendation model.

[0022] On the other hand, the present invention provides an electronic device comprising a memory, a processor and a computer program stored in the memory, wherein when the processor executes the program, the steps of the hypergraph-based citation recommendation method described in any one of the above technical solutions are implemented.

[0023] This invention significantly improves the accuracy and relevance of citation recommendations, demonstrating its advancement and practicality in academic citation recommendation tasks. Regarding recommendation accuracy, intent decoupling enables the model to more accurately understand the semantic meaning of the citing and cited papers, significantly improving the accuracy of recommendation results. The introduction of a hypergraph neural network structure effectively models the topological characteristics of the paper co-citation network, making recommendation results more consistent with citation patterns with specific intent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 Flowchart of a hypergraph-based citation recommendation method according to an embodiment of the present invention;

[0026] Figure 2 A diagram showing the reference intention in an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a hypergraph cited in an embodiment of the present invention;

[0028] Figure 4 Schematic diagram of hypergraph information propagation in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0030] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatus.

[0031] Citation recommendation is an important tool for assisting scholars in writing papers efficiently. However, because academic citations contain complex citation intent and semantic connections, fully mining and leveraging the multiple relationships between papers to provide high-quality personalized recommendations has always been a challenge in this field. This paper proposes a citation recommendation model based on an intent-aware hypergraph attention network. This model aims to capture the multiple citation intents and high-order relationships implicit in academic citation collections, thereby improving the accuracy and relevance of citation recommendations.

[0032] The hypergraph-based citation recommendation method disclosed in the embodiment of the present invention performs intent encoding on the paper embedding vector, decomposes multiple sub-vectors corresponding to potential citation intentions, and constructs a citation hypergraph based on this. Subsequently, a hypergraph attention network is used to perform micro-intent decoding to enhance the representation of papers related to specific intents, and then macro-intent decoding is performed through the intent classification task to improve the intent discrimination. On this basis, the relevance score between the new manuscript and the existing paper is calculated, and the N candidate citations with the highest scores are screened as the recommendation results. Finally, the model performance is optimized through the dual optimization goals of citation recommendation and intent classification. Such a method improves the semantic relevance of citation recommendation through intent perception, while improving the accuracy and efficiency of recommendation. Example 1

[0033] The citation recommendation method based on hypergraph has a flowchart as follows: Figure 1 As shown, it includes the steps of building a model and citation recommendation, wherein building a model includes:

[0034] S1. Collect existing papers to obtain the target paper set, preprocess the text content of each paper, and generate an initial embedding vector.

[0035] This step aims to generate an initial embedding vector for each paper as the basis for subsequent intent encoding. The existing papers mentioned above, including published papers and their reference data, are processed using pre-trained language models (such as BERT) or traditional natural language processing techniques (such as LDA) The text content, including at least one of the title, abstract, keywords, and author, generates a high-dimensional initial embedding vector .

[0036] S2. Decompose the initial embedding vector of each paper into several sub-vectors under potential citation intentions, namely intention embedding vectors; cluster all the sub-vectors of the paper according to the intention to generate the intention prototype vector. The citation intention example is shown in Figure 2 .

[0037] The purpose of this step is to extract the semantic model of the reference intention, which is divided into two steps:

[0038] S2.1. Intent Slicing: Slice the embedding vector according to the reference intent and divide it into K independent sub-vectors, formalized as a set of K intents , each subvector The embedding vector corresponding to the kth potential citation intent semantics of the paper.

[0039] S2.2. Generate intent prototypes: To identify the citation pattern of each intent, cluster the paper embedding vectors under each intent. Specifically, for the kth potential intent, optionally process the embedding vectors of all papers through a clustering algorithm (such as K-means, mean) , generate several cluster centers, which are called intention "prototype vectors" , represents the semantic prototype under the kth potential reference intention.

[0040] This step outputs a set of intention sub-vectors and intent prototype vector , providing support for the hypergraph construction in step S3.

[0041] S3. Construct a citation hypergraph to reflect the relationship between citing papers and cited papers. The hypergraph includes co-citation hyperedges and intention hyperedges. Adjust the weights of paper nodes in the hypergraph information propagation to perform micro-intention decoupling. Classify citation intentions based on macro-semantics to perform macro-intention decoupling.

[0042] S3.1 Construct a citation hypergraph: For each paper’s reference set s, construct a citation hypergraph , the reference hypergraph includes a node set and a hyperedge set, where is a set of nodes, each node represents a cited paper in the set s. is a hyperedge set used to represent complex relationships between papers, such as Figure 3 shown.

[0043] Specifically, hyperedges are divided into two types:

[0044] The first type is co-citation hyperedge , also known as cited papers, connects all papers cited by the same paper, reflecting the high-order correlation of papers under co-citation relationships.

[0045] The second type is intentional super-edge , reflecting the intention-related features of the papers in the citation set. For example, the hyperedge of intention k connects all papers under this intention.

[0046] As a preference, in order to reduce the computational complexity, this step usually filters out the sub-hyperedges that are not related to the intent, specifically, setting a threshold , when the paper The cosine value of a certain intention k is greater than the threshold When , there is a hyperedge between the two, otherwise it does not exist. It can be expressed by formula (1).

[0047] (1)

[0048] S3.2 Micro-intent decoupling: In each subgraph of a specific intent In

[15] , a hypergraph attention network (HGAN) is deployed to adaptively adjust the weights of nodes in the hypergraph information propagation and highlight nodes that are highly relevant to specific intents.

[0049] Specifically, in each subgraph of a specific intent In the paper, the representation of the updated paper is mainly through two steps: node to hyperedge and hyperedge to node. Figure 4 The specific process is demonstrated.

[0050] (1) Node to hyperedge. Through aggregation and hyperedge The relevant node features are used to update the intention embedding of the hyperedge, as shown in formula (2).

[0051] (2)

[0052] in, Indicates that under the k-th intention, the hyperedge The embedding vector of is an aggregation function defined on a hyperedge, Represents a hyperedge midpoint The attention weight of the node directly reflects Under the intention k, the hyperedge contribution.

[0053] To calculate , first each hyperedge The embedding representation of (The embedding of the co-citation hyperedge is equivalent to the embedding of the citing paper.) The closer the distance between a node and its hyperedge, the more representative the node is in this semantic context. Therefore, the attention weight of the node is It can be embedded through and hyperedge embedding The similarity is determined as shown in formula (3).

[0054] (3)

[0055] in, is a learnable attention vector, represents the element-wise product, and the Softmax function ensures that the sum of all attention weights is 1. This process is based on the semantic relevance between nodes and hyperedges, reflecting the difference in the contribution of nodes to the different intentions of hyperedges.

[0056] (2) Hyperedge to node: By integrating the hyperedge information associated with the node, the intent embedding of the node is updated; finally, each node obtains a decoupled embedding for different reference intents.

[0057] The decoupled embedding for different citation intentions can be formalized as formula (4):

[0058] (4)

[0059] in, It is a super edge For Node The attention weight can be obtained by formula (5).

[0060] (5)

[0061] in, is another learnable attention vector under the kth intent. This process considers the relevance of the hyperedge embedding and the intent of the citation set, so that the hyperedge with a high degree of match with the current node's semantics and intent is given a greater weight.

[0062] S3.3 Macro-intent decoupling: Through a multi-layer perceptron, features that are helpful for intent classification are extracted from the decoupled embedding, and then the predicted probability of each reference intent is calculated.

[0063] Specifically, after completing the decoupling of micro-intentions in the citation set hypergraph, each paper node obtains a series of decoupled embeddings for different citation intentions. Although these embeddings reveal the performance of papers under different citation intentions, considering that the embeddings between different citation intentions are not completely isolated, there may be a certain degree of semantic overlap between them.

[0064] In order to improve the discrimination of citation intent embedding at a broader semantic level and reduce the redundancy between different intent embeddings, this paper introduces a macro-intent decoupling method, which aims to improve the model's overall ability to discriminate different citation intents through an intent classification task.

[0065] Specifically, this method is based on the decoupled embedding of paper nodes in the citation set s To predict the reference intent category of the entire collection. This process can be formalized as formula (6).

[0066] (6)

[0067] in, represents the predicted probability of each reference intent, and MLP(·) is a multi-layer perceptron used to extract features that are helpful for intent classification from the disentangled embedding.

[0068] In order to optimize this intent classification task, this method adopts the cross entropy loss function, which can be expressed as formula (7).

[0069] (7)

[0070] here, is an indicator function that takes the value of 1 when the true citation intent label matches the predicted label, and 0 otherwise. The design of this loss function can effectively improve the accuracy of the model in predicting different citation intents, thereby ensuring that the embedding vector of each citation intent is more semantically distinguishable.

[0071] S4. Process the new manuscript using the model to generate an intent embedding vector, calculate the relevance score between the manuscript and existing papers, sort them from high to low according to the relevance score, and generate a list of recommended papers.

[0072] Specifically, for a new manuscript ,First, generate its intent embedding through S1 and S2.1 Then calculate its relevance score with all published papers in the system, which can be expressed as formula (8).

[0073] (8)

[0074] in, For existing candidate papers Embedding on intent k. Comprehensively evaluated the manuscript by integrating different perspectives on intent With existing candidate papers correlation.

[0075] Finally, the top N papers are sorted by relevance score from high to low, and the recommended citations are selected. This step outputs the recommended citation list, realizing the core recommendation function.

[0076] Preferably, this embodiment also includes a model optimization step, which adds the loss function of the citation recommendation step to the loss function of the macro-intention decoupling step to obtain the loss function of the intent-aware citation recommendation model.

[0077] Specifically, in the loss function optimization of citation recommendation, the model contains two optimization objectives: citation recommendation and intent classification, which promotes the performance of the final model in the intent-aware citation recommendation task.

[0078] Formally, the citation recommendation task adopts the cross entropy loss function, as shown in formula (9).

[0079] (9)

[0080] Among them, when the literature is actually cited, The value is 1; otherwise, it is 0. Combining the two losses, we can obtain the overall optimization goal of the intent-aware citation recommendation model proposed in this embodiment, which can be formalized as formula (10).

[0081] (10)

[0082] Among them, λ is the cross-term coefficient, which controls the proportion of the intent classification task in the overall training.

[0083] The method in this example utilizes innovative structures and mechanisms to significantly improve the accuracy and relevance of citation recommendations, demonstrating its advancement and practicality in academic citation recommendation tasks. Regarding recommendation accuracy, intent decoupling enables the model to more accurately understand the semantic intent between citing and cited papers, significantly improving the accuracy of recommendation results. The introduction of a hypergraph neural network effectively models the topological characteristics of the paper co-citation network, making recommendation results more consistent with citation patterns with specific intent. Example 2

[0084] The present invention provides an electronic device comprising a memory, a processor and a computer program stored in the memory, wherein when the processor executes the program, the steps of the hypergraph-based citation recommendation method described in any one of the above technical solutions are implemented.

[0085] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0086] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0087] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A citation recommendation method based on hypergraph, characterized in that: The method includes the steps of building a model and recommending citations, wherein the building of the model includes: S1. Collect existing papers to obtain the target paper set, preprocess the text content of each paper, and generate the initial embedding vector; S2. Decompose the initial embedding vector of each paper into several sub-vectors under potential citation intentions, namely, intention embedding vectors; cluster all the sub-vectors of the paper according to the intention to generate the intention prototype vector; S3. Utilize the reference set of each paper to construct a citation hypergraph to reflect the relationship between citing papers and cited papers. The citation hypergraph includes a node set and a hyperedge set. Each node in the node set represents a cited paper. The hyperedge set includes co-citation hyperedges and intention hyperedges. The co-citation hyperedges connect all papers cited by the same paper, and the intention hyperedges connect all papers under the same intention. Adjust the weights of the paper nodes in the hypergraph information propagation to perform micro-intention decoupling. Specifically, deploy a hypergraph attention network in each subgraph with a specific intention to adaptively adjust the weights of the nodes in the hypergraph information propagation. Update the intention embedding of the hyperedge by aggregating the node features related to the hyperedge, and update the intention embedding of the node by integrating the hyperedge information associated with the node. Finally, each node obtains a decoupled embedding for different citation intentions. Citation intentions are classified macro-semantically to perform macro-intention decoupling. Specifically, features that contribute to intent classification are extracted from the decoupled embedding through a multi-layer perceptron, and then the predicted probability of each citation intention is calculated. The citation recommendation step includes: S4. Process the new manuscript using the model to generate an intent embedding vector, calculate the relevance score between the manuscript and the existing papers, sort them from high to low according to the relevance score, and generate a list of recommended papers.

2. The hypergraph-based citation recommendation method according to claim 1, characterized in that: The preprocessing of the text content of each paper includes using a pre-trained language model or traditional natural language processing technology to process the text content of each paper, and the text content includes at least one of the title, abstract, keywords, and author.

3. The hypergraph-based citation recommendation method according to claim 1, characterized in that: Decomposing the embedding vector of each paper into several sub-vectors under potential citation intentions specifically includes slicing the embedding vector according to the citation intention and dividing it into K independent sub-vectors, each sub-vector corresponding to the embedding vector of the paper under the corresponding citation intention.

4. The hypergraph-based citation recommendation method according to claim 1, characterized in that: It also includes filtering sub-hyperedges that are not related to intent. Specifically, when the cosine value between a paper and a certain intent is greater than a set threshold, there is a hyperedge between the two, otherwise it does not exist.

5. The hypergraph-based citation recommendation method according to claim 1, characterized in that: The calculation of the relevance score between the manuscript and the existing paper in step S4 is specifically as follows: ,in Indicates manuscript Embedding on intent k, Indicates existing papers Embedding on intent k, Comprehensively evaluate manuscripts by integrating different perspectives of intent Compared with existing papers correlation.

6. The hypergraph-based citation recommendation method according to claim 5, characterized in that: It also includes a model optimization step, specifically, adding the loss function of the citation recommendation step and the loss function of the macro-intention decoupling step to obtain the loss function of the intent-aware citation recommendation model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the processor executes the program, the steps of the hypergraph-based citation recommendation method as described in any one of claims 1 to 6 are implemented.

Images