Dynamic graph understanding method, system and equipment based on large language model and medium
By aligning the message passing paradigm of dynamic graph neural networks with large language models, and utilizing attention, graph structure, and timing hint learning methods, we can fuse node text features, graph high-order signals, and timing information in the natural language space, solving the problem of semantic information loss in dynamic graph models and improving task prediction performance and interpretability.
Patent Information
- Application Number
- CN202510721817.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-09-12
AI Technical Summary
When processing dynamic graphs with text attributes, existing dynamic graph neural network models have compatibility issues between text features and numerical representation space, resulting in the loss of key semantic information and making it difficult to effectively and collaboratively integrate node semantic features, high-order graph structure signals and temporal dynamic characteristics.
By aligning the message passing paradigm of dynamic graph neural networks based on large language models, and utilizing attention, graph structure, and timing hint learning methods, we unify node text features, graph high-order signals, and timing information into the natural language space, and apply them to downstream tasks through instruction fine-tuning.
It avoids the key information loss when converting text features into numerical features, improves task prediction effect, transferability and interpretability, and enhances the overall performance of dynamic graph understanding.
Smart Images
Figure CN120633846A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of natural language processing and graph neural networks, and in particular to a dynamic graph understanding method, system, device and medium based on a large language model. Background Art
[0002] Graph-structured data, due to its ability to effectively represent complex relationships between entities, has found widespread application in a variety of fields, including social networks, recommendation systems, and academic networks. To model this type of non-Euclidean spatial data, graph neural networks (GNNs) have become a mainstream solution, fusing node features and topological structure through a neighborhood aggregation mechanism.
[0003] However, graph data in real-world scenarios often has dynamic evolution characteristics, requiring the simultaneous integration of structural and temporal dimension information. To this end, Dynamic Graph Neural Networks (DGNNs) have achieved joint modeling of spatiotemporal dependencies by introducing methods such as recurrent neural networks and time-series point processes. However, existing DGNNs models generally use numerical representation methods to implicitly integrate node features. In particular, when processing dynamic graphs with text attributes (Text Attributed Dynamic Graphs, TADGs), traditional methods rely on tools such as Word2Vec (word vectors) or BERT (Bidirectional Encoder Representation) to convert text into numerical representations. This processing method can easily lead to compatibility issues between text features and the numerical representation space, resulting in the loss of key semantic information.
[0004] It is worth noting that large language models (LLMs) provide new ideas for solving the above problems with their excellent text understanding capabilities. Some early works mainly focused on using natural language to describe the structure of static graphs, hoping to combine graph structure information with the text features of nodes and solve related downstream tasks through the understanding capabilities of large language models. Recently, there have also been some works that apply large language models to dynamic graphs. These works use large language models to generate auxiliary information to solve the problem of sparse node text features. However, none of these methods have effectively solved the key challenge of how to maintain the coordinated integration of node semantic features, high-order graph structure signals and temporal dynamic characteristics. There is still a significant information loss problem, which leads to poor task performance.
[0005] In view of this, the present invention is proposed. Summary of the Invention
[0006] The purpose of the present invention is to provide a dynamic graph understanding method, system, device and medium based on a large language model, which can align the message passing paradigm of dynamic graph neural networks and thus obtain better task prediction effects.
[0007] The purpose of the present invention is achieved through the following technical solutions:
[0008] A dynamic graph understanding method based on a large language model, comprising:
[0009] The message passing paradigm of DGNNs is aligned through the first language model, including: inputting a dynamic graph into the first language model, guiding the first language model to sample neighbor nodes of each node based on the text representation of the node through attention prompts; guiding the first language model through graph structure prompts to fuse the text representation of each node with the sampled neighbor nodes to obtain a fused text representation of each node; capturing the changes in the neighbor nodes of each node, guiding the first language model through time sequence prompts to combine the fused text representation of each node and the changes in the corresponding neighbor nodes, and updating the text representation of each node on the timeline to finally obtain an aligned dynamic graph; wherein DGNNs are dynamic graph neural networks;
[0010] The second largest language model is fine-tuned using the instruction fine-tuning method, and the fine-tuned second largest language model is combined with the aligned dynamic graph for task prediction.
[0011] A dynamic graph understanding system based on a large language model, used to implement the aforementioned method, includes:
[0012] The text-based dynamic graph propagation module is used to align the message passing paradigm of DGNNs through the first language model, including: inputting the dynamic graph into the first language model, guiding the first language model to sample the neighbor nodes of each node based on the text representation of the node through attention prompts; guiding the first language model through graph structure prompts to fuse the text representation of each node with the sampled neighbor nodes to obtain a fused text representation of each node; capturing the changes in the neighbor nodes of each node, guiding the first language model through time sequence prompts to combine the fused text representation of each node and the changes in the corresponding neighbor nodes, and updating the text representation of each node on the timeline to finally obtain an aligned dynamic graph; wherein DGNNs are dynamic graph neural networks;
[0013] The text-based dynamic graph prediction module is used to fine-tune the second largest language model using the instruction fine-tuning method, and then use the fine-tuned second largest language model combined with the aligned dynamic graph to perform task prediction.
[0014] A processing device comprising: one or more processors; a memory for storing one or more programs;
[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.
[0016] A readable storage medium stores a computer program, which implements the aforementioned method when the computer program is executed by a processor.
[0017] It can be seen from the technical solution provided by the present invention that a large language model is used to fully align the dynamic graph neural network from the natural language space, and the text semantic features of the nodes, the high-order neighbor signals on the graph, and the timing information are integrated in the natural language space under the framework of prompt learning, thereby avoiding the key information loss of the traditional method of converting text features into numerical features and reducing incompatibility; in addition, the data of the non-Euclidean space such as the graph structure is mapped to the natural language space and applied to downstream tasks (for example, node classification and link prediction, etc.) through the instruction fine-tuning method, thereby improving the task execution effect, the portability and interpretability of the entire solution. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 A flowchart of a dynamic graph understanding method based on a large language model provided by an embodiment of the present invention;
[0020] Figure 2 A schematic diagram of a framework of a dynamic graph understanding method based on a large language model provided by an embodiment of the present invention;
[0021] Figure 3 A schematic diagram of a dynamic graph understanding system based on a large language model provided by an embodiment of the present invention;
[0022] Figure 4 A schematic diagram of a processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0024] First, the following terms may be used in this article:
[0025] The terms "include," "comprises," "contains," "has," or other similar expressions should be interpreted as non-exclusive. For example, "including certain technical features (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products, or manufactured articles, etc.) should be interpreted as including not only the technical features explicitly listed, but also other technical features known in the art that are not explicitly listed.
[0026] The term "consisting of" excludes any technical features not explicitly listed. If used in a claim, this term renders the claim closed, excluding any technical features other than those explicitly listed, except for conventional impurities associated with them. If this term appears only in a clause of a claim, it limits only the elements explicitly listed in that clause; elements listed in other clauses are not excluded from the claim as a whole.
[0027] The following describes in detail a method, system, device, and medium for understanding a dynamic graph based on a large language model provided by the present invention. Any content not described in detail in the embodiments of the present invention belongs to the prior art known to professionals in the field. In the embodiments of the present invention, if specific conditions are not specified, the conditions are carried out according to conventional conditions in the field or the conditions recommended by the manufacturer. For reagents or instruments used in the embodiments of the present invention, if the manufacturer is not specified, they are all conventional products that can be purchased commercially.
[0028] Example 1
[0029] The embodiment of the present invention provides a dynamic graph understanding method based on a large language model, such as Figure 1 As shown, it mainly includes the following steps:
[0030] Step 1: Text-based dynamic graph propagation process.
[0031] In an embodiment of the present invention, a message passing paradigm of DGNNs (dynamic graph neural networks) is aligned through a first large language model, including: inputting a dynamic graph into the first large language model, guiding the first large language model to sample neighbor nodes of each node based on the text representation of the node through attention prompts; guiding the first large language model through graph structure prompts, fusing the text representations of each node with the sampled neighbor nodes, and obtaining a fused text representation of each node; capturing changes in the neighbor nodes of each node, guiding the first large language model through timing prompts to combine the fused text representation of each node and the corresponding neighbor node changes, updating the text representation of each node on the timeline, and finally obtaining an aligned dynamic graph.
[0032] Step 2: Text-based dynamic graph prediction process.
[0033] In an embodiment of the present invention, the second largest language model is fine-tuned using an instruction (natural language word unit composition) fine-tuning method, and the fine-tuned second largest language model is combined with the aligned dynamic graph to perform task prediction.
[0034] The above solution provided by the embodiment of the present invention is a new technical framework that uses a large language model to align dynamic graphs with text attributes, unifying node text features, graph high-order signals and temporal information into the natural language space, thereby avoiding the loss of key semantic information. Specifically, in the above text-based dynamic graph propagation process, three types of prompt learning are designed to allow the large language model to understand the dynamic graph from the perspective of natural language: 1) Attention prompt learning, based on the text representation of the node, the large language model selects important neighboring nodes to update the text representation of its own node; 2) Structural prompt learning, high-order neighborhood information aggregation on the graph is achieved through the large language model; 3) Temporal prompt learning, the node text representation is dynamically updated by the large language model according to the evolution of the graph structure. This full-text space processing method fully preserves the semantic features and successfully aligns the message passing paradigm of DGNNs; in addition, various downstream tasks are achieved through instruction fine-tuning technology.
[0035] The above three types of prompt learning mainly refer to the fact that there is no need to train the corresponding large language model, but rather to guide the corresponding large language model to answer the corresponding questions by designing appropriate instructions (prompt words) to obtain more reasonable answers.
[0036] Those skilled in the art will understand that the first and second in the first and second language models mainly serve as identifiers to distinguish the two large language models; for example, the first language model can use a closed-source large language model (e.g., GPT-4, DeepSeek-V3), and the second language model can use an open-source large language model (e.g., LLAMA); however, this is only an example. In actual applications, users can select the corresponding large language model based on actual conditions or experience. Among them, GPT-4 is the large language model launched by OpenAI, DeepSeek-V3 is the large language model launched by DeepSeek, and LLAMA is the large language model launched by Meta.
[0037] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the method provided by the embodiment of the present invention is described in detail below with reference to specific embodiments.
[0038] 1. Overall overview of the plan.
[0039] The present invention provides a task prediction method based on DGNNs alignment, which integrates the text features of nodes, high-order signals on the graph, and temporal information in a unified manner in the text space through prompt learning, and applies it to various tasks (for example, downstream tasks such as node classification and link prediction) through instruction fine-tuning.
[0040] The present invention mainly includes two parts: (1) In the first part, attention prompt learning is designed to automatically select important neighbor nodes using a large language model to prepare for subsequent neighborhood information aggregation; next, the sampled neighbor node information obtained by attention prompt learning is used to update the node information. To achieve this, the present invention introduces a graph structure prompt learning method to allow the large language model to update the text representation of the node and obtain high-order neighbor information on the graph through multiple iterations. Subsequently, the present invention designs temporal prompt learning to capture the changes in the neighbors around the node, thereby updating the text representation of the node on the time axis and integrating temporal information. Based on this, the present invention successfully aligns the propagation process of the dynamic graph network in the text space using the large language model, integrating the text semantic features of the node, the high-order signals of the graph structure and the temporal dynamic information. (2) In the second part, the instruction fine-tuning method is used to fine-tune the large language model, converting traditional downstream tasks such as node classification and link prediction into question-answering tasks in the natural language space, completely aligning the traditional dynamic graph neural network method from the natural language space, abandoning the numerical-centric modeling method, and improving the transferability and interpretability of the method.
[0041] 2. Detailed introduction of the plan.
[0042] like Figure 2 As shown, the overall framework of the present invention is presented, which mainly includes: text-based dynamic graph propagation and text-based dynamic graph prediction. The above two parts are introduced separately below.
[0043] 1. Text-based dynamic graphic communication.
[0044] (1) Node text representation initialization.
[0045] In the dynamic graph G tIn the text representation, each node has a corresponding original text attribute (such as a paper abstract). Traditional text representation methods usually use word embedding models (such as Word2Vec and BERT) to map the text to a low-dimensional numerical vector space for subsequent modeling and calculation. However, this mapping method often leads to the loss of contextual information of the text, affecting the complete expression of the semantics of the original text. To address this problem, the present invention utilizes the powerful generalization and refinement capabilities of the first language model and proposes a summary prompt instruction to guide the first language model to extract key semantics from the original text attributes, remove redundant information, and retain the core content to the greatest extent.
[0046] Take the i-th node v at timestamp t in the dynamic graph i,t For example, the corresponding original text attribute is recorded as X i ; Set the original text attribute X i Input to the first language model, and guide the first language model to the original text attribute X through summary prompts i Perform semantic extraction to obtain the initialized text representation T i , expressed as:
[0047] T i =Prompt summ (X i )
[0048] Among them, Prompt summ This is the summary prompt input to the first language model.
[0049] In the embodiment of the present invention, the first language model processes various prompts input and outputs corresponding information. Considering that the first language model is an existing model, its internal processing process can be implemented by referring to conventional technology, so it is not described in detail. At the same time, the formula is also simplified. The various prompts involved here and in the following text are * (.) represents the process of the first language model processing according to the corresponding prompt. The symbol * here refers to summ, attn, struct, and temp.
[0050] Each node in the dynamic graph uses the above method to obtain text representation and complete the initialization work.
[0051] (2) Attention-cued learning.
[0052] After obtaining the text representation of each node initialization through the aforementioned method, the present invention designs three prompt learning methods: attention prompt learning, graph structure prompt learning, and timing prompt learning to align the propagation process of the dynamic graph neural network.
[0053] First, for each node, a sample is taken from its first-order neighbor set to obtain its local structural information. However, different neighbors may have varying degrees of influence on the target node, so it is necessary to weigh the importance of neighbors. This paper introduces attention-cued learning, leveraging the adaptive learning capabilities of the first language model to automatically select the most relevant neighbors for information aggregation. This method, similar to the graph attention mechanism in graph neural networks, guides the information propagation process.
[0054] Take the i-th node v at timestamp t in the dynamic graph i,t As an example, its first-order neighbor set is recorded as In the dynamic graph, the first-order neighbors of each layer are the same. Of course, during sampling, the first-order neighbors of the samples may be slightly different due to randomness. attn , using the adaptive learning ability of the first language model, the most relevant neighbor nodes are automatically sampled based on the text representation of the node, expressed as:
[0055]
[0056] in, Indicates sampling to obtain the set of neighbor nodes in layer l, Prompt attn is the attention cue input to the first language model.
[0057] like Figure 2 As shown, for each timestamp t=0,1,..,T, there are L layers in the dynamic graph. Through the above method, the set of neighbor nodes of each node in each layer is sampled; T is the maximum timestamp and L is the total number of layers.
[0058] For example, the text content of the attention prompt can be: This is the text representation of the current node<node_rep> , the following is the text representation of the first-order neighbor nodes of the current node:<node_id:one_hop_rep> , please output the most relevant neighbor nodes.
[0059] (3) Graph structure prompts learning.
[0060] After completing neighbor screening based on the attention mechanism, we use graph structure hint learning to guide the structure aggregation process, guiding the first language model to learn the topological structure information of the dynamic graph to ensure the effective fusion of high-order structural signals and node semantic features.
[0061] Take the i-th node v at timestamp t in the dynamic graph i,t As an example, the set of neighbor nodes in the lth layer obtained by sampling is recorded as Prompt through diagram structure diagram struct Guide the first language model, combined with node v i,tThe set of neighbor nodes in the lth layer The fusion of text representation is expressed as:
[0062]
[0063] Among them, Prompt struct This is a graph structure hint for input to the first language model. Represents the node v at the l-1th layer at timestamp t i,t Text representation, when l=1, is the node v obtained through timing hint at timestamp t-1 i,t The updated text representation And when t=0, For node v i,t Initialized text representation; j is the set of neighbor nodes in the lth layer A single neighbor node in represents the text representation of the neighbor node j at the l-1th layer of timestamp t. When l=1, is the text representation of neighbor node j obtained by timing prompt update at timestamp t-1, and when t=0, is the initialized text representation of neighbor node j; The node v at the lth layer is timestamp t i,t Text representation, l = 1, 2, ..., L, where L is the total number of layers.
[0064] After L layers of stacking, get the i-th node v with timestamp t i,t Fusion text representation That is, the fusion representation of the L-layer high-order neighbor signal and the node semantic feature (node text feature, that is, the text information of the node itself) is obtained, where The node v at the Lth layer is timestamp t i,t Text representation.
[0065] Similarly, based on the above method, the fused text representation of each node at each timestamp can be obtained.
[0066] For example, the text content of the graph structure prompt can be: This is the text representation of the current node<node_rep> , the following are the text representations of the most relevant neighbor nodes of the current node:<one_hop_rep> , please update the text representation of the current node according to the text representations of these neighboring nodes and output the fused text representation.
[0067] (4) Timing cue learning.
[0068] To update text representations across timestamps, we designed temporal hint learning to adapt to the dynamic changes in neighborhood structure in dynamic graphs. This method captures changes in node representations by focusing on newly added neighborhood information at each timestamp. This reduces word-metabyte space consumption while enabling the largest language model to efficiently update text representations based on newly added neighborhoods.
[0069] Take the i-th node v at timestamp t+1 in the dynamic graph i,t+1 For example, the neighbor nodes newly sampled from timestamp t+1 compared to timestamp t are recorded as the set Prompt by timing temp Bootstrap the first language model, combined with the i-th node v at timestamp t i,t Fusion text representation T i,t , and the collection Update the text representation, expressed as:
[0070]
[0071] Among them, Prompt temp is the temporal prompt input to the first language model, j′ is the set A single neighbor node in T j′ ,t is the fused text representation of neighbor node j′ at timestamp t; For node v i,t+1 Updated text representation.
[0072] For example, the text content of the timing prompt can be: This is the text representation of the current node<node_rep> , the following is the text representation of the newly added most relevant neighbor node at the new timestamp:<new_hop_rep> , please update the text representation of the current node with the new timestamp based on the text representations of these newly added neighbor nodes.
[0073] Those skilled in the art will understand that neighbor nodes represent the correlation between nodes, and the text representations involved in the neighbor nodes are also obtained through the above three types of prompts.
[0074] In the embodiment of the present invention, the text-based dynamic graph propagation solution can be used to obtain the fused text representation of each node at different timestamps, such as Figure 2 As shown in Figure 2, the process of obtaining the fused text representation of a single node at each time stamp is demonstrated.
[0075] 2. Text-based dynamic image prediction.
[0076] Based on the fused text representations of each node in the dynamic graph at different timestamps, the second language model can be used to reason about various tasks. The following mainly introduces node classification and link prediction tasks as examples. By constructing specific prompt instructions to guide the second language model to process dynamic graph tasks, this paper effectively transforms the traditional dynamic graph neural network training paradigm into a natural language question-answering task, enabling the second language model to learn and reason in a more flexible manner and have the ability to transfer across data sets.
[0077] For node classification tasks, the corresponding task instructions are classification instructions; through the node classification instruction template prompt cls , build classification instructions:
[0078]
[0079] in, For the constructed classification instructions, T i,t is the i-th node v at timestamp t i,t Fusion text representation.
[0080] For the link prediction task, the corresponding task instruction is the link prediction instruction, which is constructed as follows:
[0081]
[0082] Among them, prompt lo For link prediction instruction template, For the constructed link prediction instructions, are the i-th node v at timestamp t i,t , positive sample node Negative Sample Node The fusion text representation, the positive sample node refers to the node v i,t There are actually connected nodes, and negative sample nodes refer to nodes v i,t There are no connected nodes.
[0083] For example, the content of the classification task instruction can be: the text representation of the current node is<node_rep> , then, which category should the node be classified into? The link prediction task instruction content can be: The text representation of the current node is<node_rep> , given the text representation of two nodes:<positice_rep> ,<negative_rep> , which of the nodes will be connected to the current node.
[0084] In the second language model fine-tuning phase, the task instructions are recorded as (i.e., as a general instruction for various tasks), the task prediction result of the second language model is the target output sequence, denoted as Y. The following loss function Loss is constructed to fine-tune the second language model:
[0085]
[0086] in, Indicates a given task instruction and historical output Y <k (i.e. including Y1,…,Y k-1 ), the kth word Y in the target output sequence Y k The conditional probability of , |Y| is the number of tokens in the target output sequence Y.
[0087] The specific fine-tuning process can refer to conventional technology and will not be described in detail in the present invention.
[0088] Because task instructions are composed entirely of natural language tokens, this invention is fully readable, making the decision-making process more explainable. Compared to traditional dynamic graph neural networks, this invention no longer relies on complex graph structure design. Instead, it converts dynamic graph tasks into question-answering tasks in natural language processing, fully leveraging the reasoning and generalization capabilities of the large language model. Furthermore, the fine-tuned second large language model can be effectively transferred to unseen datasets.
[0089] Through the description of the above embodiments, those skilled in the art will clearly understand that the above embodiments can be implemented through software or by using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) and includes a number of instructions for causing a computer device (such as a personal computer, a server, or a network device) to execute the methods described in the various embodiments of the present invention.
[0090] Example 2
[0091] The present invention also provides a dynamic graph understanding system based on a large language model, which is mainly used to implement the method provided in the above embodiment, such as Figure 3 As shown, the system mainly includes:
[0092] The text-based dynamic graph propagation module is used to align the message passing paradigm of DGNNs through the first language model, including: inputting the dynamic graph into the first language model, guiding the first language model to sample the neighbor nodes of each node based on the text representation of the node through attention prompts; guiding the first language model through graph structure prompts to fuse the text representation of each node with the sampled neighbor nodes to obtain a fused text representation of each node; capturing the changes in the neighbor nodes of each node, guiding the first language model through time sequence prompts to combine the fused text representation of each node and the changes in the corresponding neighbor nodes, and updating the text representation of each node on the timeline to finally obtain an aligned dynamic graph; wherein DGNNs are dynamic graph neural networks;
[0093] The text-based dynamic graph prediction module is used to fine-tune the second largest language model using the instruction fine-tuning method, and then use the fine-tuned second largest language model combined with the aligned dynamic graph to perform task prediction.
[0094] Considering that the main processing details involved in the above modules have been introduced in detail in the previous embodiments, they will not be repeated here.
[0095] Those skilled in the art will clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.
[0096] Example 3
[0097] The present invention also provides a processing device, such as Figure 4 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the aforementioned embodiment.
[0098] Furthermore, the processing device further includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.
[0099] In the embodiment of the present invention, the specific types of the memory, input device, and output device are not limited; for example:
[0100] The input device can be a touch screen, image acquisition device, physical button or mouse;
[0101] The output device may be a display terminal;
[0102] The memory may be a random access memory (RAM) or a non-volatile memory, such as a disk memory.
[0103] Example 4
[0104] The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.
[0105] In the embodiments of the present invention, the computer-readable storage medium may be provided in the aforementioned processing device, for example, as a memory in the processing device. Alternatively, the computer-readable storage medium may be a USB flash drive, a removable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk, among other media capable of storing program code.
[0106] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art.
Claims
1. A dynamic graph understanding method based on a large language model, characterized in that: include: The message passing paradigm of DGNNs is aligned through the first language model, including: inputting a dynamic graph into the first language model, guiding the first language model to sample neighbor nodes of each node based on the text representation of the node through attention prompts; guiding the first language model through graph structure prompts to fuse the text representation of each node with the sampled neighbor nodes to obtain a fused text representation of each node; capturing the changes in the neighbor nodes of each node, guiding the first language model through time sequence prompts to combine the fused text representation of each node and the changes in the corresponding neighbor nodes, and updating the text representation of each node on the timeline to finally obtain an aligned dynamic graph; wherein DGNNs are dynamic graph neural networks; The second largest language model is fine-tuned using the instruction fine-tuning method, and the fine-tuned second largest language model is combined with the aligned dynamic graph for task prediction.
2. A dynamic graph understanding method based on a large language model according to claim 1, characterized in that: Also includes: Initialize the text representation of the node based on the first language model, including: For the i-th node v at timestamp t in the dynamic graph i,t , and its corresponding original text attribute is recorded as X i ; Set the original text attribute X i Input to the first language model, and guide the first language model to the original text attribute X through summary prompts i Perform semantic extraction to obtain the initialized text representation T i , expressed as: T i =Prompt summ (X i ) Among them, Prompt summ A summary prompt is input to the first language model.
3. A dynamic graph understanding method based on a large language model according to claim 1, characterized in that: The method of guiding the first language model to sample neighbor nodes of each node based on the text representation of the node through attention prompts includes: For the i-th node v at timestamp t in the dynamic graph i,t , and its first-order neighbor set is recorded as By introducing attention cues and leveraging the adaptive learning capabilities of the first language model, the most relevant neighboring nodes are automatically sampled based on the text representation of the node, expressed as: in, Indicates sampling to obtain the set of neighbor nodes in layer l, Prompt attn is the attention cue input to the first language model.
4. A dynamic graph understanding method based on a large language model according to claim 1, characterized in that: The method of guiding the first language model through the graph structure prompt to fuse the text representations of each node with the sampled neighboring nodes to obtain the fused text representation of each node includes: For the i-th node v at timestamp t in the dynamic graph i,t , the set of neighbor nodes in the lth layer obtained by sampling is recorded as Guide the first language model through the graph structure prompt, combined with the node v i,t The set of neighbor nodes in the lth layer The fusion of text representation is expressed as: Among them, Prompt struct This is a graph structure hint for input to the first language model. Represents the node v at the l-1th layer at timestamp t i,t Text representation, when l=1, is the node v obtained through timing hint at timestamp t-1 i,t The updated text representation And when t=0, For node v i,t Initialized text representation; j is the set of neighbor nodes in the lth layer A single neighbor node in represents the text representation of the neighbor node j at the l-1th layer of timestamp t. When l=1, is the text representation of neighbor node j obtained by timing prompt update at timestamp t-1, and when t=0, is the initialized text representation of neighbor node j; The node v at the lth layer is timestamp t i,t Text representation, l = 1, 2, ..., L, where L is the total number of layers; After L layers of stacking, get the i-th node v with timestamp t i,t Fusion of text representations in, The node v at the Lth layer is timestamp t i,t Text representation.
5. The method for understanding dynamic graphs based on a large language model according to claim 1, wherein: The capturing of changes in neighboring nodes of each node, guiding the first language model to combine the fused text representation of each node and the changes in the corresponding neighboring nodes through time sequence prompts, and updating the text representation of each node on the timeline includes: For the i-th node v at timestamp t+1 in the dynamic graph i,t+1 , record the neighbor nodes newly sampled from timestamp t+1 compared to timestamp t as the set Guide the first large language model through temporal hints, combined with the i-th node v at timestamp t i,t Fusion text representation T i,t , and the collection Update the text representation, expressed as: Among them, j′ is a set A single neighbor node in T j′,t is the fused text representation of neighbor node j′ at timestamp t; For node v i,t+1 Updated text representation.
6. A dynamic graph understanding method based on a large language model according to claim 1, characterized in that: The method of fine-tuning the second language model using instruction fine-tuning includes: The task instruction is recorded as The task prediction result of the second largest language model is the target output sequence, denoted as Y. The following loss function Loss is constructed to fine-tune the second largest language model: in, Indicates a given task instruction and historical output Y ＜k When the target output sequence Y is the kth word Y k The conditional probability of , |Y| is the number of tokens in the target output sequence Y.
7. A dynamic graph understanding method based on a large language model according to claim 6, characterized in that: The tasks include: node classification tasks and link prediction tasks; For node classification tasks, the corresponding task instructions are classification instructions; through the node classification instruction template prompt cls , build classification instructions: in, For the constructed classification instructions, T i,t is the i-th node v at timestamp t i,t Fusion text representation of For the link prediction task, the corresponding task instruction is the link prediction instruction, which is constructed as follows: Among them, prompt lp For link prediction instruction template, For the constructed link prediction instructions, are the i-th node v at timestamp t i,t , positive sample node Negative Sample Node The fusion text representation, the positive sample node refers to the node v i,t There are actually connected nodes, and negative sample nodes refer to nodes v i,t There are no connected nodes.
8. A dynamic graph understanding system based on a large language model, characterized in that: The method for implementing any one of claims 1 to 7 comprises: The text-based dynamic graph propagation module is used to align the message passing paradigm of DGNNs through the first language model, including: inputting the dynamic graph into the first language model, guiding the first language model to sample the neighbor nodes of each node based on the text representation of the node through attention prompts; guiding the first language model through graph structure prompts to fuse the text representation of each node with the sampled neighbor nodes to obtain a fused text representation of each node; capturing the changes in the neighbor nodes of each node, guiding the first language model through time sequence prompts to combine the fused text representation of each node and the changes in the corresponding neighbor nodes, and updating the text representation of each node on the timeline to finally obtain an aligned dynamic graph; wherein DGNNs are dynamic graph neural networks; The text-based dynamic graph prediction module is used to fine-tune the second largest language model using the instruction fine-tuning method, and then use the fine-tuned second largest language model combined with the aligned dynamic graph to perform task prediction.
9. A processing device, characterized in that include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.