Method, device, equipment and storage medium for constructing continuous-time dynamic graph model

Abstract

The present application discloses a method, device, equipment and storage medium for constructing a continuous-time dynamic graph model, the method comprising sampling a historical neighbor sequence, determining a historical neighbor embedding sequence corresponding to the historical neighbor sequence through an initial continuous-time dynamic graph model; determining an enhanced historical neighbor embedding sequence corresponding to the historical neighbor embedding sequence through an initial enhanced model; determining a node representation of a target node based on the enhanced historical neighbor embedding sequence through the initial continuous-time dynamic graph model; and training an initial continuous-time dynamic graph model and an initial enhanced model based on the node representation, the enhanced historical neighbor embedding sequence and the historical neighbor embedding sequence. The present application processes the historical neighbor embedding sequence in a latent space through an initial enhanced model to obtain an enhanced historical neighbor embedding sequence, thereby generating a variety of historical neighbor embedding sequences and improving the model performance of the continuous-time dynamic graph model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for constructing a continuous-time dynamic graph model. Background Art

[0002] Continuous-time dynamic graphs are common in real-world applications such as social networks, physical systems, and e-commerce. Recently, continuous-time dynamic graph models have attracted increasing attention because they can directly learn the representation of consecutive events in continuous-time dynamic graphs, more effectively utilize temporal information, and obtain stronger representation capabilities. During the construction process, existing continuous-time dynamic graph models are affected by the accuracy of continuous-time dynamic graphs and the amount of historical data, which requires graph data enhancement methods to compensate. However, existing graph data enhancement methods are rarely used to enhance dynamic graph data, and are also affected by prior knowledge, which in turn affects the model performance of continuous-time dynamic graph models.

[0003] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a method, device, equipment and storage medium for constructing a continuous-time dynamic graph model in view of the deficiencies in the prior art.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for constructing a continuous-time dynamic graph model, wherein the method for constructing a continuous-time dynamic graph model specifically includes:

[0006] Sampling the historical neighbor sequence of the target node in the continuous-time dynamic graph;

[0007] Inputting the historical neighbor sequence into an initial continuous-time dynamic graph model, and determining a historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model;

[0008] Inputting the historical neighbor embedding sequence into an initial enhancement model, and outputting an enhanced historical neighbor embedding sequence through the initial enhancement model;

[0009] Inputting the enhanced historical neighbor embedding sequence into an initial continuous-time dynamic graph model, and determining a node representation of the target node through the initial continuous-time dynamic graph model;

[0010] Based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation, the initial continuous-time dynamic graph model and the initial enhanced model are trained to obtain a trained continuous-time dynamic graph model.

[0011] The method for constructing the continuous-time dynamic graph model, wherein the historical neighbor sequence of the target node in the sampled continuous-time dynamic graph specifically includes:

[0012] Sampling a node in the continuous-time dynamic graph as a target node;

[0013] The historical neighbor information of the target node is obtained according to the continuous-time dynamic graph to obtain a historical neighbor sequence of the target node.

[0014] The method for constructing a continuous-time dynamic graph model, wherein the initial enhancement model includes a diffusion model; the step of inputting the historical neighbor embedding sequence into the initial enhancement model and outputting an enhanced historical neighbor embedding sequence through the initial enhancement model specifically includes:

[0015] Determining a first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence;

[0016] Performing forward diffusion on the first sub-history neighbor embedding sequence through the diffusion model to obtain a sub-noise embedding sequence, and merging the sub-noise embedding sequence with the second sub-history neighbor embedding sequence to obtain a noise embedding sequence;

[0017] The noise embedding sequence is reversely denoised by the diffusion model to obtain an enhanced historical neighbor embedding sequence.

[0018] The method for constructing a continuous-time dynamic graph model, wherein determining the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence specifically includes:

[0019] Get the preset division ratio;

[0020] The historical neighbor embedding sequence is divided into a first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence according to the division ratio.

[0021] The method for constructing a continuous-time dynamic graph model, wherein the temporal position of each sub-noise embedded in the sub-noise embedding sequence in the noise embedding sequence is consistent with the temporal position of its corresponding first sub-historical neighbor embedded, wherein the temporal position of the first sub-historical neighbor embedded is the temporal position of the first sub-historical neighbor embedded in the historical neighbor embedding sequence.

[0022] The method for constructing the continuous-time dynamic graph model, wherein the training of the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain the trained continuous-time dynamic graph model specifically includes:

[0023] Constructing a dynamic graph loss function term based on the node representation;

[0024] Constructing a diffusion loss function term based on the enhanced historical neighbor embedding sequence and the historical neighbor embedding sequence;

[0025] The initial continuous-time dynamic graph model and the initial enhanced model are trained based on the dynamic graph loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model.

[0026] The method for constructing a continuous-time dynamic graph model, wherein the training process of the initial enhancement model and the training process of the initial continuous-time dynamic graph model are performed alternately; the initial continuous-time dynamic graph model and the initial enhancement model are trained based on the dynamic graph loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model, specifically comprising:

[0027] During the training process of the initial enhancement model, the model parameters of the continuous-time dynamic graph model are frozen, and the initial enhancement model is trained using a diffusion loss function term;

[0028] During the training process of the initial continuous-time dynamic graph model, the model parameters of the initial enhancement model are frozen, and the initial continuous-time dynamic graph model is trained using a dynamic graph loss function term.

[0029] The method for constructing a continuous-time dynamic graph model, wherein the initial enhancement model also includes a variational autoencoder; the variational autoencoder includes a variational encoder and a variational decoder, and the diffusion model is located between the variational encoder and the variational decoder.

[0030] The method for constructing a continuous-time dynamic graph model, wherein determining the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence specifically includes:

[0031] Performing dimensionality reduction processing on the historical neighbor embedding sequence by using the variational encoder to obtain a low-dimensional historical neighbor embedding sequence;

[0032] A first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence are determined based on the low-dimensional historical neighbor embedding sequence.

[0033] The method for constructing the continuous-time dynamic graph model, wherein the reverse denoising of the noise embedding sequence by the diffusion model to obtain the enhanced historical neighbor embedding sequence specifically includes:

[0034] Using the diffusion model to reversely denoise the noise embedding sequence to obtain a denoised historical neighbor embedding sequence;

[0035] The denoised historical neighbor embedding sequence is reconstructed by the variational decoder to obtain an enhanced historical neighbor embedding sequence.

[0036] The method for constructing the continuous-time dynamic graph model, wherein the training of the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain the trained continuous-time dynamic graph model specifically includes:

[0037] Constructing a dynamic graph loss function term based on the node representation;

[0038] Constructing a diffusion loss function term based on the denoised historical neighbor embedding sequence and the low-dimensional historical neighbor embedding sequence, wherein the low-dimensional historical neighbor embedding sequence is obtained by performing dimensionality reduction processing on the historical neighbor embedding sequence through a variational encoder;

[0039] Constructing a variational loss function term based on the low-dimensional historical neighbor embedding sequence and the historical neighbor embedding sequence;

[0040] The initial continuous-time dynamic graph model and the initial enhanced model are trained based on the dynamic graph loss function term, the variational loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model.

[0041] The method for constructing a continuous-time dynamic graph model, wherein the training process of the initial enhancement model and the training process of the initial continuous-time dynamic graph model are performed alternately; the initial continuous-time dynamic graph model and the initial enhancement model are trained based on the dynamic graph loss function term, the variational loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model, specifically comprising:

[0042] During the training process of the initial enhancement model, the model parameters of the continuous-time dynamic graph model are frozen, and the initial enhancement model is trained using the diffusion loss function term and the variational loss function term;

[0043] During the training process of the initial continuous-time dynamic graph model, the model parameters of the initial enhancement model are frozen, and the initial continuous-time dynamic graph model is trained using a dynamic graph loss function term.

[0044] The second aspect of the present application provides a device for constructing a continuous-time dynamic graph model, wherein the device for constructing a continuous-time dynamic graph model specifically includes:

[0045] A sampling module, used to sample the historical neighbor sequence of the target node in the continuous-time dynamic graph;

[0046] A control module, used to input the historical neighbor sequence into an initial continuous-time dynamic graph model, determine the historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model; input the historical neighbor embedding sequence into an initial enhancement model, output an enhanced historical neighbor embedding sequence through the initial enhancement model; and input the enhanced historical neighbor embedding sequence into the initial continuous-time dynamic graph model, determine the node representation of the target node through the initial continuous-time dynamic graph model;

[0047] A training module is used to train the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain a trained continuous.

[0048] A third aspect of the present application provides an electronic device, the electronic device comprising a processor and a memory;

[0049] The memory stores a computer-readable program executable by the processor;

[0050] When the processor executes the computer-readable program, the method for constructing the continuous-time dynamic graph model as described above is implemented.

[0051] The fourth aspect of the present application provides a computer-readable storage medium, which stores one or more computer-readable programs. The one or more computer-readable programs can be executed by one or more processors. When the computer-readable program is executed by the processor, the method for constructing a continuous-time dynamic graph model as described above is implemented.

[0052] Beneficial effects:

[0053] (1) The initial enhancement model in this application is specifically used for graph data enhancement of continuous-time dynamic graphs, and the initial enhancement model is seamlessly embedded in the continuous-time dynamic graph model. The initial enhancement model is used to enhance the historical neighbor embedding sequence used in the training phase of the continuous-time dynamic graph model to generate a variety of enhanced historical neighbor embedding sequences, thereby reducing the dependence of the continuous-time dynamic graph model on prior knowledge and improving the model performance of the constructed continuous-time dynamic graph model;

[0054] (2) This application processes the neighbor history embedding sequence in the latent space through the initial enhancement model to improve the quality of the generated enhanced history neighbor embedding sequence.

[0055] (3) This application constructs an initial enhancement model by using a variational autoencoder and a diffusion model. By using a variational autoencoder to reduce the dimension of the historical neighbor embedding sequence, the computational complexity of the extended model can be reduced, thereby reducing the computational complexity of the initial enhancement model and improving the construction speed of the continuous-time dynamic graph model.

[0056] (4) By alternately training the initial enhancement model and the initial continuous-time dynamic graph model, the present application can better enhance the historical stage embedding sequence used in the training process of the initial continuous-time dynamic graph model, thereby further improving the model performance of the constructed continuous-time dynamic graph model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 A flowchart of a method for constructing a continuous-time dynamic graph model provided in an embodiment of the present application.

[0059] Figure 2 is an example of a continuous-time dynamic graph.

[0060] Figure 3 A principle flow chart of an example of a method for constructing a continuous-time dynamic graph model provided in an embodiment of the present application.

[0061] Figure 4 A structural schematic diagram of a device for constructing a continuous-time dynamic graph model provided in an embodiment of the present application.

[0062] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application provide a method, device, equipment and storage medium for constructing a continuous-time dynamic graph model. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0065] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0066] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0067] Research has found that continuous-time dynamic graphs are applied to many fields. For example, continuous-time dynamic graphs are common in practical applications such as social networks, physical systems, and e-commerce. The continuous-time dynamic graph model can directly learn the representation of continuous events in the continuous-time dynamic graph, more effectively utilize time information to obtain stronger representation capabilities, and is receiving more and more attention. However, the construction of existing continuous-time dynamic graph models relies on continuous-time dynamic graphs, which have problems of inaccuracy and insufficient quantity. Among them, the inaccuracy problem is caused by the fact that the continuous-time dynamic graph is observed, and there may be inaccurate measurements, threshold errors, or human errors in the observation process of the continuous-time dynamic graph, resulting in the observed continuous-time dynamic graph being unable to accurately represent the real underlying process it is trying to model. The problem of insufficient quantity is caused by the fact that it is unrealistic to obtain a large amount of historical data in many applications, especially in the case of cold start. For example, an emerging trading platform may only have a few days of user asset interaction data, which makes the continuous-time dynamic graph include insufficient data.

[0068] At present, graph data enhancement methods are widely used to solve the above problems. Existing graph data enhancement methods mainly include structure-based enhancement methods and feature-based enhancement methods. Among them, the structure-based enhancement method adjusts the connectivity of the graph by adding or deleting edges or nodes. Although it can enhance the graph data, it is heavily dependent on domain knowledge and needs to select different enhancement strategy combinations according to the specific graph data set. There will be problems of over-enhancement or under-enhancement. This is because the input data is processed by a mapping-independent deep neural network, so that the features generated by the deep neural network are not specially calibrated. The feature-based enhancement method directly modifies or creates the original features of the nodes or edges in the graph, focusing on converting the features of the nodes / edges, and showing significant optimization improvements under appropriate feature enhancement. However, it cannot handle graph data that lacks or even has no node features and edge features.

[0069] To this end, the embodiment of the present application provides a method for constructing a continuous-time dynamic graph model, which samples the historical neighbor sequence of the target node in the continuous-time dynamic graph, and inputs the historical neighbor sequence into the initial continuous-time dynamic graph model, and determines the historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model; inputs the historical neighbor embedding sequence into the initial enhancement model, and outputs the enhanced historical neighbor embedding sequence through the initial enhancement model; inputs the enhanced historical neighbor embedding sequence into the initial continuous-time dynamic graph model, and determines the node representation of the target node through the initial continuous-time dynamic graph model; based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation, the initial continuous-time dynamic graph model and the initial enhancement model are trained to obtain a trained continuous-time dynamic graph model. The initial enhancement model in the embodiment of the present application is dedicated to the graph data enhancement of the continuous-time dynamic graph, and the initial enhancement model is seamlessly embedded into the continuous-time dynamic graph model, and the historical neighbor embedding sequence used in the training stage of the continuous-time dynamic graph model is enhanced by the initial enhancement model to generate a variety of enhanced historical neighbor embedding sequences, which reduces the dependence of the continuous-time dynamic graph model on prior knowledge and improves the model performance of the constructed continuous-time dynamic graph model. At the same time, the present application processes the neighbor history embedding sequence in the latent space through the initial enhancement model, which can more subtly model the transitions in the continuous-time dynamic graph and improve the quality of the generated enhanced history neighbor embedding sequence.

[0070] The content of this application is further illustrated below through the description of embodiments in conjunction with the accompanying drawings.

[0071] The present application embodiment provides a method for constructing a continuous-time dynamic graph model, such as Figure 1 As shown, the method for constructing the continuous-time dynamic graph model specifically includes steps S10-S50.

[0072] Step S10: sampling the historical neighbor sequence of the target node in the continuous-time dynamic graph.

[0073] Specifically, a continuous-time dynamic graph can be represented as a series of interaction events between node pairs that occur in chronological order: ,in, Indicates the timestamp, Indicates the timestamp Nodes that interact at all times , Represents a node set. Each node in All have node characteristics , and each interaction There are edge features , and Represent the dimensions of node and edge features respectively. For example, Figure 2 The continuous-time dynamic graph shown in the figure has a node set of {a,b,c,d,e,f,g,u,v}, and the timestamps are .

[0074] The target node is a node in the node set V corresponding to the continuous time dynamic graph, and the historical neighbor sequence is obtained by sampling the historical neighbor information of the target node in the continuous time dynamic graph before a certain time. In other words, the historical neighbor sequence includes the historical neighbor information of the target node before a certain time, wherein the historical neighbor information includes the target node, the interaction node, and the timestamp between the target node and the interaction node, that is, the historical neighbor information is a triple consisting of the target node, the interaction node, and the timestamp between the target node and the interaction node. For example, Figure 2 As shown, the target node is Figure 2 Nodes in , a certain time is , then the historical neighbor sequence of the target node may include the node exist All historical neighbor information before time, that is, the historical neighbor sequence includes , , as well as , where a, b, c, d are the interaction nodes of the target node, Is the timestamp.

[0075] Exemplarily, the historical neighbor sequence of the target node in the sampled continuous-time dynamic graph specifically includes:

[0076] S11, sampling a node in the continuous-time dynamic graph as a target node;

[0077] S12. Acquire historical neighbor information of the target node according to the continuous-time dynamic graph to obtain a historical neighbor sequence of the target node.

[0078] Specifically, the target node is sampled in the continuous time dynamic graph. After the target node is sampled, the nodes that interact with the target node before the target moment are selected in the continuous time dynamic graph, and then the historical neighbor information of the target node is determined based on the selected nodes that interact with the target node before the target moment to obtain a historical neighbor sequence. Among them, the target moment can be set before sampling; it can also be determined based on the continuous time dynamic graph and the target node. Specifically, the target moment is a timestamp among all timestamps corresponding to the target node. In an embodiment of the present application, the target moment is the latest timestamp among all timestamps corresponding to the target node. For example, the target node is Figure 2 Nodes in ,node The corresponding timestamps include , , and , then the target time is the latest timestamp among all timestamps corresponding to the target node, that is, the target time is , and then select the target node in the continuous time dynamic graph The historical neighbor information before the time, that is, the historical neighbor information is selected , , as well as , and finally the sequence of selected historical neighbor information is used as the historical neighbor sequence of the target node. When sampling the target node in the continuous time dynamic graph, a random sampling method or a point-by-point sampling method can be used. There is no specific restriction on the sampling method here, as long as the target node can be sampled in the continuous time dynamic graph.

[0079] After obtaining the historical neighbor information of the target node in the continuous time dynamic graph, the obtained historical neighbor information is sorted in chronological order (i.e., timestamp order) to obtain a historical neighbor sequence. In other words, the historical neighbor sequence of the target node may include one or more historical neighbor information, and when the historical neighbor sequence includes multiple historical neighbor information, for any two historical neighbor information, the historical neighbor information with an earlier timestamp is located before the historical neighbor information with a later timestamp. For example, Figure 2 The target node in All historical neighbor information including historical neighbor information , Historical Neighborhood Information , Historical Neighborhood Information and historical neighbor information , then the target node The historical neighbor sequence of .

[0080] It should be noted that, in practical applications, in order to avoid a large amount of calculation due to excessive historical neighbor information, after obtaining all the historical neighbor information of the target node, some historical neighbor information can be selected from all the historical neighbors of the target node to form a historical neighbor sequence of the target node. In other words, the historical neighbor sequence can include all the historical neighbor information of the target node before a certain time, and can also include some historical neighbor information of the target node before a certain time, wherein, when the historical neighbor sequence includes some historical neighbor information of the target node before a certain time, some historical neighbor information can be selected from all the historical neighbor information according to the order of timestamps.

[0081] Step S20: input the historical neighbor sequence into an initial continuous-time dynamic graph model, and determine a historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model.

[0082] Specifically, the initial continuous-time dynamic graph model is a continuous-time dynamic graph model using initial model parameters, and a trained continuous-time dynamic graph model can be obtained by training the model parameters of the initial continuous-time dynamic graph model. The continuous-time dynamic graph model includes a coding layer, and the coding layer is used to encode the historical neighbor sequence to obtain a historical neighbor embedding sequence. Specifically, the coding layer is used to encode the interactive nodes, timestamps, and edge features between the target node and the interactive node in each historical neighbor information in the historical neighbor sequence of the target node, and obtain the interactive node feature coding, timestamp coding, and edge feature coding between the target node and the interactive node, and then splice the interactive node feature coding, timestamp coding, and edge feature coding between the target node and the interactive node to obtain the historical neighbor embedding corresponding to the historical neighbor information, and finally arrange the historical neighbor embedding corresponding to each historical neighbor information in the column direction to obtain the historical neighbor embedding sequence. The historical neighbor embedding sequence is an embedding matrix, the number of rows of which is equal to the length of the historical neighbor embedding, and the number of columns of which is equal to the number of historical neighbor information included in the historical neighbor embedding.

[0083] For example, given an interaction event and time After considering the previous historical interaction data, the computational paradigm of the encoding layer can be expressed as:

[0084] ;

[0085] in, represents the historical neighbor embedding sequence, which is a matrix, represents the length of the historical neighbor embedding sequence, Represents the dimension of historical neighbor embeddings. Indicates at time Previously with node The set of neighbors that interact, Indicates Neighbors, Representation Node and neighbor nodes The edge features of Indicates The timestamp of the edge, Represents a splicing operation, represents the coding layer in the continuous time graph model, such as an MLP network. In the embodiment of the present application, for the sake of simplicity and to avoid ambiguity, The subscript node in and superscript timestamp Omit, use To express.

[0086] Step S30: input the historical neighbor embedding sequence into an initial enhancement model, and output an enhanced historical neighbor embedding sequence through the initial enhancement model.

[0087] Specifically, the initial enhancement model is used to enhance the historical neighbor embedding sequence in the latent space. That is, the enhanced historical neighbor embedding sequence is obtained by enhancing the historical neighbor embedding sequence in the latent space through the initial enhancement model. Such enhancement in the latent space can model the transition in the dynamic graph more subtly, thereby improving the data quality of the enhanced historical neighbor embedding sequence and improving the model performance of the trained continuous-time dynamic graph model.

[0088] The enhanced historical neighbor embedding sequence is obtained by enhancing the historical neighbor embedding sequence of the target node, and there is at least one historical neighbor information in the historical neighbor sequence corresponding to the target node, and the historical neighbor embedding of the historical neighbor information in the historical neighbor embedding sequence is different from the enhanced historical neighbor embedding in the enhanced historical neighbor embedding sequence. For example, Figure 2 The target node in The historical neighbor sequence of , where historical neighbor information The historical neighbor embedding in the historical neighbor embedding sequence is A, and the historical neighbor information The enhanced historical neighbor embedding in the enhanced historical neighbor embedding sequence is B, and A is different from B. In the embodiment of the present application, an enhanced historical neighbor embedding sequence different from the corresponding historical neighbor embedding sequence can be generated for the target node through the initial enhanced model, thereby avoiding the problem of poor model performance of the continuous-time dynamic graph model caused by insufficient historical data.

[0089] In one implementation of the embodiment of the present application, the initial enhancement model may adopt a diffusion model to enhance the historical neighbor embedding sequence. The diffusion model includes a forward denoising process and a reverse denoising process. The diffusion principle is: for a given distribution , the input data is sampled from this distribution , and then gradually move to Introduce Gaussian noise to get noisy output , and then gradually remove the noise output by learning to eliminate the added noise through the reverse denoising process to reconstruct the original data Specifically, the forward noise addition process gradually moves towards Introducing Gaussian noise to generate a series of Markov chain-like variables with increasing noise ratios , noise output Follows Gaussian distribution , does not carry any identifiable information about the original data point. Specifically, the transition probability of each time is given by the conditional probability Decision, among which Control in the n The scale of the noise added in the first step. The reverse denoising process starts from Start sampling by learning a neural network To implement the denoising process, in order to obtain the reconstructed original data .

[0090] Exemplarily, the step of inputting the historical neighbor embedding sequence into an initial enhancement model and outputting the enhanced historical neighbor embedding sequence through the initial enhancement model specifically includes:

[0091] S31, determining a first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence;

[0092] S32, forward diffusion is performed on the first sub-history neighbor embedding sequence through the diffusion model to obtain a sub-noise embedding sequence, and the sub-noise embedding sequence and the second sub-history neighbor embedding sequence are combined to obtain a noise embedding sequence;

[0093] S33. Perform reverse denoising on the noise embedding sequence through the diffusion model to obtain an enhanced historical neighbor embedding sequence.

[0094] Specifically, in step S31, the first sub-historical neighbor embedding sequence is used as the conditional information of the diffusion model, and the second sub-historical neighbor embedding sequence is used to be subjected to a forward diffusion operation to form a sub-noise embedding sequence. That is, the second sub-historical neighbor embedding sequence is used as the sub-historical neighbor embedding sequence to be enhanced, and the first sub-historical neighbor embedding sequence is used as the conditional information, and then the reverse denoising performed on the sub-noise embedding sequence is controlled by the first sub-historical neighbor embedding sequence to obtain an enhanced historical neighbor embedding sequence. The present application partially adds noise to the historical neighbor embedding sequence, so that an enhancement operation is performed in the latent space, thereby improving the authenticity of the enhanced historical neighbor embedding sequence. At the same time, by using part of the historical neighbor embedding sequence of the historical neighbor embedding sequence as conditional information and performing forward diffusion and reverse denoising on part of the historical neighbor embedding sequence, the enhancement amplitude of the historical neighbor embedding sequence can be controlled, and the reverse denoising can be controlled by the first historical neighbor embedding sequence to further improve the data quality of the generated enhanced reverse historical neighbor embedding sequence.

[0095] In one implementation, the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence are both sub-sequences of the historical neighbor embedding sequence, that is, the first sub-historical neighbor embedding sequence includes some historical neighbor embeddings in the historical neighbor embedding sequence, and the second sub-historical neighbor embedding sequence includes some historical neighbor embeddings in the historical neighbor embedding sequence. Among them, each historical neighbor embedding included in the first sub-historical neighbor embedding sequence is different from each historical neighbor embedding included in the second sub-historical neighbor embedding sequence, and the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence constitute a historical neighbor embedding sequence. In other words, the sequence length of the first sub-historical neighbor embedding sequence is The length of the sequence embedded with the second sub-history neighbor The sum of is equal to the sequence length of the historical neighbor embedding sequence ,Right now .

[0096] Exemplarily, determining the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence specifically includes:

[0097] Get the preset division ratio;

[0098] The historical neighbor embedding sequence is divided into a first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence according to the division ratio.

[0099] Specifically, the division ratio is used to reflect the proportional relationship between the first sub-history neighbor embedding sequence and the second sub-history neighbor embedding sequence, that is, the sequence length of the first sub-history neighbor embedding sequence The length of the sequence embedded with the second sub-history neighbor The ratio is equal to the division ratio. For example, if the division ratio is 1:7, then : =1:7. The division ratio may be preset (for example, 1:7), and the historical neighbor embedding sequence is divided according to the division ratio in each training process. Alternatively, a division ratio set may be preset, and the division ratio set includes multiple division ratios. In each training process, a division ratio is randomly selected from the division ratio set to divide the historical neighbor embedding sequence. Of course, other methods may also be used to divide the ratio, which is not limited here.

[0100] Further, after obtaining the division ratio, the historical neighbor embedding sequence is divided according to the division ratio, wherein the division can be performed according to the interaction time sequence. Specifically, when the historical neighbor embedding sequence is arranged according to the interaction time sequence, the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence are both continuous subsequences of the historical neighbor embedding sequence. For example, the historical neighbor embedding sequence , the first sub-history neighbor embedding sequence obtained by partitioning for , the second sub-history neighbor embedding sequence obtained by division for ,in , . Among them, some historical neighbor embeddings can be selected as conditional information to perform forward diffusion and reverse denoising on the unselected historical neighbor embeddings, that is, the first sub-historical neighbor embedding sequence can be used as conditional information to perform forward diffusion and reverse denoising on the second sub-historical neighbor embedding sequence; the second sub-historical neighbor embedding sequence can also be used as conditional information to perform forward diffusion and reverse denoising on the first sub-historical neighbor embedding sequence. In an embodiment of the present application, the first sub-historical neighbor embedding sequence can be selected as conditional information to perform forward diffusion and reverse denoising on the second sub-historical neighbor embedding sequence.

[0101] It should be noted that when selecting some historical neighbor embeddings as conditional information to perform forward diffusion and reverse denoising on some unselected historical neighbor embeddings, it is also possible to select a continuous subsequence in the historical neighbor embedding sequence as conditional information, select another continuous subsequence for forward diffusion and reverse denoising, etc. As long as some historical neighbor embeddings in the historical neighbor embedding sequence are used as conditional information, the scheme of performing forward diffusion and reverse denoising on some historical neighbor embeddings is within the protection scope of this application.

[0102] Further, in step S32, after obtaining the first sub-historical neighbor embedding sequence and the first subhistory neighbor embedding sequence After that, Forward diffusion, that is, Add noise to get the sub-noise embedded sequence Among them, the forward diffusion can adopt the existing diffusion process, which is not specifically limited here, and only two specific implementation methods are given for illustration.

[0103] In a specific implementation, The determination process can be expressed as:

[0104] ;

[0105] in To control the n The scale of the Gaussian noise added in the step, I represents the identity matrix, , N is a hyperparameter, is a Gaussian distribution, Indicates n -1 step of sub-noise embedding sequence, Indicates n The sub-noise embedding sequence of steps.

[0106] In another specific implementation, since the transfer kernel is a Gaussian distribution, it can be directly obtained from Sampling to get any step Therefore, let , , The determination process can also be expressed as:

[0107] ;

[0108] Among them, the distribution To reconstruct the parameters ,get .

[0109] Furthermore, to adjust the added noise, A linear noise setting can be used, that is It can be expressed as:

[0110] , ;

[0111] Among them, the hyperparameters Two hyperparameters are used to control the noise scale. Indicates the upper and lower limits of the added noise.

[0112] After obtaining the sub-noise embedding sequence, the sub-noise embedding sequence is merged with the second sub-historical neighbor embedding sequence to obtain the noise embedding sequence . Among them, the time position of each sub-noise in the sub-noise embedding sequence embedded in the noise embedding sequence is consistent with the time position of its corresponding first sub-historical neighbor embedded, and the time position of the first sub-historical neighbor embedded is the time position of the first sub-historical neighbor embedded in the historical neighbor embedding sequence. Specifically, when the sub-noise embedding sequence and the second sub-historical neighbor embedding sequence are combined to obtain the noise embedding sequence, the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence corresponding to the sub-noise embedding sequence can be spliced ​​in the time order of the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence in the historical neighbor embedding sequence, that is, the sub-noise embedding sequence and the second sub-history neighbor embedding sequence Concatenate in the order of [sub-noise embedding sequence, second sub-historical neighbor embedding sequence] to get the noise embedding sequence . The embodiment of the present application ensures that the temporal order of the noise embedding sequence remains unchanged by making the temporal position of each noise embedding in the noise embedding sequence consistent with the temporal position of each historical neighbor embedding in the historical neighbor embedding sequence, thereby making the temporal order of the enhanced historical neighbor embedding sequence consistent with the temporal order of the historical neighbor embedding sequence, thereby making the initial continuous-time dynamic graph model learn accurate temporal information, thereby improving the accuracy of the node representation determined by the initial continuous-time dynamic graph model.

[0113] It should be noted that, in practical applications, the noise embedding sequence can also be obtained by concatenating the sub-noise embedding sequence with the second sub-historical neighbor embedding sequence in other ways, for example, in the order of second sub-historical neighbor embedding sequence-sub-noise embedding sequence.

[0114] Further, in step S33, after obtaining the noise embedding sequence After that, the noise embedding sequence Perform reverse denoising to reconstruct the historical neighbor embedding sequence , that is, we get the enhanced historical neighbor embedding sequence. Specifically, Including forward diffusion and the unnoised second subhistory neighbor embedding sequence ,Will The denoising network included in the input diffusion model is used to Denoising is performed, wherein the denoising network is used to When denoising, embed the second sub-history neighbor into the sequence As the conditional input, the forward diffusion De-noising is performed to obtain an enhanced historical neighbor embedding sequence. Based on this, the reverse denoising process can be expressed as:

[0115] ;

[0116] ;

[0117] in, represents the conditional probability distribution, express The probability distribution of and The denoising network (e.g., a simple MLP network) parameterizes the predicted mean and standard deviation of the output, represents the probability distribution of the enhanced historical neighbor embedding sequence, , N is a hyperparameter.

[0118] It should be noted that when the enhanced historical neighbor embedding sequence is determined by the diffusion model, the diffusion process of the diffusion model requires a lot of computing resources, which will increase the training time and training cost of the continuous time dynamic graph model. Figure 3 As shown, in addition to the diffusion model, the initial enhancement model may also include a variational autoencoder, which includes a variational encoder and a variational decoder. The variational encoder, the diffusion model and the variational decoder are composed of a sandwich structure, that is, the diffusion module is located between the variational encoder and the variational decoder, and the historical neighbor embedding sequence obtained by the continuous-time dynamic graph model is first subjected to dimensionality reduction processing by the variational encoder before being input into the diffusion model, so as to encode the historical neighbor embedding sequence into a low-dimensional historical neighbor embedding sequence, and then the low-dimensional historical neighbor embedding sequence is used as the processing object of the diffusion model to reduce the computational complexity of the diffusion model, thereby reducing the training time and training cost of the continuous-time dynamic graph model. Of course, in order to ensure that the continuous-time dynamic graph model can obtain an enhanced historical neighbor embedding sequence with the same dimension as the historical neighbor embedding sequence, when determining the enhanced historical neighbor embedding sequence based on the low-dimensional historical neighbor embedding sequence through the diffusion model, it is necessary to first obtain the noise embedding sequence through the forward diffusion of the diffusion model according to the low-dimensional historical neighbor embedding sequence, and then input the noise embedding sequence into the extended model for reverse denoising to obtain the denoised historical neighbor embedding sequence, and then input the denoised historical neighbor embedding sequence into the variational decoder in the variational autoencoder for reconstruction to obtain the enhanced historical neighbor embedding sequence.

[0119] Exemplarily, determining the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence specifically includes:

[0120] Performing dimensionality reduction processing on the historical neighbor embedding sequence by the variational encoder to obtain a low-dimensional historical neighbor embedding sequence;

[0121] A first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence are determined based on the low-dimensional historical neighbor embedding sequence.

[0122] Specifically, a variational encoder is used to encode the historical neighbor embedding sequence to compress the historical neighbor embedding sequence into a low-dimensional historical neighbor embedding sequence, wherein the sequence length of the low-dimensional historical neighbor embedding sequence is equal to the sequence length of the historical neighbor embedding sequence, but the embedding dimension of the low-dimensional historical neighbor embedding in the low-dimensional historical neighbor embedding sequence is smaller than the embedding dimension of each historical neighbor embedding in the historical neighbor embedding sequence. For example, the historical neighbor embedding sequence , low-dimensional historical neighbor embedding sequence ,in, represents the embedding dimension of the low-dimensional historical neighbor embedding, represents the embedding dimension of historical neighbor embeddings, , represents the sequence length of the historical neighbor embedding sequence. For example, the variational encoder MLP networks can be used to embed historical neighbors into the sequence Input Variational Encoder , through the variational encoder Predicted mean and covariance , then based on and Determining the variational distribution , to obtain a low-dimensional historical neighbor embedding sequence In addition, the process of determining the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the low-dimensional historical neighbor embedding sequence is the same as the process of determining the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence, and will not be described in detail here.

[0123] Exemplarily, the reverse denoising of the noise embedding sequence by using the diffusion model to obtain an enhanced historical neighbor embedding sequence specifically includes:

[0124] Using the diffusion model to reversely denoise the noise embedding sequence to obtain a denoised historical neighbor embedding sequence;

[0125] The denoised historical neighbor embedding sequence is reconstructed by the variational decoder to obtain an enhanced historical neighbor embedding sequence.

[0126] Specifically, the variational decoder is used to restore the dimension of the denoised historical neighbor embedding sequence generated by the diffusion model based on the low-dimensional historical neighbor embedding sequence to the dimension of the historical neighbor embedding sequence, that is, the dimension is Restore to The process of obtaining the enhanced historical neighbor embedding sequence can be specifically as follows: the denoised historical neighbor embedding sequence generated by the diffusion model based on the low-dimensional historical neighbor embedding sequence Input Variational Decoder , variational decoder pass Get the enhanced history neighbor embedding sequence.

[0127] S40, inputting the enhanced historical neighbor embedding sequence into an initial continuous-time dynamic graph model, and determining the node representation of the target node through the initial continuous-time dynamic graph model.

[0128] Specifically, the node representation is the representation of the target node at a certain time. For example, the historical neighbor embedding sequence as described in the above example is the node exist The historical neighbor embedding sequence before time, then the node representation is the target node in The node representation can be used to predict the target node. For example, the target node category can be predicted based on the node representation. The node representation can be determined by the backbone layer in the continuous-time dynamic graph model, that is, the continuous-time dynamic graph model includes an encoding layer and a backbone layer. The encoding layer is used to encode the historical neighbor sequence to obtain the historical neighbor embedding sequence, and the backbone layer is used to determine the node representation of the target node based on the historical neighbor embedding sequence.

[0129] Further, if Figure 3 As shown, the initial enhancement model is integrated into the continuous-time dynamic graph model, that is, the coding layer in the continuous-time dynamic graph model, the initial enhancement model and the backbone layer in the continuous-time dynamic graph model are combined in a sandwich structure, wherein the initial enhancement model is located between the coding layer and the backbone layer, the historical neighbor embedding sequence encoded by the coding layer is the input item of the initial enhancement model, and the enhanced historical neighbor embedding sequence output by the initial enhancement model is the input item of the backbone layer, so that the backbone layer determines the node representation of the target node based on the enhanced historical neighbor embedding sequence. Specifically, when the initial enhancement model includes a diffusion model, the model structure formed by the continuous-time dynamic graph model and the initial enhancement model can be coding layer-diffusion model-backbone layer; when the initial enhancement model includes a variational autoencoder and a diffusion model, the model structure formed by the continuous-time dynamic graph model and the initial enhancement model can be coding layer-variational encoder-diffusion model-variational decoder-backbone layer.

[0130] The embodiment of the present application generates a variety of historical neighbor embedding sequences for each node through an initial enhancement model, thereby avoiding the impact of insufficient historical data on the continuous-time dynamic graph model. On the other hand, by generating historical neighbor embedding sequences in the latent space through the initial enhancement model, the transitions in the dynamic graph can be modeled more subtly, thereby providing a high-quality historical neighbor embedding sequence for the backbone layer, thereby improving the model performance of the continuous-time dynamic graph model.

[0131] Exemplarily, using the symbols of the computing paradigm of the coding layer, the computing paradigm of the backbone layer can be expressed as:

[0132] ;

[0133] in, Representation Node In time Node representation, represents the backbone layer in the continuous-time graph model, Represents a historical embedding sequence.

[0134] S50. Based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation, train the initial continuous-time dynamic graph model and the initial enhanced model to obtain a trained continuous-time dynamic graph model.

[0135] Specifically, the initial enhancement model is embedded into the initial continuous-time dynamic graph model, and cooperates with the initial continuous-time dynamic graph model to obtain a trained continuous-time dynamic graph model, and the historical neighbor embedding sequence is enhanced by the initial enhancement model to generate a variety of enhanced historical neighbor embedding sequences for the continuous-time dynamic model. To this end, when training the initial continuous-time dynamic graph model and the initial enhancement model, it is necessary to jointly train the initial continuous-time dynamic graph model and the initial enhancement model, that is, it is necessary to construct a loss function term of the initial enhancement model used for training and a loss function term of the initial continuous-time dynamic graph model used for training.

[0136] Based on this, the training of the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain the trained continuous-time dynamic graph model specifically includes:

[0137] Constructing a dynamic graph loss function term based on the node representation;

[0138] Constructing a diffusion loss function term based on the enhanced historical neighbor embedding sequence and the historical neighbor embedding sequence;

[0139] The initial continuous-time dynamic graph model and the initial enhanced model are trained based on the dynamic graph loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model.

[0140] In one implementation, training the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence, and the node representation to obtain a trained continuous-time dynamic graph model specifically includes:

[0141] Constructing a dynamic graph loss function term based on the node representation;

[0142] Constructing a diffusion loss function term based on the denoised historical neighbor embedding sequence and the low-dimensional historical neighbor embedding sequence, wherein the low-dimensional historical neighbor embedding sequence is obtained by performing dimensionality reduction processing on the historical neighbor embedding sequence through a variational encoder;

[0143] Constructing a variational loss function term based on the low-dimensional historical neighbor embedding sequence and the historical neighbor embedding sequence;

[0144] The initial continuous-time dynamic graph model and the initial enhanced model are trained based on the dynamic graph loss function term, the variational loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model.

[0145] Specifically, the diffusion loss function term is used as the loss function term of the initial enhancement model, and the initial enhancement model is trained through the diffusion loss function term. The dynamic graph loss function term is used as the loss function term of the initial continuous-time dynamic graph model, and the initial continuous-time dynamic graph model is trained through the dynamic graph loss function term. Wherein, when the initial continuous-time dynamic graph model and the initial enhancement model need to be jointly trained, an alternating training strategy or a simultaneous training strategy can be adopted, wherein the alternating training strategy refers to training the model parameters of the initial continuous-time dynamic graph model in one training, and training the model parameters of the initial enhancement model in the next training, alternating in sequence until the training is completed. The simultaneous training strategy refers to training the model parameters of the initial continuous-time dynamic graph model and the model parameters of the initial enhancement model at the same time in one training.

[0146] In one implementation of the embodiment of the present application, an alternating training strategy is used to train the initial enhancement model and the continuous-time dynamic graph model. The training process of the initial enhancement model and the training process of the initial continuous-time dynamic graph model are performed alternately. Wherein, when the initial enhancement model includes a diffusion model, during the training process of the initial enhancement model, the model parameters of the continuous-time dynamic graph model are frozen, and the initial enhancement model is trained using the diffusion loss function term; during the training process of the initial continuous-time dynamic graph model, the model parameters of the initial enhancement model are frozen, and the initial continuous-time dynamic graph model is trained using the dynamic graph loss function term. When the initial enhancement model includes a variational autoencoder and a diffusion model, when the initial enhancement model is trained, during the training process of the initial enhancement model, the model parameters of the continuous-time dynamic graph model are frozen, and the initial enhancement model is trained using the diffusion loss function term and the variational loss function term; during the training process of the initial continuous-time dynamic graph model, the model parameters of the initial enhancement model are frozen, and the initial continuous-time dynamic graph model is trained using the dynamic graph loss function term. In addition, when the initial enhancement model and the initial continuous-time dynamic graph model are alternately trained, a training end condition is preset, wherein the preset end condition may be that the diffusion loss function term and the variational loss function term are both less than a first preset threshold, the dynamic graph loss function term is less than a second preset threshold, or the number of trainings of the initial enhancement model and the initial continuous-time dynamic graph model both reach a preset number threshold. Of course, in practical applications, other training end conditions may also be used, such as the dynamic graph loss function term is less than a second preset threshold, or the number of trainings of the initial continuous-time dynamic graph model both reach a preset number threshold, etc.

[0147] Furthermore, the dynamic graph loss function term can be determined according to the task performed by the continuous-time dynamic graph model. There is no specific limitation here, and only a specific example is given.

[0148] For example: Assuming that the continuous-time dynamic graph model is used for a binary classification task, the loss function term corresponding to the continuous-time dynamic graph model can adopt the cross entropy loss function, that is, the loss function term corresponding to the continuous-time dynamic graph model can be expressed as:

[0149] ;

[0150] in, For Node The annotation label, Represents a binary classification neural network with the input item being , , the output item is the node The predicted label of .

[0151] The loss function term of the initial enhancement model may include a diffusion loss function term. In addition, when the initial enhancement model includes a variational autoencoder, the loss function term of the initial enhancement model may include a diffusion loss function term, and may also include a variational loss function term and a diffusion loss function term. Here, the initial enhancement model includes a variational autoencoder and a diffusion model, and the loss function term for training the initial enhancement model includes a variational loss function term and a diffusion loss function term.

[0152] The variational loss function term is used to optimize the parameters of the variational autoencoder. The variational loss function term The parameters of the VAE encoder can be optimized by maximizing the evidence lower bound (ELBO), the variational loss function term The calculation formula can be:

[0153] ;

[0154] Among them, the first is the reconstruction term, which is used to measure the likelihood of the decoder's reconstruction from the variational distribution. The second term is a priori term that measures the similarity between the learned variational distribution and the prior on the latent variable. is the low-dimensional historical neighbor embedding sequence obtained by compression in the above variational encoder, Embed the sequence of historical neighbors, represents the variational encoder, represents the variational decoder, represents the variational distribution, represents conditional probability.

[0155] Maximizing the ELBO is equivalent to maximizing the first term and minimizing the second term. Since the KL divergence term of the ELBO can be calculated analytically, and the reconstruction term can be approximated using Monte Carlo estimation, the variational loss function term can be rewritten as:

[0156] ;

[0157] Among them, from the variational distribution Sampling in samples, i.e., latent variables .

[0158] The diffusion loss function term is used to optimize the parameters of the diffusion model. Specifically, the variational lower bound (VLB) can be used to optimize the historical neighbor embedding sequence The log-likelihood of The calculation formula can be:

[0159] ;

[0160] in, Express expectations, represents the noise embedded sequence, Represents model parameters.

[0161] Next, we will explain in detail how to estimate the variational lower bound. make Approximating computable distributions . By Bayes' theorem, we know that given the successor and initial Any intermediate value Probability:

[0162] ;

[0163] in, , .

[0164] In actual training, in order to maintain the stability of training and simplify calculation, of learning and directly set Then, the diffusion loss function term It can be expressed as:

[0165] .

[0166] In a specific implementation, after adopting the loss function term of the initial continuous-time dynamic graph model and the loss function term of the initial enhancement model, the joint training process of the initial continuous-time dynamic graph model and the initial enhancement model can be specifically:

[0167] S1. Input the continuous-time dynamic graph into the initial continuous-time dynamic graph model to obtain the historical neighbor embedding sequence, determine the enhanced historical neighbor embedding sequence based on the historical neighbor embedding sequence and the initial enhanced model, and determine the node representation based on the enhanced historical neighbor embedding sequence and the initial continuous-time dynamic graph model; then determine the loss function of the initial continuous-time dynamic graph model ; Finally, freeze the model parameters of the initial enhancement model by minimizing the loss function of the initial continuous-time dynamic graph model To train the continuous time dynamic graph model, Wheel training;

[0168] S2, input the continuous-time dynamic graph into the initial continuous-time dynamic graph model trained in step S1 to obtain a historical neighbor embedding sequence, determine an enhanced historical neighbor embedding sequence based on the historical neighbor embedding sequence and the initial enhanced model, and determine a node representation based on the enhanced historical neighbor embedding sequence and the initial continuous-time dynamic graph model; then determine the loss function term of the initial enhanced model , Represents the preset weight coefficient; finally, the parameters of the initial continuous-time dynamic graph model are frozen, through train , , conduct Wheel training;

[0169] S3. Re-execute step S1 until the loss function term of the above-mentioned initial enhancement model and the loss function term of the initial continuous-time dynamic graph model both meet the preset training end conditions to obtain a trained continuous-time dynamic graph model.

[0170] In summary, the present embodiment provides a method for constructing a continuous-time dynamic graph model, which samples a historical neighbor sequence of a target node in a continuous-time dynamic graph, and inputs the historical neighbor sequence into an initial continuous-time dynamic graph model, and determines the historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model; inputs the historical neighbor embedding sequence into an initial enhanced model, and outputs an enhanced historical neighbor embedding sequence through the initial enhanced model; inputs the enhanced historical neighbor embedding sequence into the initial continuous-time dynamic graph model, and determines the node representation of the target node through the initial continuous-time dynamic graph model; based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation, the initial continuous-time dynamic graph model and the initial enhanced model are trained to obtain a trained continuous-time dynamic graph model. The embodiment of the present application processes the historical neighbor embedding sequence of the target node in the latent space through the initial enhanced model to obtain an enhanced historical neighbor embedding sequence, which can more subtly model the transition in the continuous-time dynamic graph, generate a variety of historical neighbor embedding sequences, and improve the model performance of the constructed continuous-time dynamic graph model.

[0171] Based on the above-mentioned method for constructing a continuous-time dynamic graph model, the present application embodiment also provides a device for constructing a continuous-time dynamic graph model, such as Figure 4 As shown, the construction device of the continuous-time dynamic graph model specifically includes:

[0172] A sampling module 100, for sampling a historical neighbor sequence of a target node in a continuous-time dynamic graph;

[0173] The control module 200 is used to input the historical neighbor sequence into an initial continuous-time dynamic graph model, and determine the historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model; input the historical neighbor embedding sequence into an initial enhancement model, and output an enhanced historical neighbor embedding sequence through the initial enhancement model; and input the enhanced historical neighbor embedding sequence into the initial continuous-time dynamic graph model, and determine the node representation of the target node through the initial continuous-time dynamic graph model;

[0174] The training module 300 is used to train the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain a trained continuous-time dynamic graph model.

[0175] The present application also provides an electronic device, such as Figure 5 As shown, it includes at least one processor 20, a memory 22, and may also include a communications interface 23 and a bus 21. The processor 20, the memory 22, and the communications interface 23 may communicate with each other through the bus 21. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0176] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0177] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0178] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0179] An embodiment of the present application provides a computer-readable storage medium, which stores one or more computer-readable programs. The one or more computer-readable programs can be executed by one or more processors to implement the steps in the computing power scheduling method based on the computing power application system as described above, and / or implement the functions of the computing power sharing APP as described above.

[0180] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the electronic device has been described in detail in the above-mentioned method, and will not be described here one by one.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing a continuous-time dynamic graph model, characterized in that: The method for constructing the continuous-time dynamic graph model specifically includes: Sampling the historical neighbor sequence of the target node in the continuous-time dynamic graph; Inputting the historical neighbor sequence into an initial continuous-time dynamic graph model, and determining a historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model; Inputting the historical neighbor embedding sequence into an initial enhancement model, and outputting an enhanced historical neighbor embedding sequence through the initial enhancement model; Inputting the enhanced historical neighbor embedding sequence into an initial continuous-time dynamic graph model, and determining a node representation of the target node through the initial continuous-time dynamic graph model; Based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation, the initial continuous-time dynamic graph model and the initial enhanced model are trained to obtain a trained continuous-time dynamic graph model, wherein the initial continuous-time dynamic graph model and the initial enhanced model are jointly trained.

2. The method for constructing a continuous-time dynamic graph model according to claim 1, characterized in that: The historical neighbor sequence of the target node in the sampled continuous-time dynamic graph specifically includes: Sampling a node in the continuous-time dynamic graph as a target node; The historical neighbor information of the target node is obtained according to the continuous-time dynamic graph to obtain a historical neighbor sequence of the target node.

3. The method for constructing a continuous-time dynamic graph model according to claim 1, characterized in that: The initial enhancement model includes a diffusion model; the step of inputting the historical neighbor embedding sequence into the initial enhancement model and outputting the enhanced historical neighbor embedding sequence through the initial enhancement model specifically includes: Determining a first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence; Performing forward diffusion on the first sub-history neighbor embedding sequence through the diffusion model to obtain a sub-noise embedding sequence, and merging the sub-noise embedding sequence with the second sub-history neighbor embedding sequence to obtain a noise embedding sequence; The noise embedding sequence is reversely denoised by the diffusion model to obtain an enhanced historical neighbor embedding sequence.

4. The method for constructing a continuous-time dynamic graph model according to claim 3, characterized in that: The determining of the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence specifically includes: Get the preset division ratio; The historical neighbor embedding sequence is divided into a first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence according to the division ratio.

5. The method for constructing a continuous-time dynamic graph model according to claim 3, characterized in that: The temporal position of each sub-noise in the sub-noise embedding sequence embedded in the noise embedding sequence is consistent with its temporal position embedded in the corresponding first sub-historical neighbor, wherein the temporal position of the first sub-historical neighbor embedding is the temporal position of the first sub-historical neighbor embedded in the historical neighbor embedding sequence.

6. The method for constructing a continuous-time dynamic graph model according to any one of claims 1 to 5, characterized in that: The training of the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain a trained continuous-time dynamic graph model specifically includes: Constructing a dynamic graph loss function term based on the node representation; Constructing a diffusion loss function term based on the enhanced historical neighbor embedding sequence and the historical neighbor embedding sequence; The initial continuous-time dynamic graph model and the initial enhanced model are trained based on the dynamic graph loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model.

7. The method for constructing a continuous-time dynamic graph model according to claim 6, characterized in that: The training process of the initial enhancement model and the training process of the initial continuous-time dynamic graph model are performed alternately; the initial continuous-time dynamic graph model and the initial enhancement model are trained based on the dynamic graph loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model, specifically including: During the training process of the initial enhancement model, the model parameters of the continuous-time dynamic graph model are frozen, and the initial enhancement model is trained using a diffusion loss function term; During the training process of the initial continuous-time dynamic graph model, the model parameters of the initial enhancement model are frozen, and the initial continuous-time dynamic graph model is trained using a dynamic graph loss function term.

8. The method for constructing a continuous-time dynamic graph model according to any one of claims 3 to 5, characterized in that: The initial enhancement model also includes a variational autoencoder; the variational autoencoder includes a variational encoder and a variational decoder, and the diffusion model is located between the variational encoder and the variational decoder.

9. The method for constructing a continuous-time dynamic graph model according to claim 8, characterized in that: The determining of the first sub-historical neighbor embedding sequence and the second sub-historical neighbor embedding sequence based on the historical neighbor embedding sequence specifically includes: Performing dimensionality reduction processing on the historical neighbor embedding sequence by using the variational encoder to obtain a low-dimensional historical neighbor embedding sequence; A first sub-historical neighbor embedding sequence and a second sub-historical neighbor embedding sequence are determined based on the low-dimensional historical neighbor embedding sequence.

10. The method for constructing a continuous-time dynamic graph model according to claim 9, characterized in that: The reverse denoising of the noise embedding sequence by using the diffusion model to obtain an enhanced historical neighbor embedding sequence specifically includes: Using the diffusion model to reversely denoise the noise embedding sequence to obtain a denoised historical neighbor embedding sequence; The denoised historical neighbor embedding sequence is reconstructed by the variational decoder to obtain an enhanced historical neighbor embedding sequence.

11. The method for constructing a continuous-time dynamic graph model according to claim 10, characterized in that: The training of the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain a trained continuous-time dynamic graph model specifically includes: Constructing a dynamic graph loss function term based on the node representation; Constructing a diffusion loss function term based on the denoised historical neighbor embedding sequence and the low-dimensional historical neighbor embedding sequence; Constructing a variational loss function term based on the low-dimensional historical neighbor embedding sequence and the historical neighbor embedding sequence; The initial continuous-time dynamic graph model and the initial enhanced model are trained based on the dynamic graph loss function term, the variational loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model.

12. The method for constructing a continuous-time dynamic graph model according to claim 11, characterized in that: The training process of the initial enhancement model and the training process of the initial continuous-time dynamic graph model are performed alternately; the initial continuous-time dynamic graph model and the initial enhancement model are trained based on the dynamic graph loss function term, the variational loss function term and the diffusion loss function term to obtain a trained continuous-time dynamic graph model, specifically including: During the training process of the initial enhancement model, the model parameters of the continuous-time dynamic graph model are frozen, and the initial enhancement model is trained using the diffusion loss function term and the variational loss function term; During the training process of the initial continuous-time dynamic graph model, the model parameters of the initial enhancement model are frozen, and the initial continuous-time dynamic graph model is trained using a dynamic graph loss function term.

13. A device for constructing a continuous-time dynamic graph model, characterized in that: The device for constructing the continuous-time dynamic graph model specifically includes: A sampling module, used to sample the historical neighbor sequence of the target node in the continuous-time dynamic graph; A control module, configured to input the historical neighbor sequence into an initial continuous-time dynamic graph model, determine a historical neighbor embedding sequence corresponding to the historical neighbor sequence through the initial continuous-time dynamic graph model; input the historical neighbor embedding sequence into an initial enhancement model, output an enhanced historical neighbor embedding sequence through the initial enhancement model; and input the enhanced historical neighbor embedding sequence into an initial continuous-time dynamic graph model, determine a node representation of the target node through the initial continuous-time dynamic graph model; A training module is used to train the initial continuous-time dynamic graph model and the initial enhanced model based on the historical neighbor embedding sequence, the enhanced historical neighbor embedding sequence and the node representation to obtain a trained continuous, wherein the initial continuous-time dynamic graph model and the initial enhanced model are jointly trained.

14. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, it implements the method for constructing a continuous-time dynamic graph model as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more computer-readable programs, and the one or more computer-readable programs can be executed by one or more processors. When the computer-readable program is executed by the processor, the method for constructing a continuous-time dynamic graph model as described in any one of claims 1-12 is implemented.

Images