A relationship structure distillation method for large-scale heterogeneous mismatched graph node classification

Abstract

The application discloses a relationship structure distillation method for large-scale heterogeneous incompatible graph node classification, comprising the following steps: acquiring heterogeneous graph data, wherein the heterogeneous graph data comprises nodes, edges, node attributes, node types, relationship types and target node labels; taking a node to be classified as a target node, organizing the neighborhood connection information of the target node according to the relationship types, and constructing a target node sample; training a heterogeneous graph teacher model, wherein the teacher model is used for encoding the node attributes and relationship structures in the heterogeneous graph, and outputting a teacher node representation; and generating a relationship structure summary according to the neighborhood connection distribution of the target node under each relationship type; the method can make full use of the discriminative information in the multi-type relationships and incompatible connections, and can reduce the dependence on multi-hop neighbor sampling, large-scale graph access and complex graph neural network calculation in the online reasoning stage, so that the deployable lightweight node classification is realized.

Description

Technical Field

[0001] This invention relates to the fields of graph neural networks and knowledge distillation technology, and in particular to a relational structure distillation method and system for classifying nodes in large-scale heterogeneous graphs. Background Technology

[0002] Currently, in the field of heterogeneous graph node classification technology, to handle graph data containing multiple types of nodes and relationships, existing technologies have proposed node classification schemes based on meta-paths, relationship type encoding, type-aware aggregation, or heterogeneous graph Transformers. These schemes, by distinguishing different node and relationship types, model the multi-type structures in heterogeneous graphs, which can improve the expressive power of node classification models for heterogeneous structures to a certain extent. However, in large-scale graph scenarios, these heterogeneous graph node classification schemes typically require accessing the target node's multi-hop neighbors, performing neighbor sampling, maintaining multi-type edge indexes, and performing complex attention calculations or relationship-level message passing during the online inference phase, resulting in high memory usage, computational overhead, and graph data access costs.

[0003] To reduce the deployment cost of complex models, knowledge distillation techniques are used to transfer knowledge from teacher models to lightweight student models. Existing distillation methods typically use the class probabilities, logits, or intermediate node representations output by the teacher model as distillation signals, enabling the student model to learn the classification boundary or representation space of the teacher model. In the field of heterogeneous graph knowledge distillation, HIRE (High-order Relational Knowledge; Liu J, Zheng T, Hao Q. HIRE: Distilling High-order Relational Knowledge From Heterogeneous Graph Neural Networks[J]. Neurocomputing, 2022. DOI:10.48550 / arXiv.2207.11887. and CN115115862A) is the first high-order relational knowledge distillation framework on heterogeneous graphs. HIRE performs first-order node-level knowledge distillation (encoding semantic information with the predicted logits of the teacher HGNN) and second-order relation-level knowledge distillation (mimicking the relational correlations between different types of node embeddings generated by the teacher HGNN). The student model is trained using node features as input and the classification results and relational relevance of the teacher model as distillation targets.

[0004] However, the above-mentioned heterogeneous graph node classification and knowledge distillation techniques still have the following shortcomings: First, graph neural network models that use uniform neighbor smoothing or simple aggregation methods struggle to effectively represent the discriminative structural information in heterogeneous connections. In scenarios where heterogeneous connections are prevalent, the classification semantics carried by different relation types are not consistent, and simple relation aggregation or weakening of key relation structures may lead to unstable classification results for target nodes.

[0005] Second, online inference of complex heterogeneous graph models in large-scale graph scenarios relies on multi-hop neighbor sampling, large-scale graph structure access, and relation-level computation. As the graph size, the number of relation types, and the number of sampled neighbors increase, the inference latency and system deployment difficulty increase significantly.

[0006] Third, existing knowledge distillation methods, when used for classifying nodes in heterogeneous and mismatched graphs, only align the classification output or node representation, making it difficult to explicitly constrain the student model's learning of the contribution of each relation type to the classification of the target node. Even with relation-level distillation introduced by schemes such as HIRE, the relation-level knowledge still exists in the form of the distillation target, and the student model cannot directly perceive the neighborhood distribution information of the target node under each relation type during the inference phase. Furthermore, the aforementioned schemes are not specifically designed for heterogeneous scenarios; their relation-level knowledge definitions are based on "correlation between different types of nodes," rather than addressing the characteristic of "different neighbor labels" in heterogeneous connections. Summary of the Invention

[0007] In view of this, the purpose of this invention is to propose a relational structure distillation method for classifying nodes in large-scale heterogeneous graphs, which enables the model to make full use of discriminative information in multi-type relations and heterogeneous connections, while reducing the dependence on multi-hop neighbor sampling, large-scale graph access and complex graph neural network computation in the online inference stage, thereby achieving deployable lightweight node classification.

[0008] According to one aspect of the present invention, a relation structure distillation method for classifying nodes in large-scale heterogeneous graphs is provided, comprising the following steps: Obtain heterogeneous graph data, which includes nodes, edges, node attributes, node types, relationship types, and target node labels; Using the node to be classified as the target node, organize the neighborhood connection information of the target node according to the relationship type to construct the target node sample; Train a heterogeneous graph teacher model, which is used to encode the node attributes and relational structure in the heterogeneous graph and output teacher node representations; Based on the neighborhood connection distribution of the target node under each relation type, a relation structure summary is generated; Based on the teacher node representation, the representation difference energy between the target node and its neighboring nodes within the same relation type is calculated, and statistics are performed according to relation type to generate a relation structure target; A lightweight student model is constructed, which takes the node features of the target node and the relation structure summary as input, and outputs node classification results and relation structure prediction results. The lightweight student model is jointly trained using the target node labels and the relation structure target through node classification supervision loss and relation structure distillation loss. During the online inference phase, the node features and relational structure summary of the target node to be classified are obtained, input into the trained lightweight student model, and the node classification result is output. The training of the heterogeneous graph teacher model, the generation of relation structure summaries and relation structure objectives, and the training of the lightweight student model are performed offline. The online inference stage is separate from the offline stage and does not depend on the heterogeneous graph teacher model or access to the original graph structure.

[0009] The above technical solution provides a relation structure distillation method for classifying nodes in large-scale heterogeneous graphs. After acquiring heterogeneous graph data, the method organizes the neighborhood information of target nodes according to relation types. In the offline stage, it completes the training of the heterogeneous graph teacher model, the generation of relation structure summaries and relation structure targets, and the distillation training of a lightweight student model. In the online inference stage, the target node features and relation structure summaries are used as inputs to the student model, and the node classification results are output. Moreover, the online stage does not rely on the teacher model or access to the original graph structure.

[0010] Compared with the closest prior art HIRE, the essential features of this claim are reflected in three aspects.

[0011] First, this scheme uses the neighborhood connection distribution statistically analyzed by relation type as a relation structure summary, which is directly used as an input feature for the student model, while HIRE's relation-level knowledge exists only in the form of distillation targets. Therefore, the student model can perceive the neighborhood distribution information under each relation type during the inference phase, without needing to query the original graph structure during the online phase. This difference decouples the computational complexity of online inference from the graph size, and the inference latency no longer increases with the number of neighbor hops or relation types.

[0012] Secondly, this scheme calculates the representational difference energy between the target node and its neighbors within the same relation type based on the teacher node representation, and statistically generates relation structure targets according to relation type. HIRE's relation-level distillation mimics the correlation between different types of node embeddings. In heteromigration scenarios, the distance between the target node and its neighbors in the teacher representation space itself reflects the discriminative strength of heteromigration connections—the greater the distance, the stronger the heteromigration signal. This design allows the distillation target to directly adapt to the physical characteristics of heteromigration scenarios. Compared to correlation energy, difference energy has a stronger characterizing ability for the classification contribution of heteromigration neighbors.

[0013] Third, after completing teacher model training, summary and target generation, and student model training offline, this scheme relies solely on node features and pre-computed relational structure summaries for reasoning in the online phase, without depending on the teacher model or access to the original graph structure. While existing schemes such as HIRE also employ the basic paradigm of offline teacher training and online student reasoning, the student model lacks input features that can directly perceive relational structure information. In the online phase, it still needs to acquire structural information through other means or rely on knowledge learned during distillation, failing to achieve complete decoupling from this scheme's independence from access to the original graph structure. This scheme, by pre-encoding relational structure information as summary features and using it as input to the student model, establishes data dependency isolation between online reasoning and the original graph structure.

[0014] Based on the above three points, this scheme achieves a balance between complex structure modeling and efficient online reasoning. Its various features work together—the relation structure summary provides the relation information carrier of the input layer, the representation difference energy provides the supervision signal for adapting heterogeneous scenarios in the distillation layer, and the offline-online separation architecture provides the deployment framework for the system layer—forming a complete technical link from structure learning to lightweight reasoning. This method can effectively reduce the dependence of online reasoning on large-scale graph storage access and complex graph model computation.

[0015] In summary, this claim effectively solves the technical problem of balancing complex structure modeling and online inference efficiency in large-scale heterogeneous graph node classification by organizing neighborhood information according to relation type to generate relation structure summaries as input to the student model, and by conducting joint distillation training in conjunction with relation structure objectives based on teacher representation difference energy. Combined with an offline-online separated inference architecture, it achieves the decoupling of the original graph structure and complex teacher model in the online stage while maintaining the knowledge transfer effect of heterogeneous relation structure.

[0016] In some embodiments, the relation structure summary includes statistics on the number of neighbors of the target node under each relation type.

[0017] In the above technical solution, the relation structure summary is further defined to include the neighbor count statistics of the target node under each relation type. These statistics numerically represent the neighborhood connection distribution of the target node across each relation dimension, serving as an input feature for the lightweight student model.

[0018] Compared to the closest existing technology, HIRE, the essential feature of this claim is that it uses the "number of neighbors counted by relation type" as the input feature of the student model, rather than using structural information only as a distillation supervision signal as in HIRE and its subsequent works (such as HG2M and HGKD). This formal transformation results in the student model being able to directly perceive the neighborhood distribution density of the target node under each relation type without accessing the original graph structure during the inference phase. For example, in a risk detection scenario, there are 5 neighboring devices under the "login" relation and 20 neighboring accounts under the "transfer" relation; this quantitative information can be directly input into the student model to participate in classification decisions.

[0019] The neighbor count statistic boasts extremely low computational cost—only one-hop neighbor counts are required in the online phase, without involving multi-hop sampling, attention calculations, or graph Transformer forward propagation. This scheme pre-encodes relational structure information as a statistical measure and uses it as input features, reducing the graph data access overhead for online inference from O(multi-hop subgraph loading + relation-level message passing) to O(one-hop neighbor counting or cache reading). Existing heterogeneous graph knowledge distillation schemes (HIRE, HG2M, etc.) use node features as the sole input in the student model during the inference phase. The learned structural knowledge is implicitly stored in the model parameters, and the differences in neighborhood distribution of target nodes under different relation types cannot be explicitly perceived in the online phase. This scheme uses the aforementioned statistical measure as input features, allowing the student model's classification decision to explicitly depend on relation-level neighborhood distribution information, while maintaining decoupling between the online phase and the original graph structure.

[0020] In heterogeneous match scenarios, the number of neighbors under different relationship types inherently carries discriminative information—an abnormally high or low number of neighbors under a certain relationship type may indicate that this relationship dimension contributes differently to the classification. This solution directly inputs this statistic, eliminating the need for the student model to extract this information from the hidden layer representation.

[0021] In some embodiments, the neighbor count is a logarithmic count of the number of neighbors under each relationship type.

[0022] In the above technical solution, the neighbor count is further defined as a logarithmic count of the number of neighbors under each relation type. This logarithmic count represents the original number of neighbors of the target node under each relation type. Transform into The value after compressing the dynamic range is used as the specific value of the relation structure summary.

[0023] Compared to the closest existing technology, HIRE, the essential feature of this claim lies in using a logarithmic transformation of the number of neighbors under each relation type as the summary component. The technical significance of this logarithmic transformation is that in heterogeneous graphs, the degree distribution of nodes under different relation types typically exhibits long-tail characteristics. For example, a "transfer" relation may connect hundreds of neighbors, while a "shared device" relation may only connect a few. Without this transformation, the summary component of high-degree relation types will dominate the student model input, while the contribution of low-degree but potentially highly discriminative relation types (such as the "login" relation, which is often more discriminative than the "transfer" relation in risk detection) to classification decisions is relatively diluted.

[0024] This solution adopts The form, where adding 1 ensures that when When =0, the logarithm is 0, avoiding mathematical undefined problems, while allowing small degree changes in sparse relation types to have a relatively larger contribution fluctuation in the summary. For example, When increasing from 1 to 2 The increment is approximately 0.405. The increment from 100 to 101 is approximately 0.005—this transformation makes the student model more sensitive to changes in the number of neighborhoods for sparse but key relation types, while maintaining a certain degree of saturation for highly connected relation types so as not to overwhelm signals from other relation dimensions.

[0025] While logarithmic transformation is a well-known mathematical operation in the field, its application to the generation of heterogeneous graph relation structure summaries, with the aim of compressing long-tailed distributions and balancing the contributions of different relation types to classification decisions, is not revealed in existing literature on heterogeneous graph knowledge distillation. HIRE's student model cannot directly input any form of neighborhood quantity information, nor does it have a logarithmic transformation design for long-tailed distributions.

[0026] In some embodiments, the neighbor count is a normalized proportion of the number of neighbors under each relationship type.

[0027] In the above technical solution, the neighbor count statistic is further defined as a normalized ratio of the neighbor count under each relation type. This normalized ratio represents the number of neighbors of the target node under each relation type. Perform normalization processing, that is This ensures that the sum of the summary components corresponding to each relation type is 1 (in...). When negligible, the distribution of neighborhood connections of the target node is represented in the form of a relative proportion.

[0028] Compared to the aforementioned logarithmic counting form, this claim uses a normalized ratio as the statistical measure. The essence of the normalized ratio is that it represents the relative distribution of the connection density of the target node across different relation types, rather than an absolute quantity. Specifically, its core lies in emphasizing relative relationships—when... Include When dealing with different relation types, the normalized proportional vector Construct a probabilistic simplex ( Its physical meaning is: the proportion of the target node's connection tendency in each relation dimension in the overall connection behavior.

[0029] The technical advantages of normalization ratio are as follows: First, the summary vectors between different target nodes can be directly compared by cosine similarity or Euclidean distance, unaffected by the difference in their total degree. This allows the student model to identify the similarity of "connection behavior patterns" during classification, rather than being dominated by the absolute number. Second, normalization ratio has inherent scale invariance. In heterogeneous mapping, the absolute number of neighbors of two target nodes may differ by a factor of hundreds (e.g., active users and inactive users), but their relationship type distribution patterns (e.g., 80% of connections come from "login" relationships) may be highly similar. Normalization ratio can capture this pattern-level similarity, while logarithmic counting still retains absolute quantity information. They are suitable for different scenarios. Logarithmic counting is suitable for scenarios that need to retain information on the difference in magnitude, while normalization ratio is suitable for scenarios that need to emphasize the relationship distribution pattern.

[0030] HIRE's student model uses node features as its sole input, and HG2M's student MLP also uses only node features as input. Their structural knowledge is implicitly stored in the model parameters in the form of distillation targets, and there is no design that explicitly uses the normalization ratio as an input feature. This normalization ratio and logarithmic count serve different scenario needs—logarithmic count is suitable for scenarios with high "absolute connection strength of relation types" and high classification correlation, while the normalization ratio is suitable for scenarios with high "distribution structure of relation types" and high classification correlation.

[0031] In some embodiments, the difference energy is the norm distance between teacher node representations.

[0032] In the above technical solution, the representation difference energy is further defined as the norm distance between the teacher node representations. That is, for the target node... and its neighboring nodes of the same relation type The difference energy is specifically defined as follows: (Square of Euclidean distance), which measures the degree of discriminative separation between the target node and its neighbors in the teacher representation space using the geometric distance between them.

[0033] Compared to the closest prior art, HIRE, the essential feature of this claim lies in the difference in the definition of "relational knowledge." HIRE's relational distillation objective is "relational correlation between different types of node embeddings," which physically measures the similarity of two node representations in a direction—the higher the correlation, the more similar the representations; while the representation difference energy in this scheme is the norm distance between teacher node representations, which physically measures the absolute separation of two node representations in space—the greater the distance, the greater the representation difference.

[0034] This difference has substantial technical significance in heteromigration scenarios. In heteromigration graphs, connected nodes often belong to different categories, and their teacher representations should ideally be separated—that is, the greater the distance between the heteromigration neighbor and the target node in the representation space, the more fully the teacher model captures the discriminative signal of the heteromigration relationship. Norm distance directly quantifies this degree of separation: the greater the distance, the stronger the discriminative information carried by the heteromigration neighbor. In contrast, correlation measures directional consistency. In heteromigration scenarios, the node representations of different categories may not be completely orthogonal in direction, making it difficult for correlation to effectively distinguish between the two different structural semantics of "similar homomigration" and "dissimilar heteromigration." This scheme uses norm distance as a specific form of representing the difference energy, allowing the relation structure target to directly reflect the discriminative strength of heteromigration connections—when the norm distance between neighbors and target nodes under the same relation type in the teacher representation space is generally large, it indicates that the relation type carries a strong heteromigration discriminative signal; conversely, it indicates that the discriminative contribution of the relation type is weak.

[0035] While norm distance is a commonly used similarity / difference metric in this field, its use as the core definition of a relation-level distillation objective in heterogeneous graphs, replacing the relevance metric used in existing schemes such as HIRE, is not explicitly revealed in existing literature on heterogeneous graph knowledge distillation. HIRE explicitly chose the relevance path rather than the distance path when defining relation-level knowledge, indicating the existence of multiple optional metric methods in this technical field. This scheme chooses norm distance due to considerations of adapting to the physical characteristics of heterogeneous scenarios—distance metric corresponds to "degree of separation," which matches the characteristic of "different neighbor labels" in heterogeneous connections.

[0036] In some embodiments, the statistical analysis by relation type includes: calculating the mean of the representational difference energy of all neighbors within the same relation type.

[0037] In the above technical solution, the statistical analysis by relation type is further defined to include calculating the mean of the representation difference energy of all neighbors within the same relation type. For the target node and relation types This statistic is That is, the arithmetic mean of the representation difference energy of all neighbors within the same relation type is logarithmically transformed and used as the target component of the relation structure corresponding to that relation type.

[0038] Compared to the closest existing technology, HIRE, the essential feature of this claim is that HIRE's relation-level distillation target mimics the correlation between different types of node embeddings—this statistic does not involve the aggregation statistics of multiple neighbors within the same type of relation; while this solution further limits the average difference energy of all neighbors within the same relation type. This average statistic characterizes the central tendency of the degree of difference between the target node and all neighbors under this relation type—its physical meaning is: how strong a discriminative signal the relation type as a whole provides for the classification of the target node.

[0039] The mechanism of this design lies in the fact that in heterogeneous mapping, the discriminative contributions of different neighbors under the same relation type to the target node may differ, but excessively high differential energy of individual strong-signal neighbors may lead to statistical instability. Mean statistics, by averaging the differential energy of all neighbors within the same relation type, eliminate the influence of individual noise and extreme values, ensuring that the target component of the relation structure for that relation type reflects the overall discriminative strength of that relation dimension, rather than being dominated by individual strong-signal or noisy neighbors. Simultaneously, the logarithmic transformation compresses the dynamic range of the mean, making the target components between different relation types numerically comparable.

[0040] The core contribution of the mean is to aggregate relation-level knowledge from the differential energy at the node pair level into a comprehensive discriminative strength index at the relation type level. This is an information abstraction process from "micro" (each pair of neighbors) to "macro" (aggregation by type), enabling the student model to learn the structural discriminative knowledge of the teacher model at the granularity of relation type.

[0041] In some embodiments, the statistical analysis by relation type includes: calculating the maximum value of the representational difference energy of all neighbors within the same relation type.

[0042] In the above technical solution, it is further specified that: the statistical analysis by relation type includes calculating the maximum value of the representation difference energy of all neighbors within the same relation type. For the target node and relation types This statistic is That is, the maximum value of the difference energy among all neighbors within the same relation type is logarithmically transformed and used as the target component of the relation structure corresponding to that relation type.

[0043] Compared to the aforementioned mean statistics, this claim uses the maximum value as the statistical measure, and the two differ in their information dimensions. The mean reflects the central tendency of the differential energy of all neighbors within the same relation type, while the maximum value captures the peak signal generated by the most discriminative neighbor within that relation type. In the scenario of heterogeneous graph node classification, the discriminative contribution of different neighbors under the same relation type to the target node is often not uniformly distributed—a few "strongly heterogeneous" neighbors (indicating high separation from the target node) may carry the main discriminative information of that relation dimension, while the differential energy of most neighbors is at a medium or low level. In this case, mean statistics may weaken the discriminative signal strength of that relation type due to the dilution by a large number of low-differential-energy neighbors, while maximum value statistics can directly capture this peak signal, making the distillation target more sensitive to reflect the most discriminative structural information within the relation type.

[0044] The complementarity between the maximum and mean values ​​is reflected in the following ways: when the discriminative contributions of neighbors within a relation type are relatively uniform, the mean can stably reflect the overall level, while the maximum value may be biased due to a single outlier; when there are a few highly discriminative neighbors within a relation type, the maximum value can capture the peak signal, while the mean may underestimate the discriminative strength of that relation type due to the dilution effect. They serve different application scenarios—the mean is suitable for situations where neighbor contributions are relatively uniformly distributed, while the maximum value is suitable for situations where a few strong-signal neighbors dominate the discriminative process—those skilled in the art can choose the appropriate statistical method based on the data distribution characteristics during actual implementation, or use them separately at different stages.

[0045] According to another aspect of the present invention, a relational structure distillation system for classifying nodes in large-scale heterogeneous graphs is provided, based on the above method, comprising: The data processing module is used to acquire heterogeneous graph data, which includes nodes, edges, node attributes, node types, relationship types, and target node labels. The target node construction module is used to construct target node samples by taking the node to be classified as the target node and organizing the neighborhood connection information of the target node according to the relationship type. The teacher structure learning module is used to train a heterogeneous graph teacher model, which encodes the node attributes and relational structures in the heterogeneous graph and outputs teacher node representations. The relation structure knowledge generation module is used to generate a relation structure summary based on the neighborhood connection distribution of the target node under each relation type, and to calculate the representation difference energy between the target node and its neighboring nodes in the same relation type based on the teacher node representation, and to generate the relation structure target by relation type. The student distillation training module is used to construct a lightweight student model. The lightweight student model takes the node features of the target node and the relation structure summary as input, outputs the node classification result and the relation structure prediction result, and uses the target node label and the relation structure target to jointly train the lightweight student model through node classification supervision loss and relation structure distillation loss. The online node classification module is used to obtain the node features and relation structure summary of the target node to be classified during the inference stage, input the trained lightweight student model, and output the node classification result. The teacher structure learning module, the relation structure knowledge generation module, and the student distillation training module are deployed offline, while the online node classification module is deployed online. The relation structure summary generated by the relation structure knowledge generation module is cached for use by the online node classification module.

[0046] In order to better utilize the above method, this application proposes a relational structure distillation system for classifying nodes in large-scale heterogeneous graphs. Each module corresponds to a step in the above method, and its specific principle has been described above and will not be repeated here.

[0047] According to another aspect of the present invention, a relational structure distillation apparatus for classifying nodes in large-scale heterogeneous and heterogeneous graphs is provided, comprising: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0048] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0049] According to another aspect of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method.

[0050] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the overall framework of an embodiment of the relational structure distillation method for classifying nodes in large-scale heterogeneous graphs according to the present invention. Figure 2 This is a flowchart illustrating an embodiment of a relational structure distillation method for classifying nodes in large-scale heterogeneous graphs according to the present invention. Figure 3 This is a block diagram illustrating the relational structure knowledge generation principle of an embodiment of the relational structure distillation method for classifying nodes in large-scale heterogeneous graphs according to the present invention. Figure 4 This is a block diagram illustrating the principle of lightweight student model training and online inference in an embodiment of a relational structure distillation method for classifying nodes in large-scale heterogeneous graphs according to the present invention. Figure 5 This is a schematic diagram of an embodiment of a relational structure distillation system for classifying nodes in a large-scale heterogeneous graph according to the present invention. Detailed Implementation

[0053] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1 This invention provides a relational structure distillation method and system for classifying nodes in large-scale heterogeneous graphs. Combined with... Figure 1 As shown, the overall framework of this invention comprises five parts: data processing, teacher structure learning, relational structure knowledge generation, student distillation training, and online node classification reasoning. Its core idea is as follows: In the offline stage, a heterogeneous graph teacher model is used to fully learn the node attributes, node types, relation types, and heterogeneous connection structures in a large-scale heterogeneous graph; then, the relational structure knowledge learned by the teacher model is transformed into relational structure summaries and relational structure targets; finally, a lightweight student model is trained using the target node labels and relational structure targets, enabling the student model to output node classification results online based on the target node features and relational structure summaries.

[0055] Please see Figure 2 A relational structure distillation method for classifying nodes in large-scale heterogeneous graphs includes the following steps: S1. Obtain heterogeneous graph data, which includes nodes, edges, node attributes, node types, relationship types, and target node labels; For example, the original business data is constructed into heterogeneous graph data. The original business data includes a node table, an edge table, a node attribute table, and a label table. The node table includes node identifiers and node types, the edge table includes source node identifiers, target node identifiers, and relationship types, the node attribute table includes numerical features, category features, or statistical features of the nodes, and the label table includes classification labels for target nodes.

[0056] This leads to the construction of a heterogeneous graph. ;in, For a set of nodes, Let be the set of edges. A collection of node types For a set of relation types, Mapping for node types, For relation type mapping, For node attribute features, The target node label.

[0057] When different node types have inconsistent attribute dimensions, a type mapping layer can be set up separately to map different types of nodes to a unified feature space: ;in, For nodes The original characteristics, and This refers to the mapping parameters corresponding to this node type. For nodes lacking original attributes, node type embeddings, statistical features, or learnable initialization vectors can be used as input features.

[0058] S2. Using the node to be classified as the target node, organize the neighborhood connection information of the target node according to the relationship type to construct the target node sample; For example, target node samples are constructed using the node to be classified as the target node. For the target node... Extract its node features Node labels And organize the neighborhood connection information according to the relation type. For any relation type Target node The set of relational neighbors is .

[0059] Therefore, a target node sample can be represented as .

[0060] S3. Train a heterogeneous graph teacher model, which is used to encode the node attributes and relational structure in the heterogeneous graph and output the teacher node representation; For example, a heterogeneous graph teacher model is trained offline. This teacher model learns discriminative information from multiple node types, multiple relation types, and heterogeneous connection structures. The teacher model can be implemented using type-aware encoding, relation type encoding, heterogeneous graph attention, or a graph Transformer structure.

[0061] Teacher Model For the target node Output teacher node representation Teacher classification results :

[0062]

[0063] The teacher model is trained under supervised supervision using target node labels:

[0064] in, This represents the set of target nodes with labels. After the teacher model is trained, the teacher model parameters are fixed, and the teacher node representations of the target nodes and their neighboring nodes are derived. This is used to generate relational structure targets in the future.

[0065] S4. Generate a relation structure summary based on the neighborhood connection distribution of the target node under each relation type; In this embodiment, the relation structure summary includes the statistics of the number of neighbors of the target node under each relation type.

[0066] In this embodiment, the neighbor count is the logarithmic count of the number of neighbors under each relationship type.

[0067] In this embodiment, another calculation method is provided, wherein the neighbor count is the normalized ratio of the number of neighbors under each relationship type.

[0068] For example, please refer to Figure 3 The relation structure knowledge generation module receives target node samples and teacher node representations output by the trained teacher model, and generates relation structure summaries, relation masks, and relation structure targets.

[0069] For the target node and relation types Count the number of neighbors under this relationship type. This generates relational structure summary components. These relational structure summary components can be in logarithmic form. Normalized proportions can also be used. ,in, To prevent constants with zero denominators, the summary components corresponding to each relation type are concatenated in a fixed order to obtain the relation structure summary. ;in, This represents the number of relation types. The relation structure summary is used to provide lightweight relation distribution information to the student model.

[0070] At the same time, a relation mask is generated: The relation mask is used to identify whether the target node has neighbors of the corresponding relation type, and to exclude invalid supervision that does not have a relation type in the distillation loss calculation.

[0071] S5. Based on the teacher node representation, calculate the representation difference energy between the target node and its neighboring nodes within the same relation type, and perform statistics according to relation type to generate a relation structure target; In this embodiment, the difference energy represents the norm distance between teacher node representations.

[0072] In this embodiment, the statistical analysis by relation type includes: calculating the mean of the representational difference energy of all neighbors within the same relation type.

[0073] In this embodiment, another calculation method is provided, wherein the statistics by relation type include: calculating the maximum value of the representation difference energy of all neighbors within the same relation type.

[0074] For example, a relational structure goal is generated based on the teacher node representation. For neighbor nodes... Calculate the energy of the difference in teacher representations between the target node and its neighboring nodes. Then, calculate the average differential energy within the same relation type. and maximum difference energy .

[0075] when When this occurs, the target value corresponding to that relation type is set to zero and excluded using a relation mask. The statistical results for all relation types are then concatenated to obtain the relation structure target. .

[0076] After the above processing, each target node forms a student model training sample. ,in, This is a summary of the relational structure. For relational structure objectives, This is the distillation loss mask obtained by expanding the relation mask.

[0077] S6. Construct a lightweight student model, which takes the node features of the target node and the relation structure summary as input, and outputs node classification results and relation structure prediction results. S7. Using the target node labels and the relation structure target, train the lightweight student model jointly using node classification supervision loss and relation structure distillation loss. For example, please refer to Figure 4 During the training phase, the lightweight student model uses the features of the target node. Summary of relational structure As input, the two are concatenated as The student model outputs node classification results. Relationship structure prediction results :

[0078]

[0079] in, For the classification output header, Output header for relational structure. These are the parameters for the student model. The dimension of the classification output head is the same as the number of node categories, and the dimension of the relation structure output head is the same as the relation structure target. Dimensional consistency (i.e.) dimension).

[0080] The student model can be implemented using a multilayer perceptron or other lightweight feedforward networks. It can have 2 to 6 hidden layers, with hidden dimensions of 64, 128, 256, or 512, and activation functions such as ReLU, GELU, or LeakyReLU can be used.

[0081] The student model is trained using a joint loss. The node classification supervision loss is... The distillation loss in the relational structure is... The total loss is ,in, For a set of target nodes with labels, This is the distillation loss mask obtained by expanding the relation mask. For element-wise multiplication, The weights for distillation loss in relational structures.

[0082] Through the above training, the student model learns both the target node classification boundary and the relational structure knowledge extracted by the teacher model.

[0083] S8. In the online inference stage, obtain the node features and relational structure summary of the target node to be classified, input the trained lightweight student model, and output the node classification result.

[0084] For example, after training, a distilled lightweight student model is exported. During the online inference phase, the system only acquires the node features of the target nodes to be classified. Summary of relational structure Input the distilled lightweight Student model and output the node classification results. The output can be the target node category, category probability, risk score, or anomaly score.

[0085] The relation structure summary in the online phase can be obtained either through offline pre-computation caching or through online lightweight queries. Online lightweight queries only need to count the number or proportion of next-hop neighbors of the target node under different relation types; they do not need to load the complete multi-hop subgraph or call the heterogeneous graph teacher model.

[0086] Through the above embodiments, those skilled in the art can implement the present invention in the following order: constructing heterogeneous graph data; constructing target node samples; offline training of the heterogeneous graph teacher model; and so on. Figure 3 Generate relation structure summary, relation mask, and relation structure target; according to Figure 4 A lightweight student model is trained and exported; in the online stage, node classification results are output based on target node features and relational structure summaries. This implementation allows complex heterogeneous and heterogeneous structure modeling to be completed in the offline stage, while the online node classification stage uses a lightweight model for inference.

[0087] To more clearly illustrate the implementation of this invention, the following description uses a risk node classification scenario as an example. Entities such as users, devices, transactions, and addresses are constructed as different types of nodes, and behaviors such as login, binding, transfer, and device sharing are constructed as edges of different relationship types. User nodes are selected as target nodes, and their classification labels can be normal users or risky users.

[0088] Following the steps above, a heterogeneous graph teacher model is trained offline, and teacher node representations are derived. The teacher model can learn discriminative structural information such as: if neighboring devices in a "login relationship" have a large number of abnormal login records, the user is considered high-risk; if a neighboring transaction account in a "transfer relationship" is a newly registered account, it may pose a risk. Then, a relational structure summary is generated based on the distribution of neighbor connections for user nodes under different relation types—for example, if a user has 5 device neighbors in the "login" relationship, 20 transaction neighbors in the "transfer" relationship, and 3 device neighbors in the "shared device" relationship, then their relational structure summary can be represented as follows: The teacher model generates relational structure objectives based on the difference energy of teacher representations—for example, a large average difference energy between the teacher representation of a user under the "login" relation and its five device neighbors indicates that the teacher model considers this relation type to be highly discriminative for classification. A lightweight student model is then trained.

[0089] In online operations, when a user to be evaluated enters the system, only the user's node features and a summary of the relationship structure are input, and the classification result or risk score can be output. There is no need to load the user's complete multi-hop subgraph, nor to call a complex heterogeneous graph teacher model.

[0090] In one alternative implementation, relation energy screening can be performed before training the teacher model. (Regarding relation type...) bottom edge Calculate relational energy based on node structure representation ;in, and This is represented by a node structure. The system can filter connections within the same relation type based on energy distribution to reduce the impact of redundant or noisy edges on the teacher model training. This step is an offline enhancement step and does not change the input and execution methods of the student model during the online inference phase.

[0091] In another alternative implementation, the teacher structure learning module can also combine node-level supervision and edge-level weak supervision. Node-level supervision is used to constrain the teacher model's output of target node classification results, while edge-level weak supervision is used to assist the teacher model in identifying the contribution of different relational connections to the target node classification. This approach enhances the teacher model's ability to characterize heterogeneous and mismatched relations, enabling the subsequently generated relational structure targets to contain more stable relational discriminative information.

[0092] The heterogeneous graph teacher model in this invention is not limited to a specific network name. Any graph learning model capable of modeling node types, relation types, and heterogeneous neighborhood structures and outputting teacher node representations can be used as an implementation of the teacher structure learning module. Similarly, the lightweight student model in this invention is not limited to a fixed number of layers or a fixed hidden dimension. Any lightweight model capable of receiving target node features and relation structure summaries and outputting node classification results and relation structure prediction results can be used as an implementation of the student distillation training module.

[0093] Through the above technical solution, this invention concentrates the modeling of complex heterogeneous and heterogeneous graphs in the offline teacher model stage, transforming relational structure knowledge into a learnable relational structure summary and relational structure objective for the student model, and then using a lightweight student model to complete online node classification. Therefore, knowledge distillation is no longer a compression step added after the node classification model, but a necessary component for the online deployment of large-scale heterogeneous and heterogeneous graph node classification schemes.

[0094] Compared with the prior art, the present invention has at least the following beneficial effects: First, this invention can improve the utilization of relational structure information in the classification of nodes in large-scale heterogeneous graphs. By organizing the neighborhood connection information of the target node according to the relation type and generating a relational structure summary, the model can distinguish the different influences of neighboring nodes on the classification of the target node under different relation types, avoiding the weakening of discriminative structure information caused by simply mixing and aggregating multiple types of relations.

[0095] Second, this invention can transfer the relational structure knowledge learned by the teacher model of complex heterogeneous graphs to a lightweight student model. By generating relational structure targets based on the difference energy of teacher node representations, the student model not only learns the classification boundaries corresponding to the target node labels, but also learns the structural differences and discriminative information under different relation types. Compared with the conventional distillation method that only aligns the teacher classification output or intermediate representation, it is more suitable for node classification tasks of heterogeneous graphs.

[0096] Third, this invention can reduce the dependence on large-scale graph structure access during the online node classification stage. During online inference, the lightweight student model takes the target node features and relation structure summary as input, without having to fully call the heterogeneous graph teacher model, nor repeatedly perform complex multi-hop neighbor sampling, relation-level message passing, or graph Transformer forward computation, thereby reducing the graph data access pressure and computational complexity during the online inference process.

[0097] Fourth, this invention can improve the deployment adaptability of large-scale heterogeneous graph node classification schemes. The complex heterogeneous structure learning process is mainly completed in the offline teacher model stage, while the online stage is performed by the distilled lightweight student model. Therefore, it is more suitable for real-time classification, batch target node prediction, and business deployment environments with limited computing resources.

[0098] Fifth, this invention forms a complete technical chain from offline structure learning to online lightweight inference. The relation structure summary is used to provide lightweight relation information that the student model can input, and the relation structure target is used to provide a supervisory signal for the teacher model's relation structure knowledge. Together, they enable a deep integration of knowledge distillation and node classification tasks.

[0099] Example 2 Please see Figure 5 A relational structure distillation system for classifying nodes in large-scale heterogeneous graphs, based on the method described in one embodiment, includes: The data processing module is used to acquire heterogeneous graph data, which includes nodes, edges, node attributes, node types, relationship types, and target node labels. The target node construction module is used to construct target node samples by taking the node to be classified as the target node and organizing the neighborhood connection information of the target node according to the relationship type. The teacher structure learning module is used to train a heterogeneous graph teacher model, which encodes the node attributes and relational structures in the heterogeneous graph and outputs teacher node representations. The relation structure knowledge generation module is used to generate a relation structure summary based on the neighborhood connection distribution of the target node under each relation type, and to calculate the representation difference energy between the target node and its neighboring nodes in the same relation type based on the teacher node representation, and to generate the relation structure target by relation type. The student distillation training module is used to construct a lightweight student model. The lightweight student model takes the node features of the target node and the relation structure summary as input, outputs the node classification result and the relation structure prediction result, and uses the target node label and the relation structure target to jointly train the lightweight student model through node classification supervision loss and relation structure distillation loss. The online node classification module is used to obtain the node features and relation structure summary of the target node to be classified during the inference stage, input the trained lightweight student model, and output the node classification result. The teacher structure learning module, the relation structure knowledge generation module, and the student distillation training module are deployed offline, while the online node classification module is deployed online. The relation structure summary generated by the relation structure knowledge generation module is cached for use by the online node classification module.

[0100] In this embodiment, in order to better utilize the method described in one of the embodiments, this application proposes a relational structure distillation system for classifying nodes in large-scale heterogeneous graphs. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0101] Example 3 A relational structure distillation apparatus for classifying nodes in large-scale heterogeneous and heterogeneous graphs, comprising: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in one of the embodiments.

[0102] In this embodiment, to better run and process the method described in one of the embodiments, the above method is stored in a memory, and the stored method is executed using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0103] Example 4 A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in one of the embodiments.

[0104] In this embodiment, to better operate and use the method described in one of the embodiments, the above method is stored in a computer-readable storage medium, and the above method is implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0105] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A relational structure distillation method for classifying nodes in large-scale heterogeneous graphs, characterized in that, Includes the following steps: Obtain heterogeneous graph data, which includes nodes, edges, node attributes, node types, relationship types, and target node labels; Using the node to be classified as the target node, organize the neighborhood connection information of the target node according to the relationship type to construct the target node sample; Train a heterogeneous graph teacher model, which is used to encode the node attributes and relational structure in the heterogeneous graph and output teacher node representations; Based on the neighborhood connection distribution of the target node under each relation type, a relation structure summary is generated; Based on the teacher node representation, the representation difference energy between the target node and its neighboring nodes within the same relation type is calculated, and statistics are performed according to relation type to generate a relation structure target; A lightweight student model is constructed, which takes the node features of the target node and the relation structure summary as input, and outputs node classification results and relation structure prediction results. The lightweight student model is jointly trained using the target node labels and the relation structure target through node classification supervision loss and relation structure distillation loss. During the online inference phase, the node features and relational structure summary of the target node to be classified are obtained, input into the trained lightweight student model, and the node classification result is output. The training of the heterogeneous graph teacher model, the generation of relation structure summaries and relation structure objectives, and the training of the lightweight student model are performed offline. The online inference stage is separate from the offline stage and does not depend on the heterogeneous graph teacher model or access to the original graph structure.

2. The relational structure distillation method for classifying nodes in large-scale heterogeneous graphs as described in claim 1, characterized in that, The relation structure summary includes the number of neighbors of the target node under each relation type.

3. The relational structure distillation method for classifying nodes in large-scale heterogeneous graphs as described in claim 2, characterized in that, The neighbor count is the logarithmic count of the number of neighbors under each relation type.

4. The relational structure distillation method for classifying nodes in large-scale heterogeneous graphs as described in claim 2, characterized in that, The neighbor count statistic is the normalized proportion of the number of neighbors under each relation type.

5. The relational structure distillation method for classifying nodes in large-scale heterogeneous graphs as described in claim 1, characterized in that, The difference energy is represented as the norm distance between teacher node representations.

6. The relational structure distillation method for classifying nodes in large-scale heterogeneous graphs as described in claim 1, characterized in that, The statistical analysis by relation type includes: calculating the mean of the representational difference energy of all neighbors within the same relation type.

7. The relational structure distillation method for classifying nodes in large-scale heterogeneous graphs as described in claim 1, characterized in that, The statistical analysis by relation type includes: calculating the maximum value of the representation difference energy of all neighbors within the same relation type.

8. A relational structure distillation system for classifying nodes in large-scale heterogeneous and heterogeneous graphs, characterized in that, The method according to any one of claims 1 to 7 includes: The data processing module is used to acquire heterogeneous graph data, which includes nodes, edges, node attributes, node types, relationship types, and target node labels. The target node construction module is used to construct target node samples by taking the node to be classified as the target node and organizing the neighborhood connection information of the target node according to the relationship type. The teacher structure learning module is used to train a heterogeneous graph teacher model, which encodes the node attributes and relational structures in the heterogeneous graph and outputs teacher node representations. The relation structure knowledge generation module is used to generate a relation structure summary based on the neighborhood connection distribution of the target node under each relation type, and to calculate the representation difference energy between the target node and its neighboring nodes in the same relation type based on the teacher node representation, and to generate the relation structure target by relation type. The student distillation training module is used to construct a lightweight student model. The lightweight student model takes the node features of the target node and the relation structure summary as input, outputs the node classification result and the relation structure prediction result, and uses the target node label and the relation structure target to jointly train the lightweight student model through node classification supervision loss and relation structure distillation loss. The online node classification module is used to obtain the node features and relation structure summary of the target node to be classified during the inference stage, input the trained lightweight student model, and output the node classification result. The teacher structure learning module, the relation structure knowledge generation module, and the student distillation training module are deployed offline, while the online node classification module is deployed online. The relation structure summary generated by the relation structure knowledge generation module is cached for use by the online node classification module.

9. A relational structure distillation apparatus for classifying nodes in large-scale heterogeneous and heterogeneous graphs, characterized in that, include: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images