Data training optimization method and system of graph basic model, medium and equipment

Abstract

The invention discloses a data training optimization method and system for a graph basic model, a medium and equipment, and belongs to the field of graph machine learning, and the method comprises the steps: obtaining each sub-graph in a graph training data set, and carrying out the coding conversion, and obtaining a sub-graph vector; based on the difference measurement between each sub-graph vector and the data distribution center, screening out a plurality of sub-graph vectors with the maximum difference measurement as first sub-graph vectors; inputting the first sub-graph corresponding to each first sub-graph vector into the original graph basic model for training to obtain a graph basic model; wherein after the loss value is calculated in each round of iteration, the first sub-images with the current loss value not larger than the dynamic loss threshold value are screened out to serve as a candidate set, gradient zero setting or re-scaling is conducted on the first sub-images in the candidate set at a preset pruning rate, and then model parameters are updated according to all latest gradients. Therefore, by implementing the method and the device, the problem of low calculation efficiency caused by data redundancy in the pre-training process of the graph basic model in the prior art can be solved.

Description

Technical Field

[0001] This invention belongs to the field of graph machine learning and relates to a data training and optimization method, system, medium, and device for graph-based models. Background Technology

[0002] Graph Foundation Models (GFM) is an emerging technique that enhances cross-task generalization capabilities through pre-training on large-scale graph data. Graph data describes complex relationships with nodes and edges, and its non-Euclidean nature requires models to have a deep understanding of the topological structure. Current GFM pre-training mainly relies on subgraph sampling strategies (such as PPR and K-hop neighbor sampling) to learn general patterns through tasks such as node reconstruction and edge prediction on massive graphs.

[0003] However, using subgraph sampling strategies suffers from drawbacks such as data redundancy and computational inefficiency: high-level nodes are frequently sampled, leading to structural overlap in the sampled subgraphs. This redundant data forces the model to repeatedly learn similar patterns, significantly increasing training overhead. Furthermore, existing techniques lack dynamic optimization mechanisms, failing to adaptively select high-value subgraphs during training, further limiting efficiency improvements. Summary of the Invention

[0004] This application provides a data training optimization method, system, medium, and device for graph-based models, which can solve the problem of low computational efficiency caused by data redundancy during the pre-training process of graph-based models in the prior art.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a data training optimization method for a graph-based model, comprising:

[0006] Retrieve each subgraph from the graph training dataset;

[0007] Each subgraph is encoded and converted to obtain the corresponding subgraph vector;

[0008] Based on the difference measure between each subgraph vector and the distribution center, several subgraph vectors with the largest difference measure are selected as the first subgraph vector according to a preset hard pruning ratio; wherein, the distribution center is the geometric center point of all subgraph vectors in the feature space.

[0009] The first subgraphs corresponding to each first subgraph vector are input into a preset original graph base model for iterative training to obtain the graph base model. After calculating the loss value in each iteration, first subgraphs whose current loss value is not greater than a dynamic loss threshold are selected as a candidate set. The gradients of the first subgraphs in the candidate set are zeroed or rescaled using a preset pruning rate. Then, the parameters of the original graph base model are updated based on the latest gradient of each first subgraph. The dynamic loss threshold is calculated based on the loss value of each first subgraph in the current iteration.

[0010] Compared with existing technologies, the embodiments of this application have the following beneficial effects: By acquiring each subgraph in the graph training dataset, the structured decomposition of the original graph data is realized, providing an operable basic unit for subsequent data optimization; by encoding and converting each subgraph to obtain the corresponding subgraph vector, the feature space of heterogeneous data is unified, providing input features with high discriminative power for difference measurement and screening; based on the difference measurement between each subgraph vector and the distribution center, and by screening the first subgraph vector with the largest difference measurement according to a preset hard pruning ratio, the amount of redundant data is directly reduced, ensuring that high-information samples participate in training first; when the first subgraph is input into the original model for iterative training, the candidate set (subgraphs with loss values ​​not greater than the threshold) is screened by calculating the dynamic threshold of the loss value of the current round, avoiding the waste of training resources on low-value samples and improving resource utilization; by setting the gradient of the candidate set subgraph to zero or rescaling and updating the model parameters with a preset pruning rate, the importance of the training data is dynamically adjusted, the stability of the gradient distribution is maintained, and training divergence is prevented. The overall solution overcomes the inherent limitations of static sampling by removing redundancy based on distribution differences and dynamically filtering low-value samples during training, effectively reducing the computational overhead of pre-training and solving the problem of low computational efficiency caused by data redundancy.

[0011] In some embodiments of the first aspect of this application, the step of encoding and converting each subgraph to obtain the corresponding subgraph vector includes:

[0012] Based on the pre-defined language model, the structural information and attribute features of the edges in each subgraph, as well as the text attributes of the nodes, are semantically encoded to obtain the corresponding feature vectors.

[0013] Based on a pre-defined graph isomorphic network, the feature vectors are structurally encoded to obtain subgraph vectors.

[0014] Compared with existing technologies, the above embodiments have the following beneficial effects: by using a preset language model to semantically encode the edge structure information, attribute features and node text attributes of the subgraph, cross-modal semantic alignment (such as mapping text attributes to vector space) is achieved; further, by using a preset graph isomorphic network to structurally encode the semantically encoded feature vectors, the topological relationship between nodes and edges is aggregated to generate subgraph vectors, thereby improving the integrity of the subgraph representation and providing highly discriminative input features for subsequent removal of redundant samples.

[0015] In some embodiments of the first aspect of this application, the distribution center is the geometric center point of all subgraph vectors in the feature space, including:

[0016] Calculate the arithmetic mean of all subgraph vectors as the initial distribution center;

[0017] The initial distribution center is iterated according to the preset Deep SVDD algorithm to obtain the distribution center.

[0018] Compared with existing technologies, the above embodiments have the following advantages: by calculating the arithmetic mean of all subgraph vectors as the initial distribution center, the core of the feature space can be quickly located; further, the distribution center is iteratively optimized through the Deep SVDD algorithm to enhance the robustness of the difference measure and improve the accuracy of redundant data identification.

[0019] In some embodiments of the first aspect of this application, the step of selecting several subgraph vectors with the largest difference measure as the first subgraph vector based on the difference measure between each subgraph vector and the distribution center, according to a preset hard pruning ratio, includes:

[0020] Calculate the Euclidean distance from each of the subgraph vectors to the distribution center;

[0021] Sort the Euclidean distances from largest to smallest, and select several subgraph vectors corresponding to the top-ranked Euclidean distances as the first subgraph vectors according to a preset hard pruning ratio.

[0022] Compared with existing technologies, the above embodiments have the following beneficial effects: information value is quantified by calculating the Euclidean distance from the subgraph vector to the distribution center; furthermore, by sorting and filtering the subgraphs with the largest distance (e.g., retaining the Top 50%), high-information samples are given priority in training, directly reducing the amount of redundant data.

[0023] In some embodiments of the first aspect of this application, selecting a first sub-graph whose current loss value is not greater than a dynamic loss threshold as a candidate set includes:

[0024] The average loss value of each first subgraph in the current round is used as the dynamic loss threshold for the current round.

[0025] The candidate set is obtained by filtering and aggregating the first subgraphs in the current round whose loss values ​​are less than or equal to the dynamic loss threshold.

[0026] Compared with existing technologies, the above embodiments have the following beneficial effects: by setting the average loss value of each subgraph in the current round as a dynamic loss threshold, the sample value is adaptively distinguished; further, subgraphs with loss values ​​not greater than the threshold are selected as candidate sets, avoiding high-value samples being mistakenly filtered out and improving the utilization rate of training resources.

[0027] In some embodiments of the first aspect of this application, the step of gradient-zeroing or rescaling the first subgraph in the candidate set with a preset pruning rate includes:

[0028] Randomly select a portion of the first subgraphs in the candidate set according to a preset pruning rate, set their corresponding gradients to zero, and rescale the gradients of the remaining first subgraphs in the candidate set; wherein the rescaling algorithm is as follows:

[0029] in, G represents the i-th first subgraph. i The gradient, where β represents the pruning rate.

[0030] Compared with the prior art, the above embodiments have the following beneficial effects: by randomly selecting some subgraphs in the candidate set with a preset pruning rate and setting their gradients to zero, the invalid influence of low-value samples on training is reduced; the gradients of the remaining subgraphs in the candidate set are rescaled using the pruning rate, keeping the expected value of the overall gradient unchanged, preventing the optimization direction shift caused by setting some gradients to zero, maintaining the gradient distribution balance in the training process, and avoiding abnormal model convergence caused by a sudden drop in gradient.

[0031] Secondly, the present invention also provides a data training and optimization system for a graph-based model, comprising: a data acquisition module, a transformation module, a filtering module, and a training module;

[0032] The data acquisition module is used to acquire each subgraph in the graph training dataset;

[0033] The conversion module is used to encode and convert each subgraph to obtain the corresponding subgraph vector;

[0034] The filtering module is used to select several subgraph vectors with the largest difference measure as the first subgraph vector based on the difference measure between each subgraph vector and the distribution center, according to a preset hard pruning ratio; wherein, the distribution center is the geometric center point of all subgraph vectors in the feature space.

[0035] The training module is used to input the first subgraphs corresponding to each first subgraph vector into a preset original graph base model for iterative training to obtain a graph base model. Specifically, after calculating the loss value in each iteration, first subgraphs whose current loss value is not greater than a dynamic loss threshold are selected as a candidate set. The gradients of the first subgraphs in the candidate set are zeroed or rescaled using a preset pruning rate. Then, the parameters of the original graph base model are updated based on the latest gradient of each first subgraph. The dynamic loss threshold is calculated based on the loss value of each first subgraph in the current iteration.

[0036] Compared with the prior art, the above embodiments of this application have the following beneficial effects: By acquiring each subgraph in the graph training dataset, the structured decomposition of the original graph data is realized, providing an operable basic unit for subsequent data optimization; by encoding and converting each subgraph to obtain the corresponding subgraph vector, the feature space of heterogeneous data is unified, providing input features with high discriminative power for difference measurement and screening; based on the difference measurement between each subgraph vector and the distribution center, and by screening the first subgraph vector with the largest difference measurement according to the preset hard pruning ratio, the amount of redundant data is directly reduced, ensuring that high-information samples participate in training first; when the first subgraph is input into the original model for iterative training, the candidate set (subgraphs with loss values ​​not greater than the threshold) is screened by calculating the dynamic threshold of the loss value of the current round, avoiding the waste of training resources on low-value samples and improving resource utilization; by setting the gradient of the candidate set subgraph to zero or rescaling and updating the model parameters with the preset pruning rate, the importance of the training data is dynamically adjusted, the stability of the gradient distribution is maintained, and training divergence is prevented. The overall solution overcomes the inherent limitations of static sampling by removing redundancy based on distribution differences and dynamically filtering low-value samples during training, effectively reducing the computational overhead of pre-training and solving the problem of low computational efficiency caused by data redundancy.

[0037] In some embodiments of the second aspect of this application, the conversion module includes: a semantic coding unit and a structural coding unit;

[0038] The semantic encoding unit is used to perform semantic encoding on the structural information and attribute features of the edges in each subgraph, as well as the text attributes of the nodes, according to a preset language model, to obtain the corresponding feature vectors.

[0039] The structure encoding unit is used to perform structure encoding on each feature vector according to a preset graph isomorphic network to obtain each subgraph vector.

[0040] Compared with existing technologies, the above embodiments have the following beneficial effects: by using a preset language model to semantically encode the edge structure information, attribute features and node text attributes of the subgraph, cross-modal semantic alignment (such as mapping text attributes to vector space) is achieved; further, by using a preset graph isomorphic network to structurally encode the semantically encoded feature vectors, the topological relationship between nodes and edges is aggregated to generate subgraph vectors, thereby improving the integrity of the subgraph representation and providing highly discriminative input features for subsequent removal of redundant samples.

[0041] Thirdly, the present invention also provides a data training and optimization device for a graph-based model, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of any of the data training and optimization methods for graph-based models of the present invention.

[0042] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data training and optimization method for any graph-based model of the present invention. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a data training and optimization method for a graph-based model provided in some embodiments of the present invention.

[0044] Figure 2 : This is a schematic diagram of the structure of a data training and optimization system for a graph-based model provided in some embodiments of the present invention.

[0045] Figure 3 : This is a structural diagram of a data training and optimization device for a graph-based model provided in some embodiments of the present invention.

[0046] Figure 4 This is a schematic diagram of a graph-based model pre-training strategy based on subgraphs provided in some embodiments of the present invention.

[0047] Figure 5 This is an overall framework diagram of a data training and optimization method for a graph-based model provided in some embodiments of the present invention.

[0048] Figure 6 This is a comparison chart of experimental performance based on the graph-based model OFA provided in some embodiments of the present invention.

[0049] Figure 7 This is a performance comparison chart of an experimental model based on Graph CLIP provided in some embodiments of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Example 1:

[0052] In existing technologies, such as Figure 4 The diagram illustrates a subgraph-based graph-based model pre-training strategy. Existing GFM techniques primarily employ two key strategies: first, aligning heterogeneous node features to the text space by introducing a large language model to achieve cross-domain semantic transfer; and second, using a unified subgraph-based pre-training strategy to reconstruct tasks at different levels (node-level, edge-level, and graph-level) into subgraph pre-training and inference tasks. The pre-training data construction process mainly relies on subgraph sampling techniques.

[0053] Next, in the pre-training phase, GFM learns from these sampled subgraphs to capture general patterns and features of graph structures. Pre-training tasks typically include node feature reconstruction, edge prediction, and subgraph structure prediction. This pre-training strategy enables the model to learn structural invariance in a unified representation space. After pre-training, GFM can be directly deployed to various target graphs to perform different downstream tasks, such as node classification, link prediction, and graph classification, without additional fine-tuning. However, in this process, the pre-training data often suffers from significant data redundancy and noise during construction, and the lack of effective pre-training data optimization mechanisms during training leads to low computational efficiency.

[0054] Please refer to Figure 1 To address the problem of low computational efficiency caused by data redundancy during the pre-training process of graph-based models in existing technologies, an embodiment of the present invention provides a data training optimization method for graph-based models, comprising steps S1 to S4:

[0055] Step S1: Obtain each subgraph in the graph training dataset.

[0056] Step S2: Encode and convert each subgraph to obtain the corresponding subgraph vector.

[0057] Furthermore, step S2 can be implemented through the following preferred embodiments, including steps S21-S22, as detailed below:

[0058] S21: Based on the preset language model, semantically encode the structural information and attribute features of the edges in each subgraph, as well as the text attributes of the nodes, to obtain the corresponding feature vectors.

[0059] S22: Based on the preset graph isomorphic network, structural encoding is performed on each feature vector to obtain each subgraph vector.

[0060] In this preferred embodiment, the edge structure information, attribute features and node text attributes of the subgraph are semantically encoded by a preset language model to achieve cross-modal semantic alignment (such as mapping text attributes to vector space); further, the semantically encoded feature vectors are structurally encoded by a preset graph isomorphic network, and the topological relationship between nodes and edges is aggregated to generate subgraph vectors, thereby improving the integrity of the subgraph representation and providing highly discriminative input features for subsequent removal of redundant samples.

[0061] Step S3: Based on the difference measure between each subgraph vector and the distribution center, select several subgraph vectors with the largest difference measure as the first subgraph vector according to the preset hard pruning ratio; wherein, the distribution center is the geometric center point of all subgraph vectors in the feature space.

[0062] Preferably, the distribution center can be obtained through the following preferred implementation method, including steps S31-S32, as follows:

[0063] S31: Calculate the arithmetic mean of all subgraph vectors as the initial distribution center.

[0064] S32: According to the preset Deep SVDD algorithm, the initial distribution center is iterated to obtain the distribution center.

[0065] In this preferred embodiment, the core of the feature space is quickly located by calculating the arithmetic mean of all subgraph vectors as the initial distribution center; further, the distribution center is iteratively optimized by the Deep SVDD algorithm to enhance the robustness of the difference measure and improve the accuracy of redundant data identification.

[0066] Furthermore, when obtaining the first sub-graph vector in step S3, it can be implemented through the following preferred embodiments, including steps S33-S34, as follows:

[0067] S33: Calculate the Euclidean distance from each of the subgraph vectors to the distribution center;

[0068] S34: Sort the Euclidean distances from largest to smallest, and select several subgraph vectors corresponding to the top-ranked Euclidean distances as the first subgraph vectors according to a preset hard pruning ratio.

[0069] In this preferred embodiment, the information value is quantified by calculating the Euclidean distance from the subgraph vector to the distribution center; furthermore, by sorting and filtering the subgraphs with the largest distance (e.g., retaining the Top 50%), high-information samples are given priority in training, directly reducing the amount of redundant data.

[0070] In specific implementation, such as Figure 5 The diagram shows the overall framework of a data training and optimization method for a graph-based model. In this scheme, the process of selecting subgraphs in step S3 is called hard pruning, as detailed below:

[0071] After obtaining the subgraph dataset, for each node in the subgraph, a pre-trained language model such as Sentence BERT can be used to encode its textual attributes, mapping node features from different sources to a unified semantic space. Similarly, edges with text or attributes are processed in the same way, using Sentence BERT to encode edge attributes. Then, a Graph Isomorphic Network (GIN) is used as the structural encoder, taking the node and edge features obtained from the previous semantic encoding as initial features. Through multi-layer message passing and feature aggregation in the GIN, a subgraph-level representation (i.e., the subgraph vector in step S22) is generated.

[0072] Next, the initial distribution center is obtained by averaging the representations of all subgraphs. Then, the distance from each subgraph vector to the distribution center is calculated as an information score (i.e., the aforementioned difference metric). Subgraph vectors with higher scores (i.e., the first subgraph vector) are retained according to a preset hard pruning ratio. After obtaining the initial distribution center, the Deep SVDD algorithm can be used for iterative optimization, enhancing the robustness of subsequent information score calculations. This method of quantifying information value using distance effectively identifies and retains subgraphs with unique information, facilitating the removal of redundant samples.

[0073] exist Figure 5 In the diagram, the information score s of the i-th subgraph vector is... i The Euclidean distance can be used for calculation, and the formula is as follows:

[0074] s i =||GIN(g i ;W)-c|| 2 , where g i Let W represent the i-th subgraph, and W be the parameters of the GIN network. GIN(g i W) represents the representation of the subgraph, i.e., the subgraph vector, and c is the distribution center.

[0075] In this process, subgraphs farther from the distribution center receive higher information scores. In practical applications, the choice of the hard pruning ratio α has a significant impact on performance: performance is optimal when α < 0.5, indicating that the hard pruning ratio should be kept cautious to avoid removing too much information. For different types of graph data, the following parameter selections are recommended: for citation networks (such as OGBN-ARXIV) and social networks, an α value of 0.3–0.4 is recommended; for knowledge graphs (such as FB15K237), an α value of 0.3–0.5 is recommended; for molecular graphs (such as Chembl), an α value of 0.4–0.5 is recommended.

[0076] Step S4: Input the first subgraph corresponding to each first subgraph vector into the preset original graph base model for iterative training to obtain the graph base model; wherein, after calculating the loss value in each iteration, the first subgraphs whose current loss value is not greater than the dynamic loss threshold are selected as candidate sets, and the gradients of the first subgraphs in the candidate set are zeroed or rescaled with a preset pruning rate, and then the parameters of the original graph base model are updated according to the latest gradient of each first subgraph; the dynamic loss threshold is calculated based on the loss value of each first subgraph in the current round.

[0077] In specific implementation, such as Figure 5 The diagram shows the overall framework of a data training optimization method for a graph-based model. In this scheme, step S4, the process of filtering and resetting gradients, is called soft pruning, and the method is as follows:

[0078] Furthermore, in step S4, when filtering the candidate set, the following preferred implementation method can be used, including steps S41-S42, as follows:

[0079] S41: The average loss value of each first subgraph in the current round is used as the dynamic loss threshold for the current round;

[0080] S42: Filter and aggregate the first subgraphs in the current round whose loss value is less than or equal to the dynamic loss threshold to obtain the candidate set.

[0081] In this preferred embodiment, the average loss value of each subgraph in the current round is set as a dynamic loss threshold to adaptively distinguish the value of samples; further, subgraphs with loss values ​​not greater than the threshold are selected as candidate sets to avoid high-value samples being mistakenly filtered out and to improve the utilization rate of training resources.

[0082] Furthermore, when zeroing out or rescaling the gradient, this can be achieved through the following preferred implementation methods, including:

[0083] Randomly select a portion of the first subgraphs in the candidate set according to a preset pruning rate, set their corresponding gradients to zero, and rescale the gradients of the remaining first subgraphs in the candidate set; wherein the rescaling algorithm is as follows:

[0084] in, G represents the i-th first subgraph. i The gradient, where β represents the pruning rate.

[0085] In this preferred embodiment, by randomly selecting some subgraphs in the candidate set with a preset pruning rate and setting their gradients to zero, the ineffective impact of low-value samples on training is reduced; the gradients of the remaining subgraphs in the candidate set are rescaled using the pruning rate, keeping the expected value of the overall gradient unchanged, preventing the optimization direction shift caused by setting some gradients to zero, maintaining the gradient distribution balance in the training process, and avoiding abnormal model convergence caused by a sudden drop in gradient.

[0086] In practical implementation, the choice of the pruning rate β in soft pruning also has a significant impact on the model training effect. Based on experimental considerations, the system performs well overall when β > 0.5, allowing for a more aggressive pruning strategy during dynamic training. However, when implementing gradient rescaling of 1 / (1-β), numerical stability issues need to be considered. When the β value is large (e.g., β > 0.7), the rescaling factor becomes very large, potentially leading to training instability. Of course, depending on the actual situation, the pruning rate β can be adjusted in each iteration based on the current training status, within the range [0,1]. When the pruning rate is set to 1, it means that the gradients of the first subgraph in all candidate sets are set to zero without rescaling. If it is 0, it means that no gradient adjustment is performed in this round. Generally, it is recommended to use gradient pruning techniques to limit the norm of the rescaled gradient to a reasonable range (e.g., 5.0).

[0087] In one specific implementation, the model is based on the PyTorch framework and experimentally verified in an environment equipped with an NVIDIA H100 GPU. For the data-centric graph model DCGFM (i.e., the model using the method described in this application), Sentence BERT is used as the semantic encoder with a hidden layer dimension of 768; a 5-layer GIN network is used as the structural encoder with a hidden layer dimension of 128. For the training parameters of DCGFM, the number of GNN layers is set to 5, the batch size to 256, the number of training epochs to 30, the learning rate to 5e-4, and a 2-hop neighbor sampling strategy is adopted. During training, the hard pruning ratio is set to 0.5, the soft pruning rate is set to 0.3, and the parameter λ is set to 5e-4.

[0088] To comprehensively evaluate the effectiveness of this invention, two representative graph-based models, OFA and Graph CLIP, were selected as experimental benchmarks. For datasets, ogbn-arxiv, FB15K237, and Chembl were used as pre-training datasets, and tests were conducted on multiple downstream tasks, including node classification (Pub Med), link prediction (WN18RR), and graph classification (HIV, PCBA). Several baseline methods were also introduced for comparison, including hard pruning methods (random selection, K-Center, Uncertainty, and DP) and soft pruning methods (random selection, Greedy, UCB, and Info Batch).

[0089] like Figure 6 The figure shown is a comparison of experimental performance based on the graph-based model OFA. Figure 7 The figure shows an experimental performance comparison of the graph-based model Graph CLIP. The experimental results show that, with an overall pruning rate of 70%, the method of the present invention achieves a performance improvement of 1.1% and 0.2% on OFA and Graph CLIP, respectively, compared with the original model. At the same time, it reduces the training time from 240 minutes and 386 minutes to 115 minutes and 195 minutes, respectively, which fully demonstrates the significant effect of the present invention in improving the training efficiency of graph-based models.

[0090] In summary, compared with the prior art, the above embodiments of this application have the following beneficial effects: By acquiring each subgraph in the graph training dataset, the structured decomposition of the original graph data is realized, providing an operable basic unit for subsequent data optimization; by encoding and converting each subgraph to obtain the corresponding subgraph vector, the feature space of heterogeneous data is unified, providing input features with high discriminative power for difference measurement and screening; based on the difference measurement between each subgraph vector and the distribution center, and by screening the first subgraph vector with the largest difference measurement according to the preset hard pruning ratio, the amount of redundant data is directly reduced, ensuring that high-information samples participate in training first; when the first subgraph is input into the original model for iterative training, the candidate set (subgraphs with loss values ​​not greater than the threshold) is screened by calculating the dynamic threshold of the loss value of the current round, avoiding the waste of training resources on low-value samples and improving resource utilization; by setting the gradient of the candidate set subgraph to zero or rescaling and updating the model parameters with the preset pruning rate, the importance of the training data is dynamically adjusted, the stability of the gradient distribution is maintained, and training divergence is prevented. The overall solution overcomes the inherent limitations of static sampling by removing redundancy based on distribution differences and dynamically filtering low-value samples during training, effectively reducing the computational overhead of pre-training and solving the problem of low computational efficiency caused by data redundancy.

[0091] Example 2:

[0092] Please refer to Figure 2Based on the same inventive concept, the present invention discloses a data training and optimization system for a graph-based model, comprising: a data acquisition module M1, a transformation module M2, a filtering module M3, and a training module M4;

[0093] The data acquisition module M1 is used to acquire each subgraph in the graph training dataset.

[0094] The conversion module M2 is used to encode and convert each subgraph to obtain the corresponding subgraph vector.

[0095] Furthermore, the conversion module M2 includes: a semantic encoding unit and a structural encoding unit;

[0096] The semantic encoding unit is used to perform semantic encoding on the structural information and attribute features of the edges in each subgraph, as well as the text attributes of the nodes, according to a preset language model, to obtain the corresponding feature vectors.

[0097] The structure encoding unit is used to perform structure encoding on each feature vector according to a preset graph isomorphic network to obtain each subgraph vector.

[0098] In this preferred embodiment, the edge structure information, attribute features and node text attributes of the subgraph are semantically encoded by a preset language model to achieve cross-modal semantic alignment (such as mapping text attributes to vector space); further, the semantically encoded feature vectors are structurally encoded by a preset graph isomorphic network, and the topological relationship between nodes and edges is aggregated to generate subgraph vectors, thereby improving the integrity of the subgraph representation and providing highly discriminative input features for subsequent removal of redundant samples.

[0099] The filtering module M3 is used to select several subgraph vectors with the largest difference measure as the first subgraph vector based on the difference measure between each subgraph vector and the distribution center, according to a preset hard pruning ratio; wherein, the distribution center is the geometric center point of all subgraph vectors in the feature space.

[0100] Furthermore, the filtering module M3 includes: a distribution center initialization unit and a distribution center iteration unit;

[0101] The distribution center initialization unit is used to calculate the arithmetic mean of all subgraph vectors as the initial distribution center.

[0102] The distribution center iteration unit is used to iterate the initial distribution center according to a preset Deep SVDD algorithm to obtain the distribution center.

[0103] In this preferred embodiment, the core of the feature space is quickly located by calculating the arithmetic mean of all subgraph vectors as the initial distribution center; further, the distribution center is iteratively optimized by the Deep SVDD algorithm to enhance the robustness of the difference measure and improve the accuracy of redundant data identification.

[0104] Furthermore, the filtering module M3 also includes: a distance calculation unit and a first filtering unit;

[0105] The distance calculation unit is used to calculate the Euclidean distance from each of the subgraph vectors to the distribution center.

[0106] The first filtering unit is used to sort the Euclidean distances from largest to smallest, and according to a preset hard pruning ratio, filter out a number of subgraph vectors corresponding to the Euclidean distances that are ranked first as the first subgraph vector.

[0107] In this preferred embodiment, the information value is quantified by calculating the Euclidean distance from the subgraph vector to the distribution center; furthermore, by sorting and filtering the subgraphs with the largest distance (e.g., retaining the Top 50%), high-information samples are given priority in training, directly reducing the amount of redundant data.

[0108] The training module M4 is used to input the first subgraphs corresponding to each first subgraph vector into a preset original graph base model for iterative training to obtain a graph base model. After calculating the loss value in each iteration, the first subgraphs whose current loss value is not greater than a dynamic loss threshold are selected as a candidate set. The gradients of the first subgraphs in the candidate set are zeroed or rescaled using a preset pruning rate. Then, the parameters of the original graph base model are updated based on the latest gradient of each first subgraph. The dynamic loss threshold is calculated based on the loss value of each first subgraph in the current iteration.

[0109] Furthermore, the training module M4 includes: a threshold calculation unit and a second filtering unit;

[0110] The threshold calculation unit is used to use the average value of the loss values ​​of each first subgraph in the current round as the dynamic loss threshold for the current round.

[0111] The second filtering unit is used to filter and aggregate the first subgraphs in the current round whose loss values ​​are less than or equal to the dynamic loss threshold, to obtain the candidate set.

[0112] In this preferred embodiment, the average loss value of each subgraph in the current round is set as a dynamic loss threshold to adaptively distinguish the value of samples; further, subgraphs with loss values ​​not greater than the threshold are selected as candidate sets to avoid high-value samples being mistakenly filtered out and to improve the utilization rate of training resources.

[0113] Furthermore, the training module M4 also includes: a pruning unit;

[0114] The pruning unit is used to randomly select a portion of the first subgraphs in the candidate set at a preset pruning rate, set their corresponding gradients to zero, and rescale the gradients of the remaining first subgraphs in the candidate set; wherein the rescaling algorithm is as follows:

[0115] in, G represents the i-th first subgraph. i The gradient, where β represents the pruning rate.

[0116] In this preferred embodiment, by randomly selecting some subgraphs in the candidate set with a preset pruning rate and setting their gradients to zero, the ineffective impact of low-value samples on training is reduced; the gradients of the remaining subgraphs in the candidate set are rescaled using the pruning rate, keeping the expected value of the overall gradient unchanged, preventing the optimization direction shift caused by setting some gradients to zero, maintaining the gradient distribution balance in the training process, and avoiding abnormal model convergence caused by a sudden drop in gradient.

[0117] In summary, compared with the prior art, the embodiments of this application have the following beneficial effects: By acquiring each subgraph in the graph training dataset, the structured decomposition of the original graph data is realized, providing an operable basic unit for subsequent data optimization; by encoding and converting each subgraph to obtain the corresponding subgraph vector, the feature space of heterogeneous data is unified, providing input features with high discriminative power for difference measurement and screening; based on the difference measurement between each subgraph vector and the distribution center, and by screening the first subgraph vector with the largest difference measurement according to a preset hard pruning ratio, the amount of redundant data is directly reduced, ensuring that high-information samples participate in training first; when the first subgraph is input into the original model for iterative training, the candidate set (subgraphs with loss values ​​not greater than the threshold) is screened by calculating the dynamic threshold of the loss value of the current round, avoiding the waste of training resources on low-value samples and improving resource utilization; by setting the gradient of the candidate set subgraph to zero or rescaling and updating the model parameters with a preset pruning rate, the importance of the training data is dynamically adjusted, the stability of the gradient distribution is maintained, and training divergence is prevented. The overall solution overcomes the inherent limitations of static sampling by removing redundancy based on distribution differences and dynamically filtering low-value samples during training, effectively reducing the computational overhead of pre-training and solving the problem of low computational efficiency caused by data redundancy.

[0118] Example 3:

[0119] Figure 3 A structural diagram of a data training and optimization device for a graph-based model according to this application is presented. For example... Figure 3 As shown, the data training and optimization equipment for the basic model in this figure may include: processor N1, memory N2, data interface N3, and communication bus N4.

[0120] Wherein: processor N1, memory N2, and data interface N3 communicate with each other through communication bus N4; data interface N3 is used for data communication with other devices such as input devices or output devices; processor N1 is used to execute program N5, specifically it can execute the relevant steps in any of the above-mentioned data training and optimization method embodiments of graph-based models.

[0121] Specifically, program N5 may include program code, which includes computer-executable instructions.

[0122] The processor N1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The data training optimization device of the basic model includes one or more processors, which may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0123] Memory N2 is used to store program N5. Memory N2 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0124] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments in this application are not directed to any particular programming language.

[0125] Example 4:

[0126] This invention also provides a computer-readable storage medium storing at least one executable instruction that, when executed on a graph-based model data training and optimization device / system, causes the graph-based model data training and optimization device / system to perform a graph-based model data training and optimization method from any of the above method embodiments.

[0127] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. Similarly, for the purpose of simplification and aiding understanding of one or more aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0128] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

Claims

1. A data training and optimization method for a graph-based model, characterized in that, include: Retrieve each subgraph from the graph training dataset; Each subgraph is encoded and converted to obtain the corresponding subgraph vector; Based on the difference measure between each subgraph vector and the distribution center, several subgraph vectors with the largest difference measure are selected as the first subgraph vector according to a preset hard pruning ratio; wherein, the distribution center is the geometric center point of all subgraph vectors in the feature space. The first subgraphs corresponding to each first subgraph vector are input into a preset original graph base model for iterative training to obtain the graph base model. After calculating the loss value in each iteration, first subgraphs whose current loss value is not greater than a dynamic loss threshold are selected as a candidate set. The gradients of the first subgraphs in the candidate set are zeroed or rescaled using a preset pruning rate. Then, the parameters of the original graph base model are updated based on the latest gradient of each first subgraph. The dynamic loss threshold is calculated based on the loss value of each first subgraph in the current iteration.

2. The data training and optimization method for a graph-based model as described in claim 1, characterized in that, The process of encoding and converting each subgraph to obtain the corresponding subgraph vector includes: Based on the pre-defined language model, the structural information and attribute features of the edges in each subgraph, as well as the text attributes of the nodes, are semantically encoded to obtain the corresponding feature vectors. Based on a pre-defined graph isomorphic network, the feature vectors are structurally encoded to obtain subgraph vectors.

3. The data training and optimization method for a graph-based model as described in claim 2, characterized in that, The distribution center is the geometric center point of all subgraph vectors in the feature space, including: Calculate the arithmetic mean of all subgraph vectors as the initial distribution center; The initial distribution center is iterated according to the preset Deep SVDD algorithm to obtain the distribution center.

4. The data training and optimization method for a graph-based model as described in claim 3, characterized in that, The step of selecting several subgraph vectors with the largest difference metrics as the first subgraph vector based on the difference metric between each subgraph vector and the distribution center, according to a preset hard pruning ratio, includes: Calculate the Euclidean distance from each of the subgraph vectors to the distribution center; Sort the Euclidean distances from largest to smallest, and select several subgraph vectors corresponding to the top-ranked Euclidean distances as the first subgraph vectors according to a preset hard pruning ratio.

5. The data training and optimization method for a graph-based model as described in claim 1, characterized in that, The first sub-graph selected as the candidate set, whose current loss value is not greater than the dynamic loss threshold, includes: The average loss value of each first subgraph in the current round is used as the dynamic loss threshold for the current round. The candidate set is obtained by filtering and aggregating the first subgraphs in the current round whose loss values ​​are less than or equal to the dynamic loss threshold.

6. The data training and optimization method for a graph-based model as described in claim 5, characterized in that, The step of grading or rescaling the first subgraph in the candidate set with a preset pruning rate includes: A subset of first subgraphs in the candidate set are randomly selected at a preset pruning rate, their corresponding gradients are set to zero, and the gradients of the remaining first subgraphs in the candidate set are rescaled; wherein the rescaled algorithm is as follows: in, G represents the i-th first subgraph. i The gradient, where β represents the pruning rate.

7. A data training and optimization system for a graph-based model, characterized in that, include: Data acquisition module, transformation module, filtering module, and training module; The data acquisition module is used to acquire each subgraph in the graph training dataset; The conversion module is used to encode and convert each subgraph to obtain the corresponding subgraph vector; The filtering module is used to select several subgraph vectors with the largest difference measure as the first subgraph vector based on the difference measure between each subgraph vector and the distribution center, according to a preset hard pruning ratio; wherein, the distribution center is the geometric center point of all subgraph vectors in the feature space. The training module is used to input the first subgraphs corresponding to each first subgraph vector into a preset original graph base model for iterative training to obtain a graph base model. Specifically, after calculating the loss value in each iteration, first subgraphs whose current loss value is not greater than a dynamic loss threshold are selected as a candidate set. The gradients of the first subgraphs in the candidate set are zeroed or rescaled using a preset pruning rate. Then, the parameters of the original graph base model are updated based on the latest gradient of each first subgraph. The dynamic loss threshold is calculated based on the loss value of each first subgraph in the current iteration.

8. The data training and optimization system for a graph-based model as described in claim 7, characterized in that, The conversion module includes: a semantic encoding unit and a structural encoding unit; The semantic encoding unit is used to perform semantic encoding on the structural information and attribute features of the edges in each subgraph, as well as the text attributes of the nodes, according to a preset language model, to obtain the corresponding feature vectors. The structure encoding unit is used to perform structure encoding on each feature vector according to a preset graph isomorphic network to obtain each subgraph vector.

9. A data training and optimization device for a graph-based model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the steps of a data training optimization method for a graph-based model according to any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a data training and optimization method for a graph-based model according to any one of claims 1-6.

Images