Image incremental learning method based on de-noising efficient image compression

Abstract

The invention discloses a graph incremental learning method based on de-noising efficient graph compression, and the method comprises the steps: keeping the overall semantics of an original graph of each task, and compressing the original graph into a small graph as a memory for memory playback through a distribution matching method. And related entropy loss is added in the image compression process, so that the noise influence in original image data is effectively avoided. The method further provides a balance training target, and the problem of data imbalance between the new task graph and the replay graph is effectively solved. Compared with the existing image incremental learning method, the image incremental learning based on denoising efficient image compression can improve the classification accuracy and save the playback memory space.

Description

Technical Field

[0001] The present invention belongs to the field of pattern recognition and machine learning, and specifically relates to graph network learning and incremental learning technology, in particular to a graph incremental learning method based on denoising and efficient graph compression. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, graph neural networks, as a powerful modeling tool, have played an important role in social network analysis, bioinformatics, knowledge graph construction, and traffic planning. For example, in social networks, GNNs can efficiently capture the relationships between users and optimize the recommendation system; in bioinformatics, GNNs are used to predict protein interactions; in knowledge graphs, GNNs can mine potential relationships and perform knowledge reasoning. However, the success of traditional graph neural networks is due to their excellent modeling capabilities for static graph data, but their performance in dynamic graph environments is limited. Therefore, there is an urgent need for an effective technology to adapt to the ever-changing new data and knowledge.

[0003] As an important machine learning method, incremental learning can adapt the model to new data with low computational and storage overhead while retaining historical task knowledge, making incremental learning an important research direction in machine learning. Therefore, graph incremental learning has attracted more and more attention.

[0004] With the continuous development of the Internet and information technology, massive graph data has been widely used in various practical scenarios. In social networks, the interactions between users constitute dynamic graphs, which can be used to recommend friends and identify fake accounts; in bioinformatics, complex molecular structures and gene relationships can also be modeled as graphs for drug development and disease diagnosis; in urban transportation, road networks and their dynamically changing attribute data are crucial for path planning and traffic prediction. Many graph incremental learning methods have emerged to cope with the characteristics of graph data evolution. Recent studies have proposed three different methods: replay-based methods, regularization-based methods, and network architecture modification-based methods. Replay-based methods solve the forgetting problem by retaining representative data selected from previous tasks and training the model together with new data. Regularization-based methods prevent the model from over-adjusting previously learned knowledge when learning new tasks by adding additional constraints to the loss function. Network architecture modification-based methods face the forgetting problem by modifying the specific architecture, activation function, or algorithm layer of the network.

[0005] Although replay-based methods are considered to be the most intuitive and effective graph incremental learning strategies at present, these methods still face many challenges in practical applications. First, when the replay space is limited, it is difficult for these replay data methods to present the distribution of historical data. This is because current replay methods often only focus on using part of the graph data to build memory, that is, only sampling and storing a single node or part of the subgraph for replay. Second, due to the limitations of the replay memory, the new task graph data usually dominates the training data, which leads to data imbalance between the replay data and the new task graph data. When using these unbalanced data to update the graph neural network model, the model may be biased towards the data of the current task, resulting in forgetting the knowledge of the previously learned tasks. Summary of the invention

[0006] In view of the problems existing in the existing methods, the present invention proposes a new replay-based incremental graph learning method. Graph compression can effectively compress training data from a large graph to a small graph while maintaining model performance. The compressed graph of the present invention maintains distribution characteristics similar to the original large graph by learning a small graph composed of synthetic nodes. In the process of generating the compressed graph, the present invention also introduces the relevant entropy regularization loss to reduce the influence of noise in the process of compressing the graph. The compressed graph can effectively capture the overall structure and attribute characteristics of the original large graph. In addition, the compressed graph is composed of new nodes, which not only retains the characteristic distribution of the original data, but also improves the privacy protection ability of the graph data to a certain extent. In order to solve the problem of imbalance in the scale of new task data and replay data during training, when new task data arrives, we compress the new task data and replay data to the same scale for training. The new graph incremental learning training method proposed by the present invention can effectively solve the problem of data imbalance and enhance the generalization of the model.

[0007] The technical solution steps of the present invention are as follows:

[0008] Step 1: In the training phase, the graph of the new task is first compressed into a small graph through denoising and efficient graph compression technology. The influence of noise points and abnormal points in the graph dataset is alleviated through the related entropy popular loss. The distribution of the compressed graph is made close to the distribution of the new task graph through complementary loss. During this period, a small-scale graph that can retain the original graph information is obtained through the constraints of the two losses. The compressed graph is then put into the memory library, and all tasks in the memory library are compressed. Figure 1 Train together to get the final model.

[0009] Step 2: Testing phase: For the test nodes, the final classification results are obtained through the network.

[0010] Described step 1 comprises the following steps:

[0011] Step 1.1: Incremental segmentation of the dataset:

[0012] Step 1.2: Perform denoising and efficient image compression on the new task to obtain a compressed image:

[0013] Step 1.3: Put the compressed graph of the new task into the memory bank:

[0014] Step 1.4: Use the compressed image of the memory bank to balance the model training:

[0015] Described step 2 comprises the following steps:

[0016] Step 2.1: Input the test data set for testing DETAILED DESCRIPTION

[0017] Incremental segmentation of data sets: To be closer to the dispersed data streams in incremental learning scenarios, the present invention randomly divides the graph data set into different sub-data sets by category, and each sub-data set represents a subtask at a different time. For the CoraFull data set, it has graph data of 70 category nodes, and then the data set is divided according to the fact that each task contains two categories. The present invention uses a total of three data sets for experiments, namely the CoraFull, OGB-Arxiv, Reddit, and OGB-Products data sets. For the four data sets, each task contains two classes, and the categories between different tasks do not overlap.

[0018] Denoising and efficient graph compression method: Among the three major graph incremental learning methods, the replay-based method has better anti-forgetting ability because it retains the data of old tasks. When learning new tasks, the new tasks are trained and learned together with the old tasks, which effectively improves the plasticity and stability of the model. Although this method solves the forgetting problem very well, in practical applications, the sampling-based replay method requires a large replay memory to remember old knowledge, and it also causes the problem of imbalance in the scale of new task data and replay data. Compared with the sampling replay method, the compressed replay method can significantly improve the model performance.

[0019] Dataset compression technology has achieved remarkable results in the field of computer vision. By compressing the size of image datasets, computer vision models can maintain high recognition accuracy and robustness while reducing computing resource consumption. Compress and combine into a small image To reduce training costs and achieve

[0020] The same test accuracy of the model trained on the dataset, the compression target is as follows:

[0021]

[0022] in is the task-related loss function and θ is the parameter of GNN.

[0023] The present invention proposes a new graph compression method, called denoising efficient graph compression. The method of the present invention synthesizes synthetic data that is close to the distribution of the actual training data based on embedded representation. However, due to the high dimensionality of the embedded representation, it is difficult to accurately estimate the distribution of the set data set. Therefore, this paper uses a series of randomly initialized neural networks to obtain different embedded representations to represent the training data set. Each individual network provides a partial explanation of the data set, and the randomly initialized network set produces a more comprehensive representation. In practical applications, there are often certain noises and outliers in the graph data. For this reason, the present invention adds the correlation entropy regularization loss to suppress the influence of noise.

[0024] For task T k , the input image is Generate the desired synthetic image through image compression The initial synthetic graph can be obtained by random noise, random sampling or clustering algorithm. Under the distribution matching scheme, the objective function of graph compression can be expressed as:

[0025]

[0026] The Dist(·,·) function is used to calculate the distance between the original image and the compressed image. The two images are encoded by the same graph encoder GNN. θ Encoding, the distance is measured in the embedding space. Specifically, the optimal compression graph It should be as close as possible to the original image in distribution.

[0027]

[0028] The noise and outliers in the real graph data will affect the estimated distribution of the real data. For this reason, this paper introduces the relevant entropy loss when generating the compressed graph, and uses the current data set to construct a predefined supervised neighbor graph G. s , where the element in row i and column j is defined as:

[0029]

[0030] in Represents sample x j The set of second-order neighbors of the same type, G s The construction of not only considers local relations, but also utilizes label information. Given a set of training samples and a newly constructed adjacency matrix, the relevant entropy can be expressed as:

[0031]

[0032] The correlation entropy loss uses the correlation entropy metric instead of the traditional Euclidean distance to measure the distance between paired sample output responses. For the second-order neighbor sample pair (x i ,x j ), the range of the correlation entropy metric is 0 to 1, while the range of the Euclidean distance is the entire real number space. In a noisy environment, if there are outliers in the sample pair, the Euclidean distance metric may be very large, and minimizing this regularization term may cause the model to focus too much on the outliers. Due to the different ranges, the correlation entropy loss can improve the generalization ability and robustness of the model to a certain extent, especially when there are noise or outliers in the sample pair. In order to enhance the similarity between the new task graph and the synthetic graph in the embedding space, this paper also introduces complementary loss as a regularization term, and uses the well-known maximum average difference loss to calculate the distribution distance between the two graphs.

[0033]

[0034] in It is a picture The set of midpoint classes, E k,c and are the embedding matrices of the input graph and the compressed graph, respectively, c k is the label of the k-category node, It is c k is the class ratio of the class. |·| is the number of rows in the matrix and Mean(·) is the mean vector of node embeddings.

[0035] Finally, the present invention uses the stochastic gradient descent method to complete the following optimization problem to learn the compressed data set:

[0036]

[0037] Where λ is a loss balance parameter, and the present invention assigns a fixed label to each synthetic node, which remains unchanged during training.

[0038] Memory library update: put the compressed graph of the new task into the memory library

[0039]

[0040] Balanced training: In graph incremental learning, since the number of nodes in the new task is much larger than the number of nodes in the replay data, it will cause the problem of class imbalance in training. This imbalance problem will cause the model to tend to learn the knowledge of new tasks and easily forget the knowledge of old tasks. Previous methods have focused on scaling the coefficients of the loss function, and their general form is as follows:

[0041]

[0042] In this formula, most methods focus on optimizing parameters α and β, which will inevitably affect the overall performance of the model. The present invention also compresses the graph of the new task instead of directly using the entire new task graph for training. The specific method is to compress the new task graph into a small graph and save it to the playback memory, and finally use the data in the playback memory to update the model.

[0043]

[0044] Network testing: Input test datasets. The four datasets are divided into multiple tasks, each task consisting of two different categories in the dataset. For each task, 60% of the nodes in the dataset are used for the training set, 20% of the nodes are used for the validation set, and the remaining 20% ​​of the nodes are used for the test set. The budget ratio represents the ratio of the number of replay memory nodes to the total number of nodes in the entire training set, and the budget is evenly distributed in each task. By default, the budget ratio of the joint baseline is 1 because it stores all training data in memory. For other replay-based methods, the default budget ratio is 0.01, which means that the size of the replay memory is only 1% of the total training data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a specific implementation flow chart of the present invention.

[0046] Figure 2 Schematic diagram of the accuracy matrix of the method of the present invention on four data sets. DETAILED DESCRIPTION

[0048] Table 1 shows the experimental results of graph incremental learning on four datasets. Compared with different graph incremental learning methods, the final performance is measured by the average accuracy (AA) and average forgetting rate (AF).

[0049] Table 1 Experimental results under the incremental setting on four datasets

[0050]

[0051]

[0052] Table 2 shows the ablation study results on the CoraFull dataset and the OGB-Arxiv dataset.

[0053] Table 2 Ablation experiment

[0054]

[0055] Quantitative evaluation: Table 1 shows the quantitative results on four datasets. The AA average accuracy represents the average classification accuracy of the model after completing all task learning, which is used to evaluate the learning ability of the model; the AF average forgetting rate represents the average degree of forgetting of each task by the model, which is used to evaluate the anti-forgetting ability of the model. DEGC represents the method proposed by the present invention. As can be seen from the table, DEGC performs well on the four datasets, with average accuracy rates of 66.1%, 67.1%, 97.9%, and 71.9%, respectively. In addition, DEGC surpasses the most advanced method Cat, with improved accuracy rates of 1.6%, 1.1%, 0.2%, and 1.0%.

[0056] Table 2 shows the results of ablation studies on the CoraFull dataset and the OGB-Arxiv dataset. The two modules are the correlation entropy loss module and the balanced training module. Not including the correlation entropy loss module means that the loss function L is not used in graph compression. C , the absence of the balanced training module Bmt means that a typical replay-based scheme is adopted, that is, the entire input graph is used for training. As can be seen from Table 2, both modules have played a role, resulting in a significant improvement in performance. A careful observation shows that the correlation entropy loss has played a role on both the Corafull and Arxiv datasets, indicating that there are noise points in the real graph data. The use of correlation entropy loss can eliminate the influence of these outliers and noise in the process of compressing the graph data. When the balanced training method is used, the performance of the model is significantly improved. This shows that there is a problem of data imbalance in the dataset. Traditional training methods may not fully consider the differences in data distribution, while balanced training ensures that the model can fully learn the data of different tasks, enhancing the generalization performance of the model.

[0057] Figure 2 The accuracy matrices of the proposed method on four datasets are given to help gain a deeper understanding of the learning process. The colored blocks in the pth row and qth column represent the p After training the model, for task T q Each row of the task matrix shows how the model’s classification performance of learned tasks changes after learning new tasks. Each column shows how the classification performance of a specific task changes during the sequential learning of all tasks. Light colors indicate higher classification performance, while dark colors indicate lower classification performance. Throughout the training process, DEGC showed remarkable stability, with the lowest drop in accuracy for each task.

Claims

1. A graph incremental learning method based on denoising and efficient graph compression, characterized by: Step 1: In the training phase, the graph of the new task is first compressed into a small graph through the denoising and efficient graph compression technology. The influence of noise points and abnormal points in the graph dataset is alleviated through the related entropy popular loss. The distribution of the compressed graph is made close to the distribution of the new task graph through the complementary loss. During this period, a small-scale graph that can retain the original graph information is obtained through the constraints of the two losses. Then the compressed graph is put into the memory library, and the final model is obtained by training the compressed graphs of all tasks in the memory library together. Step 2: Testing phase: For the test nodes, the final classification results are obtained through the network.

2. The graph incremental learning method based on denoising and efficient graph compression according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1: Incremental segmentation of the data set: Step 1.2: Perform denoising and efficient image compression on the new task to obtain a compressed image: Step 1.3: Put the compressed graph of the new task into the memory bank: Step 1.4: Use the compressed image of the memory bank to balance the model training: Step 2.1: Input the test data set for testing.

3. The graph incremental learning method for denoising and efficient graph compression according to claim 1, characterized in that: For task T k The image passed in is Generate the desired synthetic image through image compression The initial synthetic graph is obtained by random noise, random sampling or clustering algorithm; under the distribution matching scheme, the objective function of graph compression is expressed as: The Dist(·,·) function is used to calculate the distance between the original image and the compressed image. The two images are encoded by the same graph encoder GNN. θ Encoding, distances are measured in embedding space; optimal compression graph The distribution should be as close as possible to the original image; The noise and outliers in the real graph data will affect the estimated distribution of the real data. For this reason, the relevant entropy loss is introduced when the compressed graph is generated, and the current data set is used to construct a predefined supervised neighbor graph G. s , where the element in row i and column j is defined as: in Represents sample x j The set of second-order neighbors of the same type, G s The construction of not only considers local relations, but also utilizes label information; given a set of training samples and a newly constructed adjacency matrix, the relevant entropy can be expressed as: The correlation entropy loss uses the correlation entropy metric instead of the traditional Euclidean distance to measure the distance between paired sample output responses; for the second-order neighbor sample pair (x i ,x j ), the range of the relevant entropy measure is from 0 to 1, while the range of the Euclidean distance is the entire real number space; In a noisy environment, if there are outliers in the sample pair, the Euclidean distance metric may be very large, and minimizing this regularization term may cause the model to pay too much attention to the outliers; due to the different value domains, the correlation entropy loss can improve the generalization ability and robustness of the model to a certain extent, especially when there are noise or outliers in the sample pair; in order to enhance the similarity between the new task graph and the synthetic graph in the embedding space, the complementary loss is introduced as a regularization term, and the well-known maximum average difference loss is used to calculate the distribution distance between the two graphs; in It is a picture The set of midpoint classes, E k,c and are the embedding matrices of the input graph and the compressed graph, respectively, c k is the label of the k-category node, It is c k is the class ratio of the class; |·| is the number of rows in the matrix, and Mean(·) is the mean vector of node embeddings; Finally, the stochastic gradient descent method is used to complete the following optimization problem to learn the compressed dataset: where λ is the loss balance parameter, which assigns a fixed label to each synthetic node and remains unchanged during training; Balanced training is used to solve the problem of imbalance between new and old data during model training. When the graph size of the new task is much larger than the graph size in the replay library, it is difficult for the model to balance the knowledge learned from the historical graph and the incoming graph. The new task graph is compressed and put into the memory library, and the compressed graphs of all tasks in the memory library are used to train the model.

4. The graph incremental learning method based on denoising and efficient graph compression according to claim 1, characterized in that: The network architecture is implemented using Pytorch. For the first graph compression stage described, that is, the efficient denoising graph compression part, a total of 200 rounds of training are performed with an initial learning rate of 10 -3 , every 20 rounds, the learning rate is multiplied by 0.95; the GNN encoder selected is GCN, all encoders have a 256-dimensional hidden layer and a 128-dimensional output layer; when training the prediction model, a total of 200 rounds were trained in this stage, and the initial learning rate was 10 -4 , every 20 rounds, the learning rate is multiplied by 0.

9. The selected GNN encoder is GCN, and the hidden layer dimension is 256.

Images