Unlock instant, AI-driven research and patent intelligence for your innovation.

End-to-end data-free antagonism knowledge extraction method based on graph structure data

A knowledge extraction and adversarial technology, applied in the field of image processing, can solve the problems of effectively extracting the student model is not very useful, and the accuracy of the student model is low

Pending Publication Date: 2022-05-31
BEIJING UNIV OF POSTS & TELECOMM
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, they are not very useful for effectively extracting student models, resulting in lower accuracy of the trained student models

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • End-to-end data-free antagonism knowledge extraction method based on graph structure data
  • End-to-end data-free antagonism knowledge extraction method based on graph structure data
  • End-to-end data-free antagonism knowledge extraction method based on graph structure data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] The present invention is an end-to-end data-free adversarial knowledge extraction method DFAD-GNN based on graph structure data. The structure of DFAD-GNN is as follows figure 1 As shown, it mainly consists of three parts: a generator and two discriminators. One discriminator is a pre-trained teacher model T, and the other is a compact student model S that we aim to learn. More precisely, the generator G draws samples z from the previous distribution and g...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an end-to-end data-free antagonism knowledge extraction method based on graph structure data, which is used for model compression. A generative adversarial network is adopted, the network is mainly composed of three parts, a pre-trained teacher model and a pre-trained student model are regarded as two discriminators, a training graph is generated through a generator, and knowledge in the teacher model is extracted into the student model. A large number of experiments on different reference models and six representative data sets show that the method provided by the invention obviously exceeds the latest data-free baseline in the aspect of a graph classification task. Under the condition of no actual data, errors are successfully reduced, a student model with relatively good performance is obtained, and the method can be effectively applied to different network system structures. According to the method, an effective end-to-end data-free graph knowledge distillation framework is provided for the first time, and a solid foundation is laid for future research work.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to an end-to-end data-free adversarial knowledge extraction method based on graph structure data. Background technique [0002] Graph neural networks (GNNs) have been widely used in graph-structured data modeling due to their excellent performance in a wide range of practical applications. In recent years, knowledge distillation (KD) techniques for GNNs have made remarkable progress in model compression and knowledge transfer. [0003] Knowledge Distillation (KD) aims to transfer knowledge from the (larger) teacher model to the (smaller) student model. It was originally introduced to reduce the size of models deployed on devices with limited computing resources. Since then, this series of studies has attracted a lot of attention. Recently, there have been some attempts to combine knowledge distillation with graph convolutional networks (GCNS). In the field of computer v...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G06N5/02
CPCG06N3/04G06N3/08G06N5/022G06N3/048
Inventor 石川庄远鑫
Owner BEIJING UNIV OF POSTS & TELECOMM