The invention discloses a multi-
modal classification method fusing a graph
convolutional neural network and a
capsule graph neural network, and solves the problem of how to apply an
artificial intelligence technology to an appearance patent. The method comprises the following steps: coding a patent text sequence by utilizing BERT, segmenting semantic graph
convolution by utilizing PCNN, constructing a
dependency tree by utilizing grammar graph
convolution, carrying out interactive attention and relationship classification, and finally carrying out image
feature extraction through a
capsule network. According to the method, the appearance
patent classification technology fusing heterogeneous multi-
modal information is researched, the
feature extraction and fusion problems of multi-
modal information such as characters, images and networks are solved, a heterogeneous multi-modal appearance design
patent classification model is constructed, and the classification accuracy is greatly improved relative to pure text content. And meanwhile, an appearance design patent AI standard
data set is constructed on the basis of the
intellectual property hypergraph network, so that the blank of patent types in a traditional Lankyo classification
system is filled, and the
bottleneck of a traditional appearance
patent classification method is broken through.