Variational expectation maximization routing algorithm based on a capsule network

Abstract

The invention discloses a variational expectation maximization routing algorithm based on a capsule network. A pose matrix of a low-level capsule is regarded as a data point of a GMM; A pose matrix ofthe high-level capsules is regarded as Gaussian distribution, data points are clustered into Gaussian distributions through a VBEM routing algorithm, distribution parameters of the Gaussian distributions are calculated, that is, the low-level capsules are grouped to form the high-level capsules during operation, and then an activation value a is updated and calculated according to the Gaussian distribution parameters. According to the invention, a VBEM algorithm is used in a capsule network; Comparing EM algorithms, According to the method, the extra calculation amount is almost not needed; the calculation problem in the maximum likelihood method is solved; each decomposition factor is optimized based on variation inference to complete the overall optimization process; According to the method, the approximate solution can be obtained, singularity generated when a Gaussian component degenerates to a specific data point can be avoided, the hidden variable class number k can be automatically determined in the algorithm, and overfitting is avoided when k is large.

Description

technical field [0001] The invention relates to the field of variational inference and Bayesian, in particular to a variational expectation maximization routing algorithm based on capsule network, which is used for clustering low-level capsules into high-level capsules. Background technique [0002] Capsule network is a new neural network model. Capsule is a spatial concept, which can be a vector or a matrix plus scalar. The specific form of the capsule can be determined according to the characteristics of the input data. The capsule network only needs 3 layers in the field of handwritten digit recognition. The hidden layer can achieve the effect of a deep neural network, or even better. [0003] The expectation maximization (EM) algorithm is a general method to find the maximum likelihood solution of the probability model with potential variables. The Gaussian mixture model (GMM) refers to the linear combination of multiple Gaussian distribution functions, which is widely u...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the non-computable problem that the routing algorithm in the existing capsule network solves some practical problems, the present invention discloses a variational expectation-maximization routing algorithm based on the capsule network. The purpose of the variational expectation-maximization routing algorithm is to group capsules into a The relationship between the part and the whole: the pose matrix of the low-level capsule is regarded as the data point of GMM, and the pose matrix of the high-level capsule is regarded as a Gaussian distribution. The VBEM routing algorithm clusters the data points into a Gaussian distribution and calculates its distribution parameters, namely Group low-level capsules to form a high-level capsule at runtime, and then calculate the activation value a according to the Gaussian distribution parameter update

Images