A Method for Extracting Hidden Node Semantics in Deep Belief Networks

Abstract

The invention discloses a method for extracting hidden node semantics in a deep belief network, and proposes a brand-new method for obtaining semantic information of hidden nodes in a hidden layer in a deep belief network. The key point of this method is to use the combination of semi-structured topic model and deep belief network, and learn the model parameters at the same time, so that the explicit semantic information of hidden nodes in different hidden layers in deep belief network can be obtained. Another key point of the present invention is that the present invention combines two different network types, the Bayesian network and the deep neural network, and uses the topic model to perform semantic analysis on the deep belief network. Compared with the prior art, in the technical solution proposed by the present invention, a method for obtaining the semantic information of the hidden nodes inside the hidden layer in the deep belief network is constructed. This scheme can use the Bayesian topic model to model the hidden nodes in the deep belief network and obtain its specific semantic information.

Description

technical field [0001] The present invention relates to node semantic extraction technology, and more specifically, relates to a method for extracting hidden node semantics in a deep belief network. Background technique [0002] With the wide application of deep neural network technology in data mining, artificial intelligence, etc., more and more applications use deep neural network technology to process text, image, voice and video data. In text modeling tasks, using Deep Belief Networks (Deep Belief Networks) and extended models, as a type of deep neural network, has also become an effective means in document modeling. A Deep Belief Network is a deep generative network consisting of multiple layers of restricted Boltzmann machines. The Restricted Boltzmann Machine (RBM for short) is a generative random neural network, which is a network mapping structure mainly composed of a visible layer and a hidden layer. The hidden layer contains several hidden nodes, and each hidde...

AI Technical Summary

Problems solved by technology

[0005] However, the above existing technologies have the following disadvantages: when modeling documents, we cannot obtain the semantic information of each hidden node in the hidden layer

That is to say, in the document mapping process, although we can obtain the value of each hidden node, we cannot determine the specific semantics represented by each hidden node in each hidden layer

Therefore, when the deep belief network maps text data, the vector expression of the hidden layer is a black box operation, and cannot explicitly express specific semantic information

Images