Neural network for mining implicit relationship between features based on short connection

Abstract

A neural network for mining implicit relations between features based on short connection comprises: an input layer provided with an input space; a first embedding layer which is used for subsequentlyextracting nonlinear low-order feature interaction; a second embedding layer which is used for subsequently extracting linear features; a nonlinear interaction pooling layer which is used for extracting nonlinear low-order feature interaction from the dense feature matrix output by the first embedding layer; a layer loss neural network which is used for converting the low-order interaction features output by the nonlinear interaction pooling layer into high-order feature interaction output; a linear model which is used for extracting linear features from the dense feature vectors output by the second embedding layer; and a combination layer which is used for fusing the high-order feature interaction output by the layer loss neural network and the linear features output by the linear modelto obtain a final prediction value. According to the method, more characteristics can be fully utilized, the low-order characteristic interaction vector which is more effective for the target task isconstructed, and the prediction capability of the model is further improved.

Description

technical field [0001] The present invention relates to a neural network. In particular, it involves a neural network based on short connections to mine the implicit relationship between features. Background technique [0002] Sparse predictive analysis is an important problem in the field of machine learning. When data features are in sparse representation, machine learning techniques can also be applied to predict the relationship between features and targets. Most prediction tasks in reality are to estimate a real-valued feature vector x ∈ R n The mapping function y:R to the predicted target T (regression: T=R, classification: T=(+,-)) n →T. In supervised learning, suppose there is a data set D={(x 1 ,y 1 ),...,(x i ,y i ),...,(x num ,y num )}, where i represents the i-th sample, x i is the feature vector, y i For the prediction target, num represents the number of samples. The problem that the present invention deals with is: x i is a highly sparse eigenvect...

AI Technical Summary

Problems solved by technology

However, they only use the features of the current layer when learning feature interactions, and do not make full use of the historical features generated in the middle process of the model, that is to say, they ignore the transferability and reusability of features.

Images