Knowledge representation learning framework based on multi-class cross entropy comparison completion coding

Abstract

The invention discloses a knowledge representation learning framework based on multi-class cross entropy comparison completion coding. The framework mainly comprises a semantic structure feature extraction module (S) and an automatic comparison completion coding module (G). The semantic structure feature extraction module (S) is responsible for extracting low-level and high-level semantic structure features from entities and relationships and fusing the low-level and high-level semantic structure features to obtain low-level and high-level semantic structure features; the automatic comparison completion coding module (G) is responsible for predicting an entity context vector, setting positive and negative samples and a sampling method (C3NCE) of the positive and negative samples, calculating a multi-class cross entropy comparison loss function, obtaining vector representation of knowledge graph entities and relations by optimizing a target function training model, and completing a triple completion task. The framework provided by the invention can quickly, stably and accurately complement the triple of the missing information in the knowledge graph, well completes the knowledge representation learning task, greatly improves the accuracy and efficiency of knowledge graph construction, and is wide in application prospect.

Description

technical field [0001] The invention relates to the fields of natural language processing and machine learning, in particular to a knowledge representation learning framework based on multi-class cross-entropy contrastive complement coding. Background technique [0002] The advancement of neural networks has greatly promoted the development of natural language processing and changed the research landscape in the field of natural language processing. The thinking mode of traditional feature engineering is replaced by the way of neural network model learning natural language features. The knowledge graph was proposed by Google in 2012, and has been widely used in academia and industry since then. It is an important form of natural language knowledge storage. Many downstream tasks of natural language processing, such as question answering systems, relation extraction, and entity classification, are increasingly dependent on the construction of knowledge graphs and the quality ...

AI Technical Summary

Problems solved by technology

However, the biggest problem with this type of method is that the scoring function is not learnable and is limited by human prior knowledge, and the edge loss function cannot perform adaptive weighted learning on positive and negative samples, which greatly limits the effect of representation learning.

Some recently proposed knowledge representation learning methods parametrically design the scoring function, and use the logistic regression loss function or the binary cross-entropy loss function, so that the scoring function can be learned and optimized through the loss function to obtain a more reasonable scoring function. , to project the representation vectors of knowledge map entities and relationships into a more reasonable representation space for comparison and calculation, but the logistic regression loss function also cannot perform adaptive weighted learning on positive and negative samples, and the binary cross-entropy loss function is due to positive and negative There is a significant sample imbalance problem with large differences in sample size

Images