Keyword joint type generation method and system based on semantics and knowledge graph

Abstract

The invention relates to a keyword joint generation method and system based on semantics and a knowledge graph. The keyword joint generation method comprises a preparation stage, a training stage 1, atraining stage 2 and an inference stage, wherein the preparation stage comprises the following steps: according to the characteristics of an application field, manually constructing a corresponding knowledge graph M (such as a tree, an undirected graph and a directed graph) structure to represent the structural relationship (such as inclusion and causality) of required keywords; preparing unlabeled Chinese corpora related to the field; installing a sentence vector tool kit (such as python3.5 and sent2vec). The method is high in accuracy, high in semantic relevancy, high in semantic divergenceand wide in coverage range, and the complementarity of the keyword extraction method, the keyword distribution method and the keyword generation method is better.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and in particular relates to a method and system for jointly generating keywords based on semantic and knowledge graphs. Background technique [0002] With the explosive growth of Internet text data, it is often necessary to extract keywords that can summarize the core ideas of articles in business, so as to achieve accurate recommendation and improve reading efficiency. This type of business has the characteristics of strong standard subjectivity and difficulty in obtaining available annotation corpus, which leads to low accuracy of traditional methods and high calculation time for some methods. In related technologies, keyword extraction can be realized through two methods: keyword extraction (for words that have appeared in the text) and keyword generation (for words that have not appeared in the text). The main methods of keyword extraction are: statistics-based methods, grap...

AI Technical Summary

Problems solved by technology

[0005] Through the analysis of the above-mentioned published patent documents, it is found that there are the following disadvantages: when selecting candidate keywords, relying on the "preset vocabulary" requires additional manual workload; the above method is highly dependent on the semantic similarity between the text and the title Assume that it is impossible to deal with the keyword extraction problem of headline party articles and untitled articles; the above method uses the attention weight method when calculating the semantic relevance, and establishes the relationship between candidate words and each character in the title, lacking the overall and The understanding of topic-level semantics may lead to insufficient coverage of the extracted keywords; the above-mentioned method is a supervised method (sequence annotation method), and a supervised corpus is constructed through the semantic similarity between the title and the document, which reduces the workload of annotation. But it will also bring loss in accuracy / coverage / diversity; and the process is complicated, the LSTM neural network used is difficult to parallelize the calculation, the prediction speed is slow, and it cannot solve the problem of easy extraction of high-frequency words and redundant keywords

[0007] Through the analysis of the above-mentioned published patent documents, it is found that there are the following disadvantages: the semantic model adopted relies on manual labeling, and the amount of labeling required is relatively high, and the cost and labeling time are difficult to control; the above method is based on the graph model, and the time The complexity is high and the calculation speed is slow; the paragraph / position information based on the above method depends on the document structure, and the generality of keyword extraction for short texts is not good; the above method is based on word frequency and inverse document probability information, and cannot solve the Problem; and there are a large number of manual summary features, insufficient scalability, only suitable for travel documents, insufficient versatility

Images