Deep data processing method and system combined with knowledge base

Abstract

The invention discloses a deep layer data processing method combined with a knowledge base. The method comprises the following steps that concept tuple sets in a body base are merged; link routes of predicates in different types and corresponding logic rule sets are obtained; the logic rule sets are screened preliminarily, candidate rule sets are obtained; a deep layer probabilistic graphical model is obtained; structuring tuples are obtained based on data to be processed and are mapped to a layering concept space; target tuples are generated, semantic extension is carried out; logic rule sets and evidence tuple sets are obtained; a Markov logical net is subjected to instantiation, the conditional probability for target tuple founding is computed, and data processing results are obtained. The invention further provides a deep layer data processing system which comprises a structuring module, a conceptualization module, a target generating module, an extension module, an activating module and a probability computing module. Context and background knowledge can be fully merged, and accordingly, the purpose of semantic comprehension is really achieved.

Description

Technical field [0001] The present invention relates to the fields of natural language processing and artificial intelligence, and more specifically, to a deep data processing method combined with a knowledge base for text understanding. Background technique [0002] With the increasing popularity of network informatization, data is expanding rapidly with unprecedented breadth and depth. It is becoming more and more important to enhance the ability to analyze and extract knowledge from unstructured text. In addition to the huge scale of texts from the real world, the more important challenge is the complexity and uncertainty of knowledge representation and reasoning. The former is reflected in the high degree of heterogeneity of knowledge, including not only facts and concepts about each object, but also general rules of reasoning and ontological relations; the latter comes from the objective reality of knowledge itself and the level of subjective knowledge of knowledge, and a la...

AI Technical Summary

Problems solved by technology

At present, the natural language system developed under the framework of Markov logic cannot solve these requirements well. First, multi-level is the basic organizational structure of large-scale knowledge representation, but in the process of generating candidate rules, only the relationship between two predicates is usually considered. In the case of common concept variables, the inherent hierarchical relationship between concepts and the possible overlap and intersection of the described scope are ignored. Therefore, under the constraints of the given rule length, it is easy to lose a lot of important semantic information and affect all The quality of the generated logical rules; secondly, as an important part of knowledge representation, the scale of automatic acquisition of uncertain rules is directly limited by the complexity of parameter learning algorithms (that is, assigning appropriate weights to logical rules), although in principle both The layer is enough to express any function, but the efficiency is very low when expressing most functions, and the existing methods usually need to calculate the instantiation and value of all candidate clauses during the optimization process, which will consume a lot of time and space overhead, so it is not suitable for automatic processing of large-scale knowledge; re-generalization and activation are the concrete manifestation of knowledge application ability, but the probabilistic reasoning about complex relationships has not fully considered effective knowledge generalization at the entity or relationship level, Therefore, it is difficult to deal with the problem of incomplete knowledge such as incomplete evidence coverage. In addition, the current activation strategy is more based on the value of the instance tuple and its logical rules, and has not considered the relationship between the target tuple and the instance tuple or logical rules. degree, so it is easy to generate a large number of weakly related or even irrelevant rules or tuples, which brings huge computational overhead for uncertain reasoning and affects the efficiency of semantic analysis

Therefore, existing semantic analysis systems still have deficiencies in many aspects

Images