Deep learning-based short text dependency analysis method

Abstract

The invention discloses a deep learning-based short text dependency analysis method. The method comprises the steps of 1) obtaining an HTML file where a user query statement is located from a search engine log, thereby serving as a training data set; 2) according to the training data set, generating a dependency analysis tree of the query statement; and 3) training neural network model-based part-of-speech tagger and syntactic analyzer by using the dependency tree. By utilizing a currently used sentence-level dependency analyzer, massive short text dependency analysis data sets are automatically generated, and the generated data sets are subjected to noise reduction and optimization by using multiple methods. Based on the data sets, a short text dependency analysis model is trained; and anexperiment shows that the short text tagging effect of the model is greatly improved in comparison with that of the sentence-level dependency analyzer.

Description

technical field [0001] The invention belongs to a short text dependency analysis method based on deep learning. Background technique [0002] Phrase structure and dependency structure are the two most widely studied grammatical structures in syntactic analysis. Dependency grammar was first proposed by French linguist L. Tesniere in his book "Basis of Structural Syntax" (1959). Dependency Grammar reveals its syntactic structure by analyzing the dependency relationship between components in a language unit. It holds that the core verb in a sentence is the central component that dominates other components, but itself is not dominated by any other components. A dependent relationship is subordinate to the dominator. [0003] For example, for the text "Its apple watch charging stand is my favorite stand.", the dependency analysis tree obtained after dependency analysis is as follows figure 2 : [0004] From the dependency analysis tree, the overall grammatical structure of th...

AI Technical Summary

Problems solved by technology

[0009] 2. So far there are no linguistic rules for dependency analysis on short texts

[0014] ● Context-sensitive information: There are two main disadvantages of using only context-free information: 1) It is risky to directly consider the relationship between two words without considering the context

2) Context-independent information is often unable to describe the direct dependency type of two words, and thus cannot fully represent the semantics of the entire input

Images