Natural language information processing method

Abstract

The invention relates to the technical field of natural language processing, and discloses a natural language information processing method. The method comprises the following steps: storing natural language information by utilizing a Huffman tree, and converting the natural language information into a natural language vector; carrying out word segmentation processing on the natural language vector by utilizing a pre-trained LC-CRF model; extracting vocabulary unit vectors of ambiguous words, and forming an ambiguous word feature matrix; inputting the ambiguous word feature matrix into a pre-constructed word sense disambiguation model, and identifying semantics of ambiguous words by utilizing the word sense disambiguation model; iteratively updating the weight of each word based on the weighted node initial value of the word comprehensive weight and the node probability transfer matrix, and selecting the first N words with relatively large weights as keywords; and taking a keyword vector in the natural language information as an input vector, and identifying emotion in the natural language information by utilizing the neural network based on the segmented pooling layer. The naturallanguage information is processed.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, in particular to a method for processing natural language information. Background technique [0002] At this stage, the degree of informatization of the entire world has reached a new height with the development of Internet technology, and at the same time it has brought an unimaginable growth rate to the amount of information in human society. In daily life, massive natural language information brings convenience to human beings, but also brings great troubles, that is, how to efficiently obtain the required content from large-scale information. [0003] Due to the great complexity of natural language, on the one hand, natural language has no fixed pattern, so it has rich ways of expression. Humans have their own habits when expressing their thoughts, so different people often have different expressions when describing the same thing ; On the other hand, natural lang...

AI Technical Summary

Problems solved by technology

Since the jieba word segmentation technology is based on the Trie tree structure to scan word graphs, it generates a directed acyclic graph composed of all possible word formations of Chinese characters in a sentence. Dynamic programming is used to find the maximum probability path and find the largest segmentation combination based on word frequency. , does not take into account the interrelationship between words, so word segmentation results do not contain the intrinsic information in words; the word sense disambiguation method based on probability statistics has good flexibility and high disambiguation efficiency, but the disambiguation task is Constrained by the size and category of the corpus, its accuracy is low, while the method based on the semantic dictionary has a high accuracy rate, but it takes a lot of work to build a dictionary, the efficiency is difficult to improve, and the disambiguation method is single; the existing keyword extraction The algorithm is mainly the TextRank algorithm, which is a graph-based sorting algorithm. It uses the co-occurrence window to realize the relationship between some words, sorts the subsequent keywords, and directly extracts keywords from the text itself. However, this method does not analyze words. Whether the difference in importance will affect the weight transfer of adjacent nodes, and the overall information of the document corpus is not used, the weight information of words has no practical meaning, and the strength of the connection cannot be distinguished

Images