Text semantic processing method and system using image processing technology and semantic vector space

Abstract

The invention belongs to the technical field of text semantic information processing, and in particular relates to a method and a system for processing text semantics by utilizing an image processing technology and semantic vector space. The system comprises a text input and preprocessing module, a semantic vector construction module, a semantic information processing module and a semantic processing result display module, wherein the semantic information processing module is specifically used for semantic turning sentence extraction, semantic noise sentence detection, semantic range tracking and semantic scene segmentation. According to the method and the system, a text unit is mapped to a pixel in an image, and a semantic vector which describes the text unit is taken as pixel grayscale of the image, so that various technologies and methods in an image processing field can be introduced to process a text flexibly and intuitively without the influence of the diversification of word forms; meanwhile, the semantic vector is constructed by instructing a Word2Vec method, so that the lightweight of algorithms is ensured to meet the requirements on real-time application.

Description

technical field [0001] The invention belongs to the technical field of text semantic information processing, and in particular relates to a lightweight text semantic processing method and system utilizing image processing technology and semantic vector space. Background technique [0002] With the development of computer technology and network, we have now entered the era of information explosion - all kinds of massive data are presented in the form of electronic text. In this case, it is possible to quickly and accurately extract the information that users care about. It is against this background that the text information processing technology emerges as the demand becomes more and more urgent, and the semantic processing of text is the most important thing, which makes us move from language processing to language understanding. Text semantic processing technology has great application value in many fields, such as text semantic segmentation, automatic text summarization e...

AI Technical Summary

Problems solved by technology

[0003] However, the current text semantic processing technology still has some limitations. Taking the text semantic segmentation technology as an example, it basically starts from the perspective of word frequency statistics, by calculating the similarity of the word frequency statistical vectors of repeated words in adjacent text blocks. Degree to achieve semantic segmentation, such as the classic TextTiling algorithm, Dotplotting algorithm, but they do not take into account the semantic space contained in the word, can not deal with the impact of word variants or synonyms, so the robustness is not strong; after that Although some algorithms such as the ESA (Explicit semantic analysis) algorithm have enhanced robustness by introducing semantic vector space, they cannot meet the needs of real-time applications due to the high dimension of the semantic space and the huge computational cost; there is also the TopicTilling algorithm, although through Adding the connection between words and topics improves segmentation performance, but it requires complex topic models to intervene, etc.

Images