Movie searching entity identification method based on CRF

Abstract

The invention discloses a movie searching entity identification method based on CRF. Data automatic rough labeling is carried out on movie searching text data; through combination of artificial correction, training corpus labeling is finished; features are extracted through design of a feature template; and entity identification is carried out through utilization of the CRF. Involved technologiescomprise natural language interaction understanding, movie corpus labeling and entity identification, and network crawler technologies. According to the method provided by the invention, the entity identification is independent of a knowledge base, so an unlogged entity also can be identified; in the movie searching field, good entity identification effects can be obtained for different entity types; and the timeliness is high.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a CRF-based video retrieval entity recognition method. Background technique [0002] Named entity recognition refers to the recognition of named referents from text, including names of people, places, institutions, and specific entities in some special fields. It is an important research direction in the field of natural language processing and has a wide range of applications in engineering practice. Applications, such as: event detection, information retrieval, machine translation, question answering system and other fields. [0003] At present, English entity recognition technology has reached a relatively high level; however, Chinese entity recognition is relatively difficult. It is mainly due to several characteristics of Chinese itself: (1) There is no clear boundary label in Chinese, and the concept of words is relatively vague; (2) Chinese words are f...

AI Technical Summary

Problems solved by technology

[0003] At present, English entity recognition technology has reached a high level; however, Chinese entity recognition is relatively difficult

It is mainly due to several characteristics of Chinese itself: (1) There is no clear boundary label in Chinese, and the concept of words is relatively vague; (2) Chinese words are flexible and changeable, and the same entity has different meanings in different contexts; (3) Entities are nested, especially in organization names; (4) There are many simplified expressions in Chinese, and it is difficult to recognize the Chinese translation of English names

[0004] Most of the existing entity recognition algorithms are used to deal with long texts, while video retrieval texts are very short, without a complete syntactic structure, and usually include ambiguous phrases, which cannot provide sufficient background information. Therefore, it is very difficult to accurately determine the entity type

Images