DINFO-OEC text analysis mining method and device thereof

Abstract

The invention provides a concept-based unstructured text big data analysis mining method and a device thereof. The method comprises the following steps of: (1) performing pre-processing, including word segmentation and named entity recognition; (2) performing concept extraction and concept expression identification on an input text; (3) performing analysis mining on the concept expression of the input text according to mining rules; (4) calculating confidence levels of the mining results; (5) outputting the mining results according to the confidence levels; and (5) visually showing the mining results. A mining model of the method comprises three trees, including an ontology tree, an element tree and a concept tree. The device comprises a modelling unit (1), a pre-processing unit (2), a concept extraction and expression identifying unit (3), an analysis mining unit (4) and a visual show unit (5). The method and the device have the following advantages that: diversity of services and natural language expressions is separated in the modelling process, and investment in service maintenance is lowered; and the mining method can greatly improve accuracy of analysis mining.

Description

technical field [0001] The invention relates to the technical field of text mining, in particular to a concept-based DINFO-OEC mining method and DINFO-OEC mining equipment. Background technique [0002] 80% of social big data is unstructured data, and unstructured big data processing is the biggest challenge big data faces. Structured data analysis cannot fully mine and discover semantics in big data. [0003] The challenges of unstructured text mining are: [0004] Maintenance challenges brought about by language diversity: There are various language expressions in the text, and irregular usages such as abbreviations and shorthand are common. [0005] Maintenance challenges brought about by multiple business categories and fast-changing rules: There are many business categories and fast-changing categories. Every time a category changes, it is necessary to reorganize the language rules of all related categories. The maintenance workload is huge and the maintenance efficie...

AI Technical Summary

Problems solved by technology

[0008] However, the existing technologies generally use statistical methods for text mining, without considering the needs of business personnel, and only provide mining algorithms, which has brought a lot of trouble to business personnel

The problem faced by text mining technology is how to analyze and mine valuable information that users care about from a piece or a large amount of unstructured text, so that business personnel can define mining requirements and mining rules from a business perspective, regardless of the language expression in the text Linguistic Ambiguity Caused by the Diversity of Habits

Images