Data processing method and device, and electronic equipment

Abstract

The invention discloses a data processing method and device, and an electronic equipment. The method comprises steps of obtaining a to-be-processed text in a target web page; determining a target industry category to which the to-be-processed text belongs; the M preset classification word banks being preloaded into a cache from a database, wherein the M preset classification word banks correspondto M industry categories; dynamically matching the category name of the target industry category with lexicon names of M preset classification word banks in the cache, and determining a target classification word bank; and dynamically loading the target classification word bank from the cache to a memory, and performing word segmentation processing on the to-be-processed text based on the target classification word bank in the memory to obtain a word segmentation processing result. Based on the method, the category name of the target industry is dynamically matched with the lexicon name of thepreset classification word bank, the target classification word bank can be dynamically loaded to achieve a purpose of dynamic adaptation, the word segmentation range can also be narrowed, and word segmentation accuracy, specialty and processing efficiency are improved.

Description

technical field [0001] The present invention relates to the field of information technology, in particular to a data processing method, device and electronic equipment. Background technique [0002] With the increasing expansion of business data, the in-depth analysis of DPI log web page content is facing huge challenges, and the demand for a large amount of data analysis is showing a blowout trend. How to ensure the accuracy of word segmentation for a large amount of data has gradually become a thorny problem. [0003] The relatively common word segmentation technology in the existing large amount of data recognition is: based on the general thesaurus, perform word segmentation and statistics on the content of a large number of DPI log web pages. The commonly used high-frequency words in the word segmentation results of this word segmentation technology are all ranked first, the word segmentation accuracy and professionalism are low, and the number of commonly used high-fr...

AI Technical Summary

Problems solved by technology

[0004] Embodiments of the present invention provide a data processing method, device, and ele

Images