A method for automatic generation of electronic commerce dictionary

Abstract

The invention discloses an automatic generation method of an e-commerce dictionary. The automatic generation method comprises the following steps of 1 data crawling: crawling original commodity data from an e-commerce website and a search engine; 2 pretreatment; 3 exhaustion in a mode going forward one by one; 4 word frequency statistics; 5 merger treatment; 6 redundancy filtering; 7 regular type filtering; 8 potential word compensation; 9 low frequency word rejecting; and 10 feature word compensation. The automatic generation method mainly has the advantages of being high in dictionary generation speed, adopting algorithms such as machine learning, intelligent filtering, error correction and compensation to automatically generate the dictionary, and being capable of greatly improving generation efficiency; being high in including rate of the generated dictionary, enabling fewer entries to be leaked in a word segmentation process due to the fact that a method of exhaustion in the mode going forward one by one is used for word segmentation of a text; and being refine in the generated dictionary, combining with processing algorithms such as the error correction, the redundancy filtering and the regular type filtering, removing redundancy and errors in the dictionary, and finally enabling the generated e-commerce dictionary to be refine.

Description

technical field [0001] The invention relates to a method for automatically generating an electronic commerce dictionary. Mainly for the field of e-commerce, the e-commerce dictionary is the basis of e-commerce website applications, such as search, recommendation, semantic word segmentation, sorting weight calculation and many other aspects need to be used. Background technique [0002] At present, dictionaries for e-commerce are rare. At present, mainstream applications such as Taobao mostly use manual generation or simple statistical generation, and some use machine learning methods to collect entries to form dictionaries. However, the disadvantages of the traditional method mainly include: first, the manual processing workload is large; second, due to the endless changes in the application of new products in the e-commerce field, the update speed of the traditional method is slow; third, the accuracy of the automatic generation method is low, and the result is relatively r...

AI Technical Summary

Problems solved by technology

However, the disadvantages of the traditional method mainly include: first, the manual processing workload is large; second, due to the endless changes in the application of new products in the e-commerce field, the update speed of the traditional method is slow; third, the accuracy of the automatic generation method is low, and the results are relatively rough

Images