Text-subject-model-based data processing method for commodity classification

Abstract

The invention provides a text-subject-model-based data processing method for commodity classification. The method comprises the following steps of: importing Chinese and English vocabulary related to a service into a universal word library of a word segmentation system, and importing white name English words related to the service for brands and common commodity English; further expanding a stop word library of the word segmentation system; segmenting words for a description text part of a commodity, so that each commodity can have a bag of words which is not related to sequence; counting word segmentation results to acquire uncommon vocabulary with high frequency, and thus constructing a preferential word library; and appointing a general classification quantity, setting related parameters, executing quick Gibbs sampling, acquiring potential semantic association, comparing the latent semantic association with the preferential word library, the universal word library and the stop word library respectively, calculating comparison results to obtain the most possible classification of the commodity, and marking the classification by using the bags of words. In consideration of latent semantics, the influence of subjective factors of editorial staff is reduced, so that the commodity classification is accurate.

Description

【Technical field】 [0001] The invention relates to a data processing technology of electronic commerce, and relates to a data processing method of commodity classification based on a text topic model. 【Background technique】 [0002] In the e-commerce market of the Internet, the existing commodity classification systems are all manually classified by website editors. There are three problems: 1) A large number of products may lead to excessive human consumption; 2) A product may have multiple category attributes and can be divided into multiple categories, and manual classification will cause the editor to understand the attributes of things. 3) When classifying a product, the editor cannot accurately give the credibility of the classification. [0003] The Chinese invention patent with the publication number 102193936A disclosed on 2011-9-21 discloses a data classification method and device. The method is as follows: obtain the relevant data of each product that needs to be ...

AI Technical Summary

Problems solved by technology

There are three problems: 1) A large number of products may lead to excessive human consumption; 2) A product may have multiple category attributes and can be divided into multiple categories, and manual classification will cause the editor to understand the attributes of things. 3) When classifying a certain product, the editor cannot accurately give the credibility of the classification

[0004] 1. It only analyzes the title text of the product, but not all texts related to the product, including product brief description, purchase user comments, etc.;

[0005] 2. The text segmentation method is used to obtain keywords (word segmentation results) and determine their weight by frequency of occurrence. The result of this classification will not be very accurate;

[0006] 3. The potential semantic problems of the text are not considered, so it can only be classified and merged from the level of commodity type;

[0007] 4. It is necessary to manually edit the main attributes of each product for the word segmentation results. This method relies on the personal experience of the editors. Due to the lack of information of the editors, the results may deviate from the objective facts

Images