New word extraction method based on commodity comments
A product review and extraction method technology, applied in the design of Chinese natural language processing, can solve problems such as difficulty in identifying low-frequency new words, time-consuming and labor-intensive manual labeling of corpus, high complexity of manual writing and maintenance rules, and achieve the effect of increasing accuracy
Active Publication Date: 2020-06-26
ANHUI UNIV OF SCI & TECH
View PDF8 Cites 2 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] The existing Chinese new word recognition technology can be divided into two types: one is rule-based new word recognition, but the method is relatively complex to manually write and maintain rules; the other is statistics-based new word recognition, and the commonly used methods are: HMM, SVM, filtering rules, etc., the disadvantage of this method is that manual labeling of corpus is time-consuming and laborious; if low-frequency new words are filtered to ensure accuracy, it will make it difficult to recognize low-frequency new words
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0041] The present invention is described in detail below in conjunction with accompanying drawing:
[0042] The new word extraction method that the present invention provides for product review, such as figure 1 As shown, it can be roughly described as steps:
[0043] Step 1. Obtain a collection of product reviews: Use a crawler tool to crawl the online shopping reviews of Taobao and JD.com, and then save the crawled product review corpus locally.
[0044] Step 2, document preprocessing: perform noise reduction and word segmentation processing on the obtained review set documents to obtain a preprocessing set.
[0045] Step 3, word expansion processing: perform word expansion on the preprocessing set to obtain a potential word set.
[0046] Step 4, Preliminary Candidate Word Set Screening: Screen the preliminary candidate word set by word frequency and degree of freedom.
[0047] Step 5, final candidate word set screening: filter out candidate word sets by calculating supp...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a new word extraction method based on commodity comments. The method specifically comprises the following steps: preprocessing a commodity comment set, and calculating the co-occurrence frequency and mutual information of each word and an adjacent character to obtain a potential word set; respectively calculating the occurrence frequency and degree of freedom of the potential words under the commodity and the similar commodity; performing screening by calculating the support degree of the preliminary candidate word set to obtain a final candidate word set; and finally,comparing with a logged-in word dictionary, and screening out new words. The invention aims to provide an accurate and efficient method for extracting new words in online shopping comments, so that online shopping platforms or online merchants can conveniently carry out work needing comment emotion tendency analysis such as market analysis and user satisfaction survey, and expansion of special dictionaries is facilitated.
Description
technical field [0001] The invention relates to the technical field of Chinese natural language processing, in particular to a method for extracting new words from commodity reviews. Background technique [0002] With the continuous development and progress of the Internet, the scope of people's communication is constantly expanding, and vocabulary is also constantly developing and changing. The most obvious manifestation of this change is the continuous emergence of new words, which are also widely used in online shopping reviews, and these new words often contain a lot of important information such as emotions and attitudes of users. In order to better conduct market analysis and user satisfaction surveys on online shopping platforms. Among them, the new word extraction based on product reviews is the basis of the work, and it is also a crucial part of determining the quality of the work. Online shopping product reviews have the characteristics of high repetition rate, c...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/9535G06F16/33G06F40/289G06Q30/06
CPCG06F16/9535G06F16/3344G06Q30/0623
Inventor 张顺香朱海洋尹畅金鸣徐善山余宏斌
Owner ANHUI UNIV OF SCI & TECH