An e-commerce clothing defect assessment method based on online review text and pictures

Abstract

E-commerce defect detection and evaluation are mostly based on modeling and analysis of text. The present invention discloses a new e-commerce clothing defect evaluation method based on online comment text and pictures. The method crawls customer feedback data to establish a text dataset based on pictures and corresponding to pictures. The Faster R-CNN network is used to detect image defects, and the defects and severity in the comment text are extracted based on the sentiment dictionary. The severity of such clothing defects is calculated by combining the severity evaluation calculation model of text and pictures. This method makes full use of customer feedback data, and the results are more credible and accurate.

Description

Technical field:

[0001] The present application relates to the fields of natural language processing and image detection, and in particular to clothing defect detection and emotion extraction methods. Background technology:

[0002] With the rapid development of the Internet industry, the leapfrog evolution of computer-related technologies and the continuous expansion and innovation of the industry scale, e-commerce has gone from being distrusted at the beginning to being able to shop with just a few clicks. The impact of e-commerce platforms on people's lives is self-evident. According to the 42nd "Statistical Report on the Development of China's Internet", in the first half of 2018, the national online retail sales reached 4.08 trillion yuan, a year-on-year increase of 30.1%, of which the online retail sales of physical goods were 3.13 trillion yuan. The transaction scale of the "Double Eleven" Carnival in 2018 exceeded 200 billion yuan. It can be seen that online shopping, a new retail method, is becoming more and more popular, and online shopping has gradually become an important way for consumers to shop.

[0003] The rapid development of e-commerce has generated a large amount of online review data, where customers can talk about all the features of a product on the Internet platform. In previous e-commerce data mining technologies, product data mining was only performed through the text of customer reviews. With the development of computer technology and computer network technology, users' comment methods have become more diverse. When users give feedback on product information, they often attach pictures, some of which are user buyer shows, and some are problems presented by the product. When users cannot describe the product problem in detail, pictures are often the best means of feedback. With the development of deep learning technology, the development of natural language processing technology and image processing technology has been driven. Image detection and sentiment analysis models based on deep learning have been proposed. Based on the previous use of text to mine information about products, the use of image detection models can analyze and mine pictures in user reviews to increase the information dimension. Summary of the invention:

[0004] The purpose of the present invention is to propose a defect recognition method that integrates image detection and sentiment extraction based on the existing model that only mines defects in product review texts, so as to make fuller use of product feedback information and obtain the types and severity of defects in the products.

[0005] The present invention is achieved through the following technical solutions: an e-commerce clothing image defect recognition method based on deep learning and a product defect sentiment analysis method based on a sentiment dictionary.

[0006] The specific steps include:

[0007] 1. A method for evaluating e-commerce clothing defects based on online review text and pictures. The specific steps are as follows:

[0008] (1) Collect e-commerce customer feedback data and establish a defect dataset. Crawling the clothing review text and corresponding review images on the e-commerce platform, establish a clothing defect image dataset and a corresponding text dataset.

[0009] a. Clothing defect image dataset, including pilling, holes, discoloration, thread breakage, unstitching, wrinkles, and dirt as negative type samples. The negative sample data and normal sample data constitute the image sample dataset in a 1:1 ratio. The data volume is expanded through image cropping, flipping, and adding noise data enhancement operations.

[0010] b. Review text dataset, cleans special characters and meaningless text, and builds a defect vocabulary by annotating defects such as pilling, holes, discoloration, thread opening, thread derailment, wrinkles, and dirt.

[0011] (2) The trained VGG-16 convolutional neural network is used as the feature extraction network. After different layers of convolution calculations, two feature maps containing high-level features and low-level features are extracted from the original image. The feature maps are divided into two paths and shared by the subsequent RPN network and RoIPolling network.

[0012] (3) Train the RPN network to obtain the location of the candidate box, filter the candidate area based on the candidate box, generate a feature map of a fixed size, identify the object category to which the candidate area belongs, and finally regress the precise location of the object.

[0013] (4) Use the trained Faster R-CNN model to detect whether the input image is a clothing defect.

[0014] (5) Segment the text comments, construct repeated strings, perform frequency filtering, mutual information filtering, and left-right entropy filtering on the segmentation results, add new words to the stop word list, and build a sentiment dictionary based on multiple public dictionaries.

[0015] (6) The review text is segmented, short text and images are input to detect defects, such tags are output through the defect vocabulary and sentiment dictionary, and the severity of the defects is obtained through the sentiment calculation formula that integrates the image and text information.

[0016] 2. An e-commerce clothing defect assessment method based on online comment text and pictures, characterized in that, in step (1), the clothing pictures are derived from the clothing store comment areas of Taobao and JD.com, and the scrapy framework is used to crawl the two e-commerce websites to obtain different types of women's tops reviews. The clothing pictures with defects are screened by comment star rating, and the defective clothing pictures are divided into a training set and a test set at a ratio of 8:2. The training set is used to train the defect detection model, and the test set is used to verify the accuracy of the training model. LabelImg is used to annotate the clothing dataset with defects, and the annotation content is the defect name and defect coordinate bbox. The annotated pictures and the corresponding generated xml files are used to create a defective clothing dataset in the COCO dataset format.

[0017] 3. An e-commerce clothing defect assessment method based on online review text and pictures, characterized in that, in step (2), feature maps that have undergone multiple convolutions and a small number of convolutions are used as input to the RPN network, thereby ensuring the extraction of deep features while increasing the ability to detect small defects.

[0018] 4. A method for evaluating e-commerce clothing defects based on online review text and pictures, characterized in that in steps (1) to (4), the Faster R-CNN network is trained using an e-commerce clothing picture training data set to establish a clothing defect recognition model, and the trained model is used to detect whether an input clothing picture has defects and output the location of the calibrated defect in the picture.

[0019] 5. An e-commerce clothing defect assessment method based on online review text and pictures, characterized in that, in step (6), the defect severity ratio of different defects is obtained by hierarchical analysis, and scoring models for evaluating the severity of defects in text and image are established respectively, and the clothing defect severity assessment model based on text and image is obtained by unifying the dimensions. The model is as follows:

[0020]

[0021] Among them, Score(F i ) is the final severity score obtained by the model, Score(F i ,p) is the clothing defect severity score obtained based on the image information, Score(F i ,w) is the clothing defect severity score obtained based on text information, β is the proportion of each factor’s influence on information perception, and α is the ratio of each factor with unified dimensions.

[0022] The beneficial effects of the present invention are:

[0023] 1. Use convolutional neural networks to extract whether e-commerce clothing pictures have defects and the types of defects.

[0024] 2. Compared with the original Faster R-CNN, the accuracy of small object detection is improved.

[0025] 3. Based on the original text-based defect severity assessment method, the use of image data was added, and a model for evaluating defects based on text and images was established to improve the method for evaluating the severity of e-commerce product defects. Description of the drawings:

[0026] Figure 1 1 is a flow chart for implementing the method of the present invention;

[0027] Figure 2 Calculate the flow chart for the evaluation model;

[0028] Figure 3 PR graph for the image detection model; Specific implementation method:

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] like Figure 1 As shown in the figure, an e-commerce clothing defect assessment method based on online review text and pictures is shown in the figure. The specific steps are as follows:

[0031] Step 1: Collect e-commerce customer feedback data and build a defect dataset. Crawl the clothing review text and corresponding review images on the e-commerce platform to build a clothing defect image dataset and a corresponding text dataset.

[0032] a. Clothing defect image dataset, including pilling, holes, discoloration, thread breakage, unthreading, and dirt as negative type samples. The negative sample data and normal sample data constitute the image sample dataset in a 1:1 ratio. The data volume is expanded through image cropping, flipping, and adding noise data enhancement operations.

[0033] b. Review text dataset, cleans special characters and meaningless text, and builds a defect vocabulary by annotating pilling, holes, discoloration, thread breaks, thread derailment, and dirt defects.

[0034] Step 2: Use the trained VGG-16 convolutional neural network as the feature extraction network. After different layers of convolution calculations, two feature maps containing high-level features and low-level features are extracted from the original image. The feature map is divided into two paths and shared by the subsequent RPN network and RoIPolling network. Due to the characteristics of the data set, in some clothing defect pictures, since buyers need to show the overall clothing and the location of the defect, the defect calibration range is too small. The low-level feature map is mainly used to solve the problem that the faster R-CNN network cannot detect small areas.

[0035] Step 3: Train the RPN network, generate 9 anchors for each pixel, and the RPN network is divided into two routes: classification layer and regression layer. In the classification layer, the softmax function is used to judge the foreground and background of the anchor. The detection target is the foreground, and the background (image background) is discarded; in the regression layer, the candidate box position is fitted by adjusting the center coordinates and length and width of the anchor. The Proposal network is responsible for generating proposals (candidate regions) by integrating the foreground and candidate box positions. The input of the RoI Pooling network comes from the feature map and proposals (candidate regions). It outputs a fixed-size RoI feature map as the input of the subsequent fully connected layer.

[0036] Step 4: After the output of the fully connected layer, use the sonftmax function to classify the matrix calculation output by the fully connected layer. At the same time, the fully connected layer outputs the coordinate position bbox of the defect. The training results after 40,000 iterations are as follows:

[0037] Table 1

[0038] Performance evaluation parameters Results of training 40,000 times Classification layer loss 0.024 Regression layer loss 0.155 Total loss 0.185 AP Value 0.7945

[0039] Step 5: Segment the crawled text data set, construct phrases, and calculate new stop words and new words based on word frequency, mutual information, etc. Use CRF to extract product defect features and build a sentiment dictionary based on online open source dictionaries.

[0040] Step 6: Obtain the severity ratio of different defects through hierarchical analysis. After expert analysis, the comparison matrix is ​​shown in Table 2:

[0041] Table 2

[0042]

[0043] The weight Q is obtained after calculation.

[0044] The review text is divided into a set of short sentences according to punctuation marks, each short sentence is input into the sentiment dictionary, and the corresponding defect and mark word class are output. The scoring model for defect severity assessment based on text information is as follows:

[0045]

[0046] Among them, Score(F i ,w) is the defect severity assessment score based on the information in this article, o(x1) is the sentiment word score, is the score of degree adverbs, φ(x3) is the score of negation words, Score the conjunction.

[0047] Considering that a review contains multiple pictures, the defect severity assessment scoring model based on picture information is as follows:

[0048]

[0049] Among them, Score(F i ,p) is the defect severity assessment score based on the image information, Num is the number of images, n j,i is the number of defects i in the jth image.

[0050] After calculation, Score(F i ,w)、Score(F i ,p) value range, let Score(F i ,w)*α w ∈(0,3), Score(F i ,p)*α p ∈(0,3). Based on the research on the perceptual factors of e-commerce reviews, the ratio of the perceptual factors of images and texts to texts is obtained, which is converted into the weights β of each in the score calculation. Finally, the defect severity assessment formula is as follows:

[0051]

[0052] The text extraction information and the image extraction information are calculated through this formula to obtain the product defect severity score.

[0053] The principles and implementation methods of the present invention are described herein, and the description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those of ordinary skill in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for evaluating e-commerce clothing defects based on online review text and pictures, the specific steps are as follows: (1) Collect e-commerce customer feedback data to establish a defect dataset; crawl the clothing review text and corresponding review pictures on the e-commerce platform to establish a clothing defect image dataset and a corresponding text dataset; a. Clothing defect image dataset, including pilling, holes, discoloration, thread breakage, thread unstitching, and dirtiness as negative samples. The negative sample data and normal sample data constitute the image sample dataset in a 1:1 ratio. The data volume is expanded through image cropping, flipping, and adding noise data enhancement operations; b. Review text dataset, clean special characters and meaningless text, and build a defect vocabulary by annotating pilling, holes, discoloration, thread breakage, thread derailment, and dirt defects; (2) Use the trained VGG-16 convolutional neural network as the feature extraction network. After different layers of convolution calculations, two feature maps containing high-level features and low-level features are extracted from the original image. The feature maps are divided into two paths and shared by the subsequent RPN network and RoIPolling network. (3) Train the RPN network to obtain the location of the candidate box, filter the candidate area based on the candidate box, generate a feature map of a fixed size, identify the object category to which the candidate area belongs, and finally regress the precise location of the object; (4) Use the trained Faster R-CNN model to detect whether the input image is a clothing defect; (5) Segment the text comments, construct repeated strings, perform frequency filtering, mutual information filtering, and left-right entropy filtering on the segmentation results, add new words to the stop word list, and build a sentiment dictionary based on multiple public dictionaries; (6) The review text is segmented, short text and images are input to detect defects, such tags are output through the defect vocabulary and sentiment dictionary, and the severity of the defects is obtained through the sentiment calculation formula that integrates the image and text information.

2. According to claim 1, a method for evaluating e-commerce clothing defects based on online review text and pictures is characterized in that: In step (1), the clothing images are sourced from the clothing store review areas of Taobao and JD. The scrapy framework is used to crawl the two e-commerce websites to obtain reviews of different types of women's tops. Clothing images with defects are screened by review star ratings. The defective clothing images are divided into a training set and a test set at a ratio of 8:

2. The training set is used to train the defect detection model, and the test set is used to verify the accuracy of the training model. LabelImg is used to annotate the defects of the clothing dataset, with the annotation content being the defect name and defect coordinate bbox. The annotated images and the corresponding generated xml files are used to create a defective clothing dataset in the COCO dataset format.

3. According to claim 1, a method for evaluating e-commerce clothing defects based on online review text and pictures, characterized in that: In step (2), feature maps that have undergone multiple convolutions and a small number of convolutions are input into the RPN network to ensure the extraction of deep features while increasing the ability to detect small defects.

4. According to claim 1, a method for evaluating e-commerce clothing defects based on online review text and pictures is characterized in that In steps (1) to (4), the Faster R-CNN network is trained using the e-commerce clothing image training dataset to establish a clothing defect recognition model. The trained model is used to detect whether the input clothing image has defects and output the location of the defect in the image.

5. According to claim 1, a method for evaluating e-commerce clothing defects based on online review text and pictures, characterized in that: In step (6), the severity ratio of different defects is obtained through hierarchical analysis, and the scoring models for evaluating the severity of defects based on text and image are established respectively. The clothing defect severity evaluation model based on text and image is obtained by unifying the dimensions. The model is as follows: Among them, Score(F i ) is the final severity score obtained by the model, Score(F i ,p) is the clothing defect severity score obtained based on the image information, Score(F i ,w) is the clothing defect severity score obtained based on text information, β is the proportion of each factor’s influence on information perception, and α is the ratio of each factor with unified dimensions.

Images