Method for obtaining user demands based on user comments
Product attributes are extracted through BiLSTM-CRF model, multi-task text classification model recognizes requirements, combines SVM model to screen comments, and uses CycleGan to convert demand expression, which solves the problem of unspecific user demand information in the existing technology, and realizes the core user demands The accurate identification and efficient collection provide clear guidance for product innovative design.
Patent Information
- Application Number
- CN202411107186.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-17
AI Technical Summary
The existing comment mining methods cannot effectively obtain specific and detailed user requirements information, resulting in a lack of clear guidance during the design process.
The BiLSTM-CRF model is used to extract product attributes, combine the multi-task text classification model and the SVM model to identify user needs, and convert the demand expression through CycleGan to form an easy-to-understand design guidance.
It realizes accurate identification of users' core needs, provides efficient and accurate demand collection methods, assists in product innovation design, and improves the understanding of demand information.
Smart Images
Figure CN119149722A_ABST
Abstract
Description
Technical Field
[0001] The present invention specifically relates to a method for obtaining user needs based on user comments. Background Art
[0002] After purchasing a product, more and more users will post comments on social media and e-commerce platforms about their experience with the product, which contains user needs that are of great value to product innovation, providing the possibility for review feedback data to drive product innovation. However, the user demand information contained in the current conventional review mining methods is too broad and vague, and there is a big difference between the detailed and specific demand information actually needed in the design process, lacking specific directionality and practicality, and unable to provide clear and specific guidance for designers. Summary of the invention
[0003] The present invention provides a method for obtaining user needs based on user comments to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:
[0004] A method for obtaining user needs based on user comments, comprising:
[0005] Get user reviews for a specific product type;
[0006] Extract product attributes from user reviews;
[0007] Segment user reviews by product attributes;
[0008] Identify the most important product attributes;
[0009] Pick out a few reviews related to the most important product attributes;
[0010] Identify user needs from a selected number of reviews.
[0011] Furthermore, the specific method of extracting product attributes from user comments is:
[0012] The BiLSTM-CRF model is used for attribute extraction.
[0013] Furthermore, after extracting product attributes from user comments, the method for obtaining user needs based on user comments further comprises:
[0014] Merge the extracted product attributes.
[0015] Furthermore, the specific method for merging the extracted product attributes is:
[0016] For the extracted product attributes, the cosine value of the semantic vector of the attribute vocabulary is calculated, and those greater than a specific threshold are merged.
[0017] Furthermore, the specific method of segmenting user comments according to product attributes is:
[0018] The BiLSTM-CRF model is used for comment segmentation.
[0019] Furthermore, the specific method for determining the most important product attributes is:
[0020] The segmented comments are input into the multi-task text classification model to obtain the sentiment analysis value of each comment and whether it has product attributes;
[0021] Conduct statistical analysis on each attribute to determine the final type of product attributes;
[0022] Select one product attribute of the type with the highest priority as the most important product attribute.
[0023] Furthermore, if the product attribute of the type with the highest priority is not unique, then the product attribute that is mentioned most times in user comments is selected as the most important product attribute.
[0024] Furthermore, the specific method of identifying user needs from the selected comments is:
[0025] The selected comments are input into the SVM model, and the result of whether each comment belongs to the demand category is output to obtain the demand category comments.
[0026] Furthermore, after obtaining the comments on the demand category, the comments on the demand category are restored to the demand expression.
[0027] Furthermore, the specific method of restoring the demand expression from the demand-type comments is as follows:
[0028] Input the demand-related comments into CycleGan for conversion to obtain demand expressions.
[0029] The benefit of the present invention lies in that the method provided for obtaining user needs based on user comments can accurately identify the core needs of users, and can provide designers with an efficient and accurate method for collecting needs, thereby assisting in the innovative design of products.
[0030] The present invention is also beneficial in that the provided method for obtaining user needs based on user comments can construct user need expressions that meet design needs, making the demand information easier to understand in the actual design process. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0032] Figure 1 is a schematic diagram of a method for obtaining user needs based on user comments of the present invention;
[0033] Figure 2 Schematic diagram of the BiLSTM-CRF model of the present invention;
[0034] Figure 3 is a schematic diagram of a multi-task text classification model of the present invention;
[0035] Figure 4 Schematic diagram of the CycleGan structure of the present invention. DETAILED DESCRIPTION
[0036] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0037] like Figure 1 A method for obtaining user needs based on user comments of the present application is shown, comprising: S1: obtaining user comments of a specific product type. S2: extracting product attributes from user comments. S3: segmenting user comments according to product attributes. S4: determining the most important product attributes. S5: selecting several comments related to the most important product attributes. S6: identifying user needs from the selected comments. It can accurately identify the core needs of users and provide designers with an efficient and accurate method for collecting needs to assist in the innovative design of products. The above steps are described in detail below.
[0038] For step S1: Get user reviews of a specific product type.
[0039] The comment data used in this application comes from the Internet. Since the comment data is distributed in various social media and e-commerce platforms, crawler tools such as Octopus are needed to automatically obtain comment data under specified products and data sources to provide high-quality raw data for subsequent data mining and analysis. The obtained data is as follows:
[0040] C p ={c0,c1,c2,…c i…c n}
[0041] Among them, p is the product type, c i is the i-th comment data, and n is the total number of comments.
[0042] For step S2: extract product attributes from user reviews.
[0043] When users comment on a product, they usually evaluate the product's attributes or express their needs. This application mines and extracts product attributes from product reviews. User reviews are unstructured text descriptions. In this application, natural language processing technology based on deep learning is used to extract attributes. The specific model is the BiLSTM-CRF model. The model structure is as follows: Figure 2 shown.
[0044] Before extracting attributes, the BiLSTM-CRF model needs to be trained. The training data needs to be labeled. Each word in the comment is labeled, with non-attribute words marked as O, attribute start words marked as S, attribute middle words marked as P, and attribute end words marked as E.
[0045] After labeling the data, the BiLSTM-CRF model is trained, and its loss function is as follows:
[0046]
[0047] where X = {x1, x2, ..., x n},Y={y1,y2,...,y n},y i is the i-th output label. yi,i Predict y for the i-th word i Probability value, T yi+1,yi for y i+1 to y i The transition probability.
[0048] In an embodiment of the present application, after extracting product attributes from user comments, the method for obtaining user needs based on user comments further includes: merging the extracted product attributes.
[0049] The specific method for merging the extracted product attributes is: for the extracted product attributes, the semantic vector cosine value of the attribute vocabulary is calculated, and those greater than a specific threshold are merged. Specifically, the trained model is applied to the user review text to mine product attributes to obtain the attribute set M′. Since there are synonyms or near synonyms in the obtained attribute set M′, by calculating the semantic vector cosine value of the vocabulary, when it is greater than a specific threshold, the vocabulary is merged to finally obtain the attribute set M.
[0050] For step S3: user comments are segmented according to product attributes.
[0051] When users comment on products, they always comment on multiple attributes of the product. Therefore, before mining and analyzing comments, it is necessary to segment user comments so that each segmented text is a user's comment on a specific product attribute. Comment segmentation also uses the BiLSTM-CRF model. First, the model needs to be trained. The training data is labeled in the following way: the character at the beginning of the attribute comment is labeled S, the character in the middle of the attribute comment is labeled M, the character at the end of the attribute comment is labeled E, and other irrelevant characters are labeled O. The BiLSTM-CRF model structure is similar to the above one, and the training loss function is not repeated here. The trained model segments all comment data to obtain the segmented comment set C′.
[0052] For step S4: determine the most important product attributes.
[0053] In the implementation of this application, the specific method for determining the most important product attributes is:
[0054] The segmented comments are input into the multi-task text classification model to obtain the sentiment analysis value of each comment and whether it has product attributes.
[0055] Perform statistical analysis on each attribute to determine the final type of product attributes.
[0056] Select one product attribute of the type with the highest priority as the most important product attribute.
[0057] In an implementation of the present application, if the product attribute of the type with the highest priority is not unique, then the product attribute that is mentioned most times in user comments is selected as the most important product attribute.
[0058] It is understandable that in order to dig out the most important product attributes, the product attributes need to be classified and prioritized. It is mainly based on analyzing the impact of user needs on user satisfaction. This application adopts the KANO model, which divides product attributes into basic (essential), expected (willing), exciting (charming), indifferent, and reverse types. In order to distinguish which type the product attributes belong to, it is necessary to perform sentiment analysis on product reviews, which is mainly divided into several emotional levels such as high satisfaction, high satisfaction, zero satisfaction, low satisfaction, and low satisfaction. Product reviews are comments on whether they have a certain product attribute or not, and they need to be classified and distinguished. This application uses a multi-task text classification model to simultaneously realize the classification of the sentiment of the comments and whether they have product attributes. The model structure is as follows: Figure 3 shown.
[0059] Where the input (x1, x2, x3...xT ) is the comment text, y m is the sentiment classification output, and its value is {0,1,2,3,4}, where 0 means low satisfaction, 1 means low satisfaction, 2 means zero satisfaction, 3 means high satisfaction, and 4 means high satisfaction. n Whether the product attribute is present, its value is {0,1}, 0 means not present, 1 means present.
[0060] The training error function is as follows:
[0061]
[0062] in:
[0063] λ m ,λ n is the weight coefficient. (m) ,y′ (n) is the true class of the annotation.
[0064] After training is completed, the segmented comments are input into the model to obtain the sentiment analysis value of each comment and whether it has product attributes. According to the KANO model, statistical analysis is performed on each attribute, and the analysis and calculation are performed according to the following table:
[0065] Table 1 Attribute statistical analysis table
[0066]
[0067] By quantifying the table, the numbers in the corresponding type of table cells are added up to obtain the scores of the corresponding types, and the one with the highest score is the final type of the product attribute.
[0068] Through analysis, the needs of essential product attributes should be met first, followed by expected and exciting attributes. If there are multiple essential product attributes at the same time, the product attribute with the most mentions in user comments is selected, and the attribute with the highest frequency in all comments is extracted from the attribute set M as the most important product attribute (M1). The specific formula is as follows:
[0069]
[0070] Where T is the total number of attributes, M k is the kth attribute.
[0071] For step S5: select a number of reviews related to the most important product attributes.
[0072] Filter out S reviews containing product attribute M1 from all reviews.
[0073] For step S6: identify user needs from the selected comments.
[0074] In the implementation of the present application, the specific method for identifying user needs from the selected comments is:
[0075] The selected comments are input into the SVM model, and the result of whether each comment belongs to the demand category is output to obtain the demand category comments.
[0076] Since not all comments express needs, comments need to be classified. This application uses the SVM model to classify comments into two categories: the demand category is 1 and the non-demand category is -1.
[0077] Specifically, the comments are first preprocessed, including breaking down the comments into words and removing pause words through the Chinese word segmentation algorithm. The comments are then vectorized. This patent uses the TF-IDF algorithm to extract text features to achieve vectorization. The TF-IDF formula is as follows:
[0078] tfidf i,j =tf i,j ×idf i,j
[0079] where tf i,j for:
[0080]
[0081] Among them, n i,j The feature word t i In the comments j The number of times it appears in the comment d j The number of all feature words in idf. i,j for:
[0082]
[0083] Among them, |D| represents the total number of comments, |D ti | indicates that the comment contains the characteristic word t i The number of
[0084] After the review vectorization is completed, the SVM model is used for training, and its optimization formula is as follows:
[0085]
[0086] By solving the maximum and minimum values, the SVM weight coefficient is solved:
[0087]
[0088] Where w, b are the hyperplane coefficients, x iis the feature vector of the ith comment, y i is the classification value of the ith comment, λ is the Lagrange multiplier, and N is the total number of training comments.
[0089] After the training is completed, the SVM model is used to judge whether the comments related to the selected M1 attribute are needs, and the comments judged to be needs are selected.
[0090] In the implementation of the present application, after obtaining the reviews of the demand category, the reviews of the demand category are restored to the demand expression. Specifically, the specific method of restoring the reviews of the demand category to the demand expression is: inputting the reviews of the demand category into CycleGan for conversion to obtain the demand expression.
[0091] Since users tend to express their needs in a casual way, which is not easy for designers to understand, this application converts informal demand expressions into formal demand expressions so that designers can accept them more easily. When converting demand expressions from comments, on the one hand, the semantics of the demand expressed in the comments should be maintained, and on the other hand, the expression should be closer to formalization. This application uses CycleGan to convert the style of user comments. The overall flow chart of the algorithm is as follows: Figure 4 shown.
[0092] CycleGan consists of two Gan networks, which are used to generate and train real demand expression text and user comment text respectively. The model loss function is as follows:
[0093] L GAN (G,D Y ,X,Y)=E Y～pdata(y) [logD Y (y)]+E X～pdata(X) [log(1-D Y (G(x)))]
[0094] L GAN (F,D X ,Y,X)=E X～pdata(x) [logD X (x)]+E Y～pdata(y) [log(1-D X (G(y)))]
[0095] L cyc (G,F)=E X～pdata(x) ||F(G(x))-x||1+E Y～pdata(y) ||G(F(y))-y||1
[0096] L(G,F,D X ,D Y )=L GAN (G,D Y,X,Y)+L GAN (F,D X ,Y,X)+λL cyc (G,F)
[0097]
[0098] Among them, G is the generative network that converts user comments into demand expressions, x is the user comment, and D X is the discriminant network for converting comments into demand expressions. F is the generative network for converting demand expressions into comments, y is the real demand expression, and D Y Discriminative network for converting demand expressions into reviews.
[0099] After training CycleGan, the selected comments can be input into the G network to convert them into demand expressions that designers can understand.
[0100] Through the above steps, the method of this application can transform user comments into user needs, and then provide suggestions and assistance to designers for product innovation.
[0101] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.
Claims
1. A method for obtaining user needs based on user comments, characterized in that: Include: Get user reviews for a specific product type; Extract product attributes from user reviews; Segment user reviews by product attributes; Identify the most important product attributes; Pick out a few reviews related to the most important product attributes; Identify user needs from a selected number of reviews.
2. The method for obtaining user needs based on user comments according to claim 1, characterized in that: The specific method for extracting product attributes from user comments is: The BiLSTM-CRF model is used for attribute extraction.
3. The method for obtaining user needs based on user comments according to claim 1, characterized in that: After extracting product attributes from user comments, the method for obtaining user needs based on user comments further includes: Merge the extracted product attributes.
4. The method for obtaining user needs based on user comments according to claim 3 is characterized in that: The specific method for merging the extracted product attributes is: For the extracted product attributes, the cosine value of the semantic vector of the attribute vocabulary is calculated, and those greater than a specific threshold are merged.
5. The method for obtaining user needs based on user comments according to claim 1, characterized in that: The specific method of segmenting user comments according to product attributes is as follows: The BiLSTM-CRF model is used for comment segmentation.
6. The method for obtaining user needs based on user comments according to claim 5, characterized in that: The specific method for determining the most important product attributes is: The segmented comments are input into the multi-task text classification model to obtain the sentiment analysis value of each comment and whether it has product attributes; Conduct statistical analysis on each attribute to determine the final type of product attributes; Select one product attribute of the type with the highest priority as the most important product attribute.
7. The method for obtaining user needs based on user comments according to claim 6, characterized in that: If the product attribute of the highest priority type is not unique, the product attribute that is mentioned most times in user reviews is selected as the most important product attribute.
8. The method for obtaining user needs based on user comments according to claim 1, characterized in that: The specific method of identifying user needs from the selected comments is: The selected comments are input into the SVM model, and the result of whether each comment belongs to the demand category is output to obtain the demand category comments.
9. The method for obtaining user needs based on user comments according to claim 8, characterized in that: After obtaining the comments on demand, restore the comments on demand to demand expression.
10. The method for obtaining user needs based on user comments according to claim 9, characterized in that: The specific method of restoring the demand expression from the demand class comments is: Input the demand-related comments into CycleGan for conversion to obtain demand expressions.