Automatic labeling method for quality problem scenario labels based on category
Through the automatic labeling method of category-based quality problem scenario labels, the problem that multi-classification algorithm is difficult to expand after the label category is determined in massive corpus, and the clear labeling and cause analysis of specific product quality problems is realized, which improves the scalability of the model and wide application.
Patent Information
- Application Number
- CN202011519354.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-21
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-12-21
AI Technical Summary
When the existing multi-classification algorithms process massive after-sales chat corpus, it is difficult to expand after the label category is determined, the model is poor in scalability, and it is impossible to effectively deal with scenarios with unknown classification quantities, and the quality problems of specific products cannot be determined.
Using the automatic labeling method of quality problem scenario labels based on category, through quality descriptor mining, product category construction, product and quality descriptor mapping, classification label annotation and database construction, a basic support database is formed to judge the modification relationship between product and quality descriptor , add and change classification tags flexibly.
It realizes clear labeling and cause analysis of specific product quality problems. The model execution process can be explained, and the classification labels can be flexibly added and changed. It is suitable for scenarios with unknown classification numbers, improving the breadth and efficiency of applications.
Smart Images

Figure CN112579776B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing technology, and in particular to an automatic labeling method for quality problem scenario labels based on categories. Background Art
[0002] With the development of network technology, various industries have chosen to conduct e-commerce transactions through the Internet. The advantage is that transactions are no longer limited to limited space and time, which greatly improves efficiency. E-commerce often has its corresponding after-sales customer service, which can notify merchants to improve and perfect the corresponding products based on user feedback on product quality issues, and finally form a complete virtuous cycle chain. However, if the quality problems of different products are extracted from the massive after-sales chat corpus by manual means alone, it is not only inefficient, but also requires a lot of time and labor costs. Regarding the automatic extraction of product quality problems, the classification of product quality problems in massive texts can be automatically identified by classification. The advantage is that it can replace manual work and process large quantities of after-sales chat corpus. However, the disadvantages are also obvious, mainly manifested in the number of classification labels for quality problems has been determined. When a new classification of quality problems needs to be added, the model needs to be readjusted and the training corpus needs to be added for retraining. However, the operation is cumbersome due to the complexity of the model, that is, the scalability of the model is poor.
[0003] Neural network algorithms are usually used for multi-classification of text. The multi-classification problem of the neural network algorithm must first obtain training data through a large number of annotated quality problems, then confirm the label category of the quality problem through the annotated data, and finally transform the output layer of the neural network so that it no longer outputs a unique probability, but instead corresponds to a probability for each label category. A probability greater than 0.5 indicates that the input text contains the quality problem, otherwise it indicates that the input text does not contain the quality problem.
[0004] Multi-classification algorithms such as neural network algorithms will encounter the same problem, that is, the label category must be determined at the beginning. Otherwise, if labels are added after training is completed, the model needs to be modified and retrained, which is very troublesome. Moreover, when annotating the corpus, it is not certain whether all labels of quality problems have been exhausted. Therefore, for application scenarios where it is uncertain whether to add label categories, conventional multi-classification methods will appear somewhat powerless. In addition, conventional multi-classification algorithms can only determine which quality problems are contained in the input text, but cannot determine which specific products in the text these quality problems belong to. Summary of the invention
[0005] An object of the present invention is to solve at least the above problems and to provide at least the advantages which will be described hereinafter.
[0006] Another object of the present invention is to provide an automatic labeling method for quality problem scenario labels based on categories, which can determine the quality problems of specific commodities according to the quality descriptors corresponding to the commodities, avoid the application of label exhaustive mode, and can freely and flexibly increase label categories, meeting the application of classification scenarios with unknown number of categories.
[0007] In order to achieve these purposes and other advantages according to the present invention, a method for automatically labeling quality problem scenario labels based on categories is provided, comprising the following steps:
[0008] S1. Mining quality description words: Based on a pre-built basic dictionary that stores basic description words, all words in the known corpus are compared with the basic description words as candidate words, quality description words are screened out from the candidate words, and the quality description words are added to the basic dictionary to gradually expand the basic dictionary and form a quality description word library;
[0009] S2. Commodity category construction: Classify commodities into commodity categories;
[0010] S3, mapping of goods and quality descriptors: traverse the known corpus and map each product to a quality descriptor through the descriptive relationship between the goods and quality descriptors;
[0011] S4. Classification labeling of quality descriptors: taking commodity categories as units, label each quality descriptor under the commodity category with a corresponding classification label;
[0012] S5. Construction of basic support database: The commodities, commodity categories to which the commodities belong, and quality descriptors marked with classification labels under the commodity categories are stored in the form of a graph structure to form a basic support database;
[0013] S6, determining the modification relationship between the commodity and the quality descriptor: judging whether the commodity and the quality descriptor in the input text are in a modification relationship by the distance between the commodity name and the quality descriptor; if the distance is less than a predetermined threshold M, it is determined that the commodity and the quality descriptor are in a modification relationship, and the process proceeds to S7; otherwise, the input text is discarded;
[0014] S7. Confirmation of product quality issues: Find the classification label corresponding to the product contained in the input text in the basic support database constructed by S5.
[0015] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, before S1, it also includes:
[0016] S0. Unknown word mining: Use NLP to mine known corpus to discover unknown words; wherein, the unknown words refer to new words developed over time and proprietary words in a specific field.
[0017] Preferably, in the automatic labeling method of the quality problem scenario label based on the category, all the words in the known corpus are compared with the basic descriptive words as candidate words, the quality descriptive words in the candidate words are screened out, and the quality descriptive words are added to the basic dictionary to gradually expand the basic dictionary to form a quality description word library, which includes: after removing the entity nouns in all the words in the known corpus, the remaining words are used as candidate words and the basic descriptive words are respectively calculated with cosine similarity to obtain a similarity set between each candidate word and the basic descriptive word, when the number of words in the similarity set that are greater than a preset threshold T exceeds a preset limit value N, it is determined that the candidate word corresponding to the similarity set is a quality descriptive word, and the quality descriptive word is added to the basic dictionary to gradually expand the basic dictionary to form a quality description word library;
[0018] Among them, X cycles are preset to screen out quality descriptors from the candidate words; T and N are set to high values in the first X-1 cycles, and the quality descriptors screened out by the high values of T and N are directly added to the basic dictionary, and T and N are set to low values before the Xth cycle, and the candidate words are compared with the basic descriptors in the basic dictionary to which the quality descriptors screened out in the first X-1 cycles have been added.
[0019] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, classifying the commodities into commodity categories in S2 specifically includes:
[0020] The required product names and corresponding categories are obtained from the existing e-commerce platform, and then the categories are adjusted to form the product categories.
[0021] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, traversing the known corpus in S3 and corresponding each commodity with a quality descriptor through the descriptive relationship between the commodity and the quality descriptor specifically includes:
[0022] Through dependency syntax analysis, determine whether the product and quality descriptor are in a modification relationship:
[0023] S3-1, perform syntactic analysis on the input text to generate a primary syntactic tree;
[0024] S3-2, determining one by one whether the primary syntax tree includes other subject-predicate relationships except the corresponding relationship between the commodity and the quality descriptor, and whether there is a parallel relationship with the components in other subject-predicate relationships; if yes, the syntax tree is cut according to the subject-predicate relationship to form a final syntax tree; if no, the syntax tree is used as the final syntax tree;
[0025] S3-3. Determine whether the distance between the product and the quality descriptor on the final syntax tree is less than a preset distance threshold; if yes, determine that there is a modification relationship between the product and the quality descriptor; if no, determine that there is no modification relationship between the product and the quality descriptor.
[0026] Preferably, in the automatic labeling method of quality problem scenario labels based on categories, in S4, before labeling each quality descriptor under the commodity category with a corresponding classification label, the quality descriptors that have appeared under each commodity category are counted, and the quality descriptors are combined with the characteristics of the commodity category and all possible quality problems under the commodity category to form a classification label.
[0027] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, in S5, the goods, the commodity categories to which the goods belong, and the quality descriptors labeled with classification labels under the commodity categories are stored in the form of a graph structure. Before forming a basic support database, the classification labels corresponding to all commodities under the commodity category need to be shared with the quality descriptors, specifically including:
[0028] S5-1. Count all possible quality problems of all products in each product category, and map the products in the product category to classification labels using quality descriptors;
[0029] S5-2. The quality descriptors under the same classification label in each product category are shared among all products under the corresponding product category.
[0030] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, in S5, the graph structure includes the following two connection relationships:
[0031] Connection relationship 1: the sequential connection relationship between products, product categories, classification labels, and quality descriptors; and
[0032] Connection relationship 2: The connection relationship between product categories and corresponding quality descriptors.
[0033] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, in S6, before determining the modification relationship between the product and the quality descriptor, it is necessary to find the parallel relationship between the products and confirm the validity of the quality descriptor, which specifically includes:
[0034] S6-1. Parallel relationship search between commodities: The basic support database is used to search for commodities of known commodity categories in the input text after word segmentation, and when multiple commodities are retrieved, it is determined whether there are conjunctions between the names of the multiple commodities that indicate parallel meanings. If yes, the multiple commodities are classified as parallel commodities; if not, it is determined that the input text contains multiple commodities; and when only one commodity is retrieved in the input text or the multiple commodities retrieved are parallel commodities, and the distance between the name of the commodity or any commodity in the parallel commodities and the quality descriptor is less than M, it is determined that the quality descriptor and the commodity or all commodities in the parallel commodities are in a modifying relationship; if the input text contains multiple commodities, the input text is segmented according to punctuation marks, and the matching operation of commodities and quality descriptors is performed in sequence to determine the quality descriptor that has a modifying relationship with each commodity;
[0035] S6-2. Confirmation of the validity of quality descriptors: The quality description word library is used to search whether the input text contains quality descriptors. If the input text contains quality descriptors, the validity of the quality descriptors contained in the input text is determined by a preset negative word library. If the quality descriptors are preceded or followed by any negative word stored in the negative word library, the quality descriptors are determined to be invalid.
[0036] Preferably, in the automatic labeling method for quality problem scenario labels based on categories, the classification labels corresponding to the commodities contained in the input text found in the basic support database constructed by S5 in S7 specifically include:
[0037] S7-1, determining the product category of the product contained in the input text through the connection relationship between the product and the product category in the connection relationship 1;
[0038] S7-2, judging through connection relationship 2 whether the quality descriptor in the commodity and quality descriptor with a modification relationship in the input text is a quality descriptor under the commodity category determined in S7-1; if yes, proceeding to S7-3; otherwise, judging that the modification relationship between the commodity and the quality descriptor contained in the input text is wrong, and discarding the corresponding modification relationship;
[0039] S7-3. Determine all classification labels under the commodity category determined in S7-1 through the sequential connection relationship among the commodity category, classification label and quality descriptor in connection relationship 1, and select from all the classification labels the classification labels that have a mapping relationship with the commodity having a modification relationship in the input text and the quality descriptor in the quality descriptor.
[0040] The present invention has at least the following beneficial effects:
[0041] The automatic labeling method of the quality problem scenario label based on the category of the present invention can not only complete the classification task, but also conduct a clear reason analysis to determine why the classification label is selected, and the execution process of the model is understandable and explainable. At the same time, the goods will be extracted during the classification process, and the classification label is determined based on the goods, that is, the quality problem finally determined is the quality problem of the specific goods, which can help users fully understand the quality problems encountered by the goods, and facilitate the improvement of the specific goods in the subsequent production and sales process.
[0042] The ability to flexibly add and change classification labels avoids the drawback of traditional models where classification labels cannot be changed once model training is completed, and has a wider range of application scenarios.
[0043] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a flow chart of the automatic labeling method of the quality problem scenario label based on the category according to the present invention;
[0045] Figure 2 This is a relationship diagram of labeling using large green mango, winter jujube and oven as examples, using the automatic labeling method for quality problem scenario labels based on categories described in this application. DETAILED DESCRIPTION
[0046] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.
[0047] It should be understood that terms such as “having”, “including” and “comprising” used herein do not exclude the existence or addition of one or more other elements or combinations thereof.
[0048] like Figure 1 As shown, the present invention provides an automatic labeling method for quality problem scenario labels based on categories, comprising the following steps:
[0049] S1. Mining quality description words: Based on a pre-built basic dictionary that stores basic description words, all words in the known corpus are compared with the basic description words as candidate words, quality description words are screened out from the candidate words, and the quality description words are added to the basic dictionary to gradually expand the basic dictionary and form a quality description word library;
[0050] S2. Commodity category construction: Classify commodities into commodity categories;
[0051] S3, mapping of goods and quality descriptors: traverse the known corpus and map each product to a quality descriptor through the descriptive relationship between the goods and quality descriptors;
[0052] S4. Classification labeling of quality descriptors: taking commodity categories as units, label each quality descriptor under the commodity category with a corresponding classification label;
[0053] S5. Construction of basic support database: The commodities, commodity categories to which the commodities belong, and quality descriptors marked with classification labels under the commodity categories are stored in the form of a graph structure to form a basic support database;
[0054] S6, determining the modification relationship between the commodity and the quality descriptor: judging whether the commodity and the quality descriptor in the input text are in a modification relationship by the distance between the commodity name and the quality descriptor; if the distance is less than a predetermined threshold M, it is determined that the commodity and the quality descriptor are in a modification relationship, and the process proceeds to S7; otherwise, the input text is discarded;
[0055] S7. Confirmation of product quality issues: Find the classification label corresponding to the product contained in the input text in the basic support database constructed by S5.
[0056] In the above scheme, the automatic labeling method of the quality problem scenario label based on category actually includes two stages. The first is to build a basic support database, and the second is to classify the product quality problems using the built basic support database. Specifically, it is necessary to manually build a small basic dictionary of descriptive words, and then expand the descriptive words with the basic dictionary as the core. Then, the products are obtained from the known corpus, and the products are classified into multiple product categories. Then, the known corpus is traversed, and each product is matched with the quality descriptor through the descriptive relationship between the product and the quality descriptor. Then, taking the product category as the unit, for each quality descriptor under the product category, The corresponding classification labels are added, and finally the goods, the commodity categories to which the goods belong, and the quality descriptors marked with the classification labels under the commodity categories are stored in the form of a graph structure to form a basic support database; then, the distance between the name of the goods and the quality descriptors is used to determine whether there is a modification relationship between the goods and the quality descriptors in the input text. If the distance is less than a predetermined threshold M, it is determined that there is a modification relationship between the goods and the quality descriptors, otherwise the input text is discarded; finally, the classification label corresponding to the goods contained in the input text is found in the constructed basic support database, that is, the specific quality problem of the goods is determined.
[0057] In addition to completing the classification task, this method can also conduct a clear cause analysis to determine why the classification label is selected. The execution process of the model is understandable and explainable. At the same time, the goods will be extracted during the classification process, and the classification label is determined based on the goods. In other words, the quality problem finally determined is the quality problem of the specific goods, which can help users fully understand the quality problems encountered by the goods and facilitate the improvement of specific goods in the subsequent production and sales process.
[0058] Secondly, this model can flexibly add and change classification labels, avoiding the disadvantage of traditional models that classification labels cannot be changed once model training is completed, and its application scenarios are more extensive.
[0059] In a preferred solution, before S1, the following is also included:
[0060] S0. Unknown word mining: Use NLP to mine known corpus to discover unknown words; wherein, the unknown words refer to new words developed over time and proprietary words in a specific field.
[0061] In the above scheme, taking quality description words as an example, users often use many colloquial description words in chat corpus. These words do not appear in the vocabulary of general fields, but contain specific meanings, so they need to be extracted by means of new word (unregistered word) discovery. The quality description words are extracted through the subsequent mining means, making the quality description words included in the known corpus more comprehensive, and thus making the quality description words covered in the formed quality description vocabulary more comprehensive, so that the quality description words extracted from the input text are more accurate and comprehensive. At the same time, after the new word is discovered, the text is cut into words through word segmentation, and the words are mapped to high-dimensional space to form word vectors through Word2vec training, so as to reflect the semantic relevance of words with spatial distance, thereby turning the study of word semantics to a simpler study of spatial distance.
[0062] In a preferred solution, all the words in the known corpus are compared with the basic descriptive words as candidate words, quality descriptive words among the candidate words are screened out, and the quality descriptive words are added to the basic dictionary to gradually expand the basic dictionary to form a quality description word library, which includes: after removing entity nouns from all the words in the known corpus, the remaining words are used as candidate words and the basic descriptive words are respectively calculated with cosine similarity to obtain a similarity set between each candidate word and the basic descriptive word, when the number of words in the similarity set that are greater than a preset threshold T exceeds a preset limit value N, it is determined that the candidate word corresponding to the similarity set is a quality descriptive word, and the quality descriptive word is added to the basic dictionary to gradually expand the basic dictionary to form a quality description word library;
[0063] Among them, X cycles are preset to screen out quality descriptors from the candidate words; T and N are set to high values in the first X-1 cycles, and the quality descriptors screened out by the high values of T and N are directly added to the basic dictionary, and T and N are set to low values before the Xth cycle, and the candidate words are compared with the basic descriptors in the basic dictionary to which the quality descriptors screened out in the first X-1 cycles have been added.
[0064] In the above scheme, all words in the chat corpus are used as candidate words, and entity nouns such as names of people, places, and products are removed from the candidate words to eliminate interference items. The remaining candidate words are respectively subjected to cosine similarity calculation with the descriptive words in the basic dictionary.
[0065] Simi(word1, ..., wordk) represents the similarity set between the ith candidate word and all basic descriptive words. When the number of words in the set Simi(word1, ..., wordk) that are greater than the threshold T exceeds N, it means that the candidate word is a quality descriptor. You can customize X loops as needed. For the first X-1 loops, you can set the thresholds T and N a little higher. The screened quality descriptors can be directly added to the basic dictionary, and the new basic dictionary will be used for calculation in the next loop. For the last loop, you can appropriately adjust the thresholds T and N slightly lower in order to recall as many descriptive words with quality issues as possible.
[0066] In a preferred solution, the classification of commodities into commodity categories in S2 specifically includes:
[0067] The required product names and corresponding categories are obtained from the existing e-commerce platform, and then the categories are adjusted to form the product categories.
[0068] In the above scheme, the required product names and corresponding categories are obtained from existing e-commerce platforms such as Taobao, and then some categories are manually adjusted. For example, categories such as snacks, food, and meals can be merged to reduce the subsequent workload. Some categories are not reasonably classified and need to be adjusted. The outdoor fitness category includes road bikes, combat pants, mountaineering bags, etc. Obviously, it is unreasonable to classify these things in the same category. Through manual correction, road bikes are classified under the bicycle category, combat pants belong to the clothing category, mountaineering bags belong to the luggage category, and so on, so as to quickly obtain reasonable product categories.
[0069] In a preferred solution, traversing the known corpus in S3 and matching each product with a quality descriptor through the description relationship between the product and the quality descriptor specifically includes:
[0070] Through dependency syntax analysis, determine whether the product and quality descriptor are in a modification relationship:
[0071] S3-1, perform syntactic analysis on the input text to generate a primary syntactic tree;
[0072] S3-2, determining one by one whether the primary syntax tree includes other subject-predicate relationships except the corresponding relationship between the commodity and the quality descriptor, and whether there is a parallel relationship with the components in other subject-predicate relationships; if yes, the syntax tree is cut according to the subject-predicate relationship to form a final syntax tree; if no, the syntax tree is used as the final syntax tree;
[0073] S3-3. Determine whether the distance between the product and the quality descriptor on the final syntax tree is less than a preset distance threshold; if yes, determine that there is a modification relationship between the product and the quality descriptor; if no, determine that there is no modification relationship between the product and the quality descriptor.
[0074] In the above scheme, the text is first parsed and a syntax tree is generated. The distance between the product and the descriptive word on the syntax tree is judged. The closer the distance, the more likely it is a modification relationship. If the text after syntax analysis contains other SBV subject-predicate relationships in addition to the relationship between the product and the descriptive word, and there is a COO parallel relationship with the components in the previous relationship, the syntax tree that meets the conditions will be cut according to the subject-predicate relationship to form two or more new syntax trees. Finally, the distance between the product and the descriptive word is judged on the new syntax tree. The closer the distance, the modification relationship is formed. Finally, all the corpus is traversed to count the quality descriptive words corresponding to all the products through the description relationship between the product and the descriptive word.
[0075] In a preferred solution, S4 takes commodity categories as units, and before assigning corresponding classification labels to each quality descriptor under the commodity category, it also counts the quality descriptors that have appeared under each commodity category, and combines the quality descriptors with the characteristics of the commodity category and all possible quality problems under the commodity category to form classification labels.
[0076] In the above scheme, statistics are performed with categories as the core. The descriptive words that have appeared under each category are counted, and classification labels are formed by combining the characteristics of the combination of descriptive words and the category and all possible quality problems under the category. Under the category, corresponding classification labels are manually marked for each quality descriptor.
[0077] In a preferred solution, in S5, the goods, the commodity categories to which the goods belong, and the quality descriptors marked with classification labels under the commodity categories are stored in the form of a graph structure. Before forming a basic support database, the classification labels corresponding to all goods under the commodity category need to be shared with the quality descriptors, specifically including:
[0078] S5-1. Count all possible quality problems of all products in each product category, and map the products in the product category to classification labels using quality descriptors;
[0079] S5-2. The quality descriptors under the same classification label in each product category are shared among all products under the corresponding product category.
[0080] In the above scheme, firstly, all possible quality problems of all commodities under the category are counted, and each commodity is mapped to a classification label through the descriptor of the quality problem, that is, each classification label reflects the quality problem of a commodity, and then all classification labels contained in each category can be determined, and the quality descriptors covered by all classification labels under each category can also be determined; secondly, the descriptors under the same classification label in each category are shared with the commodities under the category. For example: commodity x in the same category has not had a modification relationship with descriptor a, but commodity x has had a quality problem with classification label A, and descriptor a has a mapping relationship with classification label A in this category, then we believe that commodity x and descriptor a can be modified and mapped to the quality problem represented by classification label A. That is, it makes up for the lack of modification relationship between commodities and quality descriptors due to insufficient corpus.
[0081] In a preferred solution, in S5, the graph structure includes the following two connection relationships:
[0082] Connection relationship 1: the sequential connection relationship between products, product categories, classification labels, and quality descriptors; and
[0083] Connection relationship 2: The connection relationship between product categories and corresponding quality descriptors.
[0084] In the above scheme, connection relationship 2 can be used to determine all quality descriptors under the specified category, connection relationship 1 can be used to determine the category of the product through the "product->category" relationship, and all classification labels under the specified category can be determined through "category->classification label". Furthermore, the classification labels corresponding to all quality descriptors under the specified category can be determined through connection relationships 1 and 2, and the combined effect of connection relationships 1 and 2 can also determine the classification labels through the modification relationship between the product and the quality descriptor, thereby determining the real quality problem of the product.
[0085] In a preferred solution, in S6, before determining the modification relationship between the product and the quality descriptor, it is necessary to find the parallel relationship between the products and confirm the validity of the quality descriptor, which specifically includes:
[0086] S6-1. Parallel relationship search between commodities: The basic support database is used to search for commodities of known commodity categories in the input text after word segmentation, and when multiple commodities are retrieved, it is determined whether there are conjunctions between the names of the multiple commodities that indicate parallel meanings. If yes, the multiple commodities are classified as parallel commodities; if not, it is determined that the input text contains multiple commodities; and when only one commodity is retrieved in the input text or the multiple commodities retrieved are parallel commodities, and the distance between the name of the commodity or any commodity in the parallel commodities and the quality descriptor is less than M, it is determined that the quality descriptor and the commodity or all commodities in the parallel commodities are in a modifying relationship; if the input text contains multiple commodities, the input text is segmented according to punctuation marks, and the matching operation of commodities and quality descriptors is performed in sequence to determine the quality descriptor that has a modifying relationship with each commodity;
[0087] S6-2. Confirmation of the validity of quality descriptors: The quality description word library is used to search whether the input text contains quality descriptors. If the input text contains quality descriptors, the validity of the quality descriptors contained in the input text is determined by a preset negative word library. If the quality descriptors are preceded or followed by any negative word stored in the negative word library, the quality descriptors are determined to be invalid.
[0088] In the above scheme, the conjunctions that represent parallel meanings include "and", "with", "with", "same", "and", "or", and "and", etc. If there is a parallel relationship between commodities, the parallel commodities are in the same position in the following operation, that is, equivalent to the same commodity. If there is only one commodity name or a parallel relationship consisting of multiple commodity names in the input text, the description word and the unique commodity name or all commodity names in the parallel relationship have a modification relationship. If there are multiple commodity names in the input text (all commodities in the parallel relationship are counted as one), the input text is divided according to punctuation marks, and the matching operation of commodities and description words is performed in sequence. First, the distance between each description word and the commodity in the first small block of text is calculated. If the distance meets the threshold M, the description word belongs to the commodity with the closest distance. That is to say, a description word can only modify one commodity (except for commodities in a parallel relationship), and a commodity can be modified by multiple description words. After the calculation of the first text block is completed, the same method is used to calculate the next text block. At the same time, the description words in the first text block are no longer calculated, but the commodities will still be added to the next calculation. If there is no product in the first small block of text, the descriptive word is immediately combined with the descriptive word in the next text block to perform distance calculation with the product in the text block to finally determine the modification relationship; then, the accumulated quality description word library is used to retrieve whether the text contains quality descriptive words. If the text contains descriptive words, a negative word library is also needed to judge the validity. For example, the quality descriptive word "broken" is a description of the classification label of "broken and deformed", but if the input text is modified by the negative word "no" before "broken", it means that "no breakage" is no longer a quality problem, so the quality descriptive word here is an invalid descriptive word. The composition of the negative word library includes explicit negative words such as "no", "not", "no", "cannot", "no longer", "no more", etc., among which the explicit negative words are divided into two parts according to the modification position. For example, negative words such as "no" are generally modified before the descriptive word, while "no more" and so on are modified after the descriptive word. Implicit negative words include: "beware", "avoid", "eliminate", "prevent", "clear", etc. Although these words are not negative words, they express negative meanings, which can further ensure the accuracy of quality description word extraction.
[0089] In a preferred solution, the classification label corresponding to the commodity contained in the input text is found in the basic support database constructed by S5 in S7 specifically includes:
[0090] S7-1, determining the product category of the product contained in the input text through the connection relationship between the product and the product category in the connection relationship 1;
[0091] S7-2, judging through connection relationship 2 whether the quality descriptor in the commodity and quality descriptor with a modification relationship in the input text is a quality descriptor under the commodity category determined in S7-1; if yes, proceeding to S7-3; otherwise, judging that the modification relationship between the commodity and the quality descriptor contained in the input text is wrong, and discarding the corresponding modification relationship;
[0092] S7-3. Determine all classification labels under the commodity category determined in S7-1 through the sequential connection relationship among the commodity category, classification label and quality descriptor in connection relationship 1, and select from all the classification labels the classification labels that have a mapping relationship with the commodity having a modification relationship in the input text and the quality descriptor in the quality descriptor.
[0093] In the above scheme, the quality descriptors and the goods they modify can be found from the input text through the above steps. The goods and descriptors with modification relationships are input. First, the category of the goods is determined through the graph relationship (connection relationship 2) "goods->category"; then the graph relationship (connection relationship 1) "category->quality descriptor" is used to confirm whether the descriptor is a real modifier under the category. If the descriptor is in the graph relationship, the next step can be directly performed. If the descriptor is not in the graph relationship, it means that the descriptor cannot represent the quality problem of this category. It also means that the modification relationship between the goods and the descriptor is misjudged. It is necessary to abandon the goods and descriptors with this modification relationship in time. Finally, all the classification labels under the category of the goods are determined through the graph relationship (connection relationship 1) "category->classification label->quality descriptor". Among these classification labels, the classification label that has a mapping relationship with the input quality descriptor is the final result we need.
[0094] Example
[0095] like Figure 2 As shown, very raw, unripe, rotten, unpalatable, fake and not real are all quality descriptors included in the quality description vocabulary; large green mango and winter jujube are classified into the fruit category, and oven is classified into the home appliance category; the fruit category contains classification labels: too ripe, rotten and deteriorated, bad taste, etc., and the two quality descriptors of very raw and unripe belong to the classification label too ripe; unpalatable belongs to the classification label bad taste; the home appliance category contains classification labels: fake and damaged and deformed, etc.; the two quality descriptors of fake and not real belong to the classification label; rotten belongs to both the classification label rotten and deteriorated and the classification label damaged and deformed.
[0096] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.
Claims
1. A method for automatically labeling quality problem scenarios based on categories, characterized in that: The following steps are involved: S1. Mining quality description words: Based on a pre-built basic dictionary that stores basic description words, all words in the known corpus are compared with the basic description words as candidate words, quality description words are screened out from the candidate words, and the quality description words are added to the basic dictionary to gradually expand the basic dictionary and form a quality description word library; S2. Commodity category construction: Classify commodities into commodity categories; S3, mapping of goods and quality descriptors: traverse the known corpus and map each product to a quality descriptor through the descriptive relationship between the goods and quality descriptors; S4. Classification labeling of quality descriptors: taking commodity categories as units, label each quality descriptor under the commodity category with a corresponding classification label; S5. Construction of basic support database: The commodity category to which the commodity belongs and the quality descriptors marked with classification labels under the commodity category are stored in the form of a graph structure to form a basic support database; S6, determining the modification relationship between the commodity and the quality descriptor: judging whether the commodity and the quality descriptor in the input text are in a modification relationship by the distance between the commodity name and the quality descriptor; if the distance is less than a predetermined threshold M, it is determined that the commodity and the quality descriptor are in a modification relationship, and the process proceeds to S7; otherwise, the input text is discarded; S7, confirmation of product quality issues: finding the classification label corresponding to the product contained in the input text in the basic support database constructed by S5; Among them, in S5, the graph structure includes the following two connection relationships: Connection relationship 1: the sequential connection relationship between products, product categories, classification labels, and quality descriptors; and Connection 2: the connection between product categories and quality descriptors; In S6, before determining the modification relationship between the product and the quality descriptor, it is necessary to find the parallel relationship between the products and confirm the validity of the quality descriptor, including: S6-1. Parallel relationship search between commodities: The basic support database is used to search for commodities of known commodity categories in the input text after word segmentation, and when multiple commodities are retrieved, it is determined whether there are conjunctions between the names of the multiple commodities that indicate parallel meanings. If yes, the multiple commodities are classified as parallel commodities; if not, it is determined that the input text contains multiple commodities; and when only one commodity is retrieved in the input text or the multiple commodities retrieved are parallel commodities, and the distance between the name of the commodity or any commodity in the parallel commodities and the quality descriptor is less than M, it is determined that the quality descriptor and the commodity or all commodities in the parallel commodities are in a modifying relationship; if the input text contains multiple commodities, the input text is segmented according to punctuation marks, and the matching operation of commodities and quality descriptors is performed in sequence to determine the quality descriptor that has a modifying relationship with each commodity; S6-2, quality descriptor validity confirmation: search the quality description word library to see if the input text contains the quality descriptor, and if the input text contains the quality descriptor, determine the validity of the quality descriptor contained in the input text through the preset negative word library; if the quality descriptor contains any negative word stored in the negative word library before or after the quality descriptor, determine that the quality descriptor is invalid; The classification labels corresponding to the commodities contained in the input text found in the basic support database constructed by S5 in S7 specifically include: S7-1, determining the product category of the product contained in the input text through the connection relationship between the product and the product category in the connection relationship 1; S7-2, judging through connection relationship 2 whether the quality descriptor in the commodity and quality descriptor with a modification relationship in the input text is a quality descriptor under the commodity category determined in S7-1; if yes, proceeding to S7-3; otherwise, judging that the modification relationship between the commodity and the quality descriptor contained in the input text is wrong, and discarding the corresponding modification relationship; S7-3. Determine all classification labels under the commodity category determined in S7-1 through the sequential connection relationship among the commodity category, classification label and quality descriptor in connection relationship 1, and select from all the classification labels the classification labels that have a mapping relationship with the commodity having a modification relationship in the input text and the quality descriptor in the quality descriptor.
2. The automatic labeling method for quality problem scenario labels based on category according to claim 1, characterized in that: S1 also includes: S0. Unknown word mining: Use NLP to mine known corpus to discover unknown words; wherein, the unknown words refer to new words developed over time and proprietary words in a specific field.
3. The automatic labeling method for quality problem scenario labels based on category according to claim 1 is characterized in that: Comparing all the words in the known corpus as candidate words with the basic description words, screening out the quality description words among the candidate words, and adding the quality description words to the basic dictionary to gradually expand the basic dictionary to form a quality description word library, including: after removing the entity nouns from all the words in the known corpus, the remaining words are used as candidate words and the basic description words are respectively calculated with cosine similarity to obtain a similarity set between each candidate word and the basic description word, when the number of words in the similarity set that are greater than a preset threshold value T exceeds a preset limit value N, determining that the candidate word corresponding to the similarity set is a quality description word, and adding the quality description word to the basic dictionary to gradually expand the basic dictionary to form a quality description word library; Among them, X cycles are preset to screen out quality descriptors from the candidate words; T and N are set to high values in the first X-1 cycles, and the quality descriptors screened out by the high values of T and N are directly added to the basic dictionary, and T and N are set to low values before the Xth cycle, and the candidate words are compared with the basic descriptors in the basic dictionary to which the quality descriptors screened out in the first X-1 cycles have been added.
4. The automatic labeling method for quality problem scenario labels based on category according to claim 1 is characterized in that: In S2, the categories of commodities are classified into: The required product names and corresponding categories are obtained from the existing e-commerce platform, and then the categories are adjusted to form the product categories.
5. The automatic labeling method for quality problem scenario labels based on category according to claim 1, characterized in that: In S3, the known corpus is traversed, and each product is matched with a quality descriptor through the descriptive relationship between the product and the quality descriptor. Specifically, the following are included: Through dependency syntax analysis, determine whether the product and quality descriptor are in a modification relationship: S3-1, perform syntactic analysis on the input text to generate a primary syntactic tree; S3-2, determining one by one whether the primary syntax tree includes other subject-predicate relationships except the corresponding relationship between the commodity and the quality descriptor, and whether there is a parallel relationship with the components in other subject-predicate relationships; if yes, the syntax tree is cut according to the subject-predicate relationship to form a final syntax tree; if no, the syntax tree is used as the final syntax tree; S3-3. Determine whether the distance between the product and the quality descriptor on the final syntax tree is less than a preset distance threshold; if yes, determine that there is a modification relationship between the product and the quality descriptor; if no, determine that there is no modification relationship between the product and the quality descriptor.
6. The automatic labeling method for quality problem scenario labels based on category according to claim 1, characterized in that: In S4, before assigning a corresponding classification label to each quality descriptor under a commodity category, the quality descriptors that have appeared under each commodity category are counted, and the quality descriptors are combined with the characteristics of the commodity category and all possible quality problems under the commodity category to form a classification label.
7. The automatic labeling method for quality problem scenario labels based on category according to claim 1 is characterized in that: In S5, the goods, the categories to which the goods belong, and the quality descriptors marked with classification labels under the categories are stored in the form of a graph structure. Before forming a basic support database, the classification labels corresponding to all goods under the categories need to be shared with the quality descriptors, including: S5-1. Count all possible quality problems of all products in each product category, and map the products in the product category to classification labels using quality descriptors; S5-2. The quality descriptors under the same classification label in each product category are shared among all products under the corresponding product category.
Citation Information
Patent Citations
Commodity labeling method suitable for electronic commerce Chinese website
CN105320778A
Commercial quality emotional dictionary construction system based on big data text mining
CN105760502A
Comment analysis method based on word vectors and syntactic features and visual interactive interface
CN110175325A
Customs declaration commodity intelligent classification method based on historical data mining
CN110471948A
Commodity short title generation method and device
CN111191022A
