A method for extracting data from text to generate product SPU
By extracting data from text to generate product SPU, using association rule algorithms and frequent item set mining technology, the problem of extracting information on the e-commerce platform is solved, and fast and accurate product information extraction is achieved, supporting the brand to better Statistical delivery results and competitive product analysis.
Patent Information
- Application Number
- CN202110785144.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-12
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-07-12
AI Technical Summary
The existing technology is difficult to quickly and accurately extract product information published on e-commerce platforms, which leads to a large amount of workload when placing advertisements and it is difficult for brands to understand the effectiveness of competing products.
Design a method to extract data from text to generate product SPU. By regularly collecting notes, calling brand vocabulary, product vocabulary and description vocabulary, using association rule algorithms and frequent item set mining technology, extracting brand vocabulary and product vocabulary and key descriptive words to form a standardized commodity unit SPU.
It has achieved rapid and accurate extraction of information issued by e-commerce platforms, and can be better applied to brand placement performance statistics and competitive advertising placement voice surveys, reducing the workload and difficulty of review of the brand.
Smart Images

Figure CN113935761B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a method for extracting data from text to generate a commodity SPU. Background Art
[0002] With the rise of e-commerce platforms in China, online shopping has become increasingly important in people's lives, and has become an important consumption channel for consumers.
[0003] At present, Taobao, Douyin, Kuaishou, Xigua Video, Xiaohongshu and other platforms have gradually become new growth drivers for e-commerce platforms and content platforms due to their high conversion rates and good marketing effects, accelerating consumption conversion and bringing higher traffic to merchants. For example, the rise of Xiaohongshu's ecosystem has gathered a large number of beauty evaluation enthusiasts, and later gradually developed to evaluate other industries such as maternal and child care, food, men's clothing, shoes and bags, and many other fields, deriving a large number of product evaluation advertisements, and gradually attracted more brands to settle in Xiaohongshu to place commercial advertisements. For brands, searching for the advertisements placed by the brand through the vast Xiaohongshu posts is undoubtedly a huge workload. At the same time, there is also a need for brands to understand the volume of competitive products placed in the circle. Therefore, how to accurately extract the brand product information with relatively comprehensive information in Xiaohongshu posts is of great significance for each brand to count the effect of brand placement or investigate the volume of competitive product advertising placement. Summary of the invention
[0004] The purpose of the present invention is to design a method for extracting data from text to generate a commodity SPU in view of the existing technology, which is reasonable and can quickly and accurately extract commodity information.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A method for extracting data from text to generate a commodity SPU comprises the following steps:
[0007] 1) Regularly collect new note texts, store the new note texts in a pre-created SQL Server database, and store the IDs of the new note texts in the note queue;
[0008] 2) For the note queue, retrieve the corresponding note text from the SQL Server database one by one according to the note text ID, and call the brand words in the pre-created brand word library to scan the note text one by one, and check whether the note text has the corresponding brand word. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 3);
[0009] 3) Call the pre-created product word library to scan the note text one by one, and check whether the note text contains the corresponding product word. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 4);
[0010] 4) Determine whether the product category of the product represented by the retrieved product word and the product category of several products operated by the retrieved brand word have an intersection. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 5);
[0011] 5) Check whether the product word is associated with the pre-created description word library with the description word set W1 of the product word. If not, save the brand word and product word as the product SPU. If yes, execute step 6);
[0012] 6) Intercept the character string S1 between the brand word and the product word in the note text, and match the key description word with the description word set W1. If the key description word is not matched, save the brand word and product word as the product SPU. If the key description word is matched, save the brand word, key description word and product word as the product SPU.
[0013] Preferably, the method for matching key descriptive words in step 6) comprises the following steps:
[0014] 6.1) Traverse the description words in the description word set W1 to match the note text and obtain the description word subset W2;
[0015] 6.2) Determine whether the length of the description word subset W2 is less than 2, if so, repeat step 6.1), if not, execute step 6.3);
[0016] 6.3) Obtain the intersection S2 of the description word subset W2 and the string S1; traverse the intersection S2, find the index position idx in the string S1 corresponding to each character in the intersection S2, and obtain the dictionary D1;
[0017] 6.4) Sort the dictionary D1 from small to large values to obtain sorting T1;
[0018] 6.5) Obtain a set S3 of consecutive adjacent strings from sort T1;
[0019] 6.6) Traverse the string set S3, filter the strings whose length is less than 2 and not in the string S1, and obtain the key description words.
[0020] Preferably, the method for obtaining the continuous adjacent character string set S3 from T1 in step 6.5) comprises the following steps:
[0021] 6.5.1) Initialize and obtain an empty list L1;
[0022] 6.5.2) Determine whether the length of sort T1 is empty. If so, return list L1. If not, add an empty list to list L1 and set the initialization value i=0, where i is a natural number.
[0023] 6.5.3) Determine whether i is less than the length of T1. If not, return to list L1. If so, execute step 6.5.4);
[0024] 6.5.4) Determine whether T[i][1] is equal to T[i-1][1]+1. If not, execute step 6.5.5). If yes, execute step 6.5.6);
[0025] 6.5.5) After inserting T[i-1][0] into the last list in list L1 and inserting an empty list into list L1, execute step 6.5.6);
[0026] 6.5.6) Insert T[i-1][0] into the last list in list L1, i is automatically increased by 1, and execute step 6.5.3).
[0027] Preferably, the method for pre-creating the SQL Server database in step 1) is: collecting a large number of note texts reviewing the products, setting an ID for each note text, and creating a SQL Server database based on Microsoft SQL Server.
[0028] Preferably, the method for pre-creating a brand vocabulary in step 2) is: performing brand mining analysis on the SQL Server database note text, obtaining brand words used to represent brand names and product categories of several products sold by each brand, and associating each brand word with its corresponding product category to create a brand vocabulary.
[0029] Preferably, the method for pre-creating a product vocabulary in step 3) is: performing product mining analysis on the SQL Server database note text, obtaining product words used to represent product names and the product category of the product represented by each product word, and associating each product word with its corresponding product category to create a product vocabulary.
[0030] Preferably, the method for pre-creating a description word library in step 5) is: performing description word mining on the SQL Server database note text, mainly through an association rule algorithm, obtaining a set of description words used to represent product description information, and creating a description word library.
[0031] Preferably, the description word updating method of the description word library comprises the following steps:
[0032] 5.1) Regularly traverse the brand word library to search whether each brand word is associated with at least one descriptive word. If so, execute step 5.2); if not, execute step 5.3);
[0033] 5.2) Retrieve the latest N note texts related to the brand word from the SQL Server database, where N is a natural number greater than 10, and execute step 5.4);
[0034] 5.3) Retrieve the full amount of notes related to the brand word from the SQL Server database and execute step 5.4);
[0035] 5.4) Traverse the retrieved note texts and extract the product word related text A related to the brand word;
[0036] 5.5) Perform descriptive word mining on the product word related text A: filter out stop words and product related words, obtain the descriptive words corresponding to the product words, and store them in the description word library.
[0037] Furthermore, it also includes mining key descriptive words for the commodity word related text A. The method is: after sorting the commodity word related texts of each commodity word, traverse M commodity word related texts for each commodity word in order, where M is a natural number greater than 1, and then calculate the relevance of the words according to the GSP algorithm, and output the description word with the maximum support as the key descriptive word.
[0038] Furthermore, it also includes denoising and saving the product word related text A, the method is: determine whether the product word related text A exists in the brand word, if so, match the product word related text A through the product word associated with the brand word, if the product word is matched, determine whether the product category managed by the brand word and the product category corresponding to the product word have any intersection, if so, extract the text between the brand word and the product word, and save it after filtering out stop words.
[0039] The present invention adopts the above technical scheme, regularly collects a large number of note texts commenting on products on the network platform, and performs data mining of brand words, product words, descriptive words, etc. on the note text based on the association rule algorithm, and mines the key descriptive words that are strongly related to the products through frequent item sets, and intelligently extracts the "brand word + key descriptive word + product word" phrase sentence pattern as the standardized product unit SPU, so as to more comprehensively and accurately extract the product information in the posting information of platforms such as Xiaohongshu, so that it can be better used in text-based product analysis scenarios, which is of great significance for each brand to count the brand delivery effect or investigate the volume of competing product advertising. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 The flowchart of the method for extracting data from text to generate commodity SPU of the present invention. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solutions and advantages of the present invention more concise and clear, the present invention will explain the exemplary embodiments in detail, and its examples are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are only examples of devices and methods consistent with some aspects of the present invention as detailed in the attached claims.
[0043] like Figure 1 As shown, the method of extracting data from text to generate a commodity SPU of the present invention comprises the following steps:
[0044] 1) Regularly collect new note texts, store the new note texts in a pre-created SQL Server database, and store the IDs of the new note texts in the note queue; preferably, collect the new note texts every 1 day;
[0045] 2) For the note queue, retrieve the corresponding note text from the SQL Server database one by one according to the note text ID, and call the brand words in the pre-created brand word library to scan the note text one by one, and check whether the note text has the corresponding brand word. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 3);
[0046] 3) Call the pre-created product word library to scan the note text one by one, and check whether the note text contains the corresponding product word. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 4);
[0047] 4) Determine whether the product category of the product represented by the retrieved product word and the product category of several products operated by the retrieved brand word have an intersection. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 5);
[0048] 5) Check whether the product word is associated with the pre-created description word library with the description word set W1 of the product word. If not, save the brand word and product word as the product SPU. If yes, execute step 6);
[0049] 6) Intercept the character string S1 between the brand word and the product word in the note text, and match the key description word with the description word set W1. If the key description word is not matched, save the brand word and product word as the product SPU. If the key description word is matched, save the brand word, key description word and product word as the product SPU.
[0050] The above-mentioned "moving the note text to a pre-created manual review library" of the present invention enables it to promptly discover new brands, new products, or new products under existing brands that appear in the industry through manual review in the later stage, so as to timely update and improve the brand vocabulary, product vocabulary and description vocabulary, thereby facilitating the circulation operation of the entire system.
[0051] Preferably, the product category of the product represented by the product word and the product category of the several products managed by the brand word share a common classification system.
[0052] Preferably, the method for matching key descriptive words in step 6) comprises the following steps:
[0053] 6.1) Traverse the description words in the description word set W1 to match the note text and obtain the description word subset W2;
[0054] 6.2) Determine whether the length of the description word subset W2 is less than 2, if so, repeat step 6.1), if not, execute step 6.3);
[0055] 6.3) Obtain the intersection S2 of the description word subset W2 and the string S1; traverse the intersection S2, find the index position idx in the string S1 corresponding to each character in the intersection S2, and obtain the dictionary D1;
[0056] 6.4) Sort the dictionary D1 from small to large values to obtain sorting T1;
[0057] 6.5) Obtain a set S3 of consecutive adjacent strings from sort T1;
[0058] 6.6) Traverse the string set S3, filter the strings whose length is less than 2 and not in the string S1, and obtain the key description words.
[0059] Preferably, the method for obtaining the continuous adjacent character string set S3 from T1 in step 6.5) comprises the following steps:
[0060] 6.5.1) Initialize and obtain an empty list L1;
[0061] 6.5.2) Determine whether the length of sort T1 is empty. If so, return list L1. If not, add an empty list to list L1 and set the initialization value i=0, where i is a natural number.
[0062] 6.5.3) Determine whether i is less than the length of T1. If not, return to list L1. If so, execute step 6.5.4);
[0063] 6.5.4) Determine whether T[i][1] is equal to T[i-1][1]+1. If not, execute step 6.5.5). If yes, execute step 6.5.6);
[0064] 6.5.5) After inserting T[i-1][0] into the last list in list L1 and inserting an empty list into list L1, execute step 6.5.6);
[0065] 6.5.6) Insert T[i-1][0] into the last list in list L1, i is automatically increased by 1, and execute step 6.5.3).
[0066] Preferably, the method for pre-creating the SQL Server database in step 1) is: collecting a large number of note texts reviewing the products, setting an ID for each note text, and creating a SQL Server database based on Microsoft SQL Server.
[0067] Preferably, the method for pre-creating a brand vocabulary in step 2) is: performing brand mining analysis on the SQL Server database note text, obtaining brand words used to represent brand names and product categories of several products sold by each brand, and associating each brand word with its corresponding product category to create a brand vocabulary.
[0068] Preferably, the method for pre-creating a product vocabulary in step 3) is: performing product mining analysis on the SQL Server database note text, obtaining product words used to represent product names and the product category of the product represented by each product word, and associating each product word with its corresponding product category to create a product vocabulary.
[0069] Preferably, the method for pre-creating a description word library in step 5) is: performing description word mining on the SQL Server database note text, mainly through an association rule algorithm, obtaining a set of description words used to represent product description information, and creating a description word library.
[0070] Preferably, the description word updating method of the description word library comprises the following steps:
[0071] 5.1) Regularly traverse the brand word library to check whether each brand word is associated with at least one descriptive word. If so, execute step 5.2); if not, execute step 5.3); preferably, traverse the brand word library every 5 days;
[0072] 5.2) Retrieve the latest N note texts related to the brand word from the SQL Server database, where N is a natural number greater than 10, and execute step 5.4); preferably, N is equal to 2000;
[0073] 5.3) Retrieve the full amount of notes related to the brand word from the SQL Server database and execute step 5.4);
[0074] 5.4) Traverse the retrieved note texts and extract the product word related text A related to the brand word;
[0075] 5.5) Perform descriptive word mining on the product word related text A: filter out stop words and product related words, obtain the descriptive words corresponding to the product words, and store them in the description word library.
[0076] Furthermore, it also includes mining key descriptive words for the commodity word related text A, the method is: after sorting the commodity word related text of each commodity word, traverse M commodity word related texts in order for each commodity word, where M is a natural number greater than 1, and then calculate the relevance of the words according to the GSP algorithm, and output the descriptive word with the maximum support as the key descriptive word. Preferably, the M is equal to 20.
[0077] Preferably, the relevance of words is calculated according to the GSP algorithm, which includes the following steps:
[0078] 5.6.1) Count the frequency of each character in the text list, filter out characters whose frequency is less than the minimum support, and obtain a character list L2;
[0079] 5.6.2) Merge the characters in list L2 into new words in pairs, count the frequency of each new word, filter out the new words whose frequency is less than the minimum support, and obtain a list of new words;
[0080] 5.6.3) Filter out words with less than the minimum support in the list of new words to obtain a list of words L3;
[0081] 5.6.4) Determine whether the number of words in list L3 is equal to 0. If so, return an empty set. If not, execute step 5.6.5);
[0082] 5.6.5) Determine whether the number of words in list L3 is equal to 1. If so, return to list L3. If not, execute step 5.6.6);
[0083] 5.6.6) After merging similar strings, return to list L3.
[0084] Preferably, the method for merging similar character strings comprises the following steps:
[0085] 5.7.1) Initialize array A1, and list L3 receives the incoming list L2;
[0086] 5.7.2) Determine whether to traverse the list L3 until it is exhausted. If so, return to array A1. If not, take out word Wi, remove the first character, and execute step 5.7.3);
[0087] 5.7.3) Determine whether to traverse the list L3 until it is exhausted. If so, execute step 5.7.2). If not, take out the word Ni and remove the last character of Ni, and then execute step 5.7.4);
[0088] 5.7.4) Determine whether Wi with the first character removed is equal to Ni with the last character removed. If not, execute step 5.7.3). If not, execute step 5.7.5);
[0089] 5.7.5) Concatenate the last character of Wi and Ni and save them into array A1.
[0090] Furthermore, it also includes denoising and saving the product word related text A, the method is: determine whether the product word related text A exists in the brand word, if so, match the product word related text A through the product word associated with the brand word, if the product word is matched, determine whether the product category managed by the brand word and the product category corresponding to the product word have any intersection, if so, extract the text between the brand word and the product word, and save it after filtering out stop words.
[0091] The present invention adopts the above technical scheme, regularly collects a large number of note texts commenting on products on the network platform, and performs data mining of brand words, product words, descriptive words, etc. on the note text based on the association rule algorithm, and mines the key descriptive words that are strongly related to the products through frequent item sets, and intelligently extracts the "brand word + key descriptive word + product word" phrase sentence pattern as the standardized product unit SPU, so as to more comprehensively and accurately extract the product information in the posting information of platforms such as Xiaohongshu, so that it can be better used in text-based product analysis scenarios, which is of great significance for each brand to count the brand delivery effect or investigate the volume of competing product advertising.
[0092] For the convenience of understanding, the following semantic interpretations are given for the words such as notes, brand words, product words, key descriptive words, etc. involved in the present invention (this interpretation cannot be understood as limiting the scope of protection of the present invention):
[0093] Brand words: For example, Estee Lauder is the name of a beauty brand and is a word used to represent the brand name.
[0094] Brand vocabulary: a collection library used to pre-collect and continuously supplement and update various brand words, such as: creating a brand vocabulary library including brand words such as Lancome, Estee Lauder, Perfect Diary, etc.
[0095] Brand classification: refers to the industry category or commodity category of the products sold by a commercial brand. For example, Estee Lauder mainly sells skin care and makeup, so its categories are skin care and makeup. There is a one-to-many relationship between brands and categories.
[0096] Product word: For example, Little Brown Bottle is the name of a facial cream under the Estee Lauder brand. Little Brown Bottle is a product word, which is a word used to represent the name of a product.
[0097] Product word classification: indicates the industry category or product category to which the product belongs, and shares the same classification system with brand words. Product words and categories have a one-to-one relationship.
[0098] Product vocabulary: a collection library used to pre-collect and continuously supplement and update various product terms, such as creating a product vocabulary library including product terms in various fields such as small brown bottle, facial mask, makeup remover, toner, etc.
[0099] Descriptive words: For example: seventh generation. In Estee Lauder’s seventh generation little brown bottle, the seventh generation belongs to the description of the little brown bottle.
[0100] Key descriptive words: For example: Perfect Diary Mist Dream Matte Lip Glaze, where "Perfect Diary" is a brand word, "Mist Dream Matte" is a key descriptive word, and "Lip Glaze" is a product word.
[0101] Notes: The name of Xiaohongshu’s posts, generally including but not limited to text, video, and pictures; it may be a combination of multiple forms, and this patent focuses on analyzing text content. Notes can be collected in real time through data collection technology.
[0102] The present invention generates a correspondence between brand words and product words, and an example is given below:
[0103] Dior:
[0104] Estee Lauder: {'Little Brown Bottle': ['Facial Essence Seventh Generation', 'Eighth Generation', 'New Year Limited Edition', 'Anti-Aging', 'Multi-Effect Intelligence', 'Sixth Generation'], 'Cleansing Milk': ['Fresh and Bright Red Pomegranate Two-in-One', 'Bright and Rejuvenating Foam', 'Pomegranate Foam', 'Purifying and Softening Skin', 'Skin Transparent Repair', 'Pink', 'Purifying Face', 'Soft Silk'], 'Facial Cleanser': ['New Red Pomegranate', 'Pomegranate Cleansing', 'Only Amino Acid', 'Skin Transparent Repair', 'Pink Rejuvenating Face'], 'Moisturizing Cream': ['Enriching']}
[0105] For ease of understanding, the present invention briefly describes a specific embodiment: For example, a new note text "Through evaluation of multiple products, it is found that the seventh generation of Estee Lauder's recently released small brown bottle is very useful", first analyze that there is a brand word (Estee Lauder) in this sentence, and then there is a product word (small brown bottle), the classification of the small brown bottle in the product word library is skin care, and the brand word Estee Lauder in the brand library is skin care, makeup, so there is an intersection between the brand word and the product word classification, based on this premise, the description phrase (string) between the brand word and the product word is analyzed, first the description word library corresponding to Estee Lauder mined and analyzed is taken out from the database, and the description word list corresponding to the small brown bottle is found, and the one that best matches the description phrase between the brand word and the product word in the description word list (that is, the key descriptive word that is strongly related to the product) is found, and finally the product SPU of brand word + key descriptive word + product word can be generated, or if no descriptive word is found, the product SPU without descriptive word such as brand word + product word is generated. The focus of the method of the present invention is how to determine that the key descriptive word is "seventh generation" instead of "recently released".
[0106] The above is a preferred specific embodiment of the present invention. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all equivalent changes, modifications, substitutions and variations made by a person skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A method for extracting data from text to generate product SPU, Features: It includes the following steps: 1) Regularly collect new note texts, store the new note texts in a pre-created SQL Server database, and store the IDs of the new note texts in the note queue; 2) For the note queue, retrieve the corresponding note text from the SQL Server database one by one according to the note text ID, and call the brand words in the pre-created brand word library to scan the note text one by one, and check whether the note text has the corresponding brand word. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 3); 3) Call the pre-created product word library to scan the note text one by one, and check whether the note text contains the corresponding product word. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 4); 4) Determine whether the product category of the product represented by the retrieved product word and the product category of several products operated by the retrieved brand word have an intersection. If not, move the note text to the pre-created manual review library and end the task. If yes, execute step 5); 5) Check whether the product word is associated with the pre-created description word library with the description word set W1 of the product word. If not, save the brand word and product word as the product SPU. If yes, execute step 6); 6) Intercept the string S1 between the brand word and the product word in the note text, and perform key description word matching with the description word set W1. If no key description word is matched, save the brand word and product word as the product SPU. If a key description word is matched, save the brand word, key description word and product word as the product SPU. The method for matching key description words in step 6) comprises the following steps: 6.1) Traverse the description words in the description word set W1 to match the note text and obtain the description word subset W2; 6.2) Determine whether the length of the description word subset W2 is less than 2, if so, repeat step 6.1), if not, execute step 6.3); 6.3) Obtain the intersection S2 of the description word subset W2 and the string S1; traverse the intersection S2, find the index position idx in the string S1 corresponding to each character in the intersection S2, and obtain the dictionary D1; 6.4) Sort the dictionary D1 from small to large values to obtain sorting T1; 6.6) Obtain a set S3 of consecutive adjacent strings from sort T1; 6.6) Traverse the string set S3, filter the strings whose length is less than 2 and not in the string S1, and obtain the key description words; The method for obtaining the continuous adjacent character string set S3 from T1 in step 6.5) comprises the following steps: 6.5.1) Initialize and obtain an empty list L1; 6.5.2) Determine whether the length of sort T1 is empty. If so, return list L1. If not, add an empty list to list L1 and set the initialization value i=0, where i is a natural number. 6.5.3) Determine whether i is less than the length of T1. If not, return to list L1. If so, execute step 6.5.4); 6.5.4) Determine whether T[i][1] is equal to T[i-1][1]+1. If not, execute step 6.5.5). If yes, execute step 6.5.6); 6.5.5) After inserting T[i-1][0] into the last list in list L1 and inserting an empty list into list L1, execute step 6.5.6); 6.5.6) Insert T[i-1][0] into the last list in list L1, i is automatically increased by 1, and execute step 6.5.3).
2. The method for generating a commodity SPU by extracting data from a text according to claim 1, Features: The method for pre-creating the SQL Server database in step 1) is: collect a large number of note texts of product reviews, set an ID for each note text, and create a SQL Server database based on Microsoft SQL Server.
3. The method for generating commodity SPU by extracting data from text according to claim 1, Features: The method for pre-creating a brand vocabulary in step 2) is: perform brand mining analysis on the SQL Server database note text, obtain brand words used to represent brand names and product categories of several products operated by each brand, and associate each brand word with its corresponding product category to create a brand vocabulary.
4. The method for generating a commodity SPU by extracting data from a text according to claim 1, Features: The method for pre-creating a product vocabulary in step 3) is: perform product mining analysis on the SQL Server database note text, obtain product words used to represent product names and the product category of the product represented by each product word, and associate each product word with its corresponding product category to create a product vocabulary.
5. The method for generating commodity SPU by extracting data from text according to claim 1, Features: The method for pre-creating a description word library in step 5) is: mining description words on the SQL Server database note text and obtaining a description word set for representing product description information through an association rule algorithm, and creating a description word library.
6. The method for generating a commodity SPU by extracting data from text according to claim 5, Features: The method for updating description words in a description word library comprises the following steps: 5.1) Regularly traverse the brand word library to search whether each brand word is associated with at least one descriptive word. If so, execute step 5.2); if not, execute step 5.3); 5.2) Retrieve the latest N note texts related to the brand word from the SQL Server database, where N is a natural number greater than 10, and execute step 5.4); 5.3) Retrieve the full amount of notes related to the brand word from the SQL Server database and execute step 5.4); 5.4) Traverse the retrieved note texts and extract the product word related text A related to the brand word; 5.5) Perform descriptive word mining on the product word related text A: filter out stop words and product related words, obtain the descriptive words corresponding to the product words, and store them in the description word library.
7. The method for generating a commodity SPU by extracting data from a text according to claim 6, Features: It also includes mining key descriptive words for the commodity word related text A. The method is: after sorting the commodity word related texts of each commodity word, traverse M commodity word related texts for each commodity word in order, where M is a natural number greater than 1, and then calculate the relevance of the words according to the GSP algorithm, and output the description word with the maximum support as the key descriptive word.
8. The method for generating a commodity SPU by extracting data from a text according to claim 7, Features: It also includes denoising and saving the product word related text A, the method is: determine whether the product word related text A exists in the brand word, if so, match the product word related text A through the product word associated with the brand word, if the product word is matched, determine whether the product category managed by the brand word and the product category corresponding to the product word have an intersection, if so, extract the text between the brand word and the product word, and save it after filtering out stop words.
