Commodity label mining method and device
By performing text recognition on product description images and matching tag words in the tag dictionary tree, the problem of low efficiency in mining product tags from unstructured data is solved, achieving efficient and accurate product tag extraction.
Patent Information
- Application Number
- CN202310582392.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-22
AI Technical Summary
Existing technologies struggle to effectively mine product tags from unstructured data, especially for products from e-commerce platforms, where tag mining is both inefficient and costly.
By acquiring product description images, using optical character recognition technology to extract description text, and performing tag word matching in a pre-built tag dictionary tree, combined with the AC automaton algorithm and suffix tree optimization, product tags can be obtained quickly and accurately.
It improves the efficiency and accuracy of product tag mining, saves computing costs, and ensures the query time and accuracy of tag matching.
Smart Images

Figure CN119003748A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and in particular to a method and device for mining product labels. Background Art
[0002] At present, there are two main ways to mine labels for e-commerce products. One is to label products manually. However, with the development of technology, the number of products is increasing. Labeling products manually consumes a lot of manpower and material resources and is inefficient. The other is to mine product labels with the help of AI technology and other means. The platform can automatically mine product label collections to minimize manual maintenance costs.
[0003] The general AI labeling technology uses deep learning algorithms to mine product labels from text data. The target data information is often structured data, so label mining can be performed on structured data; however, the method of mining product labels from structured data is not applicable to unstructured data, and unstructured data is difficult to obtain; therefore, how to obtain unstructured product description information and perform label mining from it is a problem that needs to be solved urgently. Summary of the invention
[0004] The embodiments of the present application provide a method and device for mining product labels.
[0005] The first embodiment of the present application proposes a method for mining product tags, including:
[0006] Get the first description image of the target product;
[0007] Extracting a first description text of the target product from the first description picture;
[0008] In a pre-established tag dictionary tree, tag word matching is performed on the first description text to obtain one or more target tag words of the target product, wherein the tag dictionary tree is generated by first candidate tag words extracted from second description texts of multiple candidate products.
[0009] In one embodiment of the present application, before obtaining the first description picture of the target product, the method further includes:
[0010] Extracting a second description text of the candidate product from the second description picture of the candidate product;
[0011] Segmenting the second description text to obtain multiple text strings;
[0012] Determining a representation value of the text string according to the mutual information and left-right information entropy of the text string;
[0013] Selecting a text string whose representation value is greater than a preset threshold from the multiple text strings as a second candidate label word;
[0014] The tag dictionary tree is generated according to the selected second candidate tag word.
[0015] In one embodiment of the present application, generating the tag dictionary tree according to the selected second candidate tag word includes:
[0016] Perform common word filtering on the second candidate tag words to obtain the first candidate tag words;
[0017] The tag dictionary tree is generated based on the first candidate tag word.
[0018] In one embodiment of the present application, the tag word matching of the first description text in the pre-established tag dictionary tree to obtain one or more target tag words of the target product includes:
[0019] Determine each node on the tag dictionary tree and establish a suffix tree for each node;
[0020] Searching the first description text in the tag dictionary tree based on the AC automaton algorithm to obtain a candidate text string;
[0021] The first digit of the candidate text string is swapped, and the candidate text string is searched based on the suffix tree to obtain one or more target tag words of the target product.
[0022] In one embodiment of the present application, after obtaining one or more target tag words of the target product, the method further includes:
[0023] Determine whether each target tag word is compatible with the target product;
[0024] Target label words that are not suitable for the target product are removed to obtain a first target label word for the target product.
[0025] In one embodiment of the present application, after removing the target label words that are not suitable for the target product to obtain the first target label word for the target product, the method further includes:
[0026] Supplementing the first target label word based on synonym replacement to obtain a second target label word;
[0027] The second target label word is checked for duplication based on phrase replacement, and repeated label words are deleted to obtain a third target label word for the target product.
[0028] In one embodiment of the present application, after obtaining the third target tag word of the target product, the method further includes:
[0029] The product name of the target product is obtained, and the product name is combined with each of the third target tags to obtain a final fourth target product tag of the target product.
[0030] In one embodiment of the present application, the determining whether each target tag word is compatible with the target product includes:
[0031] Extracting features of the target product to obtain wide features of the target product, wherein the wide features include cross features, discrete features, and partial continuous features of the target product;
[0032] The target label word is semantically encoded by a pre-trained language model to obtain an embedded vector representation of the target label word;
[0033] Inputting the embedding vector representation into a long short-term memory network to extract semantic information in forward and backward directions, thereby obtaining an encoding vector carrying context information;
[0034] The wide feature and the encoding vector are concatenated to obtain a concatenated vector, and binary classification is performed based on the concatenated vector to obtain a classification label of the target label word, where the classification label is used to indicate whether the target label word is adapted to the target product.
[0035] The second embodiment of the present application provides a device for mining commodity labels, including:
[0036] A first acquisition module, used to acquire a first description image of a target product;
[0037] A second acquisition module, configured to extract a first description text of the target product from the first description picture;
[0038] A target tag word acquisition module is used to match the first description text with tags in a pre-established tag dictionary tree to obtain one or more target tag words for the target product, wherein the tag dictionary tree is generated by first candidate tag words extracted from second description texts of multiple candidate products.
[0039] In one embodiment of the present application, before the first acquisition module, it is also used to:
[0040] Extracting a second description text of the candidate product from the second description picture of the candidate product;
[0041] Segmenting the second description text to obtain multiple text strings;
[0042] Determining a representation value of the text string according to the mutual information and left-right information entropy of the text string;
[0043] Selecting a text string whose representation value is greater than a preset threshold from the multiple text strings as a second candidate label word;
[0044] The tag dictionary tree is generated according to the selected second candidate tag word.
[0045] In one embodiment of the present application, before the first acquisition module, it is used to:
[0046] Perform common word filtering on the second candidate tag words to obtain the first candidate tag words;
[0047] The tag dictionary tree is generated based on the first candidate tag word.
[0048] In one embodiment of the present application, the target tag word acquisition module is used to:
[0049] Determine each node on the tag dictionary tree and establish a suffix tree for each node;
[0050] Searching the first description text in the tag dictionary tree based on the AC automaton algorithm to obtain a candidate text string;
[0051] The first digit of the candidate text string is swapped, and the candidate text string is searched based on the suffix tree to obtain one or more target tag words of the target product.
[0052] In one embodiment of the present application, after the target tag word acquisition module, it is also used to:
[0053] Determine whether each target tag word is compatible with the target product;
[0054] Target label words that are not suitable for the target product are removed to obtain a first target label word for the target product.
[0055] In one embodiment of the present application, after the target tag word acquisition module, it is also used to:
[0056] Supplementing the first target label word based on synonym replacement to obtain a second target label word;
[0057] The second target label word is checked for duplication based on phrase replacement, and repeated label words are deleted to obtain a third target label word for the target product.
[0058] In one embodiment of the present application, after the target tag word acquisition module, it is also used to:
[0059] The product name of the target product is obtained, and the product name is combined with each of the third target tags to obtain a final fourth target product tag of the target product.
[0060] In one embodiment of the present application, after the target tag word acquisition module, it is used to:
[0061] Extracting features of the target product to obtain wide features of the target product, wherein the wide features include cross features, discrete features, and partial continuous features of the target product;
[0062] The target label word is semantically encoded by a pre-trained language model to obtain an embedded vector representation of the target label word;
[0063] Inputting the embedding vector representation into a long short-term memory network to extract semantic information in forward and backward directions, thereby obtaining an encoding vector carrying context information;
[0064] The wide feature and the encoding vector are concatenated to obtain a concatenated vector, and binary classification is performed based on the concatenated vector to obtain a classification label of the target label word, where the classification label is used to indicate whether the target label word is adapted to the target product.
[0065] The third aspect of the present application provides an electronic device, including: a commodity tag mining device provided in the second aspect of the present application.
[0066] The fourth aspect embodiment of the present application proposes an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a product tag mining method proposed in the first aspect embodiment of the present application.
[0067] The fifth aspect embodiment of the present application proposes a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of an electronic device, the electronic device can execute the method proposed in the first aspect embodiment of the present application.
[0068] The sixth aspect embodiment of the present application proposes a computer program product, including a computer program, which implements the method proposed in the first aspect embodiment of the present application when executed by a processor in a communication device.
[0069] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:
[0070] In an embodiment of the present application, text recognition is performed on the first description image of the target product to extract the description information therein as the first description text. The first description text embodies the label word of the target product. Therefore, label word matching is performed on the first description text based on the AC automaton algorithm to obtain the target label word of the target product. In the embodiment of the present application, the AC automaton algorithm is used to perform label word matching, which can save the query time of label word matching and save computing costs. The first description text obtained based on text recognition is used as the basis for label word matching, thereby ensuring the accuracy of target label word extraction of the target product.
[0071] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0073] Figure 1 A flowchart of a method for mining product labels provided in an embodiment of the present application;
[0074] Figure 1a A structural diagram of a text detection framework provided in an embodiment of the present application;
[0075] Figure 1b A structural diagram of a text recognition framework provided in an embodiment of the present application;
[0076] Figure 2 A flowchart of another method for mining product tags provided in an embodiment of the present application;
[0077] Figure 2a A schematic diagram of obtaining a text string provided in an embodiment of the present application;
[0078] Figure 2b A schematic diagram of obtaining a second candidate tag word provided in an embodiment of the present application;
[0079] Figure 3 A flowchart of another method for mining product tags provided in an embodiment of the present application;
[0080] Figure 3a A schematic diagram of a tag dictionary tree and a suffix tree of a node provided in an embodiment of the present application;
[0081] Figure 4 A flowchart of another method for mining product tags provided in an embodiment of the present application;
[0082] Figure 4aA schematic diagram of a rationality judgment of a target tag word for a target commodity provided in an embodiment of the present application;
[0083] Figure 4b A schematic diagram of a fourth target product label provided in an embodiment of the present application;
[0084] Figure 4c A schematic diagram of label mining for multiple target products provided in an embodiment of the present application;
[0085] Figure 5 A schematic diagram of the structure of a device for mining commodity labels provided in an embodiment of the present application;
[0086] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0087] Figure 7 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application;
[0088] Figure 8 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0089] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.
[0090] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0091] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at..." or "when..." or "in response to determination".
[0092] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0093] It should be noted that the product tag mining method provided in any embodiment of the present application can be executed alone, or in combination with possible implementation methods in other embodiments, or in combination with any technical solution in related technologies.
[0094] A method and device for mining product labels according to an embodiment of the present application will be described below with reference to the accompanying drawings.
[0095] Figure 1 A flowchart of a method for mining product tags provided in an embodiment of the present application is shown below. Figure 1 As shown, the method includes but is not limited to the following steps:
[0096] S101, obtaining a first description picture of a target product.
[0097] Optionally, the target products may be electronic products, food, chemicals, clothing, hardware, and other items that can be sold online.
[0098] Obtain the first description image of the target product. For example, in an e-commerce platform or e-commerce server, when searching for the target product, the page including the product description is used as the first description image of the target product, or the first description image of the target product is obtained from a pre-stored database, or the first description image of the target product can be captured on the Internet.
[0099] S102: extracting a first description text of the target product from the first description picture.
[0100] Optionally, an optical character recognition method (OCR) is used to extract unstructured information such as text description from the first description image, and the extracted unstructured information such as text description is used as the first description text of the target product.
[0101] In some implementations, the description of the target product may include important product selling point information such as function description, appearance description, and feature description.
[0102] The key to ORC text recognition lies in text detection and text recognition, which is also where deep learning technology can fully exert its effectiveness. Optionally, ORC text recognition can include a text detection framework and a text recognition framework, wherein the text detection frame is shown in 1a, and the text detection frame is used to detect irregular shaped text, and the text recognition framework is shown in 1b, and the text recognition framework is used to identify the text content therein and obtain the first description text of the target product.
[0103] Among them, the text detection framework can use the DBNet algorithm, which is a segmentation-based text detection algorithm. Among various text detection algorithms, the segmentation-based detection algorithm can better handle irregular-shaped texts such as curved ones, so it can achieve better detection results. However, the process of converting the segmentation results into detection boxes in the processing steps after segmentation is complicated and time-consuming. Therefore, the DBNet algorithm proposes a differentiable binarization module (Differentiable Binarization, referred to as DB), which adds the binarization threshold to the training and learning, and can obtain more accurate detection boundaries, thereby simplifying the post-processing process.
[0104] The text recognition framework can adopt the CRNN+Attention method. The CRNN network architecture includes convolutional layer, recurrent layer and transcription layer. The convolutional layer adopts deep convolutional neural network CNN to extract features of the first description image. The recurrent layer uses Attention+bidirectional recurrent neural network to predict the feature sequence and output the predicted label distribution. The transcription layer uses CTC loss to convert a series of label distributions obtained from the recurrent layer into the final label sequence.
[0105] S103, performing tag word matching on the first description text in a pre-established tag dictionary tree to obtain one or more target tag words of the target product, wherein the tag dictionary tree is generated by first candidate tag words extracted from second description texts of multiple candidate products.
[0106] In the example of the present application, there are multiple types of candidate products in the e-commerce platform or e-commerce server, and a label dictionary tree can be pre-established through the label words of the candidate products. In some implementations, the second description picture corresponding to each candidate product is obtained, and the second description text corresponding to the candidate product is extracted from the second description picture. Further, the first candidate label word is extracted from the second description text of each candidate product, and a label dictionary tree is constructed based on the first candidate label word. Optionally, the characters in the first candidate label word are used as nodes, and the relationship between the nodes is obtained, and edges can be established between the nodes with relationships to form a label dictionary tree. Furthermore, in order to improve query efficiency, a suffix tree can be pre-established for each node on the label dictionary tree.
[0107] The suffix tree contains all the suffixes of one or more strings, and the empty string is also considered as one of the suffixes. For example, for the string bananas, all its suffixes are: bananas, ananas, nanas, anas, nas, as, s, $ (indicating empty). Optionally, in the embodiment of the present application, based on the AC automaton algorithm, the first description text corresponding to the target product is matched with the label word in the label dictionary tree to obtain the target label word of the target product.
[0108] In some implementations, a search is performed in the tag dictionary tree based on the AC automaton algorithm to obtain a candidate text string, the first position of the candidate text string is swapped, and the candidate text string is searched based on the suffix tree to obtain one or more target tag words for the target product. Through the finite automaton of the AC automaton algorithm, multiple character comparisons are converted into state transitions, and when there is a tag word match, the suffix tree is searched again to obtain the final search result, and no backtracking is required, saving a lot of query time.
[0109] In an embodiment of the present application, text recognition is performed on the first description image of the target product to extract the description information therein as the first description text. The first description text embodies the label word of the target product. Therefore, label word matching is performed on the first description text based on the AC automaton algorithm to obtain the target label word of the target product. In the embodiment of the present application, the AC automaton algorithm is used to perform label word matching, which can save the query time of label word matching and save computing costs. The first description text obtained based on text recognition is used as the basis for label word matching, thereby ensuring the accuracy of target label word extraction of the target product.
[0110] Figure 2 A flow chart of another method for mining product tags provided in an embodiment of the present application. Figure 2 As shown, the method includes but is not limited to the following steps:
[0111] S201, extracting a second description text of the candidate commodity from a second description picture of the candidate commodity.
[0112] Optionally, the candidate commodities may be electronic products, food, chemicals, clothing, hardware and other items that can be sold online, and a tag dictionary tree is generated according to the tags of the candidate commodities as a basis for subsequent tag matching.
[0113] The second description image of each candidate product is obtained. Optionally, the second description image of the candidate product may be a product information description page of the candidate product.
[0114] Optionally, the ORC text recognition method can be used to perform text scanning and other unstructured information extraction on the second description image, and the extracted unstructured information such as text description can be used as the second description text of the candidate product. The second description text can be important product selling point information such as function description, appearance description and feature description.
[0115] S202, segment the second description text to obtain multiple text strings.
[0116] Optionally, the method for segmenting the second description text may be: jieba word segmentation, reverse maximum matching algorithm, forward matching method, etc., which are not limited here.
[0117] The purpose of segmenting the second description text to obtain text strings is to obtain the label words of the product, so as to facilitate the mining of product labels and assist in the generation of subsequent label dictionary trees.
[0118] For example, assuming that the initial sentence of the first description text is "anti-fouling and durable, more skin-friendly and natural, long-lasting excellent touch and elasticity, real-time comfort", the text string obtained by using jieba word segmentation can be "anti-fouling / durable / , more / skin-friendly / natural / , / long-lasting / excellent / touch / and / elasticity / , / real-time / comfortable", such as Figure 2a shown.
[0119] S203: Determine a representation value of the text string according to the mutual information and left-right information entropy of the text string.
[0120] Mutual information is used to reflect the degree of mutual dependence between variables. Binary mutual information refers to the amount of correlation between two events. The higher the value of mutual information, the higher the correlation between the two events. In the embodiment of the present application, for any two adjacent text strings, if the value of the mutual information between the two text strings is high, the correlation between the two text strings is high, and the possibility of their corresponding formation of new words is greater; conversely, if the value of the mutual information between the two text strings is small, the correlation between the two text strings is small, and the possibility of their corresponding formation of new words is smaller.
[0121] The mutual information is defined as follows:
[0122]
[0123] Where MI(X,Y) represents the mutual information between two random variables X and Y; p(x,y) represents the joint probability distribution function of the two random variables X and Y; p(x) and p(y) represent the marginal probability distribution functions of the two random variables X and Y, respectively.
[0124] In the embodiment of the present application, the point-to-point mutual information is used to reflect the mutual information between two specific text strings. The mutual information between text strings can be understood as the internal cohesion between the text strings. Optionally, the calculation of the point-to-point mutual information is:
[0125]
[0126] Among them, PMI(x,y) represents the mutual information between text string x and text string y, p(x,y) represents the joint probability distribution function between text string x and text string y; p(x) and p(y) represent the marginal probability distribution functions between text string x and text string y respectively.
[0127] When text string x and text string y are independent of each other, that is, text string x and text string y are unrelated, then p(x, y) = p()(), and the corresponding mutual information PMI(x, y) between text string x and text string y = 0; when the correlation between text string x and text string y is greater, the joint probability distribution function p(x, y) between text string x and text string y is larger than p(x)p(y), and the value of mutual information PMI(x, y) is larger.
[0128] Furthermore, the left and right information entropy represents the measure of the uncertainty of random variables. The larger the entropy value, the greater the information content, the higher the corresponding uncertainty, and the more difficult it is to predict. For a random variable X, the corresponding information entropy value is calculated as:
[0129] H(X)=-∑ x∈X p(x)log2p(x)
[0130] Among them, H(X) represents the information entropy of random variable X; p(x) represents the probability of event x occurring.
[0131] In the embodiment of the present application, the point-to-point mutual information can be understood as the degree of solidification of two text strings. In the specific calculation of the point-to-point mutual information, p(x, y) actually represents the probability of text string x and text string y appearing together, and p(x) and p(y) represent the probability of text string x and text string y appearing together, respectively. For example, assuming that in a document, "deep learning" appears 10 times, "depth" appears 15 times, and "learning" appears 20 times, and the total number of words in the document library is a fixed value N, then the point-to-point mutual information of "deep learning" on "depth" and "learning" is The greater the mutual information between points, the more often the two text strings appear together, which means that the two text strings are more cohesive and the greater the possibility that they form a new word.
[0132] In the embodiment of the present application, the left and right information entropy can be understood as the degree of freedom of a text string. For example, suppose that in a certain website, the word "quilt" appears 956 times, and the word "life" appears 2330 times. The left neighboring words of "quilt" are relatively rich in usage, such as "sun quilt", "quilt", "strip quilt", "in quilt", "bed quilt", "fold quilt", "lift quilt", "new quilt" and "grab quilt" and more than 100 different usages. The left information entropy of its left neighboring words can be calculated to be H1 accordingly, while the left neighboring words of "life" are relatively few, generally "last life", "next life", There are more than a dozen usages of "quilt", "a lifetime", "eight lifetimes", "several lifetimes" and "which lifetime", and the left information entropy of its left neighboring character can be calculated accordingly as H2. Since information entropy is used to reflect the degree of uncertainty, the left information entropy H1 must be greater than the left information entropy H2, which means that the uncertainty of the word "quilt" is obviously greater, and it is very likely that "quilt" will be used as a word; therefore, the larger the left and right entropy values are, the richer the surrounding words of the text string are, which means that the text string has a greater degree of freedom and the possibility of it being called an independent word is greater.
[0133] In some implementations, in order to facilitate the description of the features of the text string, the left and right information entropies and mutual information of each text string are normalized respectively.
[0134] It should be noted that the left and right information entropies refer to the left information entropy and right information entropy of a text string. The left information entropy is obtained based on the left neighboring word of the text string, and the right information entropy is obtained based on the right neighboring word of the text string.
[0135] The minimum value of the left information entropy and the right information entropy corresponding to the normalized text string is selected, and the minimum value is added to the normalized mutual information to obtain the representation value of the corresponding text string.
[0136] S204: Select a text string whose representation value is greater than a preset threshold from the multiple text strings as a second candidate tag word.
[0137] After obtaining the representation value of each text string, the larger the representation value, the greater the degree of solidification and freedom of the text string, and the greater the possibility that the text string is a separate word. Therefore, a text string with a representation value greater than a preset threshold is selected from multiple text strings as the second candidate label word.
[0138] S205: Determine a first candidate tag word from the second candidate tag words, and generate the tag dictionary tree based on the first candidate tag word.
[0139] Optionally, when determining the second candidate tag word, the second candidate tag word may be filtered for common words to obtain the first candidate tag word.
[0140] In some implementations, the filtering of common words may be based on a pre-established common word list, and the second candidate tag words may be filtered for common words.
[0141] For example, for the initial sentence "anti-fouling and durable, more skin-friendly and natural, long-lasting excellent touch and elasticity, real-time comfortable", the initial sentence is segmented to obtain multiple text strings, such as "anti-fouling / durable / , more / skin-friendly / natural / , / long-lasting / excellent / touch / and / elasticity / , / real-time / comfortable", after new word mining through mutual information and left-right entropy, the text string result may be "anti-fouling and durable / , more / skin-friendly and natural / , / long-lasting / excellent / touch / and / elasticity / , / real-time / comfortable", and the result of the second candidate label word after further filtering the common words is "anti-fouling and durable / skin-friendly and natural / excellent touch / elasticity / comfortable", such as Figure 2b shown.
[0142] Furthermore, a tag dictionary tree is generated based on the first candidate tag word.
[0143] Optionally, the characters in the first candidate label word are used as nodes, and the relationship between the nodes is obtained, and edges can be established between the nodes with relationships to form a label dictionary tree. Furthermore, in order to improve query efficiency, a suffix tree can be pre-established for each node on the label dictionary tree. The second candidate label word is a more accurate word found by processing the text string. The first candidate label word is obtained after filtering the second candidate label word with common vocabulary. A label dictionary tree is generated based on the selected first candidate label word. The label dictionary tree can be understood as a set of product labels, which is used as a reference basis for subsequent matching of product label words.
[0144] S206, obtaining a first description picture of the target product.
[0145] In the embodiment of the present application, the implementation method of step S206 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0146] S207: extracting a first description text of the target product from the first description picture.
[0147] In the embodiment of the present application, the implementation method of step S207 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0148] S208, performing tag word matching on the first description text in a pre-established tag dictionary tree to obtain one or more target tag words of the target product, wherein the tag dictionary tree is generated by first candidate tag words extracted from second description texts of multiple candidate products.
[0149] In the embodiment of the present application, the implementation method of step S208 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0150] In an embodiment of the present application, the first description text of the candidate product is extracted, and the first description text is segmented to obtain a text string, the mutual information and left and right entropies of the text string are calculated to obtain a characterization value, and a second candidate label word is selected from the text string based on the characterization value to generate a label dictionary tree. In an embodiment of the present application, the characterization value calculated by mutual information and left and right entropy reflects the degree of solidification and freedom of the text string. The second candidate label word selected by the characterization value is more accurate and reasonable, and after obtaining the second candidate label word, the second candidate label word is filtered for common vocabulary to obtain the first candidate label word, and the label dictionary tree is generated using the filtered first candidate label word, so that the label dictionary tree is more meaningful for reference, and the matching of label words for the target product based on the label dictionary tree is more accurate, and the target label word of the target product obtained is also more accurate.
[0151] Figure 3 A flow chart of another method for mining product tags provided in an embodiment of the present application. Figure 3 As shown, the method includes but is not limited to the following steps:
[0152] S301, determine each node on the tag dictionary tree, and establish a suffix tree for each node.
[0153] like Figure 3a As shown, it shows a label dictionary tree and a suffix tree of a node. Since the label dictionary tree is generated by the first candidate label word, each node on the label dictionary tree corresponds to a character in the first candidate label word, that is, for the first candidate label word "comfortable and breathable", each character "comfortable", "suitable", "permeable" and "breathable" is a node.
[0154] In order to save search time, a suffix tree is established for each node; for example, when using the tag dictionary tree to scan the phrase "comfortable and breathable", "comfortable", "comfortable", "comfortable and breathable" are scanned starting from "comfortable", and then "comfortable and breathable" are scanned, and then "suitable" is returned to "suitable" and "suitable and breathable", and then "suitable and breathable" is returned to "air" and "air-permeable", "air-permeable and comfortable", "air-permeable and comfortable and comfortable", etc.; if while scanning "comfortable and breathable", you can know whether "suitable and breathable", "breathable" and "air" are in the tag dictionary tree, you can omit these three queries.
[0155] Since "breathable", "ventilated", and "air" have a shared progressive suffix, after reversing the order of the beginning and the end, "air", "air-ventilated", and "air-ventilated-breathable" can恰好利用另一颗前缀树索引,因此将其称之为后缀树。
[0156] S302, search the first description text in the label dictionary tree based on the Aho-Corasick automaton algorithm to obtain candidate text strings.
[0157] The Aho-Corasick automaton algorithm is a multi-pattern matching algorithm, that is, based on the Aho-Corasick automaton algorithm, search and extract product labels in the first description text to obtain candidate text strings.
[0158] The core idea of the Aho-Corasick automaton algorithm is to convert character comparison into state transition through a finite automaton. When reaching certain specific states, it indicates that pattern matching occurs; the Aho-Corasick automaton algorithm is based on the structure of the trie tree and combines the idea of KMP (Knuth-Morris-Pratt). It consists of a trie tree + mismatch information. There are two steps to establish the Aho-Corasick automaton algorithm: (1) The basic trie tree structure: construct a trie tree from all pattern strings; (2) The KMP idea: construct mismatch pointers for all nodes on the trie tree to establish a suffix tree, saving a large amount of queries.
[0159] The principle of the Aho-Corasick automaton algorithm is to build a suffix tree for each node on the prefix tree based on the prefix tree. The suffix tree of each node is composed of all suffixes of that node.
[0160] Exemplarily, assume that a passage containing m characters is given, and then n words are given, where both m and n are positive integers. The function of the Aho-Corasick automaton algorithm is to find out how many words appear in the passage; that is, in the embodiments of the present application, the Aho-Corasick automaton algorithm is used to find the candidate label words that appear in the label dictionary tree, and the found candidate label words are used as candidate text strings.
[0161] It should be noted that when searching the first description text based on the Aho-Corasick automaton algorithm, the first description text should be a text string obtained after segmentation, and the characterization value should be determined based on the mutual information and left-right information entropy of the text string, and the candidate label words in the first description text of the target product should be screened according to the characterization value. That is to say, the Aho-Corasick automaton algorithm searches for the candidate label words in the first description text, and the basis for the search is the label dictionary tree. The candidate label words that can be searched in the label dictionary tree are used as candidate text strings.
[0162] S303, reverse the order of the beginning and the end of the candidate text string, and search the candidate text string based on the suffix tree to obtain one or more target label words of the target product.
[0163] The AC automaton algorithm is a multi-pattern matching algorithm, and the purpose of building a suffix tree for each node is to avoid a large amount of backtracking and reduce the query time. Since the suffix tree is built based on all the suffixes of each node, the candidate text string is swapped at the beginning and the end, and the swapped candidate text string is searched in the suffix tree to obtain the target label word corresponding to the target product, which can save search time and cost.
[0164] In the actual search process of the AC automaton algorithm, while recording the candidate text strings corresponding to the complete path, all path suffixes and their text strings can also be found. In other words, the AC automaton algorithm can search for candidate text strings based on the suffix tree corresponding to each node.
[0165] In an embodiment of the present application, based on the prefix tree, a suffix tree is established for each node of the prefix tree, and the AC automaton algorithm is used to extract candidate label words in the first description text of the target product for query to obtain the target label word of the target product. In an embodiment of the present application, by establishing a suffix tree, a lot of query time and cost are saved, and the target label word mining of the target product is faster.
[0166] Figure 4 A flow chart of another method for mining product tags provided in an embodiment of the present application. Figure 4 As shown, the method includes but is not limited to the following steps:
[0167] S401, obtaining a first description image of a target product.
[0168] In the embodiment of the present application, the implementation method of step S401 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0169] S402: Extract a first description text of the target product from the first description picture.
[0170] In the embodiment of the present application, the implementation method of step S402 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0171] S403, performing tag word matching on the first description text in a pre-established tag dictionary tree to obtain one or more target tag words of the target product, wherein the tag dictionary tree is generated by first candidate tag words extracted from second description texts of multiple candidate products.
[0172] In the embodiment of the present application, the implementation method of step S403 can be implemented by any method in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0173] S404: Determine whether each target tag word is compatible with the target product.
[0174] Optionally, after performing label word matching on the first description text to obtain a target label word, a rationality judgment may be performed on the target label word of the target product to determine whether the target label word matches the attribute of the corresponding target product.
[0175] Optionally, feature extraction can be performed on the target product to obtain wide features of the target product, wherein the wide features include cross features, discrete features and some continuous features of the target product; the target label word is semantically encoded by a pre-trained language model to obtain an embedded vector representation of the target label word; the embedded vector representation is input into a long short-term memory network to extract semantic information in the forward and backward directions to obtain an encoding vector carrying context information; the wide features and the encoding vector are concatenated to obtain a concatenated vector, and binary classification is performed based on the concatenated vector to obtain a classification label of the target label word, and the classification label is used to indicate whether the target label word is compatible with the target product.
[0176] In some implementations, the wide features of the target product may specifically include the target product's attribute words, product words, part of speech, word frequency, label length, and PV value, etc., which are relatively broad features used to characterize the target product, and the wide features are expressed in vector form.
[0177] Among them, the product word of the target product can be the product name of the label mined from the product description information, which is used to represent a certain type of product corresponding to the label; the attribute word can reflect the characteristics of a certain type of product and can be used to represent the attribute characteristics of this type of product; the word frequency can reflect the rationality of the target product label. The larger the word frequency, the more times it is mentioned, and the more it can represent the target product; the label length can filter out some unintentional labels; the PV value can represent the number of occurrences of the target label word, which can also be understood as the search popularity of the label. The higher the search popularity, the stronger the possibility of becoming a word.
[0178] Optionally, the pre-trained language model may be a BERT model, that is, the target label word of the target product is input into the BERT model for semantic encoding to obtain an embedding vector of the target label word.
[0179] Furthermore, the embedded vector is input into the long short-term memory network for semantic information extraction in the forward and backward directions. Since the long short-term memory network can learn what information to remember and what information to forget during the training process, the long short-term memory network can better capture dependencies over longer distances. In order to facilitate the extraction of semantic information in the forward and backward directions of the embedded vector, the embodiment of the present application uses a BiLSTM composed of a forward long short-term memory network and a backward long short-term memory network to extract semantic information and obtain a coding vector carrying context information.
[0180] Concatenate the wide feature with the encoding vector. Vector concatenation refers to the concatenation of elements in the vector to obtain a concatenated vector. The concatenated vector contains the feature information of the target product and the feature information of the target label word. Based on the concatenated vector, the association between the features is extracted and the binary classification result is output.
[0181] Optionally, the binary classification result can be 0 and 1, that is, the classification labels are 0 and 1, 1 indicates that the target label word is compatible with the target product, and 0 indicates that the target label word is not compatible with the target product.
[0182] like Figure 4a As shown, it shows a schematic diagram of outputting binary classification results based on wide features and encoding vector concatenation.
[0183] S405: Remove target label words that are not suitable for the target product to obtain a first target label word for the target product.
[0184] The target label words that are not suitable for the target product are removed, and all the target label words that are suitable for the target product are retained as the first target label words.
[0185] Optionally, after obtaining the first target label word, the first target label word can be supplemented based on synonym replacement to obtain a second target label word; the second target label word can be checked for duplication based on phrase replacement, and repeated label words can be deleted to obtain a third target label word for the target product.
[0186] For example, assuming that the target product is canvas shoes, the first target label words of the target product are "comfortable foot feel", "breathable and perspiration-wicking", "comfortable foot feel" and "fine workmanship". Taking any first target label word "comfortable foot feel" as an example, the first target label word "comfortable foot feel" is supplemented based on synonym replacement. The supplemented label words can be "comfortable foot feel", "comfortable on feet", etc. The supplemented label words are merged with the original first target label words to obtain the second target label words.
[0187] Furthermore, the phrase replacement in the second target label word is checked for duplicates. In the embodiment of the present application, the phrase replacement refers to a sequential replacement. For example, the phrase replacement of "comfortable foot feel" obtains "comfortable foot feel". The duplicate label word "comfortable foot feel" is checked for duplicates in the second target label word. Therefore, the duplicate label word is deleted to obtain the third label word of the target product, that is, the first target label word "comfortable foot feel", "breathable and perspiration-wicking", "comfortable foot feel" and "fine workmanship" are deleted from "comfortable foot feel", and the third target label words are "comfortable foot feel", "breathable and perspiration-wicking" and "fine workmanship".
[0188] In some implementations, after obtaining the third target label word of the target product, the product name of the target product may be obtained, and the product name may be combined with each third target label to obtain a final fourth target product label of the target product.
[0189] For example, Figure 4b As shown, assuming that the product name of the target product is canvas shoes, the third target label words of the target product include: "comfortable foot feeling", "breathable and perspiration-wicking" and "fine workmanship", and the fourth target product labels obtained by combining the product name with each third target label word are: "comfortable foot feeling canvas shoes", "breathable and perspiration-wicking canvas shoes" and "fine workmanship canvas shoes".
[0190] In the embodiment of the present application, after obtaining the target label word of the target commodity, it is determined whether the target label word is compatible with the actual attribute of the target commodity to screen out the first target label word that is compatible, and further, the first target label word is screened for a second time by phrase replacement to obtain the third target label word of the target commodity, and the fourth target commodity label of the target commodity can also be obtained based on the third target label word of the target commodity and the commodity name of the target commodity. In the embodiment of the present application, by extracting features of the target commodity and semantically encoding the target label word to obtain a classification label of whether the target label word is compatible with the target commodity, the accuracy of the first target label word corresponding to the target commodity is ensured, and interference information that cannot characterize the selling point information of the target commodity is screened; further, the first target label word is screened for a second time by phrase replacement to obtain the third target label word, so that the third target label word describing the target commodity is more accurate and reliable, significantly improving the quality of the label word describing the selling point information of the target commodity, and obtaining the fourth target commodity label based on the more accurate third target label word and the commodity name of the target commodity, the fourth target commodity label has a high accuracy rate and accurate description, and increases the rationality of commodity label mining.
[0191] Based on the above embodiments, Figure 4cAs shown, the target product may be canvas shoes, and the first description picture corresponding to the canvas shoes is obtained, and the first description text is extracted from the first description picture. The first description text may be "comfortable foot feel, breathable lining, fine workmanship, comfortable breathable and perspiration-wicking, breathable and comfortable foot feel, versatile and fine workmanship, and lightweight". The first description text is matched with labels to obtain target label words, and the target label words may be "comfortable foot feel, breathable and perspiration-wicking, comfortable foot feel, and fine workmanship". After obtaining the target label words, it is necessary to judge the rationality of the target label words and check for duplicates and delete them. Since "comfortable foot feel" and "comfortable foot feel" are repeated label words, "comfortable foot feel" is deleted, and all target label words are reasonable label words for the target product canvas shoes. Therefore, the third target label word of the determined target product canvas shoes may be "comfortable foot feel, breathable and perspiration-wicking, and fine workmanship". The fourth target product label obtained based on the third target product label and the product name of the target product may be "comfortable foot feel canvas shoes, breathable and perspiration-wicking canvas shoes, and fine workmanship canvas shoes".
[0192] Optionally, the target product can also be a T-shirt, and the first description text corresponding to the T-shirt can be "50 machine washes without deformation, stiff and non-shrinking, antibacterial and bacteriostatic, moisture absorbent and quick-drying, rebound and no fading", and the target label word obtained by matching the label words of the first description text can be "antibacterial and bacteriostatic, stiff and non-shrinking, moisture absorbent and quick-drying", and the third target word obtained after rationality judgment and duplicate checking and deletion of the target label words can be "antibacterial and bacteriostatic, stiff and non-shrinking, moisture absorbent and quick-drying", and the fourth target product label obtained by further combining with the product name can be "antibacterial and bacteriostatic T-shirt, stiff and non-shrinking T-shirt, moisture absorbent and quick-drying T-shirt".
[0193] Optionally, the target product can also be an electric toothbrush. The first description text of the electric toothbrush can be "super cleaning power, high-density hair transplantation, full body wash". The target label word obtained by matching the first description text can be "full body wash, high-density hair transplantation, super cleaning power". After judging the rationality of the target label word and checking for duplicates and deleting it, the third target word obtained can be "full body wash, high-density hair transplantation, super cleaning power". Further combined with the product name, the fourth target product label can be "full body wash electric toothbrush, high-density hair transplantation electric toothbrush, super cleaning power electric toothbrush".
[0194] Optionally, the target product may also be a windbreaker, and the first description text of the windbreaker may be “breathable and perspiration-wicking, reflective print, UV protection, portable and storageable”, and the target label words obtained by matching the label words of the first description text may be “breathable and perspiration-wicking, reflective print, UV protection, portable and storageable”, and after judging the rationality of the target label words and checking for duplicates and deleting them, the third target word obtained may be “breathable and perspiration-wicking, reflective print, UV protection, portable and storageable”, and further combined with the product name, the fourth target product label may be “breathable and perspiration-wicking windbreaker, reflective print windbreaker, UV protection windbreaker, portable and storage windbreaker”.
[0195] Figure 5 Schematic diagram of the structure of a device for mining commodity labels according to an embodiment of the present application. Figure 5 As shown, the mining device 500 for commodity tags includes:
[0196] A first acquisition module 501 is used to acquire a first description image of a target product;
[0197] A second acquisition module 502 is used to extract a first description text of the target product from the first description picture;
[0198] The target tag word acquisition module 503 is used to match the first description text with tags in a pre-established tag dictionary tree to obtain one or more target tag words for the target product, wherein the tag dictionary tree is generated by the first candidate tag words extracted from the second description texts of multiple candidate products.
[0199] In some implementations, before the first obtaining module 501, it is further configured to:
[0200] Extracting the second description text of the candidate product from the second description picture of the candidate product;
[0201] Segmenting the second description text to obtain multiple text strings;
[0202] Determine the representation value of the text string according to the mutual information and left-right information entropy of the text string;
[0203] Selecting a text string whose representation value is greater than a preset threshold from multiple text strings as a second candidate label word;
[0204] Generate a tag dictionary tree based on the selected second candidate tag word.
[0205] In some implementations, before the first acquisition module 501, it is used to:
[0206] Perform common word filtering on the second candidate tag word to obtain the first candidate tag word;
[0207] Generate a tag dictionary tree based on the first candidate tag word.
[0208] In some implementations, the target tag word acquisition module 503 is used to:
[0209] Determine each node on the tag dictionary tree and build a suffix tree for each node;
[0210] Search the first description text in the tag dictionary tree based on the AC automaton algorithm to obtain a candidate text string;
[0211] The first digit of the candidate text string is swapped, and the candidate text string is searched based on the suffix tree to obtain one or more target tag words of the target product.
[0212] In some implementations, after the target tag word acquisition module 503, it is also used to:
[0213] Determine whether each target label word is compatible with the target product;
[0214] The target label words that are not suitable for the target product are removed to obtain the first target label word of the target product.
[0215] In some implementations, after the target tag word acquisition module 503, it is also used to:
[0216] Supplement the first target label word based on synonym replacement to obtain a second target label word;
[0217] The second target label words are checked for duplication based on phrase replacement, and repeated label words are deleted to obtain the third target label words of the target product.
[0218] In some implementations, after the target tag word acquisition module 503, it is also used to:
[0219] The product name of the target product is obtained, and the product name is combined with each third target label to obtain a final fourth target product label of the target product.
[0220] In some implementations, after the target tag word acquisition module 503, it is used to:
[0221] Extract features of the target product to obtain wide features of the target product, where the wide features include cross features, discrete features, and some continuous features of the target product;
[0222] The pre-trained language model performs semantic encoding on the target label word to obtain the embedded vector representation of the target label word;
[0223] The embedded vector representation is input into the long short-term memory network to extract semantic information in the forward and backward directions, and an encoding vector carrying context information is obtained;
[0224] The wide feature and the encoding vector are concatenated to obtain a concatenated vector, and binary classification is performed based on the concatenated vector to obtain the classification label of the target label word. The classification label is used to indicate whether the target label word is adapted to the target product.
[0225] In an embodiment of the present application, text recognition is performed on the first description image of the target product to extract the description information therein as the first description text. The first description text embodies the label word of the target product. Therefore, label word matching is performed on the first description text based on the AC automaton algorithm to obtain the target label word of the target product. In the embodiment of the present application, the AC automaton algorithm is used to perform label word matching, which can save the query time of label word matching and save computing costs. The first description text obtained based on text recognition is used as the basis for label word matching, thereby ensuring the accuracy of target label word extraction of the target product.
[0226] Figure 6 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Figure 6 As shown, the electronic device 600 includes a mining device 500 for commodity tags. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.
[0227] According to an embodiment of the present application, an electronic device is also provided, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement the method for mining product tags as described above.
[0228] In order to implement the above embodiments, the present application also proposes a storage medium.
[0229] When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the method for mining product tags as described above.
[0230] In order to implement the above embodiments, the present application also provides a computer program product.
[0231] When the computer program product is executed by a processor of an electronic device, the electronic device is enabled to execute the method as described above.
[0232] Figure 7 The present invention is a block diagram of an electronic device according to an exemplary embodiment. Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0233] like Figure 7 As shown, the electronic device 700 includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a memory 706 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0234] The following components are connected to the I / O interface 705: a memory 706 including a hard disk, etc.; and a communication part 707 including a network interface card such as a LAN (Local Area Network) card, a modem, etc., and the communication part 707 performs communication processing via a network such as the Internet; a drive 708 is also connected to the I / O interface 705 as needed.
[0235] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 707. When the computer program is executed by the processor 701, the above-mentioned functions defined in the method of the present application are executed.
[0236] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by the processor 701 of the electronic device 700 to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0237] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0238] Figure 8 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application. Figure 8 As shown, the electronic device 800 includes a processor 801 and a memory 802. The memory 802 is used to store program codes, and the processor 801 is connected to the memory 802 to read the program codes from the memory 802 to implement the method for mining product tags in the above embodiment.
[0239] Optionally, the number of processors 801 may be one or more.
[0240] Optionally, the electronic device may further include an interface 803, and the number of the interface 803 may be multiple. The interface 803 may be connected to an application program and may receive data from an external device such as a sensor.
[0241] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0242] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for mining product tags, characterized in that: include: Get the first description image of the target product; Extracting a first description text of the target product from the first description picture; In a pre-established tag dictionary tree, tag word matching is performed on the first description text to obtain one or more target tag words of the target product, wherein the tag dictionary tree is generated by first candidate tag words extracted from second description texts of multiple candidate products.
2. The method according to claim 1, characterized in that Before obtaining the first description picture of the target product, the method further includes: Extracting a second description text of the candidate product from the second description picture of the candidate product; Segmenting the second description text to obtain multiple text strings; Determining a representation value of the text string according to the mutual information and left-right information entropy of the text string; Selecting a text string whose representation value is greater than a preset threshold from the multiple text strings as a second candidate label word; The tag dictionary tree is generated according to the selected second candidate tag word.
3. The method according to claim 2, characterized in that The step of generating the tag dictionary tree according to the selected second candidate tag word includes: Perform common word filtering on the second candidate tag words to obtain the first candidate tag words; The tag dictionary tree is generated based on the first candidate tag word.
4. The method according to claim 2, characterized in that: The step of matching the first description text with tags in the pre-established tag dictionary tree to obtain one or more target tags for the target product includes: Determine each node on the tag dictionary tree and establish a suffix tree for each node; Searching the first description text in the tag dictionary tree based on the AC automaton algorithm to obtain a candidate text string; The first digit of the candidate text string is swapped, and the candidate text string is searched based on the suffix tree to obtain one or more target tag words of the target product.
5. The method according to any one of claims 1 to 4, characterized in that After obtaining one or more target tag words of the target product, the method further includes: Determine whether each target tag word is compatible with the target product; Target label words that are not suitable for the target product are removed to obtain a first target label word for the target product.
6. The method according to claim 5, characterized in that After removing the target label words that are not suitable for the target product to obtain the first target label word for the target product, the method further includes: Supplementing the first target label word based on synonym replacement to obtain a second target label word; The second target label word is checked for duplication based on phrase replacement, and repeated label words are deleted to obtain a third target label word for the target product.
7. The method according to claim 5, characterized in that After obtaining the third target tag word of the target product, the method further includes: The product name of the target product is obtained, and the product name is combined with each of the third target tags to obtain a final fourth target product tag of the target product.
8. The method according to claim 5, characterized in that The determining whether each target tag word is compatible with the target product includes: Extracting features of the target product to obtain wide features of the target product, wherein the wide features include cross features, discrete features, and partial continuous features of the target product; The target label word is semantically encoded by a pre-trained language model to obtain an embedded vector representation of the target label word; Inputting the embedding vector representation into a long short-term memory network to extract semantic information in forward and backward directions, thereby obtaining an encoding vector carrying context information; The wide feature and the encoding vector are concatenated to obtain a concatenated vector, and binary classification is performed based on the concatenated vector to obtain a classification label of the target label word, where the classification label is used to indicate whether the target label word is adapted to the target product.
9. A device for mining commodity labels, characterized in that: include: A first acquisition module, used to acquire a first description image of a target product; A second acquisition module, configured to extract a first description text of the target product from the first description picture; A target label word acquisition module is used to match the first description text with labels in a pre-established label dictionary tree based on an AC automaton algorithm to obtain one or more target label words for the target product, wherein the label dictionary tree is generated by first candidate label words extracted from second description texts of multiple candidate products.
10. The device according to claim 9, characterized in that Before the first acquisition module, it is also used for: Extracting a second description text of the candidate product from the second description picture of the candidate product; Segmenting the second description text to obtain multiple text strings; Determining a representation value of the text string according to the mutual information and left-right information entropy of the text string; Selecting a text string whose representation value is greater than a preset threshold from the multiple text strings as a second candidate label word; The tag dictionary tree is generated according to the selected second candidate tag word.
11. The device according to claim 10, characterized in that Before the first acquisition module, it is used to: Perform common word filtering on the second candidate tag words to obtain the first candidate tag words; The tag dictionary tree is generated based on the first candidate tag word.
12. The device according to claim 10, characterized in that The target tag word acquisition module is used to: Determine each node on the tag dictionary tree and establish a suffix tree for each node; Searching the first description text in the tag dictionary tree based on the AC automaton algorithm to obtain a candidate text string; The first digit of the candidate text string is swapped, and the candidate text string is searched based on the suffix tree to obtain one or more target tag words of the target product.
13. The device according to any one of claims 9 to 12, characterized in that After the target tag word acquisition module, it is also used to: Determine whether each target tag word is compatible with the target product; Target label words that are not suitable for the target product are removed to obtain a first target label word for the target product.
14. The device according to claim 13, characterized in that After the target tag word acquisition module, it is also used to: Supplementing the first target label word based on synonym replacement to obtain a second target label word; The second target label word is checked for duplication based on phrase replacement, and repeated label words are deleted to obtain a third target label word for the target product.
15. The device according to claim 13, characterized in that After the target tag word acquisition module, it is also used to: The product name of the target product is obtained, and the product name is combined with each of the third target tags to obtain a final fourth target product tag of the target product.
16. The device according to claim 13, characterized in that After the target tag word acquisition module, it is used to: Extracting features of the target product to obtain wide features of the target product, wherein the wide features include cross features, discrete features, and partial continuous features of the target product; The target label word is semantically encoded by a pre-trained language model to obtain an embedded vector representation of the target label word; Inputting the embedding vector representation into a long short-term memory network to extract semantic information in forward and backward directions, thereby obtaining an encoding vector carrying context information; The wide feature and the encoding vector are concatenated to obtain a concatenated vector, and binary classification is performed based on the concatenated vector to obtain a classification label of the target label word, where the classification label is used to indicate whether the target label word is adapted to the target product.
17. An electronic device, characterized in that: include: A device as claimed in any one of claims 9 to 16.
18. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.
19. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 8.
20. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed by a processor.
Citation Information
Patent Citations
Commodity data processing method, device and system
CN111597296A
Character string multimode fuzzy matching method based on AC automaton
CN112100361A
Invoice content generation method and system, electronic equipment and storage medium
CN112330379A
New word discovery method and device based on left and right information entropy and mutual information
CN114330336A
Automatic material marking method and device and storage medium
CN114547360A
