A method and device for extracting text labels

Abstract

The present invention discloses a method and device for extracting text labels, and relates to the field of computer technology. A specific implementation of the method includes: obtaining a target text, accessing the nodes of a pre-constructed label dictionary tree, determining the node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes, and generating an initial label set; judging whether the initial label of the initial label set exists in a set reference label set, and respectively calculating the similarity between at least one initial label not existing in the reference label set and the reference label of the reference label set; according to the similarity calculation result, selecting corresponding target reference labels for at least one initial label from the reference label set, and using the target reference labels to replace at least one initial label, and obtaining a final label set. The method realizes the automatic extraction of text labels, improves the efficiency and accuracy of label extraction, and reduces labor costs.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a method and device for extracting text labels. Background Art

[0002] On e-commerce platforms, tens of thousands of active merchants publish tens of millions of items every day. In order to get more and more accurate traffic for their items, merchants will label the items to attract relevant users and increase the conversion rate of items.

[0003] In the prior art, labels are generally generated for items in the following ways: Method 1: Backstage operators select appropriate labels based on their understanding of the items and the label system supported by the platform; Method 2: Use the label values ​​contained in the label system to match the item titles one by one to generate all possible labels; Method 3: Train the model based on artificial intelligence algorithms to automatically generate labels through the model.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] Method 1 has low processing efficiency, poor accuracy, and high labor costs; Method 2 can only extract labels that fully match the label system, and cannot extract decentralized labels or similarity labels; Method 3 relies on a large amount of training text and label training set data, and requires manual annotation. If the label system library data changes, it is necessary to collect the training set data again and retrain the model, which cannot meet business needs. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method and device for extracting text labels, which searches all possible paths for the target text according to the label dictionary tree structure to obtain an initial label set, and then determines the final label set based on the similarity between the initial label and the reference label, thereby realizing automatic extraction of text labels, improving the efficiency and accuracy of label extraction, and reducing labor costs.

[0007] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for extracting text tags is provided.

[0008] A method for extracting text labels according to an embodiment of the present invention includes: obtaining a target text, accessing nodes of a pre-constructed label dictionary tree, determining node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes, and generating an initial label set; determining whether an initial label of the initial label set exists in a set reference label set, and calculating the similarity between at least one initial label not existing in the reference label set and a reference label of the reference label set; and selecting corresponding target reference labels for the at least one initial label from the reference label set according to the similarity calculation result, and replacing the at least one initial label with the target reference label to obtain a final label set.

[0009] Optionally, according to the attribution relationship between nodes, the node value existing in the target text is determined from the label dictionary tree to generate an initial label, including: initializing an empty string, taking the currently unvisited leaf node belonging to the root node as the current node, visiting the current node, and determining the node value of the current node in the target text; judging and splicing: judging whether the current node has an unvisited leaf node, if the current node has an unvisited leaf node, and the node value of the leaf node exists in the target text, then splicing the node value of the current node and the node value of the leaf node; updating: using the spliced ​​node value to update the string, taking the leaf node as the current node; repeating the judging and splicing step and the updating step until the leaf node does not exist at the current node, and taking the updated string as an initial label.

[0010] Optionally, selecting a corresponding target benchmark tag for the current initial tag from the benchmark tag set includes: determining the maximum similarity between the current initial tag and the benchmark tags of the benchmark tag set; comparing the maximum similarity with a set threshold, and if the maximum similarity is greater than or equal to the threshold, selecting the benchmark tag corresponding to the maximum similarity as the target benchmark tag.

[0011] Optionally, the method further comprises: splitting the benchmark tags of the benchmark tag set according to a set granularity to construct the tag dictionary tree.

[0012] Optionally, the benchmark tags of the benchmark tag set are split according to a set granularity to construct the tag dictionary tree, including: adding a root node and selecting a current benchmark tag from the benchmark tag set; splitting: splitting the current benchmark tag according to the character granularity to obtain multiple single characters; judging: judging in turn whether the single characters exist in the dictionary tree path according to the original order of the single characters in the current benchmark tag; adding: if the single character does not exist in the dictionary tree path, adding a node corresponding to the single character in the dictionary tree path; selecting the next benchmark tag from the benchmark tag set as the current benchmark tag, repeating the splitting step, the judging step and the adding step until the last benchmark tag to obtain the tag dictionary tree.

[0013] Optionally, the method further includes: performing word segmentation processing on the target text according to character granularity; and determining whether a node value of the leaf node exists in the target text after word segmentation.

[0014] Optionally, calculating the similarity between at least one initial tag that does not exist in the benchmark tag set and a benchmark tag of the benchmark tag set includes: calculating the similarity between at least one initial tag that does not exist in the benchmark tag set and a benchmark tag of the benchmark tag set in any one of the following ways: calculating edit distance, cosine distance, Euclidean distance, or Jaccard distance.

[0015] To achieve the above objective, according to another aspect of an embodiment of the present invention, a device for extracting text labels is provided.

[0016] A device for extracting text labels according to an embodiment of the present invention comprises: an acquisition generation module, used to acquire a target text, access nodes of a pre-constructed label dictionary tree, determine node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes, and generate an initial label set; a similarity calculation module, used to determine whether an initial label of the initial label set exists in a set reference label set, and respectively calculate the similarity between at least one initial label not existing in the reference label set and a reference label of the reference label set; a label determination module, used to select corresponding target reference labels for the at least one initial label from the reference label set according to the similarity calculation result, and use the target reference labels to replace the at least one initial label to obtain a final label set.

[0017] Optionally, the acquisition generation module is also used to: initialize an empty string, take the currently unvisited leaf node belonging to the root node as the current node, visit the current node, and determine whether the node value of the current node exists in the target text; judge and splice: judge whether there is an unvisited leaf node for the current node, if there is an unvisited leaf node for the current node, and the node value of the leaf node exists in the target text, then splice the node value of the current node and the node value of the leaf node; update: use the spliced ​​node value to update the string, and take the leaf node as the current node; repeat the judgement and splicing steps and the update steps until the leaf node does not exist at the current node, and use the updated string as an initial label.

[0018] Optionally, the label determination module is also used to: determine the maximum similarity between the current initial label and the benchmark label of the benchmark label set; compare the maximum similarity with a set threshold, and if the maximum similarity is greater than or equal to the threshold, select the benchmark label corresponding to the maximum similarity as the target benchmark label.

[0019] Optionally, the device further comprises: a dictionary tree construction module, configured to split the benchmark tags of the benchmark tag set according to a set granularity to construct the tag dictionary tree.

[0020] Optionally, the dictionary tree construction module is also used to: add a root node and select a current benchmark tag from the benchmark tag set; split: split the current benchmark tag according to character granularity to obtain multiple single characters; judge: judge whether the single characters exist in the dictionary tree path according to the original order of the single characters in the current benchmark tag; add: if the single character does not exist in the dictionary tree path, add a node corresponding to the single character in the dictionary tree path; select the next benchmark tag from the benchmark tag set as the current benchmark tag, repeat the splitting step, the judging step and the adding step until the last benchmark tag, and obtain the tag dictionary tree.

[0021] Optionally, the device further includes: a word segmentation judgment module, which is used to perform word segmentation processing on the target text according to character granularity; and judge whether the node value of the leaf node exists in the target text after word segmentation.

[0022] Optionally, the similarity calculation module is further used to: calculate the similarity between at least one initial tag not existing in the benchmark tag set and the benchmark tag of the benchmark tag set in any one of the following ways: calculating edit distance, cosine distance, Euclidean distance, and Jaccard distance.

[0023] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0024] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement a text label extraction method according to an embodiment of the present invention.

[0025] To achieve the above objective, according to another aspect of the embodiments of the present invention, a computer readable medium is provided.

[0026] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a method for extracting a text tag according to an embodiment of the present invention is implemented.

[0027] An embodiment of the above invention has the following advantages or beneficial effects: all possible paths of the target text are searched according to the label dictionary tree structure to obtain an initial label set, and then the final label set is determined based on the similarity between the initial label and the reference label, thereby realizing the automatic extraction of text labels, improving the efficiency and accuracy of label extraction, and reducing labor costs; all possible label paths are obtained based on the depth-first traversal algorithm, improving the efficiency of path search and ensuring the integrity of path search.

[0028] An embodiment of the above invention has the following advantages or beneficial effects: through similarity and threshold comparison, the characters in the target text that do not appear in the benchmark tag set are converted, and the dispersed tags and similarity tags are successfully extracted; the benchmark tag set is reconstructed into a tag dictionary tree, which reduces the time of subsequent path search and improves query efficiency; the benchmark tags of the benchmark tag set are split at character granularity, and then a tag dictionary tree is constructed to further ensure the accuracy of the extraction of dispersed tags and similarity tags.

[0029] An embodiment of the above invention has the following advantages or beneficial effects: the target file is segmented according to the character granularity, and the characters of the target file and the node values ​​of the tag dictionary tree are compared with the same granularity to ensure the accuracy of label extraction; by calculating the similarity between the initial label that does not exist in the benchmark label set and each benchmark label, the subsequent successful extraction of dispersed labels and similarity labels is guaranteed.

[0030] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0032] Figure 1 is a schematic diagram of the main steps of the method for extracting text tags according to an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of the extraction principle of the text label extraction method according to an embodiment of the present invention;

[0034] Figure 3 is a schematic diagram of the main flow of the text label extraction method according to an embodiment of the present invention;

[0035] Figure 4 Schematic diagram of a dictionary tree path adding process according to an embodiment of the present invention;

[0036] Figure 5 Schematic diagram of the process of constructing a tag dictionary tree according to an embodiment of the present invention;

[0037] Figure 6 is a schematic diagram of the construction result of the tag dictionary tree according to an embodiment of the present invention;

[0038] Figure 7 is a schematic diagram of the main process of generating an initial tag set according to an embodiment of the present invention;

[0039] Figure 8 is a schematic diagram of main modules of a device for extracting text labels according to an embodiment of the present invention;

[0040] Fig. 9 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0041] Fig.10 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0043] The technical terms involved in the embodiments of the present invention are explained below.

[0044] Label: A perceptual classification feature abstracted from the object’s color, size, pattern, style and other information.

[0045] Dictionary tree: It is a tree structure and a variant of hash tree. Its typical application is to count, sort and save a large number of strings (but not limited to strings), so it is often used by search engine systems for text word frequency statistics. Its advantages are: using the common prefix of the string to reduce the query time, minimize unnecessary string comparisons, and the query efficiency is higher than the hash tree.

[0046] Search, recall, exposure, and conversion: these are a series of behaviors generated during the interaction between users and websites. Among them, search refers to the behavior of users entering keywords on the website according to their own needs; recall refers to the behavior of the website presenting some results that may be of interest to users after performing complex algorithm matching based on the keywords used by users in the search; exposure refers to the results exposed to users by the website through algorithm analysis; and conversion refers to the user's ordering behavior.

[0047] Edit distance: It is an algorithm for calculating text similarity. Its main theoretical basis is that if string 1 can be transformed into string 2 through operations such as addition, deletion, and modification, the fewer the number of related operations, the more similar string 1 and string 2 are.

[0048] On e-commerce platforms, merchants will label items to attract corresponding users and increase item conversion rates. For example, for the item title "xxxx casual pants men 2019 autumn and winter thick slim straight slim-fit men's pants Korean youth business suit pants elastic retro long pants men 1047 khaki 31", the corresponding labels are "Korean style", "business casual", "retro style", among which "xxxx" is the brand name. Through this label, users who are interested in "Korean style / business casual / retro style" clothing are attracted and recommended to the user to increase the product conversion rate.

[0049] In the prior art, there are many ways to add tags to items, such as manual selection by merchants, manual selection by backend operators, extracting tags by absolute matching, extracting tags by using algorithms, etc. Each method is described in detail below.

[0050] For merchants’ manual selection method, in order to attract more traffic, merchants may add some labels that do not match the item. For example, if the item in the above example is labeled “European and American style” or “Motorcycle style”, according to the search recall rules, the item will be exposed to users who are interested in “European and American style” or “Motorcycle style”. Although this attracts more user traffic, since the users themselves are not interested in such items, it is highly likely that the item conversion will not be promoted, which is invalid traffic, and at the same time it will bring a bad user experience to users. Some merchants are not familiar with the label system currently supported by the platform and find it difficult to affix appropriate labels.

[0051] For the manual selection method by backend operators, operators select appropriate labels based on their understanding of the items and the label system supported by the platform. Still taking the above item title as an example, the label "Korean version" contained in the title corresponds to the label "Business Casual", and the similar corresponding label is "Retro Style". Then, based on the business side's operational experience, it can be concluded that the labels that the item title ultimately points to are "Korean version", "Business Casual", and "Retro Style". Although this method is more reliable than the manual selection method by merchants, this method has low processing efficiency, poor accuracy, and high labor costs.

[0052] For the method of extracting labels by absolute matching, the label values ​​contained in the label system are used to match the item titles one by one to generate all possible labels. Still taking the above item title as an example, "Korean version" is a valid value of the label system and will be extracted. Although "business casual" is a valid value of the label system, since "business" and "casual" are scattered in different positions in the title text, it cannot be extracted in the form of "business casual" by forced matching; although "retro style" is also a valid value of the label system, it is not completely consistent with the keyword "retro" in the title, so it cannot be obtained by forced matching. In other words, it is impossible to extract scattered labels and similarity labels.

[0053] Regarding the method of extracting labels using algorithms, such as predicting labels based on artificial intelligence algorithms, regularization and triple relationships, and predicting labels through word2vec (a model used to generate word vectors) and label knowledge networks. The method of predicting labels by establishing a classification model based on training texts and training labels relies on a large amount of training text and label training set data, which requires a lot of manpower labeling costs. If the label system library data changes, it is necessary to collect training data again, train the corresponding model, and observe the test results to adjust the optimal parameters. It cannot adapt to the flexible and changing business characteristics of the label system; based on regularization combined with triple relationships, it is impossible to meet the requirements of existing business systems in a timely manner; the method of predicting labels based on word2vec and label knowledge networks, because word2vec itself is an unsupervised algorithm, its calculation results often have great uncertainty.

[0054] Based on the above problems, the present invention provides an automated method for extracting text tags. For merchants, more accurate user traffic can be obtained, auxiliary guidance is more effective, and the item conversion rate can be improved, so that user behavior analysis, search recommendation and other related tasks can be performed more effectively. For the e-commerce platform ecology, the accuracy and stability of search recommendations can be improved, invalid traffic can be reduced, and labor costs can be reduced. At the same time, users can also get a better experience. The text tag extraction method of the embodiment of the present invention is described in detail below.

[0055] Figure 1 FIG. 1 is a schematic diagram of the main steps of the method for extracting text tags according to an embodiment of the present invention. Figure 1 As shown, the text label extraction method of the embodiment of the present invention mainly includes the following steps:

[0056] Step S101: Obtain the target text, access the nodes of the pre-built tag dictionary tree, determine the node values ​​existing in the target text from the tag dictionary tree according to the attribution relationship between the nodes, and generate an initial tag set. The target text is the text from which the text tags are to be extracted. First, split the multiple benchmark tags of the benchmark tag set according to the set granularity, and reorganize the tag dictionary tree through the tree structure.

[0057] Then, the target text is segmented according to the set granularity, and all possible paths are searched for the segmented target text according to the structure of the label dictionary tree to obtain all label value path sequences, which constitute the initial label set.

[0058] Step S102: Determine whether the initial tag of the initial tag set exists in the set reference tag set, so as to calculate the similarity between at least one initial tag that does not exist in the reference tag set and the reference tag of the reference tag set. Determine whether the current initial tag of the initial tag set exists in the reference tag set. If it exists in the reference tag set, continue to determine the next initial tag. If it does not exist in the reference tag set, calculate the similarity between the current initial tag and multiple reference tags in the reference tag set. After the similarity calculation is completed, continue to determine the next initial tag.

[0059] Step S103: According to the similarity calculation result, select corresponding target reference tags for at least one initial tag from the reference tag set, and use the target reference tags to replace at least one initial tag to obtain a final tag set. The final tag set includes the initial tags of the initial tag set that exist in the reference tag set, and the result after the initial tags that do not exist in the reference tag set are replaced with the target reference tags. If the current initial tag does not exist in the reference tag set, determine the maximum similarity between the current initial tag and multiple reference tags in the reference tag set.

[0060] Then, the maximum similarity is compared with the set threshold. If the maximum similarity is greater than or equal to the threshold, the reference label corresponding to the maximum similarity is selected as the target reference label, and the target reference label is used to replace the current initial label of the initial label set. If the maximum similarity is less than the threshold, a prompt message is issued. In the above manner, each initial label that does not exist in the reference label set is replaced with the corresponding target reference label to obtain the final label set.

[0061] Figure 2It is a schematic diagram of the extraction principle of the text tag extraction method according to an embodiment of the present invention. As Figure 2 shown, the text tag extraction method according to an embodiment of the present invention includes two parts: prior knowledge calculation and prediction process. The prior knowledge calculation part mainly includes obtaining a benchmark tag set and generating a tag dictionary tree. In the prediction process part, first, the target text is obtained, and then, using the previously generated tag dictionary tree and combining with the depth-first traversal algorithm, an initial tag set is obtained; then, the text similarity is calculated sequentially with all the benchmark tags in the benchmark tag set, so as to obtain the final tag set as the output result of the algorithm.

[0062] Figure 3 It is a schematic diagram of the main process of the text tag extraction method according to an embodiment of the present invention. As Figure 3 shown, the text tag extraction method according to an embodiment of the present invention mainly includes the following steps:

[0063] Step S301: Obtain a benchmark tag set. Obtain multiple benchmark tags from the tag database to form a benchmark tag set. The benchmark tags can be submitted by relevant personnel and then subject to manual review. After passing the review, they can be added to the tag database, and the benchmark tags in the tag database can be manually maintained. In the embodiment, the benchmark tag set is ["Hong Kong style", "British style", "Youth casual", "Business formal", "Business casual", "European and American style", "Retro style", "Chinese style", "Military workwear", "Korean style", "Street", "Punk", "Hip-hop", "Japanese style", "Original design", "Basic mass", "Japanese and Korean style", "Ecological", "Fresh", "Casual style", "Neutral style", "Sports casual"].

[0064] It should be noted that the above benchmark tag set is only an example, and according to different application scenarios, the benchmark tag set has different values.

[0065] Step S302: Split the benchmark tags in the benchmark tag set according to the set granularity to construct a tag dictionary tree. Split each benchmark tag in the benchmark tag set according to the character granularity to construct a tag dictionary tree. Taking the benchmark tag "European and American style" as an example below, introduce how each benchmark tag participates in the construction of the tag dictionary tree.

[0066] Figure 4 It is a schematic diagram of the dictionary tree path adding process according to an embodiment of the present invention. Referring to Figure 4 , in the embodiment of the present invention, first add a root node (root), and then split "European and American style" into "Ou", "Mei", and "Feng" according to the character granularity. Then, sequentially judge whether "Ou", "Mei", and "Feng" are in the existing dictionary tree paths. If not, add corresponding nodes under the corresponding paths. According to Figure 4In the manner shown, in the tag dictionary tree, a dictionary tree path is added for each benchmark tag in the benchmark tag set to obtain a tag dictionary tree.

[0067] Step S303: Get the target text. Collect the target text of the tag to be extracted. In the embodiment, the target text is "xxxx casual pants men 2019 autumn and winter thick slim straight slim-fit men's pants Korean version youth business suit pants elastic retro long pants men 1047 khaki 31".

[0068] Step S304: Use the depth-first traversal algorithm to determine the node values ​​existing in the target text from the tag dictionary tree and generate an initial tag set. Use the depth-first search (dfs) algorithm to access the nodes of the tag dictionary tree and find all possible paths that meet the tag dictionary tree. The pseudo code is as follows:

[0069]

[0070] In the above pseudocode, L is the initial label set and s is a string.

[0071] Still taking the target text "xxxx casual pants men 2019 autumn and winter thick slim straight leg men's pants Korean version youth business suit pants elastic long retro pants men 1047 khaki 31" as an example, first the word "business" is in the leaf node of the root node of the label dictionary tree; then visit the leaf node of "business" in the label dictionary tree, which contains the word "business", and the word "business" is in the target text; then visit the node "business", whose leaf nodes have the words "normal" and "rest", and the word "rest" appears in the target text; continue to determine that the leaf node of "rest" in the label dictionary tree is the word "leisure", and the word "leisure" appears in the target text; continue to determine the leaf node of the word "leisure", and find that it has no leaf node, and the calculation process ends. That is, "business" → "business" → "rest" → "leisure" is a path that conforms to the label dictionary tree.

[0072] Similarly, "Korean" → "version" is a path that conforms to the tag dictionary tree. "Retro" → "ancient" is also a path that conforms to the tag dictionary tree. After this step, the initial tag set can be obtained as ["business casual", "Korean version", "retro"].

[0073] Step S305: Determine whether the current initial tag of the initial tag set exists in the benchmark tag set. If not, execute step S306; if yes, execute step S309. Select any initial tag from the initial tag set as the current initial tag, and query it in the benchmark tag set to determine whether the current initial tag is included in the benchmark tag set. For example, in step S304, "business casual" and "Korean version" are values ​​in the benchmark tag set, and no further similarity calculation is required.

[0074] Step S306: Calculate the similarity between the current initial label and each reference label in the reference label set respectively, and select the target reference label according to the similarity. For example, in step S304, "retro" is not a value in the reference label set, so it is necessary to calculate the similarity with each reference label in the reference label set. There are various ways to calculate similarity, such as calculating the edit distance, cosine distance, Euclidean distance, Jaccard distance, etc. Among them, the Jaccard distance is used to calculate the similarity between two individuals with symbolic measures or boolean measures.

[0075] Taking the jaro - winkler algorithm as an example, the following is an explanation. The jaro - winkler algorithm is an extension for calculating string similarity based on the edit distance. The specific calculation formula is as follows:

[0076]

[0077] In the formula, d j is the similarity, m is the number of matching characters, s 1 , s 2 are the lengths of the two strings, and t is the number of transpositions.

[0078] Taking the calculation of the jaro - winkler similarity between "retro" and "retro style" as an example: The matching characters between "retro" and "retro style" are "retro", the number of matching characters m = 2, and there is no transposition phenomenon, so the number of transpositions t = 0. The length of the "retro" string s 1 = 2, and the length of the "retro style" string s 2 = 3. Substituting into formula 1, the jaro - winkler similarity between "retro" and "retro style" = 1 / 3×(2 / 2 + 2 / 3+(2 - 0) / 2)= 0.89.

[0079] Calculate the similarity between "retro" and other reference labels in the reference label set in the above way. In the embodiment, the reference label with the maximum similarity can be used as the target reference label. If there are multiple reference labels with the maximum similarity, any one can be selected as the target reference label. It is also possible to randomly select one from the reference labels whose similarity is greater than a certain threshold as the target reference label.

[0080] Step S307: Determine whether the similarity between the target reference label and the current initial label is greater than or equal to the set threshold. If it is greater than or equal to the threshold, execute step S308; if it is less than the threshold, execute step S309. The threshold can be custom - set, assumed to be 0.8.

[0081] In a preferred embodiment, if the similarity between the target reference label and the current initial label is less than the threshold, it may be that the reference label set does not currently include similar or identical reference labels. A prompt message can be sent to facilitate the operator to manually determine whether to modify and improve the reference label set according to the current initial label.

[0082] Step S308: Replace the current initial label in the initial label set with the target reference label. In the embodiment, the reference label with the maximum similarity is used as the target reference label. Suppose that after calculation, the reference label corresponding to the maximum similarity is "retro style". Since the similarity between "retro style" and "retro" is 0.89, and 0.89 > 0.8 meets the condition. Therefore, "retro" can be converted to "retro style" as the result output.

[0083] Step S309: Determine whether there are remaining initial labels in the initial label set. If there are, execute Step S310; otherwise, end this process. The remaining initial labels are the reference labels that have not been selected.

[0084] Step S310: Select the next initial label in the initial label set as the current initial label, and execute Step S305. For each initial label in the initial label set, repeat Steps S305 - S310, and the initial label set after replacement, that is, the final label set, can be obtained.

[0085] Figure 5 It is a schematic diagram of the construction process of the label trie according to the embodiment of the present invention. As Figure 5 shown, the construction process of the label trie according to the embodiment of the present invention (that is, the implementation process of Step S302) includes the following steps:

[0086] Step S501: Add a root node and select the current reference label from the reference label set. First, add a new root node, and arbitrarily select a reference label from the reference label set as the current reference label.

[0087] Step S502: Split the current reference label according to the character granularity to obtain multiple single characters. Suppose the current reference label is "European and American style". Split it according to the character granularity, and the 3 single characters "European", "American", and "style" can be obtained.

[0088] Step S503: Determine whether the current single character exists in the trie path. If the current single character does not exist in the trie path, execute Step S504; if the current single character exists in the trie path, execute Step S505. According to the original order of the single characters in the current reference label, select the single characters in turn as the current single character in this step. That is, select "European", "American", and "style" as the current single character in turn.

[0089] Step S504: Add the node corresponding to the current single character to the trie path, and execute Step S505. Taking the current single character being "Ou" as an example, if the character "Ou" does not exist in all current trie paths, add a leaf node with the node value of "Ou" under the root node. Taking the current single character being "Mei" as another example, if the character "Mei" does not exist in all current trie paths that contain "Ou", add a leaf node with the node value of "Mei" under the "Ou" node.

[0090] Step S505: Determine whether there is a next single character. If there is, execute Step S506; if not, execute Step S507.

[0091] Step S506: Take the next single character as the current single character, and execute Step S503.

[0092] Step S507: Determine whether there is an unselected reference label in the reference label set. If there is, execute Step S508; if not, end this process.

[0093] Step S508: Select the next reference label from the reference label set as the current reference label, and execute Step S502. Select the next reference label from the reference label set as the current reference label, and repeat the execution of Step S502 - Step S508 until the last reference label, then the label trie can be obtained.

[0094] Figure 6 is a schematic diagram of the construction result of the label trie in the embodiment of the present invention. Refer to Figure 6 , taking ["European and American style", "youthful and casual", "business formal", "business casual"] as an example, a label trie is constructed. This label trie includes a root node and multiple leaf nodes, and the character in the leaf node is the node value of that leaf node. It should be noted that if new reference labels are added to the reference label set, the label trie can still be supplemented and improved in the manner shown in Figure 4 , with good flexibility.

[0095] Figure 7 is a schematic diagram of the main process for generating the initial label set in the embodiment of the present invention. As shown in Figure 7 , the process of generating the initial label set in the embodiment of the present invention (i.e., the implementation process of Step S304) includes the following steps:

[0096] Step S701: Initialize the initial label set and the string, and perform word segmentation on the target text according to the character granularity. The initial label set is represented by L, and initialize L as an empty set; the string is represented by s, and initialize s as an empty string. Split the target text into one character by one according to the granularity of Step S302.

[0097] Step S702: Determine whether the node value of the current node exists in the split target text. If it exists, execute Step S703; if not, execute Step S708. The initial value of the current node is any leaf node belonging to the root node in the label trie.

[0098] Step S703: Determine whether there are unvisited leaf nodes for the current node. If so, execute Step S704; if not, execute Step S707. Assume the current node is "business", and the unvisited leaf node of the character "business" is the character "service".

[0099] Step S704: Determine whether the node value of the leaf node exists in the split target text. If it exists, execute Step S705; if not, execute Step S708. Assume the target text is still "xxxx men's casual pants, thick style,修身直筒小脚男裤, Korean style, young business western pants, elastic long, retro pants, men's, 1047, khaki, 31", then the character "service" exists in the target text.

[0100] Step S705: Concatenate the node value of the current node and the node value of the leaf node, and update the string with the concatenated node value. The currently concatenated node value is "business service".

[0101] Step S706: Take the leaf node as the current node and execute Step S703. Then, take the character "service" as the current node and repeat Steps S703 - S706 until there are no leaf nodes for the current node and the node value concatenation ends.

[0102] Step S707: Add the updated string as an initial label to the initial label set. According to the above example, the finally obtained initial label is "business casual", and add it to the initial label set.

[0103] Step S708: Determine whether there are unvisited leaf nodes in the label trie that belong to the root node. If so, execute Step S709; if not, end this process.

[0104] Step S709: Select the next leaf node belonging to the root node as the current node and execute Step S702. For example, continue to take the character "European" as the current node and repeat Steps S702 - S709. Each time of repeating execution, an initial label can be obtained until all leaf nodes belonging to the root node are visited, and the final initial label set can be obtained.

[0105] It can be seen from the text label extraction method of the embodiment of the present invention that all possible paths are searched for the target text according to the label dictionary tree structure to obtain an initial label set, and then the final label set is determined based on the similarity between the initial label and the reference label, thereby realizing the automatic extraction of text labels, improving the efficiency and accuracy of label extraction, and reducing labor costs; all possible label paths are obtained based on the depth-first traversal algorithm, thereby improving the efficiency of path search and ensuring the integrity of path search.

[0106] It can be seen from the text label extraction method of the embodiment of the present invention that, through similarity and threshold comparison, the characters in the target text that do not appear in the benchmark label set are converted, and the dispersed labels and similarity labels are successfully extracted; the benchmark label set is reconstructed into a label dictionary tree to reduce the time of subsequent path search and improve query efficiency; the benchmark labels of the benchmark label set are split at character granularity, and then a label dictionary tree is constructed to further ensure the accuracy of the extraction of dispersed labels and similarity labels.

[0107] It can be seen from the text label extraction method of the embodiment of the present invention that the target file is segmented according to the character granularity, and the characters of the target file and the node values ​​of the label dictionary tree are compared with the same granularity to ensure the accuracy of label extraction; by calculating the similarity between the initial label that does not exist in the benchmark label set and each benchmark label, the subsequent successful extraction of dispersed labels and similarity labels is guaranteed.

[0108] Figure 8 Schematic diagram of the main modules of the text label extraction device according to an embodiment of the present invention. Figure 8 As shown, the text label extraction device 800 of the embodiment of the present invention mainly includes:

[0109] The acquisition generation module 801 is used to obtain the target text, access the nodes of the pre-built label dictionary tree, and determine the node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes to generate an initial label set. The target text is the text of the text label to be extracted. First, multiple benchmark labels of the benchmark label set are split according to the set granularity, and the label dictionary tree is generated by reorganizing the tree structure.

[0110] Then, the target text is segmented according to the set granularity, and all possible paths are searched for the segmented target text according to the structure of the label dictionary tree to obtain all label value path sequences, which constitute the initial label set.

[0111] The similarity calculation module 802 is used to determine whether the initial tag of the initial tag set exists in the set reference tag set, so as to respectively calculate the similarity between at least one initial tag that does not exist in the reference tag set and the reference tag of the reference tag set. It is determined whether the current initial tag of the initial tag set exists in the reference tag set. If it exists in the reference tag set, the next initial tag is judged; if it does not exist in the reference tag set, the similarity between the current initial tag and multiple reference tags in the reference tag set is respectively calculated. After the similarity calculation is completed, the next initial tag is judged.

[0112] The tag determination module 803 is used to select corresponding target reference tags for the at least one initial tag from the reference tag set according to the similarity calculation result, and use the target reference tags to replace the at least one initial tag to obtain a final tag set. The final tag set includes the initial tags of the initial tag set that exist in the reference tag set, and the result after the initial tags that do not exist in the reference tag set are replaced with the target reference tags. If the current initial tag does not exist in the reference tag set, the maximum similarity between the current initial tag and multiple reference tags in the reference tag set is determined.

[0113] Then, the maximum similarity is compared with the set threshold. If the maximum similarity is greater than or equal to the threshold, the reference label corresponding to the maximum similarity is selected as the target reference label, and the target reference label is used to replace the current initial label of the initial label set. If the maximum similarity is less than the threshold, a prompt message is issued. In the above manner, each initial label that does not exist in the reference label set is replaced with the corresponding target reference label to obtain the final label set.

[0114] In addition, the text tag extraction device 800 of the embodiment of the present invention may also include: a dictionary tree construction module and a word segmentation judgment module ( Figure 8 The dictionary tree construction module is used to split the benchmark tags of the benchmark tag set according to the set granularity to construct the tag dictionary tree. The word segmentation judgment module is used to perform word segmentation processing on the target text according to the character granularity; and to judge whether there is a node value of the leaf node in the target text after word segmentation.

[0115] From the above description, it can be seen that all possible paths of the target text are searched according to the label dictionary tree structure to obtain the initial label set. Subsequently, the final label set is determined based on the similarity between the initial label and the benchmark label, thereby realizing the automatic extraction of text labels, improving the efficiency and accuracy of label extraction, and reducing labor costs.

[0116] Fig. 9An exemplary system architecture 900 is shown to which the text tag extraction method or text tag extraction device according to the embodiment of the present invention can be applied.

[0117] like Fig. 9 As shown, system architecture 900 may include terminal devices 901, 902, 903, network 904 and server 905. Network 904 is used to provide a medium for communication links between terminal devices 901, 902, 903 and server 905. Network 904 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0118] Users can use terminal devices 901, 902, 903 to interact with server 905 through network 904 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 901, 902, 903, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0119] The terminal devices 901 , 902 , and 903 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0120] The server 905 may be a server that provides various services, such as a background management server that an administrator processes the target text sent by the terminal devices 901, 902, and 903. The background management server may generate an initial tag set, calculate the similarity between the initial tag and the reference tag, select a target reference tag based on the similarity, replace the initial tag with the target reference tag, and feed back the processing result (e.g., the final tag set) to the terminal device.

[0121] It should be noted that the text tag extraction method provided in the embodiment of the present application is generally executed by the server 905 , and accordingly, the text tag extraction device is generally set in the server 905 .

[0122] It should be understood that Fig. 9 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0123] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0124] The electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a text label extraction method of an embodiment of the present invention.

[0125] The computer-readable medium of the present invention stores a computer program, and when the program is executed by a processor, a method for extracting text tags according to an embodiment of the present invention is implemented.

[0126] Reference below Fig.10 , which shows a schematic diagram of the structure of a computer system 1000 suitable for implementing an electronic device of an embodiment of the present invention. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0127] like Fig.10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the computer system 1000 are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0128] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0129] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the main step diagram. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present invention are executed.

[0130] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0131] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, may be described as: a processor includes an acquisition generation module, a similarity calculation module and a label determination module. Among them, the names of these modules do not constitute a limitation on the module itself under certain circumstances. For example, the acquisition generation module may also be described as "a module that obtains the target text, accesses the nodes of a pre-built label dictionary tree, and determines the node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes, and generates an initial label set".

[0133] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining a target text, accessing the nodes of a pre-constructed tag dictionary tree, and determining the node values ​​existing in the target text from the tag dictionary tree according to the attribution relationship between the nodes, and generating an initial tag set; judging whether the initial tags of the initial tag set exist in the set reference tag set, and respectively calculating the similarity between at least one initial tag not existing in the reference tag set and the reference tag of the reference tag set; according to the similarity calculation result, selecting the corresponding target reference tag for the at least one initial tag from the reference tag set, and using the target reference tag to replace the at least one initial tag, so as to obtain a final tag set.

[0134] From the above description, it can be seen that all possible paths of the target text are searched according to the label dictionary tree structure to obtain the initial label set. Subsequently, the final label set is determined based on the similarity between the initial label and the benchmark label, thereby realizing the automatic extraction of text labels, improving the efficiency and accuracy of label extraction, and reducing labor costs.

[0135] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0136] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting text labels, characterized in that: include: The benchmark labels of the benchmark label set are split according to the set granularity to construct a label dictionary tree, including: adding a root node, selecting a current benchmark label from the benchmark label set; splitting: splitting the current benchmark label according to the character granularity to obtain multiple single characters; judging: judging in turn whether the single characters exist in the dictionary tree path according to the original order of the single characters in the current benchmark label; adding: if the single character does not exist in the dictionary tree path, adding a node corresponding to the single character in the dictionary tree path; selecting the next benchmark label from the benchmark label set as the current benchmark label, repeating the splitting step, the judging step and the adding step until the last benchmark label is reached to obtain the label dictionary tree; wherein the label dictionary tree includes a root node and multiple leaf nodes, and the characters in the leaf nodes are the node values ​​of the leaf nodes; Obtain the target text, access the nodes of the pre-constructed label dictionary tree, determine the node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes, and generate an initial label set; wherein, perform word segmentation processing on the target text according to the set granularity, perform all possible path searches on the target text after word segmentation according to the structure of the label dictionary tree, obtain all label value path sequences, and form an initial label set based on the obtained label value path sequences; determine whether there is a node value of a leaf node in the target text after word segmentation; Determining whether the initial tag of the initial tag set exists in the set reference tag set, so as to respectively calculate the similarity between at least one initial tag not existing in the reference tag set and the reference tag of the reference tag set; According to the similarity calculation result, corresponding target reference tags are respectively selected for the at least one initial tag from the reference tag set, and the at least one initial tag is correspondingly replaced with the target reference tags to obtain a final tag set.

2. The method according to claim 1, characterized in that According to the attribution relationship between nodes, the node value existing in the target text is determined from the label dictionary tree, and an initial label is generated, including: Initialize an empty string, take a currently unvisited leaf node belonging to the root node as the current node, access the current node, and determine the node value of the current node in the target text; Determine splicing: determine whether the current node has an unvisited leaf node. If the current node has an unvisited leaf node and the target text has a node value of the leaf node, then splice the node value of the current node and the node value of the leaf node; Update: Use the concatenated node value to update the string, and use the leaf node as the current node; The judging and splicing step and the updating step are repeatedly performed until the leaf node does not exist in the current node, and the updated character string is used as an initial label.

3. The method according to claim 1, characterized in that Selecting a corresponding target reference label for the current initial label from the reference label set includes: Determining a maximum similarity between the current initial tag and a reference tag of the reference tag set; The maximum similarity is compared with a set threshold value, and if the maximum similarity is greater than or equal to the threshold value, the reference tag corresponding to the maximum similarity is selected as the target reference tag.

4. The method according to claim 2, characterized in that: The word segmentation processing of the target text according to the set granularity includes: performing word segmentation processing of the target text according to the character granularity.

5. The method according to any one of claims 1 to 4, characterized in that: Calculating the similarity between at least one initial tag not existing in the reference tag set and a reference tag in the reference tag set includes: The similarity between at least one initial tag not existing in the reference tag set and the reference tag of the reference tag set is calculated by any one of the following methods: calculating edit distance, cosine distance, Euclidean distance, and Jaccard distance.

6. A text label extraction device, characterized in that: include: A dictionary tree construction module is used to split the benchmark labels of the benchmark label set according to the set granularity to construct a label dictionary tree, including: adding a root node, selecting a current benchmark label from the benchmark label set; splitting: splitting the current benchmark label according to the character granularity to obtain multiple single characters; judging: judging in turn whether the single characters exist in the dictionary tree path according to the original order of the single characters in the current benchmark label; adding: if the single character does not exist in the dictionary tree path, adding a node corresponding to the single character in the dictionary tree path; selecting the next benchmark label from the benchmark label set as the current benchmark label, repeating the splitting step, the judging step and the adding step until the last benchmark label is obtained to obtain the label dictionary tree; wherein the label dictionary tree includes a root node and multiple leaf nodes, and the characters in the leaf nodes are the node values ​​of the leaf nodes; The acquisition generation module is used to obtain the target text, access the nodes of the pre-constructed label dictionary tree, and determine the node values ​​existing in the target text from the label dictionary tree according to the attribution relationship between the nodes to generate an initial label set; wherein, the target text is segmented according to the set granularity, and all possible paths are searched for the segmented target text according to the structure of the label dictionary tree to obtain all label value path sequences, and the initial label set is formed based on the obtained label value path sequences; it is determined whether there is a node value of a leaf node in the segmented target text; a similarity calculation module, configured to determine whether an initial tag of the initial tag set exists in a set reference tag set, so as to respectively calculate a similarity between at least one initial tag not existing in the reference tag set and a reference tag of the reference tag set; The tag determination module is used to select corresponding target reference tags for the at least one initial tag from the reference tag set according to the similarity calculation result, and use the target reference tags to replace the at least one initial tag to obtain a final tag set.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images