Method and device for obtaining commodity attribute word hierarchical relationship dictionary

Abstract

The application provides an acquisition method and device of a commodity attribute word hierarchical relationship dictionary, and relates to the technical field of data processing. The method comprises the following steps: extracting candidate commodity attribute words from multiple data sources, performing clustering analysis on the candidate commodity attribute words, and obtaining a first commodity attribute word of any cluster; for the first commodity attribute word of any cluster, grouping the first commodity attribute words two by two to obtain a commodity attribute word pair; inputting any commodity attribute word pair into a target classification model to identify the hierarchical relationship type, so as to obtain the hierarchical relationship of any commodity attribute word pair and generate a hierarchical relationship dictionary of commodity attribute words. In the application, the coverage of commodity attribute words is improved, the hierarchical relationship between commodity attribute words is fully mined, rich samples are provided for subsequent full-potential hierarchical relationship classification judgment, the accuracy of acquiring commodity attribute hierarchical relationships is significantly improved, and more commodity attribute hierarchical relationships can be mined.

Claims

1. A method for obtaining a dictionary of hyponyms and hypernyms of product attribute terms, characterized in that, include: Candidate product attribute words are extracted from multiple data sources, including a target data source, and candidate product attribute words are extracted from the target data source based on preset lexical rules. Perform cluster analysis on the candidate product attribute words to obtain the first product attribute word of any cluster; For any cluster of the first product attribute words, the first product attribute words are grouped into pairs to obtain product attribute word pairs; The hyponym relationship type of any of the product attribute word pairs is identified in the target classification model to obtain the hyponym relationship of any product attribute word pair, so as to generate a hyponym relationship dictionary of product attribute words. The step of identifying the hierarchical relationship type of any of the product attribute word pairs input into the target classification model to obtain the hierarchical relationship of any product attribute word pair includes: Expand the corpus for any of the product attribute word pairs to obtain the target text corresponding to the product attribute word pairs; The target text is input into the target classification model to identify the recognition probability between two product attribute words in the product attribute word pair under each relationship type. The relationship types include superior relationship, inferior relationship and no superior-inferior relationship. The hierarchical relationship of the product attribute word pairs is determined based on the preset conditions that the recognition probability of each relationship type is satisfied.

2. The method according to claim 1, characterized in that, The extraction of candidate product attribute words from multiple data sources includes: In response to the data source being an e-commerce server, the product title is extracted from the e-commerce server, and the first candidate product attribute words are obtained from the product title based on the first language model; In response to the fact that the target data source includes multiple open databases, the corpus in the target data source is matched based on the preset lexical rules to determine the product attribute corpus as the second candidate product attribute words; In response to the data source being a preset product map, wherein the product map includes multiple preset product attribute words, the product map is identified, and the preset product attribute words are extracted from it as third candidate product attribute words.

3. The method according to claim 2, characterized in that, The step of obtaining the first candidate product attribute words from the product title based on the first language model includes: The product title is input into a first language model for encoding to obtain the feature representation of the product title; The feature representation is input into a bidirectional long short-term memory network for recognition to obtain the sequence label of the product title; The sequence label is input into a conditional random field, and the first candidate product attribute words of the product title are output.

4. The method according to claim 1, characterized in that, The target classification model includes a second language model and a fully connected layer. The step of inputting the target text into the target classification model and identifying the recognition probability of two product attribute words in each relation type includes: The target text is input into the second language model for encoding to obtain the second feature representation of the product attribute word pair; The second feature representation is input into the fully connected layer to obtain the recognition probability of the two product attribute words in the product attribute word pair under each relation type.

5. The method according to claim 1, characterized in that, The step of expanding the corpus of any of the product attribute word pairs to obtain the target text corresponding to the product attribute word pairs includes: Retrieves one or more preset description elements; Obtain the semantic dependency relationship between the description element and the product attribute word pair; Based on the semantic dependency relationship, determine the filling position of the description element relative to the product attribute word pair; The description element is filled in the filling position to obtain the target text corresponding to the product attribute word pair.

6. The method according to claim 1, characterized in that, The step of determining the hierarchical relationship of the product attribute word pair based on the preset conditions satisfied by the recognition probability of each relationship type includes: In response to the fact that the first recognition probability of the two product attribute words in the product attribute word pair under the hypernym relationship is greater than the first probability threshold, and the second recognition probability under the hyponym relationship and the third recognition probability under the absence of a hypernym relationship are both less than the second probability threshold, the product attribute word pair is confirmed to be the hypernym relationship; or In response to the second recognition probability being greater than the first probability threshold, and both the first recognition probability and the third recognition probability being less than the second probability threshold, the product attribute word pair is confirmed to be the hyponym; or In response to the third recognition probability being greater than the first probability threshold, and both the first recognition probability and the second recognition probability being less than the second probability threshold, the product attribute word pair is confirmed to have no hierarchical relationship.

7. The method according to claim 1, characterized in that, Before generating the hypernym / hypernym relation dictionary for product attribute terms, the following steps are also included: For any pair of target product attribute words with no hierarchical relationship, identify the suffix words of the target product attribute word pair and determine whether the suffix words are of the same type. In response to the suffix being of the same type, the hierarchical relationship of the target product attribute word pair is updated according to the suffix.

8. A device for acquiring a dictionary of hypernyms and hyponyms of product attribute terms, characterized in that, include: An extraction module is used to extract candidate product attribute words from multiple data sources, including a target data source, and to extract candidate product attribute words from the target data source based on preset lexical rules. The clustering module is used to perform cluster analysis on the candidate product attribute words and obtain the first product attribute word of any cluster; The grouping module is used to group the first product attribute words in pairs for any cluster to obtain product attribute word pairs; The generation module is used to identify the hierarchical relationship type of any of the product attribute word pairs input into the target classification model, so as to obtain the hierarchical relationship of any product attribute word pair and generate a dictionary of hierarchical relationships of product attribute words. The generation module is specifically used to expand the corpus of any of the product attribute word pairs to obtain the target text corresponding to the product attribute word pairs; The target text is input into the target classification model to identify the recognition probability between two product attribute words in the product attribute word pair under each relationship type. The relationship types include superior relationship, inferior relationship and no superior-inferior relationship. The hierarchical relationship of the product attribute word pairs is determined based on the preset conditions that the recognition probability of each relationship type is satisfied.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images