Method and device for extracting key attributes of data

Abstract

The present invention discloses a method and device for extracting key attributes of data, and relates to the field of computer technology. A specific implementation of the method includes: extracting structured attribute information from the description information of the data, the structured attribute information including attribute names and their corresponding attribute values; normalizing the attribute names by performing similarity calculations on the attribute values, and determining key attribute names from the normalized attribute names; constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; extracting key attributes of data based on the key attribute knowledge graph. This implementation implements efficient key attribute extraction of unstructured information based on the key attribute knowledge graph constructed by processing structured attribute information, without relying on grammatical structure and annotated data, greatly improving the efficiency and accuracy of data key attribute extraction.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for extracting key attributes of data. Background Art

[0002] Due to the wide variety of data categories and the vast differences in the key attributes that distinguish different categories of data, it becomes almost unrealistic to solve the matching problem of all data in big data scenarios using only a certain algorithm model. Instead, it is necessary to extract key information that can distinguish data based on the characteristics of different categories of data. Therefore, how to extract the key attributes of multi-category data becomes a problem that must be solved.

[0003] At present, the key attribute extraction methods in the field of data search or retrieval mainly include pattern matching-based methods, statistics-based (traditional machine learning and deep learning) methods, weak supervision / corpus expansion-based methods, and so on.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] Existing key attribute extraction methods in the field of data search or retrieval cannot efficiently extract key attributes of unstructured information, and need to rely on grammatical structure and annotated data. Therefore, the efficiency and accuracy of key attribute extraction of data are very low. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method and device for extracting key attributes of data, which can efficiently extract key attributes of unstructured information based on a key attribute knowledge graph constructed by processing structured attribute information, without relying on grammatical structure and annotated data, thereby greatly improving the efficiency and accuracy of extracting key attributes of data.

[0007] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for extracting key attributes of data is provided.

[0008] A method for extracting key attributes of data, comprising: extracting structured attribute information from descriptive information of the data, wherein the structured attribute information includes attribute names and their corresponding attribute values; normalizing the attribute names by performing similarity calculations on the attribute values, and determining key attribute names from the normalized attribute names; constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; and extracting key attributes of the data based on the key attribute knowledge graph.

[0009] Optionally, the attribute names are normalized by performing similarity calculation on the attribute values, and key attribute names are determined from the normalized attribute names, including: performing similarity calculation on the attribute values, and determining a group of attribute names whose attribute value similarities satisfy a set threshold as a similar attribute name set; for each similar attribute name set, selecting an attribute name from the similar attribute name set as the normalized attribute name, and merging and deduplicating the attribute values ​​corresponding to each attribute name in the similar attribute name set to obtain the attribute values ​​corresponding to the normalized attribute names; and determining key attribute names based on the importance of the attribute values ​​corresponding to each normalized attribute name to the data.

[0010] Optionally, before constructing the key attribute knowledge graph based on each key attribute name and its corresponding attribute value, it also includes: for each key attribute name, if the number of attribute values ​​corresponding to the key attribute name is less than a set number threshold, the attribute value is expanded according to the attribute value corresponding to the key attribute name in the external knowledge base.

[0011] Optionally, before constructing the key attribute knowledge graph according to each key attribute name and its corresponding attribute value, it also includes: for each key attribute name, if the attribute value corresponding to the key attribute name is a composite attribute value, splitting the composite attribute value and constructing a replacement value set for the composite attribute value.

[0012] Optionally, before constructing the key attribute knowledge graph according to each key attribute name and its corresponding attribute value, it also includes: based on an external knowledge base, respectively obtaining a stop word library, a synonym library, a hyponym library and a key attribute library for each key attribute name and its corresponding attribute value, wherein the key attribute library includes the key attribute name and its corresponding attribute value, and a set of replacement values ​​for the compound attribute value.

[0013] Optionally, constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value includes: constructing a key attribute knowledge graph based on a stop word library, a synonym library, a hyponym library and a key attribute library for each key attribute name and its corresponding attribute value.

[0014] Optionally, extracting key attributes of data based on the key attribute knowledge graph includes: extracting structured attribute information from the description information of the data, deleting invalid attribute information from the structured attribute information based on the stop word library to obtain first attribute information, and extracting first key attributes from the first attribute information based on the synonym library and the key attribute library; extracting unstructured attribute information from the description information of the data, and matching the unstructured attribute information based on the synonym library and the key attribute library to obtain second key attributes, the first key attributes and the second key attributes including key attribute names and corresponding attribute values; extracting data key attributes from the first key attributes and the second key attributes based on the composite attribute values ​​and their replacement value sets in the key attribute library, the synonym library and the hierarchical relationship library.

[0015] Optionally, after extracting key attributes of the data based on the key attribute knowledge graph, the method further includes: performing validity and uniqueness analysis on the extracted key attributes of the data to optimize the key attribute knowledge graph.

[0016] Optionally, if the description information of the data is picture data, text recognition is performed on the picture data to obtain text description information, and then key attributes of the data are extracted from the text description information.

[0017] According to another aspect of an embodiment of the present invention, a device for extracting key attributes of data is provided.

[0018] A device for extracting key attributes of data, comprising: an information extraction module, used to extract structured attribute information from the description information of the data, wherein the structured attribute information includes attribute names and their corresponding attribute values; an attribute processing module, used to normalize the attribute names by performing similarity calculations on the attribute values, and determine key attribute names from the normalized attribute names; a graph construction module, used to construct a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; and an attribute extraction module, used to extract key attributes of data based on the key attribute knowledge graph.

[0019] According to yet another aspect of an embodiment of the present invention, an electronic device for extracting key attributes of data is provided.

[0020] An electronic device for extracting key attributes of data includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method for extracting key attributes of data provided by an embodiment of the present invention.

[0021] According to still another aspect of the embodiments of the present invention, a computer readable medium is provided.

[0022] A computer readable medium stores a computer program, which, when executed by a processor, implements a method for extracting key attributes of data provided by an embodiment of the present invention.

[0023] An embodiment of the above invention has the following advantages or beneficial effects: by extracting structured attribute information from the description information of the data, the structured attribute information includes attribute names and their corresponding attribute values; by performing similarity calculations on the attribute values ​​to normalize the attribute names, and determining the key attribute names from the normalized attribute names; constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; extracting key attributes of data based on the key attribute knowledge graph, thereby realizing efficient key attribute extraction of unstructured information based on the key attribute knowledge graph constructed by processing the structured attribute information, without relying on grammatical structure and annotated data, greatly improving the efficiency and accuracy of key attribute extraction of data.

[0024] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0026] Figure 1 is a schematic diagram of the main steps of a method for extracting key attributes of data according to an embodiment of the present invention;

[0027] Figure 2 It is a schematic diagram of the construction process of the key attribute knowledge graph of one embodiment of the present invention;

[0028] Figure 3 It is a schematic diagram of a data key attribute extraction process according to an embodiment of the present invention;

[0029] Figure 4 is a schematic diagram of main modules of a device for extracting key attributes of data according to an embodiment of the present invention;

[0030] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0031] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0033] Data query problems are similar to search scenario problems. For example, when you need to search for a certain product or products from a large number of products on an e-commerce platform, you need to find the products the user wants as accurately and comprehensively as possible. At this time, the tolerance for wrong results or incomplete data will be lower.

[0034] The structuring of product key attribute information is a very important part of the product matching task. Its efficiency and quality will seriously affect the product matching effect. The essence of structuring product key attribute information is the extraction of product key attribute information. Although there are a large number of information extraction methods in the search field (including entity extraction, relationship extraction and time extraction, the specific information is shown below), it is currently impossible to use the same algorithm to solve cross-domain information extraction (different fields need to be labeled separately and retrained, resulting in high implementation costs for single category projects). The existing key attribute extraction methods in the field of data search or search have the following problems:

[0035] 1. The pattern matching method extracts key attributes based on the logical grammar of text data. It cannot extract key attributes for text data that is composed of phrases but does not have a grammatical structure.

[0036] 2. Statistical methods require a large amount of labeled data to train a deep learning model, which consumes a lot of text annotation costs;

[0037] 3. Methods based on weak supervision / corpus expansion require the introduction of a large amount of knowledge similar to the characteristics of text data. However, the similar knowledge to be introduced needs to be constructed and the introduction of similar knowledge will reduce the accuracy of the key feature extraction results.

[0038] In order to solve the problems existing in the prior art, the present invention provides a method for extracting key attributes of data, which can extract key attributes of text data that is composed of phrase stacks but does not have a grammatical structure, without relying on syntax or syntax tree information, and a large amount of annotation data. Taking the application scenario of extracting key information of goods on an e-commerce platform as an example, the text description information of the goods contains both structured information and unstructured information. Based on the low-quality structured information of goods on the e-commerce platform, the present invention constructs a high-quality knowledge graph of key attributes of goods to achieve efficient key attribute extraction of unstructured information of goods (such as: product title, product details description, etc.).

[0039] Figure 1 FIG. 1 is a schematic diagram of the main steps of the method for extracting key attributes of data according to an embodiment of the present invention. Figure 1 As shown, the method for extracting key attributes of data in the embodiment of the present invention mainly includes the following steps S101 to S104.

[0040] Step S101: extracting structured attribute information from the description information of the data, where the structured attribute information includes an attribute name and its corresponding attribute value;

[0041] Step S102: normalizing the attribute names by performing similarity calculation on the attribute values, and determining the key attribute names from the normalized attribute names;

[0042] Step S103: construct a key attribute knowledge graph according to each key attribute name and its corresponding attribute value;

[0043] Step S104: extracting key attributes of data based on the key attribute knowledge graph.

[0044] According to the above-mentioned steps S101 to S104, by extracting and processing the structured attribute information to obtain the key attribute names, then building a key attribute knowledge graph based on the key attribute names and attribute values, and extracting the key attributes of the data based on the key attribute knowledge graph, it is possible to efficiently extract the key attributes of the unstructured information of the data without relying on grammatical structures and annotated data, etc., thereby greatly improving the efficiency of extracting the key attributes of the data.

[0045] According to an embodiment of the present invention, structured attribute information refers to attribute information from which attribute names and attribute values ​​can be directly obtained, for example, "Brand: A", from which the attribute name "Brand" and the attribute value "A" can be directly obtained. By processing the data extracted from the big data platform, a large amount of structured attribute information can be obtained, thereby obtaining a large number of attribute names and their corresponding attribute values, so as to facilitate subsequent processing.

[0046] According to an embodiment of the present invention, in step S102, when normalizing the attribute names by performing similarity calculation on the attribute values ​​and determining the key attribute names from the normalized attribute names, the following steps may be specifically included:

[0047] Perform similarity calculation on the attribute values, and determine a group of attribute names whose attribute value similarity meets a set threshold as a synonymous attribute name set;

[0048] For each synonymous attribute name set, select an attribute name from the synonymous attribute name set as the normalized attribute name, and merge and remove duplicate attribute values ​​corresponding to each attribute name in the synonymous attribute name set to obtain the attribute value corresponding to the normalized attribute name;

[0049] The key attribute name is determined according to the importance of the attribute value corresponding to each normalized attribute name to the data.

[0050] A large number of attribute names and their corresponding attribute values ​​are obtained through step S101. In order to determine the key attributes, it is necessary to determine the key attribute names and their corresponding attribute values. When determining the key attribute names, since different data providers may name the same attribute differently, in order to more accurately and reasonably extract the key attribute names, the attribute names may be normalized according to the similarity between the attribute values ​​of each attribute name, so as to normalize similar attribute names into the same attribute name.

[0051] Specifically, when calculating the similarity between attribute values, similarity algorithms that can be used include: calculating the ratio of identical character strings between two attribute values, calculating the edit distance of text pinyin, calculating the cosine similarity or Hamming distance between vectors corresponding to two attribute values, etc. After obtaining the similarity between attribute values, a group of attribute names whose similarity meets a set threshold can be determined as a synonymous attribute name set, so that a large number of attribute names can be divided into multiple synonymous attribute name sets according to the similarity of attribute values.

[0052] For each set of synonymous attribute names, you can arbitrarily select an attribute name as the normalized attribute name, or you can select the attribute name with the highest frequency of occurrence as the normalized attribute name, or you can select the attribute name as the normalized attribute name according to other set rules. The present invention does not limit this. After that, it is necessary to determine the attribute value corresponding to the normalized attribute name. Since each attribute name in the set of synonymous attribute names corresponding to the normalized attribute name corresponds to an attribute value, these attribute values ​​are similar but not exactly the same, therefore, the attribute values ​​corresponding to each attribute name in the set of synonymous attribute names can be merged and deduplicated to obtain the attribute value corresponding to the normalized attribute name. In this way, you can obtain the normalized attribute name corresponding to each set of synonymous attribute names and its corresponding attribute value (there may be multiple).

[0053] Afterwards, the key attribute names can be determined from these normalized attribute names. When determining the key attribute names, the present invention determines them based on the importance of the attribute values ​​corresponding to the normalized attribute names to the data. Specifically, when calculating the importance of the attribute values ​​to the data, it can be calculated based on the frequency of occurrence of the attribute value in all attribute values ​​before deduplication, or the importance of different attribute values ​​can be set based on prior knowledge, and so on. Based on the importance of each attribute value, the normalized attribute name corresponding to the attribute value with higher importance can be selected as the key attribute name.

[0054] After determining the key attribute names and their corresponding attribute values, the key attribute knowledge graph can be constructed.

[0055] In one embodiment of the present invention, before constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value, the attribute value can be expanded when the attribute value is small. Specifically, for each key attribute name, if the number of attribute values ​​corresponding to the key attribute name is less than the set number threshold, the attribute value is expanded according to the attribute value corresponding to the key attribute name in the external knowledge base. At present, the attribute value is mainly expanded from the existing industry knowledge graph and search results. Specifically, the expanded value of the attribute value can be obtained by crawling.

[0056] In another embodiment of the present invention, before constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value, the attribute value can also be split more finely. Specifically, for each key attribute name, if the attribute value corresponding to the key attribute name is a composite attribute value, the composite attribute value is split and a replacement value set of the composite attribute value is constructed. There may be multiple attribute values ​​corresponding to each key attribute name, which may include composite attribute values. A composite attribute value refers to an attribute value containing sub-attribute values, which can be split into the sub-attribute values ​​it contains. For example: the attribute values ​​corresponding to the attribute name "washing machine type" include "drum pulsator washing machine, drum washing machine, pulsator washing machine", where the attribute value "drum pulsator washing machine" is a composite attribute value and needs to be split into "drum washing machine, pulsator washing machine". The present invention sets a splitting rule for attribute values ​​to determine whether the attribute value in the key attribute name is a composite attribute value and how to split the composite attribute value. The attribute value obtained after splitting the composite attribute value is the replacement value set of the composite attribute value.

[0057] In another embodiment of the present invention, before constructing a key attribute knowledge graph according to each key attribute name and its corresponding attribute value, a stop word library, a synonym library, a hyponym library and a key attribute library for each key attribute name and its corresponding attribute value can be obtained based on an external knowledge base, wherein the key attribute library includes the key attribute name and its corresponding attribute value, and a replacement value set of the composite attribute value. Among them, the synonym library of the key attribute name includes a set of synonymous attribute names corresponding to the key attribute name, and the synonyms of the key attribute name provided in the knowledge base; the stop word library and the hyponym library, as well as the synonym library of the attribute value can be obtained based on the external knowledge base.

[0058] According to an embodiment of the present invention, a key attribute knowledge graph is constructed according to each key attribute name and its corresponding attribute value, that is, a stop word library, a synonym library, a hyponym library and a key attribute library are constructed according to each key attribute name and its corresponding attribute value.

[0059] After constructing the key attribute knowledge graph, the key attributes of the data can be extracted based on the key attribute knowledge graph.

[0060] Generally speaking, the description information of data includes both structured attribute information and unstructured attribute information. For example, the product description information of an e-commerce platform includes both structured attribute information of the product, such as "brand: A", and unstructured attribute information of the product, such as "pure cotton material, multi-color matching".

[0061] When extracting key attributes of data, specifically, you can follow the following steps:

[0062] Extracting structured attribute information from the description information of the data, deleting invalid attribute information from the structured attribute information based on a stop word library to obtain first attribute information, and then extracting a first key attribute from the first attribute information based on a key attribute library;

[0063] Extracting unstructured attribute information from the description information of the data, and matching the unstructured attribute information to obtain a second key attribute based on the key attribute library, wherein the first key attribute and the second key attribute include a key attribute name and a corresponding attribute value;

[0064] The data key attribute is extracted from the first key attribute and the second key attribute based on the composite attribute value and its replacement value set in the key attribute library, the synonym library and the hyponym library.

[0065] For structured attribute information, after extracting the attribute name and attribute value therefrom, firstly, the attribute name corresponding to the stop word can be deleted from the attribute name included in the structured attribute information according to the stop word library corresponding to the attribute name, thereby deleting the invalid attribute information to obtain the first attribute information, and then, based on the key attribute names and their corresponding attribute values ​​in the key attribute library, as well as the synonym library of the key attribute names, the first key attribute can be extracted from the first attribute information.

[0066] For unstructured attribute information, the second key attribute can be obtained by matching the key attribute name and its corresponding attribute value in the key attribute library and the synonym library of the key attribute name from the unstructured attribute information.

[0067] After that, for the first key attribute and the second key attribute, we can determine whether there are composite attribute values ​​that need to be split and replace the attribute values ​​according to the composite attribute values ​​and their replacement value sets in the key attribute library; normalize the attribute values ​​according to the synonym library corresponding to the attribute values ​​to normalize them to comparable word dimensions; merge the hypernyms and hyponyms in the attribute values ​​according to the hypernymy library, delete the hypernyms and retain the hyponyms. In this way, we can get the key attribute name and its attribute value from the data description information.

[0068] After extracting key attributes of the data based on the key attribute knowledge graph, the validity and uniqueness of the extracted key attributes of the data can also be analyzed to optimize the key attribute knowledge graph.

[0069] According to another embodiment of the present invention, if the description information of the data is picture data, text recognition is performed on the picture data to obtain text description information, and then key attributes of the data are extracted from the text description information.

[0070] The following takes the extraction of key attribute information of commodities from commodity description information of an e-commerce platform as an example to introduce the specific implementation process of the key attribute extraction method of the present invention.

[0071] In the embodiment of the present invention, two steps are mainly used to significantly improve the effectiveness of extracting key attributes of commodities and to make the attribute values ​​reach the same comparable granularity as much as possible. First, by constructing a knowledge graph of key attributes of commodities, the key attribute names and attribute value ranges, hyponyms, synonyms, stop words, compound word replacement rules, and mutually exclusive relationship words that are of main concern to a certain type of commodity are clarified. Then, based on the construction of the commodity knowledge graph, the attribute values ​​corresponding to the key attribute names are extracted more efficiently from the commodity structure and non-structural information, providing data support for subsequent commodity matching.

[0072] Figure 2 FIG. 1 is a schematic diagram of the construction process of a key attribute knowledge graph according to an embodiment of the present invention. Figure 2 As shown, in the embodiment of the present invention, the construction of the key attribute knowledge graph mainly includes the following steps:

[0073] 1. Collect and extract the structured information (attribute name + attribute value) of all products under a certain last-level category on the big data platform;

[0074] 2. Extract all attribute names and their corresponding attribute values ​​from the structured information;

[0075] 3. Select key attribute names: First, normalize synonymous attribute names through the similarity algorithm of attribute value sets; second, sort the importance of the attribute values ​​corresponding to the normalized attribute names in the title and attributes, and select a specified number of attribute names that can effectively distinguish products as key attribute names;

[0076] 4. Based on the normalized results of the key attribute names given in step 3, merge and remove duplicates to obtain the attribute value corresponding to each key attribute name;

[0077] 5. When the attribute values ​​corresponding to the key attribute names obtained in step 4 are small, the attribute values ​​are expanded by introducing the attribute values ​​corresponding to the attribute names in the external knowledge base;

[0078] 6. Based on the key attribute names and their corresponding attribute values ​​collected in the above steps, determine whether there are composite attribute values ​​that need to be split and build splitting rules;

[0079] 7. According to the splitting rules, the attribute values ​​corresponding to the key attribute names of the products are split into finer granularity;

[0080] 8. Based on the key attribute names and attribute values ​​constructed in step 7, a stop word library, synonym library, hyponym library and key attribute library corresponding to each key attribute name and attribute value are constructed through an external knowledge base, wherein the key attribute library includes the key attribute name and the corresponding attribute value, and the attribute value includes both the attribute value before the split and the attribute value after the split. In addition, a mutually exclusive library of attribute values ​​can also be constructed;

[0081] 9. Build a key attribute knowledge graph based on all the data in step 8.

[0082] Figure 3 FIG. 1 is a schematic diagram of a data key attribute extraction process according to an embodiment of the present invention. Figure 3 As shown, based on Figure 2 The key attribute knowledge graph of the product constructed in is used to extract the attribute value information corresponding to the key attribute name in the product. According to the type of product description information, it can be divided into two parts: 1) structured attribute information; 2) unstructured attribute information.

[0083] 1. For 1) structured attribute information, first, based on the stop word library, invalid attribute values ​​under the corresponding attribute name are removed to obtain first attribute information; then, based on the synonym library and the key attribute library, a first key attribute is extracted from the first attribute information, including the first key attribute name and the corresponding attribute value;

[0084] 2. For 2) unstructured attribute data, a second key attribute is obtained by matching the unstructured attribute information based on the synonym library and the key attribute library, including the second key attribute name and the corresponding attribute value;

[0085] 3. Based on the composite attribute values ​​and their replacement value sets in the key attribute library, the composite attribute values ​​in the attribute values ​​obtained in steps 1 and 2 are split into finer granularity;

[0086] 4. Based on the hyponymy database, merge the hyponyms and hyponyms in the key attribute values, retain the hyponyms, and delete the hypernyms;

[0087] 5. Based on the synonym database, normalize the attribute values ​​under the key attribute names to normalize them to the same comparable word dimension;

[0088] 6. Perform validity and uniqueness analysis on the extracted key attributes of the data to optimize the key attribute knowledge graph.

[0089] According to the embodiments of the present invention, the results of simulation experiments on matching products of multiple categories show that the accuracy (non-vacancy rate) of extracting key attributes of products by the method of the present invention is increased from 69% to 92% on average. Since the technical solution of the present invention mainly solves the problem of text information extraction, when the attribute information of the product is included in the picture, it is necessary to supplement the key attribute names and attribute values ​​of the product missing in the text information by the method of text recognition R.

[0090] Figure 4 Schematic diagram of the main modules of the device for extracting key attributes of data according to an embodiment of the present invention. Figure 4 As shown, the device 400 for extracting key attributes of data according to an embodiment of the present invention mainly includes an information extraction module 401 , an attribute processing module 402 , a graph construction module 403 and an attribute extraction module 404 .

[0091] The information extraction module 401 is used to extract structured attribute information from the description information of the data, where the structured attribute information includes an attribute name and its corresponding attribute value;

[0092] The attribute processing module 402 is used to normalize the attribute names by performing similarity calculation on the attribute values, and determine the key attribute names from the normalized attribute names;

[0093] A graph construction module 403 is used to construct a key attribute knowledge graph according to each key attribute name and its corresponding attribute value;

[0094] The attribute extraction module 404 is used to extract key attributes of data based on the key attribute knowledge graph.

[0095] According to an embodiment of the present invention, the attribute processing module 402 may also be used to:

[0096] Perform similarity calculation on the attribute values, and determine a group of attribute names whose attribute value similarity meets a set threshold as a synonymous attribute name set;

[0097] For each synonymous attribute name set, select an attribute name from the synonymous attribute name set as the normalized attribute name, and merge and remove duplicate attribute values ​​corresponding to each attribute name in the synonymous attribute name set to obtain the attribute value corresponding to the normalized attribute name;

[0098] The key attribute name is determined according to the importance of the attribute value corresponding to each normalized attribute name to the data.

[0099] According to another embodiment of the present invention, the data key attribute extraction device 400 may further include an attribute value expansion module (not shown in the figure) for:

[0100] Before constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value, for each key attribute name, if the number of attribute values ​​corresponding to the key attribute name is less than the set number threshold, the attribute value is expanded according to the attribute value corresponding to the key attribute name in the external knowledge base.

[0101] According to another embodiment of the present invention, the data key attribute extraction device 400 may further include an attribute value splitting module (not shown in the figure) for:

[0102] Before constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value, for each key attribute name, if the attribute value corresponding to the key attribute name is a composite attribute value, the composite attribute value is split and a replacement value set for the composite attribute value is constructed.

[0103] According to another embodiment of the present invention, the data key attribute extraction device 400 may further include an attribute extension module (not shown in the figure) for:

[0104] Based on the external knowledge base, a stop word library, a synonym library, a hyponym library and a key attribute library are respectively obtained for each key attribute name and its corresponding attribute value, wherein the key attribute library includes the key attribute name and its corresponding attribute value, and a replacement value set of the compound attribute value.

[0105] According to one embodiment of the present invention, the graph construction module 403 may also be used to:

[0106] A key attribute knowledge graph is constructed based on the stop word library, synonym library, hyponym library and key attribute library for each key attribute name and its corresponding attribute value.

[0107] According to another embodiment of the present invention, the attribute extraction module 404 may also be used for:

[0108] Extracting structured attribute information from the description information of the data, deleting invalid attribute information from the structured attribute information based on the stop word library to obtain first attribute information, and extracting a first key attribute from the first attribute information based on the synonym library and the key attribute library;

[0109] Extracting unstructured attribute information from the description information of the data, and matching the unstructured attribute information to obtain a second key attribute based on the synonym library and the key attribute library, wherein the first key attribute and the second key attribute include a key attribute name and a corresponding attribute value;

[0110] Data key attributes are extracted from the first key attribute and the second key attribute based on the composite attribute value and its replacement value set in the key attribute library, the synonym library and the hyponym library.

[0111] According to another embodiment of the present invention, the device 400 for extracting key attributes of data may further include a graph optimization module (not shown in the figure) for:

[0112] After extracting key attributes of the data based on the key attribute knowledge graph, the extracted key attributes of the data are analyzed for validity and uniqueness to optimize the key attribute knowledge graph.

[0113] According to yet another embodiment of the present invention, if the description information of the data is picture data, text recognition is performed on the picture data to obtain text description information, and then key attributes of the data are extracted from the text description information.

[0114] According to the technical solution of the embodiment of the present invention, structured attribute information is extracted from the descriptive information of the data, and the structured attribute information includes attribute names and their corresponding attribute values; the attribute names are normalized by performing similarity calculations on the attribute values, and key attribute names are determined from the normalized attribute names; a key attribute knowledge graph is constructed according to each key attribute name and its corresponding attribute value; key attributes of the data are extracted based on the key attribute knowledge graph, thereby realizing efficient key attribute extraction of unstructured information based on the key attribute knowledge graph constructed by processing the structured attribute information, without relying on grammatical structure and annotated data, thereby greatly improving the efficiency and accuracy of key attribute extraction of data.

[0115] Figure 5 An exemplary system architecture 500 is shown to which the method for extracting key attributes of data or the apparatus for extracting key attributes of data according to the embodiment of the present invention can be applied.

[0116] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0117] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0118] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0119] The server 505 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 501, 502, and 503. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0120] It should be noted that the method for extracting key attributes of data provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the device for extracting key attributes of data is generally disposed in the server 505 .

[0121] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0122] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 6 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0123] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0124] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0125] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0126] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0127] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The units or modules involved in the embodiments of the present invention may be implemented in software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as follows: a processor includes an information extraction module, an attribute processing module, a graph construction module, and an attribute extraction module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves. For example, the information extraction module may also be described as a "module for extracting structured attribute information from the descriptive information of the data."

[0129] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: extracting structured attribute information from the description information of the data, the structured attribute information includes attribute names and their corresponding attribute values; normalizing the attribute names by performing similarity calculations on the attribute values, and determining key attribute names from the normalized attribute names; constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; extracting key attributes of data based on the key attribute knowledge graph.

[0130] According to the technical solution of the embodiment of the present invention, structured attribute information is extracted from the descriptive information of the data, and the structured attribute information includes attribute names and their corresponding attribute values; the attribute names are normalized by performing similarity calculations on the attribute values, and key attribute names are determined from the normalized attribute names; a key attribute knowledge graph is constructed according to each key attribute name and its corresponding attribute value; key attributes of the data are extracted based on the key attribute knowledge graph, thereby realizing efficient key attribute extraction of unstructured information based on the key attribute knowledge graph constructed by processing the structured attribute information, without relying on grammatical structure and annotated data, thereby greatly improving the efficiency and accuracy of key attribute extraction of data.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting key attributes of data, characterized in that: include: Extracting structured attribute information from the description information of the data, wherein the structured attribute information includes an attribute name and its corresponding attribute value; Normalizing the attribute names by performing similarity calculation on the attribute values, and determining the key attribute names from the normalized attribute names; Construct a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; Extracting key attributes of data based on the key attribute knowledge graph; The attribute names are normalized by performing similarity calculation on the attribute values, and the key attribute names are determined from the normalized attribute names to include: Perform similarity calculation on the attribute values, and determine a group of attribute names whose attribute value similarity meets a set threshold as a synonymous attribute name set; For each synonymous attribute name set, select an attribute name from the synonymous attribute name set as the normalized attribute name, and merge and remove duplicate attribute values ​​corresponding to each attribute name in the synonymous attribute name set to obtain the attribute value corresponding to the normalized attribute name; The key attribute name is determined according to the importance of the attribute value corresponding to each normalized attribute name to the data.

2. The method according to claim 1, characterized in that Before constructing the key attribute knowledge graph according to each key attribute name and its corresponding attribute value, it also includes: For each key attribute name, if the number of attribute values ​​corresponding to the key attribute name is less than a set number threshold, the attribute value is expanded according to the attribute value corresponding to the key attribute name in the external knowledge base.

3. The method according to claim 1, characterized in that Before constructing the key attribute knowledge graph according to each key attribute name and its corresponding attribute value, it also includes: For each key attribute name, if the attribute value corresponding to the key attribute name is a composite attribute value, the composite attribute value is split and a replacement value set of the composite attribute value is constructed.

4. The method according to claim 1, characterized in that Before constructing the key attribute knowledge graph according to each key attribute name and its corresponding attribute value, it also includes: Based on the external knowledge base, a stop word library, a synonym library, a hyponym library and a key attribute library are respectively obtained for each key attribute name and its corresponding attribute value, wherein the key attribute library includes the key attribute name and its corresponding attribute value, and a replacement value set of the compound attribute value.

5. The method according to claim 4, characterized in that Constructing a key attribute knowledge graph based on each key attribute name and its corresponding attribute value includes: A key attribute knowledge graph is constructed based on the stop word library, synonym library, hyponym library and key attribute library for each key attribute name and its corresponding attribute value.

6. The method according to claim 5, characterized in that Extracting key attributes of data based on the key attribute knowledge graph includes: Extracting structured attribute information from the description information of the data, deleting invalid attribute information from the structured attribute information based on the stop word library to obtain first attribute information, and extracting a first key attribute from the first attribute information based on the synonym library and the key attribute library; Extracting unstructured attribute information from the description information of the data, and matching the unstructured attribute information to obtain a second key attribute based on the synonym library and the key attribute library, wherein the first key attribute and the second key attribute include a key attribute name and a corresponding attribute value; Data key attributes are extracted from the first key attribute and the second key attribute based on the composite attribute value and its replacement value set in the key attribute library, the synonym library and the hyponym library.

7. The method according to claim 1, characterized in that After extracting the key attributes of the data based on the key attribute knowledge graph, the method further includes: The extracted data key attributes are analyzed for validity and uniqueness to optimize the key attribute knowledge graph.

8. The method according to claim 1, characterized in that If the description information of the data is picture data, text recognition is performed on the picture data to obtain text description information, and then key attributes of the data are extracted from the text description information.

9. A method for extracting key attributes of data, characterized in that: include: Extracting structured attribute information from the description information of the data, wherein the structured attribute information includes an attribute name and its corresponding attribute value; Normalizing the attribute names by performing similarity calculation on the attribute values, and determining the key attribute names from the normalized attribute names; Based on the external knowledge base, a stop word library, a synonym library, a hyponym library and a key attribute library are respectively obtained for each key attribute name and its corresponding attribute value, wherein the key attribute library includes the key attribute name and its corresponding attribute value, and a replacement value set of the composite attribute value; Constructing a key attribute knowledge graph according to each key attribute name and its corresponding attribute value, including: constructing a key attribute knowledge graph according to a stop word library, a synonym library, a hyponym library and a key attribute library of each key attribute name and its corresponding attribute value; Extracting key attributes of data based on the key attribute knowledge graph includes: extracting structured attribute information from the description information of the data, deleting invalid attribute information from the structured attribute information based on the stop word library to obtain first attribute information, and extracting first key attributes from the first attribute information based on the synonym library and the key attribute library; extracting unstructured attribute information from the description information of the data, and matching the unstructured attribute information based on the synonym library and the key attribute library to obtain second key attributes, wherein the first key attribute and the second key attribute include key attribute names and corresponding attribute values; extracting data key attributes from the first key attributes and the second key attributes based on the composite attribute value and its replacement value set in the key attribute library, the synonym library and the hierarchical relationship library.

10. The method according to claim 9, characterized in that Before constructing the key attribute knowledge graph according to each key attribute name and its corresponding attribute value, it also includes: For each key attribute name, if the number of attribute values ​​corresponding to the key attribute name is less than a set number threshold, the attribute value is expanded according to the attribute value corresponding to the key attribute name in the external knowledge base.

11. The method according to claim 9, characterized in that After extracting the key attributes of the data based on the key attribute knowledge graph, the method further includes: The extracted data key attributes are analyzed for validity and uniqueness to optimize the key attribute knowledge graph.

12. The method according to claim 9, characterized in that If the description information of the data is picture data, text recognition is performed on the picture data to obtain text description information, and then key attributes of the data are extracted from the text description information.

13. A device for extracting key attributes of data, characterized in that: include: An information extraction module, used to extract structured attribute information from the description information of the data, wherein the structured attribute information includes an attribute name and its corresponding attribute value; An attribute processing module, used for normalizing attribute names by performing similarity calculation on attribute values, and determining key attribute names from the normalized attribute names; A graph construction module is used to construct a key attribute knowledge graph based on each key attribute name and its corresponding attribute value; An attribute extraction module, used for extracting key attributes of data based on the key attribute knowledge graph; The attribute processing module is further used to perform similarity calculation on the attribute values, and determine a group of attribute names whose attribute value similarity meets a set threshold as a synonymous attribute name set; for each synonymous attribute name set, select an attribute name from the synonymous attribute name set as the attribute name after normalization processing, and merge and remove duplicate attribute values ​​corresponding to each attribute name in the synonymous attribute name set to obtain the attribute value corresponding to the attribute name after normalization processing; The key attribute name is determined according to the importance of the attribute value corresponding to each normalized attribute name to the data.

14. A device for extracting key attributes of data, characterized in that: include: An information extraction module, used to extract structured attribute information from the description information of the data, wherein the structured attribute information includes an attribute name and its corresponding attribute value; An attribute processing module, used for normalizing attribute names by performing similarity calculation on attribute values, and determining key attribute names from the normalized attribute names; The attribute expansion module is used to obtain, based on an external knowledge base, a stop word library, a synonym library, a hyponym library, and a key attribute library for each key attribute name and its corresponding attribute value, respectively, wherein the key attribute library includes the key attribute name and its corresponding attribute value, and a replacement value set of a composite attribute value; A graph construction module is used to construct a key attribute knowledge graph according to each key attribute name and its corresponding attribute value, including: constructing a key attribute knowledge graph according to a stop word library, a synonym library, a hyponym library and a key attribute library of each key attribute name and its corresponding attribute value; The attribute extraction module is used to extract key attributes of data based on the key attribute knowledge graph, including: extracting structured attribute information from the description information of the data, deleting invalid attribute information from the structured attribute information based on the stop word library to obtain first attribute information, and extracting first key attributes from the first attribute information based on the synonym library and the key attribute library; extracting unstructured attribute information from the description information of the data, and matching the unstructured attribute information based on the synonym library and the key attribute library to obtain second key attributes, wherein the first key attribute and the second key attribute include key attribute names and corresponding attribute values; extracting data key attributes from the first key attributes and the second key attributes based on the composite attribute value and its replacement value set in the key attribute library, the synonym library and the upper and lower relationship library.

15. An electronic device for extracting key attributes of data, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

16. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images