A knowledge graph expansion method, device, electronic device and storage medium
By identifying the entity type of data to be processed and extracting attribute information, and automatically adding it to the knowledge graph, the problem of low knowledge graph expansion efficiency in the prior art is solved, and efficient graph expansion is achieved.
Patent Information
- Application Number
- CN202110610082.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-01
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-06-01
AI Technical Summary
In the prior art, the knowledge graph expansion efficiency is low, and it requires manual expansion, resulting in inefficiency.
By identifying the entity type to which the data to be processed belongs, and based on the preset relationship between the entity type and the attribute type, attribute information is extracted from the data to be processed and added to the created knowledge graph to achieve automated expansion.
It improves the efficiency of knowledge graph expansion, reduces manual intervention, and improves the accuracy and speed of data addition.
Smart Images

Figure CN113254665B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a knowledge graph expansion method, device, electronic device and storage medium. Background Art
[0002] With the advent of the information age, various types of information have exploded. There may be direct or indirect relationships between different entities to which this information belongs. Various applications can provide users with various services based on the direct or indirect relationships. For example, some applications can push product information, film and television information, etc. to users based on the direct or indirect relationships.
[0003] In the prior art, the information of the entities with direct or indirect relationships is usually stored by constructing a knowledge graph. When constructing a knowledge graph, the staff usually manually constructs the knowledge graph based on the existing information, and as the information increases, the constructed knowledge graph can be expanded based on the newly added information to enrich the information contained in the knowledge graph. In the related art, the constructed knowledge graph is usually expanded manually by the staff, which leads to low efficiency in expanding the knowledge graph. Summary of the invention
[0004] The purpose of the embodiments of the present invention is to provide a knowledge graph expansion method, device, electronic device and storage medium to improve the efficiency of expanding the knowledge graph. The specific technical solution is as follows:
[0005] In a first aspect of the present invention, a knowledge graph expansion method is provided, the method comprising:
[0006] Obtaining data to be processed;
[0007] Identify the entity type of the entity to be added to which the data to be processed belongs as the target entity type;
[0008] Based on a first preset relationship between entity type and attribute type, extracting attribute information of the attribute type corresponding to the target entity type from the data to be processed as entity data of the entity to be added;
[0009] The created knowledge graph is expanded based on the entity data of the entity to be added.
[0010] In one embodiment of the present application, the expanding the created knowledge graph based on the entity data of the entity to be added includes:
[0011] Find out whether there is a target entity that is the same entity as the entity to be added in the created knowledge graph;
[0012] If so, merging the entity data of the entity to be added with the entity data of the target entity to expand the knowledge graph;
[0013] If it does not exist, create the entity to be added in the knowledge graph, and add the entity data of the entity to be added in the knowledge graph.
[0014] In one embodiment of the present application, searching whether there is a target entity in the created knowledge graph that is the same entity as the entity to be added includes:
[0015] Based on a second preset relationship between entity type and search method, determine a search method for searching an original entity library for a target entity that is the same entity as the entity to be added, wherein the original entity library contains entity data of the entity in the knowledge graph;
[0016] According to the determined search method, the original entity library is searched to see whether there is a target entity that is the same entity as the entity to be added. If so, it is determined that the target entity exists in the knowledge graph.
[0017] In one embodiment of the present application, the method further includes:
[0018] Add the entity data of the entity to be added to the original entity library.
[0019] In one embodiment of the present application, searching the original entity library for a target entity that is the same entity as the entity to be added according to the determined search method includes:
[0020] Using a preset fuzzy search method, searching the original entity library for entities similar to the entity to be added as candidate entities;
[0021] According to the determined search method, it is searched from the candidate entities whether there is a target entity that is the same entity as the entity to be added.
[0022] In one embodiment of the present application, the using of a preset fuzzy search method to search for an entity similar to the entity to be added from the original entity library as a candidate entity includes:
[0023] Search the original entity library for entities whose index information matches the target attribute information as candidate entities, wherein the index information of each entity includes an attribute field of preset attribute information in the entity data of the entity, and the target attribute information is: the name information in the attribute information of the entity to be added.
[0024] In one embodiment of the present application, the search method corresponding to each entity type is:
[0025] A search method that performs a one-to-one match on the attribute information of the attribute type corresponding to the entity type.
[0026] In one embodiment of the present application, the expanding the created knowledge graph based on the entity data of the entity to be added includes:
[0027] Performing format conversion on the entity data of the entity to be added to obtain triple information of the entity data of the entity to be added, wherein the triple information includes name information, attribute type information, and attribute information of the entity to be added;
[0028] The triple information is imported into the graph database of the knowledge graph, and the entity data of the entity to be added is added to the knowledge graph based on the graph database.
[0029] In one embodiment of the present application, based on the first preset relationship between the entity type and the attribute type, extracting attribute information of the attribute type corresponding to the target entity type from the data to be processed as the entity data of the entity to be added includes:
[0030] Determine the format type of the data to be processed as a target format;
[0031] Selecting a script for extracting data from the target format from pre-designed data extraction scripts as a target script;
[0032] Based on a first preset relationship between entity type and attribute type, determining an attribute type corresponding to the target entity type as a target attribute type;
[0033] Utilizing the target script, the attribute information of the target attribute type is extracted from the data to be processed as the entity data of the entity to be added.
[0034] In a second aspect of the present invention, a knowledge graph expansion device is provided, the device comprising:
[0035] A data acquisition module, used for acquiring data to be processed;
[0036] A type identification module, used to identify the entity type of the entity to be added to which the data to be processed belongs, as the target entity type;
[0037] A data extraction module, configured to extract attribute information of an attribute type corresponding to the target entity type from the data to be processed based on a first preset relationship between entity type and attribute type, as entity data of the entity to be added;
[0038] The graph expansion module is used to expand the created knowledge graph based on the entity data of the entity to be added.
[0039] In a third aspect of the present invention, there is also provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0040] Memory, used to store computer programs;
[0041] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.
[0042] In another aspect of the implementation of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned knowledge graph expansion methods is implemented.
[0043] In another aspect of the implementation of the present invention, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned knowledge graph expansion methods.
[0044] In the knowledge graph expansion scheme provided by the embodiment of the present invention, data to be processed can be obtained; the entity type of the entity to be added to which the data to be processed belongs is identified as the target entity type; based on the first preset relationship between the entity type and the attribute type, the attribute information of the attribute type corresponding to the target entity type is extracted from the data to be processed as the entity data of the entity to be added; and the created knowledge graph is expanded based on the entity data of the entity to be added. In this way, after obtaining the data to be processed, the entity type of the entity to which the data to be processed belongs can be first determined, and then the entity data of the entity can be extracted from the data to be processed, so as to add the entity data of the entity to the knowledge graph, thereby realizing the expansion of the knowledge graph without the need for manual expansion of the knowledge graph. It can be seen that the efficiency of expanding the knowledge graph can be improved by applying the scheme provided by the embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0046] Figure 1 A schematic diagram of a flow chart of a knowledge graph expansion method provided in an embodiment of the present invention;
[0047] Figure 2 A schematic diagram of a process for obtaining entity data provided by an embodiment of the present invention;
[0048] Figure 3 A schematic diagram of an entity matching process provided by an embodiment of the present invention;
[0049] Figure 4 A schematic diagram of a process of entity data merging provided by an embodiment of the present invention;
[0050] Figure 5 A schematic diagram of a knowledge graph expansion process provided by an embodiment of the present invention;
[0051] Figure 6 A schematic diagram of the structure of a knowledge graph expansion device provided by an embodiment of the present invention;
[0052] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0054] In order to improve the efficiency of expanding the knowledge graph, an embodiment of the present invention provides a knowledge graph expansion method, device, electronic device and storage medium.
[0055] In one embodiment of the present invention, a knowledge graph expansion method is provided, the method comprising:
[0056] Obtaining data to be processed;
[0057] Identify the entity type of the entity to be added to which the data to be processed belongs as the target entity type;
[0058] Based on a first preset relationship between entity type and attribute type, extracting attribute information of an attribute type corresponding to the target entity type from the data to be processed as entity data of the entity to be added;
[0059] Expand the created knowledge graph based on the entity data of the entity to be added.
[0060] In this way, after obtaining the data to be processed, the entity type of the entity to which the data to be processed belongs can be determined first, and then the entity data of the entity can be extracted from the data to be processed, thereby adding the entity data of the entity to the knowledge graph to expand the knowledge graph without the need for manual expansion of the knowledge graph. It can be seen that the solution provided by the embodiment of the present invention can improve the efficiency of expanding the knowledge graph.
[0061] The knowledge graph expansion method, device, electronic device and storage medium provided in the embodiments of the present invention are introduced in detail below through specific examples.
[0062] See also Figure 1 , Figure 1 A schematic flow chart of a knowledge graph expansion method provided in an embodiment of the present invention, the method comprising the following steps S101 to S104:
[0063] S101, obtaining data to be processed.
[0064] The data to be processed may be data used to describe the attribute information of an entity, and the entity may be a star, an athlete, a film, a game, a commodity, an album, etc. For example, when the entity is a star, the attribute information may include name information, age information, height information, work information, etc., and when the entity is an album, the attribute information may include album name information, release time information, publisher information, etc.
[0065] In one embodiment of the present invention, the data to be processed may be text-type data, image-type data, voice-type data, etc.
[0066] In one embodiment of the present invention, when obtaining the data to be processed, data can be obtained from a preset database as the data to be processed, or data can be obtained from a public data platform as the data to be processed.
[0067] The above data platform may be an encyclopedia data platform, such as Baidu Encyclopedia Data Platform, Sogou Encyclopedia Data Platform, Google Encyclopedia Data Platform, etc. Since the data in the encyclopedia data platform is highly accurate, rich in information, and clearly laid out, it is convenient to subsequently extract entity data of entities from the encyclopedia data. Therefore, obtaining the data to be processed from the encyclopedia data platform is conducive to improving the accuracy of the subsequently expanded knowledge graph.
[0068] In addition, the above-mentioned data platform can also be a news data platform, a video data platform, an image data platform, etc.
[0069] S102, identifying the entity type of the entity to be added to which the data to be processed belongs, as the target entity type.
[0070] The entity type mentioned above refers to the type to which the entity belongs, which may include: stars, athletes, movies, games, commodities, albums, etc. For example, when the entity to be added to the data to be processed is "Andy Lau", the target entity type mentioned above is "star"; when the entity to be added to the data to be processed is "House of Flying Daggers", the target entity type mentioned above is "movie".
[0071] In one embodiment of the present invention, when identifying the entity type to which the entity to be added belongs, multiple identification methods may be included, which are respectively introduced below:
[0072] In one method, some data fields can be extracted from the data to be processed, and then the extracted data fields are matched with preset classification rules to obtain the scores of each entity type of the entity to be added to which the data to be processed belongs, and the entity type with the largest score is selected as the target entity type to which the entity to be added belongs.
[0073] Among them, the above classification rules are: pre-set rules for distinguishing the categories of entities to which data belongs based on fields. The above partial data fields can be fields in the title part, the beginning part, the end part, etc. of the data to be processed.
[0074] In this way, the entities to which the data to be processed belongs can be divided into types according to the classification rules. The method of identifying types is simple and the efficiency of obtaining the target entity type is high.
[0075] In another way, a pre-trained binary classification model can be used to classify the entity type of the entity to be added to which the processed data belongs.
[0076] Specifically, each binary classification model is used to classify the data to be processed of a certain entity type. For example, assuming that a binary classification model is used to classify the data to be processed of the "star" type, after the data to be processed is input into the binary classification model, the binary classification model can output an output result representing "yes" or "no", indicating that the entity type of the entity described in the input data to be processed is "star" or not "star".
[0077] Multiple binary classification models can be designed, and different binary classification models are used to classify the data to be processed of different entity types. In this way, after obtaining the data to be processed, the data to be processed can be input into each binary classification model in turn until the entity type of the entity to which the data to be processed belongs is determined.
[0078] In one embodiment of the present application, for each binary classification model, when training the binary classification model, sample data can be obtained in advance, and it is manually determined whether the entity type of the entity described by the sample data is the target type to obtain labeling information, and then the sample data is input into the binary classification model to obtain the model output result. When the model output result is inconsistent with the labeling information, the parameters of the binary classification model are adjusted, and then the sample data is re-input into the binary classification model with adjusted parameters until the preset training end condition is reached, and finally a binary classification model for classifying data of the target type is obtained.
[0079] The target type mentioned above can be understood as the entity type corresponding to the data that the trained binary classification model is expected to be able to classify.
[0080] The above training end condition can be that the number of training times reaches a preset threshold, such as 50,000 times, 100,000 times, etc., or that the model output results corresponding to a preset number of sample data are consistent with the annotation information. The above preset number can be 20, 30, 50, etc.
[0081] In this way, the artificial intelligence model is used to identify the entity type of the entity to which the data to be processed belongs, and the accuracy of the recognition result is higher.
[0082] In another manner, manual identification may be used to identify the entity type of the entity to which each piece of data to be processed belongs.
[0083] In one embodiment of the present invention, the above three methods can be combined to identify the entity type of the entity to which the data to be processed belongs. For example, the entity type corresponding to the data to be processed can be first identified using a preset classification rule, and if the identification fails, the binary classification model can be used for identification. If the identification fails again, it can be switched to manual identification.
[0084] S103: Based on a first preset relationship between entity type and attribute type, extract attribute information of an attribute type corresponding to the target entity type from the data to be processed as entity data of the entity to be added.
[0085] Specifically, different entity types correspond to different attribute types. For example, taking the entity type "film" as an example, the corresponding attribute types include film name, release time, starring actor, director, film category, etc.; taking the entity type "game" as an example, the corresponding attribute types include game name, publisher, game category, etc.; taking the entity type "star" as an example, the corresponding attribute types include star name, stage name, date of birth, work, relatives and friends, etc.
[0086] For example, assuming that the entity type to which the entity to be added belongs is identified as a star, and the attribute types corresponding to "star" in the first preset relationship include the star's name, stage name, date of birth, works, and relatives and friends, then the attribute information of the above attribute types can be extracted from the data to be processed as the entity data of the star.
[0087] In one embodiment of the present invention, when extracting entity data of an entity to be added, the data to be processed and the entity type of the entity to which it belongs can be input into a pre-trained data extraction model to obtain the entity data of the entity to be added output by the above model.
[0088] In addition, the data to be processed can be converted into text format, and then semantic analysis can be performed on the data to be processed in text format, and the analysis results of information used to describe the attribute type corresponding to the above target entity type can be selected from the analysis results as the entity data of the entity to be added.
[0089] In one embodiment of the present invention, the format type of the data to be processed can be determined as the target format; from the pre-designed data extraction scripts, a script for extracting data in the target format can be selected as the target script; based on the first preset relationship between the entity type and the attribute type, the attribute type corresponding to the target entity type can be determined as the target attribute type; using the target script, attribute information of the target attribute type can be extracted from the data to be processed as the entity data of the entity to be added.
[0090] The above-mentioned format types include structured data, semi-structured data, text data, etc. Since data of different format types correspond to different data extraction methods, different data extraction scripts can be designed to extract data of different format types to be processed.
[0091] Specifically, the target format of the data to be processed can be first obtained, and then a target script for extracting data from the target format can be determined. Finally, the target script can be used to extract information on the target attribute type in the data to be processed as entity data of the entity to be added.
[0092] In one embodiment of the present application, the above-mentioned target script can perform semantic analysis on the data to be processed in the target format to obtain information described by the data to be processed, and then search for information used to describe the target attribute type from the above-mentioned information as an extraction result, thereby obtaining the entity data of the entity to be added.
[0093] See also Figure 2 , Figure 2 A schematic diagram of a process for obtaining entity data provided by an embodiment of the present invention. Figure 2 As shown, Baidu Encyclopedia data can be obtained from the Baidu Encyclopedia data platform through some public data interfaces OpenApi as the data to be processed, and then the pre-trained binary classifier is used to identify the entity type of the entity to which the above Baidu Encyclopedia data belongs. The binary classifier is the above-mentioned binary classification model, which can include a star classifier, a game classifier, etc., which are used to identify star-type entities, game-type entities, etc. After identifying the entity type of the entity to which the Baidu Encyclopedia data belongs, the Extractor script can be used to extract the entity data of the entity from the Baidu Encyclopedia data, wherein each entity type corresponds to an Extractor script. Finally, the entity data RawData of entities belonging to different entity types can be obtained.
[0094] S104: Expand the created knowledge graph based on the entity data of the entity to be added.
[0095] Specifically, a knowledge graph can be created in advance based on the entity data of existing entities, and the created knowledge graph contains entity types, corresponding attribute types, attribute information, etc. of different entities. After obtaining the entity data of the entity to be added, the entity data of the entity to be added can be added to the knowledge graph, thereby expanding the knowledge graph.
[0096] In one embodiment of the present invention, the entity data of the entity to be added can be formatted to obtain triple information of the entity data of the entity to be added, and the triple information can be imported into the graph database of the knowledge graph, and the entity data of the entity to be added can be added to the knowledge graph based on the graph database.
[0097] The triplet information includes the name information, attribute type information, and attribute information of the entity to be added. The format of the triplet can be json-ld format.
[0098] Assuming that the entity to be added is "Jay Chou", the entity type it belongs to is star, and the attribute types corresponding to "star" include star name, nickname, date of birth, work, and spouse, the triple information of the entity to be added can be shown in Table 1 below:
[0099] Table 1
[0100]
[0101] Specifically, the entity data of the entity to be added can be processed to obtain the triple information of the above entity data in json-ld format, and then the above triple information can be imported into the graph database JanusGraph. The graph database will add the above triple information to the knowledge graph to expand the knowledge graph, and can provide entity query, wandering and other services.
[0102] In the knowledge graph expansion scheme provided by the above embodiment, data to be processed can be obtained; the entity type of the entity to be added to which the data to be processed belongs is identified as the target entity type; based on the first preset relationship between the entity type and the attribute type, the attribute information of the attribute type corresponding to the target entity type is extracted from the data to be processed as the entity data of the entity to be added; and the created knowledge graph is expanded based on the entity data of the entity to be added. In this way, after obtaining the data to be processed, the entity type of the entity to which the data to be processed belongs can be first determined, and then the entity data of the entity can be extracted from the data to be processed, so as to add the entity data of the entity to the knowledge graph, thereby realizing the expansion of the knowledge graph without the need for manual expansion of the knowledge graph. It can be seen that the application of the scheme provided by the above embodiment can improve the efficiency of expanding the knowledge graph.
[0103] In one embodiment of the present invention, when expanding the knowledge graph, it is possible to search in the created knowledge graph whether there is a target entity that is the same entity as the entity to be added. If so, the entity data of the entity to be added is merged with the entity data of the target entity to expand the knowledge graph; if not, the entity to be added is created in the knowledge graph, and the entity data of the entity to be added is added to the knowledge graph.
[0104] Specifically, if there is a target entity in the knowledge graph that is the same entity as the entity to be added, there is no need to create the entity to be added in the knowledge graph again. You only need to merge the entity data of the entity to be added into the entity data of the target entity in the knowledge graph to expand the knowledge graph.
[0105] If there is no target entity in the knowledge graph that is the same entity as the entity to be added, it is necessary to create the entity to be added in the knowledge graph and add the entity data of the entity.
[0106] In one embodiment of the present invention, when searching for a target entity in a knowledge graph, a search method for searching for a target entity that is the same entity as the entity to be added in the original entity library can be determined based on a second preset relationship between the entity type and the search method. According to the determined search method, a search is performed to determine whether the original entity library exists a target entity that is the same entity as the entity to be added. If so, it is determined that the target entity exists in the knowledge graph.
[0107] The original entity database contains entity data of entities in the knowledge graph, and the original entity database may be a non-relational distributed database Hbase.
[0108] Specifically, since data of different entity types have different attribute types, data of different attribute types can be searched according to different matching methods. Therefore, different entity types correspond to different search methods.
[0109] First, according to the entity type of the entity to be added, the search method corresponding to the entity type can be determined from the above-mentioned second preset relationship, and then the target entity that is the same entity as the entity to be added can be searched from the original entity library according to the determined search method. If the target entity can be found in the original entity library, it means that the target entity exists in the knowledge graph. If the target entity cannot be found in the original entity library, it means that the target entity does not exist in the knowledge graph.
[0110] In one embodiment of the present invention, the search method corresponding to each entity type is a search method of performing a one-to-one matching on the attribute information of the attribute type corresponding to the entity type.
[0111] In this way, when searching for a target entity, the entity type of the entity to be added can be determined first, and then the attribute type corresponding to the entity type can be determined. Then, the attribute information of each attribute type can be matched one by one, and finally the target entity that is the same entity as the entity to be added can be found.
[0112] In one embodiment of the present invention, the entity data of the entity to be added can also be added to the original entity library. This ensures that the original entity library contains the entity data of all entities in the knowledge graph, thereby facilitating subsequent searches for the entity data of entities in the knowledge graph from the original entity library.
[0113] In one embodiment of the present invention, when searching for a target entity, a preset fuzzy search method can be used to search the original entity library for entities similar to the entity to be added as candidate entities, and then according to the determined search method, a search is performed among the candidate entities to determine whether there is a target entity that is the same entity as the entity to be added.
[0114] Specifically, firstly, a fuzzy search method is used to search the original entity library for entities similar to the entity to be added. If so, it means that there may be a target entity in the original entity library that is the same entity as the entity to be added. Therefore, the found entity can be used as a candidate entity, and the target entity can be further searched from the candidate entities later.
[0115] If there is no candidate entity, it means that there is no target entity in the original entity library that is the same entity as the entity to be added, so there is no need to perform further search.
[0116] In this way, candidate entities are first searched, and then the target entity is determined from the candidate entities. Since the method of searching for candidate entities is relatively simple, the number of objects to be searched can be reduced when searching for the target entity, thereby improving the efficiency of searching for the target entity.
[0117] In one embodiment of the present invention, when searching for candidate entities, entities whose index information matches the target attribute information in the original entity library may be searched as candidate entities.
[0118] The index information of each entity includes the attribute field of the preset attribute information in the entity data of the entity, and the target attribute information is: the name information in the attribute information of the entity to be added.
[0119] Specifically, an index may be created for each entity in the original entity library, and the index information of each entity includes an attribute field of its preset attribute information, and the preset attribute information may include: name attribute information, relationship attribute information, etc. The ElasticSearch program may be used to create the index information of each entity.
[0120] For example, assuming that the entity is "Liu Xiang", the preset attribute information includes name attribute information "Liu Xiang" and relationship attribute information "spouse: Wu Sha", then the index information of the entity can be "Liu Xiang; spouse: Wu Sha".
[0121] Since the target attribute information is the name information of the entity to be added, it can be understood that a fuzzy search method is used to match the name information of the entity to be added with the name information and relationship information of each entity in the original entity library to find candidate entities associated with the entity to be added from the original entity library.
[0122] In one embodiment of the present invention, when searching for a target entity from candidate entities, a pre-trained entity matching model may be used for searching.
[0123] For details, see Figure 3 , Figure 3 A schematic diagram of an entity matching process provided by an embodiment of the present invention. Figure 3 As shown, the entity data of the entity to be added and the entity data of the above-mentioned candidate entity can be input into the above-mentioned entity matching model. The entity matching model can first use the structure conversion module to perform structural conversion on the input entity data to obtain entity data in vector form, and then use the trained matching analysis module to perform matching degree analysis on the entity data after structural conversion. According to the matching degree analysis result, it is determined whether the candidate entity and the entity to be added are the same entity.
[0124] See also Figure 4 , Figure 4 A schematic diagram of a process of merging entity data provided by an embodiment of the present invention. Figure 4 As shown, after obtaining the entity data of the entity to be added, the index information of each entity in the original entity library can be used to search for candidate entities similar to the entity to be added from the original entity library in a fuzzy search manner, and then the above-mentioned entity matching model is used to determine the target entity that is the same entity as the entity to be added from the candidate entities. Then, according to the attribute type, the entity data of the entity to be added and the data belonging to the same attribute type in the entity data of the target entity in the knowledge graph are merged, thereby expanding the knowledge graph.
[0125] See also Figure 5 , Figure 5 A schematic diagram of a knowledge graph expansion process provided by an embodiment of the present invention, such as Figure 5 As shown:
[0126] First, based on iQiyi data, such as data of entities such as characters, albums, stars, and games, the entity types of entities to which iQiyi data belongs can be identified by manual intervention, and the entity data of the above entities can be extracted from iQiyi data to construct a knowledge graph;
[0127] Then, through some public data interfaces OpenApi, Baidu Encyclopedia data is obtained from the Baidu Encyclopedia data platform as the data to be processed, and then the pre-trained binary classifier is used to identify the entity type of the entity to which the above Baidu Encyclopedia data belongs. The binary classifier may include a star classifier, a game classifier, etc. After identifying the entity type of the entity to which the Baidu Encyclopedia data belongs, the Extractor script can be used to extract the entity data of the entity from the Baidu Encyclopedia data, and finally the entity data Raw Data of entities belonging to different entity types can be obtained;
[0128] Afterwards, for entity data of different entity types, a deduplicator can be used to identify entity data of each entity in the entity library that belongs to the same entity as the entity to be added, and then the entity data belonging to the same entity are merged, and the deduplicated entity data are imported into the graph database to expand the knowledge graph. For example, the above-mentioned deduplicator may include a star deduplicator to deduplicate entity data of star types, and may also include a game deduplicator to deduplicate entity data of game types.
[0129] In the knowledge graph expansion scheme provided by the above embodiment, data to be processed can be obtained; the entity type of the entity to be added to which the data to be processed belongs is identified as the target entity type; based on the first preset relationship between the entity type and the attribute type, the attribute information of the attribute type corresponding to the target entity type is extracted from the data to be processed as the entity data of the entity to be added; and the created knowledge graph is expanded based on the entity data of the entity to be added. In this way, after obtaining the data to be processed, the entity type of the entity to which the data to be processed belongs can be first determined, and then the entity data of the entity can be extracted from the data to be processed, so as to add the entity data of the entity to the knowledge graph, thereby realizing the expansion of the knowledge graph without the need for manual expansion of the knowledge graph. It can be seen that the application of the scheme provided by the above embodiment can improve the efficiency of expanding the knowledge graph.
[0130] See also Figure 6 , Figure 6 A schematic diagram of the structure of a knowledge graph expansion device provided in an embodiment of the present invention, the device comprising:
[0131] The data acquisition module 601 is used to obtain data to be processed;
[0132] A type identification module 602, used to identify the entity type of the entity to be added to which the data to be processed belongs, as a target entity type;
[0133] A data extraction module 603, configured to extract attribute information of an attribute type corresponding to the target entity type from the data to be processed based on a first preset relationship between entity type and attribute type, as entity data of the entity to be added;
[0134] The graph expansion module 604 is used to expand the created knowledge graph based on the entity data of the entity to be added.
[0135] In one embodiment of the present application, the graph expansion module 604 includes:
[0136] A target entity search submodule is used to search whether there is a target entity that is the same entity as the entity to be added in the created knowledge graph. If so, the first expansion submodule is triggered; if not, the second expansion submodule is triggered;
[0137] The first expansion submodule is used to merge the entity data of the entity to be added with the entity data of the target entity to expand the knowledge graph;
[0138] The second expansion submodule is used to create the entity to be added in the knowledge graph and add entity data of the entity to be added in the knowledge graph.
[0139] In one embodiment of the present application, the target entity search submodule includes:
[0140] A search method determining unit, configured to determine a search method for searching an original entity library for a target entity that is the same entity as the entity to be added based on a second preset relationship between the entity type and the search method, wherein the original entity library contains entity data of the entity in the knowledge graph;
[0141] The target entity search unit is used to search the original entity library for a target entity that is the same entity as the entity to be added according to the determined search method. If yes, it is determined that the target entity exists in the knowledge graph.
[0142] In one embodiment of the present application, the device further includes a data adding unit, which is used to:
[0143] Add the entity data of the entity to be added to the original entity library.
[0144] In one embodiment of the present application, the target entity search unit includes:
[0145] A candidate entity search subunit, used to search for entities similar to the entity to be added from the original entity library using a preset fuzzy search method as candidate entities;
[0146] The target entity search subunit is used to search, according to the determined search method, whether there is a target entity that is the same entity as the entity to be added from the candidate entities.
[0147] In one embodiment of the present application, the candidate entity search subunit is specifically used to:
[0148] Search the original entity library for entities whose index information matches the target attribute information as candidate entities, wherein the index information of each entity includes an attribute field of preset attribute information in the entity data of the entity, and the target attribute information is: the name information in the attribute information of the entity to be added.
[0149] In one embodiment of the present application, the search method corresponding to each entity type is:
[0150] A search method that performs a one-to-one match on the attribute information of the attribute type corresponding to the entity type.
[0151] In one embodiment of the present application, the graph expansion module 604 is specifically used to:
[0152] Performing format conversion on the entity data of the entity to be added to obtain triple information of the entity data of the entity to be added, wherein the triple information includes name information, attribute type information, and attribute information of the entity to be added;
[0153] The triple information is imported into the graph database of the knowledge graph, and the entity data of the entity to be added is added to the knowledge graph based on the graph database.
[0154] In one embodiment of the present application, the data extraction module 603 is specifically used to:
[0155] Determine the format type of the data to be processed as a target format;
[0156] Selecting a script for extracting data from the target format from pre-designed data extraction scripts as a target script;
[0157] Based on a first preset relationship between entity type and attribute type, determining an attribute type corresponding to the target entity type as a target attribute type;
[0158] Utilizing the target script, the attribute information of the target attribute type is extracted from the data to be processed as the entity data of the entity to be added.
[0159] In the knowledge graph expansion scheme provided by the above embodiment, data to be processed can be obtained; the entity type of the entity to be added to which the data to be processed belongs is identified as the target entity type; based on the first preset relationship between the entity type and the attribute type, the attribute information of the attribute type corresponding to the target entity type is extracted from the data to be processed as the entity data of the entity to be added; and the created knowledge graph is expanded based on the entity data of the entity to be added. In this way, after obtaining the data to be processed, the entity type of the entity to which the data to be processed belongs can be first determined, and then the entity data of the entity can be extracted from the data to be processed, so as to add the entity data of the entity to the knowledge graph, thereby realizing the expansion of the knowledge graph without the need for manual expansion of the knowledge graph. It can be seen that the application of the scheme provided by the above embodiment can improve the efficiency of expanding the knowledge graph.
[0160] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.
[0161] Memory 703, used for storing computer programs;
[0162] The processor 701 is used to implement the knowledge graph expansion method when executing the program stored in the memory 703.
[0163] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0164] The communication interface is used for communication between the above terminal and other devices.
[0165] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0166] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0167] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the knowledge graph expansion method described in any of the above embodiments is implemented.
[0168] In another embodiment provided by the present invention, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the knowledge graph expansion method described in any of the above embodiments.
[0169] In the knowledge graph expansion scheme provided by the above embodiment, data to be processed can be obtained; the entity type of the entity to be added to which the data to be processed belongs is identified as the target entity type; based on the first preset relationship between the entity type and the attribute type, the attribute information of the attribute type corresponding to the target entity type is extracted from the data to be processed as the entity data of the entity to be added; and the created knowledge graph is expanded based on the entity data of the entity to be added. In this way, after obtaining the data to be processed, the entity type of the entity to which the data to be processed belongs can be first determined, and then the entity data of the entity can be extracted from the data to be processed, so as to add the entity data of the entity to the knowledge graph, thereby realizing the expansion of the knowledge graph without the need for manual expansion of the knowledge graph. It can be seen that the application of the scheme provided by the above embodiment can improve the efficiency of expanding the knowledge graph.
[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0171] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0172] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, electronic device embodiment, computer-readable storage medium embodiment, and computer program product embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A knowledge graph expansion method, characterized in that: The method comprises: Obtaining data to be processed; Identify the entity type of the entity to be added to which the data to be processed belongs as the target entity type; Based on a first preset relationship between entity type and attribute type, extracting attribute information of the attribute type corresponding to the target entity type from the data to be processed as entity data of the entity to be added, where different entity types correspond to different attribute types; Expanding the created knowledge graph based on the entity data of the entity to be added; The extracting, based on the first preset relationship between the entity type and the attribute type, attribute information of the attribute type corresponding to the target entity type from the data to be processed as the entity data of the entity to be added includes: Obtain the entity data of the entity to be added in any of the following ways: Method 1: Determine the format type of the data to be processed as the target format; select a script for extracting data from the data in the target format from pre-designed data extraction scripts as the target script; determine the attribute type corresponding to the target entity type based on a first preset relationship between entity type and attribute type as the target attribute type; use the target script to extract attribute information of the target attribute type from the data to be processed as the entity data of the entity to be added; Method 2: inputting the data to be processed and the target entity type of the entity to which it belongs into a pre-trained data extraction model to obtain the entity data of the entity to be added output by the data extraction model; Method three: convert the data to be processed into text format, then perform semantic analysis on the data in text format, and select analysis results for describing the information of the attribute type corresponding to the target entity type from the analysis results as the entity data of the entity to be added.
2. The method according to claim 1, characterized in that The expanding the created knowledge graph based on the entity data of the entity to be added includes: Find out whether there is a target entity that is the same entity as the entity to be added in the created knowledge graph; If so, merging the entity data of the entity to be added with the entity data of the target entity to expand the knowledge graph; If it does not exist, create the entity to be added in the knowledge graph, and add the entity data of the entity to be added in the knowledge graph.
3. The method according to claim 2, characterized in that The searching whether there is a target entity in the created knowledge graph that is the same entity as the entity to be added includes: Based on a second preset relationship between entity type and search method, determine a search method for searching an original entity library for a target entity that is the same entity as the entity to be added, wherein the original entity library contains entity data of the entity in the knowledge graph; According to the determined search method, the original entity library is searched to see whether there is a target entity that is the same entity as the entity to be added. If so, it is determined that the target entity exists in the knowledge graph.
4. The method according to claim 3, characterized in that The method further comprises: Add the entity data of the entity to be added to the original entity library.
5. The method according to claim 3, characterized in that: The searching, according to the determined searching method, whether the original entity library has a target entity that is the same entity as the entity to be added, includes: Using a preset fuzzy search method, searching the original entity library for entities similar to the entity to be added as candidate entities; According to the determined search method, it is searched from the candidate entities whether there is a target entity that is the same entity as the entity to be added.
6. The method according to claim 5, characterized in that The using of a preset fuzzy search method to search the original entity library for an entity similar to the entity to be added as a candidate entity includes: Search the original entity library for entities whose index information matches the target attribute information as candidate entities, wherein the index information of each entity includes an attribute field of preset attribute information in the entity data of the entity, and the target attribute information is: the name information in the attribute information of the entity to be added.
7. The method according to claim 3, characterized in that The search method corresponding to each entity type is: A search method that performs a one-to-one match on the attribute information of the attribute type corresponding to the entity type.
8. The method according to any one of claims 1 to 7, characterized in that The expanding the created knowledge graph based on the entity data of the entity to be added includes: Performing format conversion on the entity data of the entity to be added to obtain triple information of the entity data of the entity to be added, wherein the triple information includes name information, attribute type information, and attribute information of the entity to be added; The triple information is imported into the graph database of the knowledge graph, and the entity data of the entity to be added is added to the knowledge graph based on the graph database.
9. A knowledge graph expansion device, characterized in that: The device comprises: A data acquisition module, used for acquiring data to be processed; A type identification module, used to identify the entity type of the entity to be added to which the data to be processed belongs, as the target entity type; A data extraction module, configured to extract attribute information of an attribute type corresponding to the target entity type from the data to be processed based on a first preset relationship between entity type and attribute type, as entity data of the entity to be added, where different entity types correspond to different attribute types; A graph expansion module, used to expand the created knowledge graph based on the entity data of the entity to be added; The data extraction module is specifically used to obtain the entity data of the entity to be added in any one of the following ways: Method 1: Determine the format type of the data to be processed as the target format; select a script for extracting data from the data in the target format from pre-designed data extraction scripts as the target script; determine the attribute type corresponding to the target entity type based on a first preset relationship between entity type and attribute type as the target attribute type; use the target script to extract attribute information of the target attribute type from the data to be processed as the entity data of the entity to be added; Method 2: inputting the data to be processed and the target entity type of the entity to which it belongs into a pre-trained data extraction model to obtain the entity data of the entity to be added output by the data extraction model; Method three: convert the data to be processed into text format, then perform semantic analysis on the data in text format, and select analysis results for describing the information of the attribute type corresponding to the target entity type from the analysis results as the entity data of the entity to be added.
10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-8 when executing a program stored in a memory.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.
