Entity disambiguation method, device, computer equipment and storage medium

Abstract

The present application relates to an entity disambiguation method, apparatus, computer equipment and storage medium. The method comprises: obtaining profile data corresponding to a plurality of entities to be disambiguated; a plurality of the entities correspond to the same entity name; inputting the profile data into a pre-trained information extraction model, extracting key attribute information corresponding to the plurality of the entities through the information extraction model; calculating the similarity of two entities to be compared in a plurality of the entities on a plurality of attribute dimensions according to the key attribute information; for each group of two entities to be compared, weightedly summing the similarities of the two entities to be compared in the plurality of attribute dimensions, and obtaining the comprehensive similarity between the two entities to be compared in each group; disambiguating the plurality of entities to be disambiguated according to the comprehensive similarity. The method based on data analysis technology can improve the accuracy of disambiguating entities including unstructured data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an entity disambiguation method, apparatus, computer device and storage medium. Background Art

[0002] Entity disambiguation, also known as semantic disambiguation, is specifically used to resolve the problem of ambiguity between entities with the same name. For example, for two entities with the same name "Zhang San", whether they refer to the same person or different people needs to be disambiguated. Entity disambiguation is widely used in search systems, question-answering systems, and knowledge bases. There are a large number of duplicate names in network resources. When analyzing and understanding texts, it is of great practical significance to perform entity disambiguation to clarify the correct reference of entities.

[0003] Traditional entity disambiguation methods are mainly for entities with structured data, but there is currently no good solution to achieve entity disambiguation for entities with poor data quality or unstructured data. However, for many data crawled from the Internet, most of them are unstructured data. How to correctly analyze these data to clarify the correct direction of the entity is an urgent problem to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide an entity disambiguation method, apparatus, computer device and storage medium that can accurately disambiguate entities including unstructured data to clarify the correct reference of the entity in response to the above technical problems.

[0005] An entity disambiguation method, the method comprising:

[0006] Obtaining profile data corresponding to a plurality of entities to be disambiguated respectively; the plurality of entities correspond to the same entity name;

[0007] Inputting the profile data into a pre-trained information extraction model, and extracting key attribute information corresponding to the plurality of entities respectively through the information extraction model;

[0008] According to the key attribute information, respectively calculating the similarity between two entities to be compared in the plurality of entities in a plurality of attribute dimensions;

[0009] For each group of two entities to be compared, weighted sum of the similarities of the two entities to be compared in the multiple attribute dimensions is performed to obtain a comprehensive similarity between the two entities to be compared in each group;

[0010] The multiple entities to be disambiguated are disambiguated according to the comprehensive similarity.

[0011] An entity disambiguation device, comprising:

[0012] An acquisition module, used to acquire profile data corresponding to a plurality of entities to be disambiguated; the plurality of entities correspond to the same entity name;

[0013] An extraction module, used for inputting the profile data into a pre-trained information extraction model, and extracting key attribute information corresponding to the plurality of entities respectively through the information extraction model;

[0014] A calculation module, used for respectively calculating the similarity between two entities to be compared in the plurality of entities in terms of a plurality of attribute dimensions according to the key attribute information;

[0015] The calculation module is also used for performing weighted summation of the similarities of the two entities to be compared in each group in the multiple attribute dimensions to obtain the comprehensive similarity between the two entities to be compared in each group;

[0016] A determination module is used to disambiguate the multiple entities to be disambiguated according to the comprehensive similarity.

[0017] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0018] Obtaining profile data corresponding to a plurality of entities to be disambiguated respectively; the plurality of entities correspond to the same entity name;

[0019] Inputting the profile data into a pre-trained information extraction model, and extracting key attribute information corresponding to the plurality of entities respectively through the information extraction model;

[0020] According to the key attribute information, respectively calculating the similarity between two entities to be compared in the plurality of entities in a plurality of attribute dimensions;

[0021] For each group of two entities to be compared, weighted sum of the similarities of the two entities to be compared in the multiple attribute dimensions is performed to obtain a comprehensive similarity between the two entities to be compared in each group;

[0022] The multiple entities to be disambiguated are disambiguated according to the comprehensive similarity.

[0023] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0024] Obtaining profile data corresponding to a plurality of entities to be disambiguated respectively; the plurality of entities correspond to the same entity name;

[0025] Inputting the profile data into a pre-trained information extraction model, and extracting key attribute information corresponding to the plurality of entities respectively through the information extraction model;

[0026] According to the key attribute information, respectively calculating the similarity between two entities to be compared in the plurality of entities in a plurality of attribute dimensions;

[0027] For each group of two entities to be compared, weighted sum of the similarities of the two entities to be compared in the multiple attribute dimensions is performed to obtain a comprehensive similarity between the two entities to be compared in each group;

[0028] The multiple entities to be disambiguated are disambiguated according to the comprehensive similarity.

[0029] The above-mentioned entity disambiguation method, device, computer equipment and storage medium input the profile data corresponding to multiple entities corresponding to the same entity name into the information extraction model, and can quickly and accurately extract the key attribute information corresponding to each entity from the profile data. Therefore, when performing similarity calculations, a large amount of invalid data can be filtered out, and only the similarity of the two entities to be compared in multiple attribute dimensions needs to be calculated based on the valid key attribute information. For each group of two entities to be compared, the similarities of each group of two entities to be compared in multiple attribute dimensions are weighted and summed to obtain a comprehensive similarity that can more comprehensively reflect the similarity situation. In this way, based on the comprehensive similarity, it is possible to accurately determine whether the entities to be disambiguated point to the same entity, and unstructured data can be well used to clarify the correct direction of the entity, greatly improving the accuracy of disambiguating entities including unstructured data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is an application scenario diagram of an entity disambiguation method in an embodiment;

[0031] Figure 2 is a flowchart of an entity disambiguation method in one embodiment;

[0032] Figure 3 A flowchart of a step of performing weighted summation of similarities of entities in multiple attribute dimensions to obtain comprehensive similarities between multiple entities to be disambiguated in one embodiment;

[0033] Figure 4 is a structural block diagram of an entity disambiguation device in one embodiment;

[0034] Figure 5 is a structural block diagram of an entity disambiguation device in another embodiment;

[0035] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] The entity disambiguation method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 110 communicates with the server 120 through the network. The terminal 110 can obtain the profile data from the network resources and send the profile data to the server 120. The server 120 executes the entity disambiguation method to determine the disambiguation categories corresponding to the multiple entities to be disambiguated, so as to clarify the correct direction of each entity. Alternatively, the terminal 110 can directly execute the entity disambiguation method to determine the disambiguation categories corresponding to the multiple entities to be disambiguated, so as to clarify the correct direction of each entity. Among them, the terminal 110 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 120 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0038] In one embodiment, Figure 2 As shown, an entity disambiguation method is provided, which is applied to Figure 1 The terminal 110 or the server 120 (both the terminal and the server may be referred to as computer devices) in the example is used for explanation, and the following steps are included:

[0039] S202, obtaining brief description data corresponding to a plurality of entities to be disambiguated; a plurality of entities correspond to the same entity name.

[0040] Entity refers to something that exists objectively and can be distinguished from each other, including specific people, things, abstract concepts or connections. Entity disambiguation, also known as semantic disambiguation, is the process of identifying whether entities with the same entity name refer to the same entity, or determining the target entity from multiple entities with the same entity name.

[0041] The multiple entities to be disambiguated refer to multiple entities corresponding to the same entity name, which can be two entities or more than two entities. By executing the entity disambiguation method, it can be identified whether the entities to be disambiguated point to the same entity. Introduction data is data that explains the entity, including introduction text, introduction video, introduction image, introduction audio and other data, which is a kind of unstructured data. Unstructured data is data with irregular or incomplete data structure, no predefined data model, and is not convenient to be represented by a two-dimensional logical table of a database.

[0042] When the entity is a person entity, the corresponding profile data may be resume data; when the entity is a business entity, the corresponding profile data may be a business introduction; when the entity is an item entity, the corresponding profile data may be the text in the entry that explains the item.

[0043] In one embodiment, the database may store multi-source data, which may come from suppliers, crawled from web pages, or data from third-party platforms, etc. There may be different data corresponding to the same entity name in the multi-source data. Therefore, whether building a knowledge base through the multi-source data or organizing the multi-source data, entity disambiguation of the multi-source data is required.

[0044] In one embodiment, the computer device may extract profile data corresponding to the same entity name from multiple source data. In one embodiment, the computer device may perform a pairwise comparison process on entities with the same entity name, that is, pairing multiple entities to be disambiguated in pairs, respectively determining whether two different entities point to the same entity, and then comprehensively determining whether multiple entities to be disambiguated point to the same entity based on the results of the pairwise determination.

[0045] For example, when the entity is a person entity, the computer device can obtain resume data corresponding to the same entity name. For example, the computer device obtains three resumes all named "Zhang San", which can be respectively called resume data 1, resume data 2, and resume data 3. Then, the computer device can execute the entity disambiguation method according to resume data 1 and resume data 2, respectively, to determine whether the entities corresponding to resume data 1 and resume data 2 are the same entity. The computer device can also execute the entity disambiguation method according to resume data 1 and resume data 3, respectively, to determine whether the entities corresponding to resume data 1 and resume data 3 are the same entity. The computer device can also execute the entity disambiguation method according to resume data 2 and resume data 3, respectively, to determine whether the entities corresponding to resume data 2 and resume data 3 are the same entity.

[0046] S204, inputting the profile data into a pre-trained information extraction model, and extracting key attribute information corresponding to the plurality of entities respectively through the information extraction model.

[0047] Among them, the information extraction model is a machine learning algorithm model that has classification or recognition capabilities after training. The full name of machine learning in English is Machine Learning, abbreviated as ML. The information extraction model can have recognition capabilities through sample learning. The information extraction model of this embodiment is used to identify key attribute information from the profile data. The information extraction model can adopt neural network models such as CNN (Convolutional Neural Network) model, RNN (Recurrent Neural Network) model, LSTM (Long Short-Term Memory) model, BiLSTM (Bi-directional Long Short-Term Memory, Bidirectional Long Short-Term Memory Network) model. In practice, the use of LSTM model can better identify key attribute information from the profile data.

[0048] Key attribute information is the information of the key attributes of an entity, that is, the information of the entity in key dimensions. Different categories of entities may correspond to different key attribute dimensions. The key attribute dimension may be a dimension that is very important to the entity, determined by experience or preset regulations after classifying the data corresponding to the entity. The key attribute information under the key attribute dimension can be used to represent or uniquely identify the entity. For example, when the entity is a person entity, the key attribute information may be information in dimensions such as school name, academic level, major name, company name, and department name; when the entity is a corporate entity, the corresponding key attribute information may be information in dimensions such as corporate legal person, corporate establishment time, corporate location, corporate scale, or main business.

[0049] Specifically, the information extraction model is trained by training data to obtain an information extraction model with identification capability. The computer device can segment the sentences in the profile data, remove the punctuation marks in the sentences, and input the profile data corresponding to each entity into the information extraction model respectively. After the information extraction model performs feature extraction and analysis on the profile data, the identification symbol corresponding to each character in the profile data is obtained. The text corresponding to the specific identification symbol is extracted as the corresponding key attribute information.

[0050] For example, when the entity is a person entity, the computer device can input the profile data corresponding to the person entity into the information extraction model, and extract the corresponding key attribute information through the information extraction model. For example, the computer device can pre-set the identifier "O" to identify invalid information; the identifier "B-ORG" to identify "school name", the identifier "I-ORG" to identify "education level", etc. Then, when the information extraction model outputs the profile data and the identifier corresponding to each character, the corresponding key attribute information can be determined according to the identifier.

[0051] In one embodiment, the entity includes a person entity; the entity name includes a person name; the profile data includes resume data; and the key attribute information includes a school name, academic level, major name, company name, and multiple department names.

[0052] The resume data is a concise written introduction of a person's education, experience, expertise, hobbies and other relevant information. When the entity includes a person entity, the profile data includes resume data, and the key attribute information includes multiple types of school name, education level, major name, company name, and department name, by executing the entity disambiguation method, it is possible to accurately determine whether entities corresponding to the same person name are the same entity based on the resume data.

[0053] S206: Calculate the similarities of two entities to be compared in multiple attribute dimensions respectively according to the key attribute information.

[0054] Among them, the attribute dimension is the dimension corresponding to the key attribute information. For example, when the key attribute information is the name of the school, the corresponding attribute dimension is the school dimension corresponding to the entity. Similarity is the degree of similarity between entities. The greater the similarity between two entities, the greater the possibility that the two entities point to the same object. The smaller the similarity between two entities, the smaller the possibility that the two entities point to the same object, and the more likely they are different entities. The similarity between entities can be determined by comparing the similarity between the key attribute information of each entity. Specifically, the computer device can calculate the similarity between each entity in multiple attribute dimensions based on the key attribute information.

[0055] In one embodiment, step S206 specifically includes: for two entities to be compared among multiple entities, in each attribute dimension, the key attribute information corresponding to the two entities to be compared is split into multiple characters to form a character set corresponding to the two entities to be compared and to each attribute dimension; from the character set, characters corresponding to both entities to be compared are selected to form a character intersection set; based on the character set, a total character set corresponding to each attribute dimension is determined; in each attribute dimension, the character overlap between the character intersection set and the total character set is calculated; and the character overlap corresponding to each attribute dimension is used as the similarity between the two entities to be compared in each attribute dimension.

[0056] Specifically, after determining the key attribute information of each entity in different attribute dimensions, the computer device can perform pairwise comparisons on the multiple entities to be compared. For two entities to be compared among the multiple entities, the following processing is performed in each attribute dimension: the key attribute information corresponding to the two entities to be compared among the multiple entities under the attribute dimension is split into single characters, and the single characters corresponding to each entity under the attribute dimension are composed of a character set corresponding to the two entities to be compared and corresponding to the attribute dimension. Further, the computer device can select characters corresponding to both entities to be compared from the character sets corresponding to the two entities to be compared, that is, to find the intersection of each character set to obtain a character intersection set. The computer device removes the duplicates of the characters corresponding to the two entities to be compared under the attribute dimension and composes a total character set, that is, the union of the character sets. The computer device uses the overlap between the character intersection set composed of the characters corresponding to the two entities to be compared and the corresponding total character set as the similarity of the entity to be compared. Accordingly, corresponding processing is performed under each attribute dimension.

[0057] In one embodiment, the degree of overlap between the character intersection set and the corresponding character total set may specifically be the ratio of the number of characters in the character intersection set to the number of characters in the character total set.

[0058] In one embodiment, the number of entities to be disambiguated is two, or the number of entities to be disambiguated is multiple, but the computer device calculates the similarity between two entities in sequence based on the pairwise combinations of the multiple entities to determine whether the two entities are the same entity. In this case, the computer device performs the following processing for each attribute dimension: the key attribute information corresponding to the two entities to be disambiguated under the dimension is split into single characters respectively. A character intersection set is formed based on the characters corresponding to both entities. The characters corresponding to the two entities are deduplicated to form a total character set. The degree of overlap between the character intersection set corresponding to the two entities to be disambiguated and the total character set is taken as the similarity of the two entities to be disambiguated under the attribute dimension.

[0059] In one embodiment, the number of entities to be disambiguated is three or more, and the computer device can simultaneously calculate the similarity between the three or more entities to determine whether the three or more entities are the same entity. In this case, the computer device performs the following processing for each attribute dimension: the key attribute information corresponding to the three or more entities to be disambiguated under the dimension is split into single characters respectively. A character intersection set is formed according to the characters corresponding to the three or more entities. The characters corresponding to the three or more entities are deduplicated to form a total character set. The overlap between the character intersection set corresponding to the three or more entities to be disambiguated and the total character set is used as the similarity between the two entities to be disambiguated under the attribute dimension.

[0060] S208 , for each group of two entities to be compared, weighted sum of the similarities between the two entities to be compared in multiple attribute dimensions is performed to obtain a comprehensive similarity between the two entities to be compared in each group.

[0061] Specifically, the computer device may obtain preset weighting coefficients corresponding to the multiple attribute dimensions, and perform weighted summation on the similarities of the two entities to be compared in the multiple attribute dimensions according to the corresponding preset weighting coefficients to obtain the comprehensive similarity between the two entities to be compared. The following formula may be used for calculation: comprehensive similarity = weighting coefficient 1*similarity 1+weighting coefficient 2*similarity 2+…+weighting coefficient N*similarity N. In one embodiment, the computer device may assign different weighting coefficients according to the importance corresponding to different attribute dimensions.

[0062] For example, if the attribute dimensions include school dimension, education dimension, major dimension and enterprise dimension, the corresponding preset weighting coefficients are 0.2, 0.3, 0.4 and 0.1. If the similarities between the entities to be disambiguated in each attribute dimension are 90%, 90%, 100%, and 70% respectively, then the comprehensive similarity = 0.2*90%+0.3*90%+0.4*100%+0.1*70%=85%.

[0063] S210, disambiguating multiple entities to be disambiguated based on the comprehensive similarities.

[0064] Specifically, the computer device can determine the disambiguation categories corresponding to the multiple entities to be disambiguated based on the comprehensive similarity. The computer device can pre-set the corresponding threshold value to determine whether the multiple entities to be disambiguated are the same entity, that is, the disambiguation category corresponding to the multiple entities to be disambiguated. Among them, the disambiguation category is the category corresponding to the multiple entities after the multiple entities to be disambiguated are judged, which is used to indicate whether the entities to be disambiguated point to the same entity. The disambiguation category may specifically include at least one of similar entities, independent entities, manual judgment is required, and it cannot be judged. Among them, similar entities refer to the multiple entities to be disambiguated as the same entity; independent entities refer to the entities to be disambiguated as independent entities, which do not point to the same entity; manual judgment is required, which means that it is impossible to judge whether the current entity is the same entity based on the current conditions, and it may be a similar entity or an independent entity; it cannot be judged, which means that the current amount of information is too small, and it is impossible to judge whether the current entity to be disambiguated is the same entity.

[0065] In one embodiment, the computer device may pre-set a first threshold, a second threshold, and a third threshold, wherein the first threshold is greater than the second threshold, and the second threshold is greater than the third threshold. When the comprehensive similarity is greater than or equal to the first threshold, the multiple entities to be disambiguated are determined to be similar entities; when the comprehensive similarity is less than the first threshold and greater than the second threshold, manual judgment is required, and the computer device cannot determine whether the multiple entities to be disambiguated are similar entities or independent entities; when the comprehensive similarity is less than or equal to the second threshold and greater than the third threshold, the multiple entities to be disambiguated are determined to be independent entities; when the comprehensive similarity is less than or equal to the third threshold, it cannot be determined (it may be that the amount of information is too little to make a judgment).

[0066] For example, when the entity is a person entity and there are currently two entities to be disambiguated, when the comprehensive similarity is >= 70%, the two person entities to be compared are judged to be similar person entities; when 40% < comprehensive similarity < 70%, manual judgment is required; when 10% < comprehensive similarity <= 40%, the two person entities to be compared are judged to be independent person entities; when the comprehensive similarity <= 10%, no judgment can be made.

[0067] The above-mentioned entity disambiguation method inputs the profile data corresponding to multiple entities with the same entity name into the information extraction model, and can quickly and accurately extract the key attribute information corresponding to each entity from the profile data. Therefore, when performing similarity calculation, a large amount of invalid data can be filtered out, and only the similarity of the two entities to be compared in multiple attribute dimensions needs to be calculated based on the valid key attribute information. For each group of two entities to be compared, the similarities of each group of two entities to be compared in multiple attribute dimensions are weighted and summed to obtain a comprehensive similarity that can more comprehensively reflect the similarity situation. In this way, based on the comprehensive similarity, it is possible to accurately determine whether the entities to be disambiguated point to the same entity, and unstructured data can be well used to clarify the correct direction of the entity, greatly improving the accuracy of disambiguating entities including unstructured data.

[0068] In one embodiment, step S202, i.e., the step of obtaining profile data respectively corresponding to the multiple entities to be disambiguated, specifically includes: obtaining the multi-source data to be processed; determining the entity name of the entity corresponding to the multi-source data; dividing the multi-source data corresponding to the same entity name into the same group of multi-source data; respectively extracting the profile data from each group of multi-source data, and using the extracted profile data as the profile data respectively corresponding to the multiple entities to be disambiguated.

[0069] Among them, multi-source data refers to data from multiple sources, with complex contents and uneven data quality. Specifically, the computer device can determine the entity name of the entity corresponding to the multi-source data, and then divide the multi-source data corresponding to the same entity name into the same group of multi-source data. Extract the brief introduction data from each group of multi-source data, and use the extracted brief introduction data as the brief introduction data corresponding to the multiple entities to be disambiguated.

[0070] Taking the fund manager entity as an example, the multi-source data corresponding to the fund manager entity include nationality, gender, resume data, fund portfolio performance data, and investment style. In order to integrate the corresponding multi-source data, the computer device must first perform person entity disambiguation. Although the computer device can extract the person's name, nationality, gender, start time, resume data, etc. from the multi-source data to be processed, due to poor data quality, the nationality, gender, and start time fields may be inaccurate or empty, and traditional disambiguation methods are difficult to accurately match duplicate fund manager entities. In response to this situation, the computer device can divide the multi-source data corresponding to the same person name into the same group of multi-source data, and then extract the resume data from each group of multi-source data, and perform corresponding entity disambiguation based on the extracted resume data.

[0071] In the above embodiment, multi-source data corresponding to the same entity name are divided into the same group of multi-source data, and profile data are extracted from each group of multi-source data, and corresponding entity disambiguation is performed based on the extracted profile data. Since the unstructured profile data in the multi-source data is the most reliable data source of information, the accuracy and practicality of entity disambiguation based on the unstructured profile data will be greatly improved.

[0072] In one embodiment, step S208, that is, for each group of two entities to be compared, performing weighted summation on the similarities of the two entities to be compared in multiple attribute dimensions to obtain the comprehensive similarity between the two entities to be compared in each group, specifically includes the following steps:

[0073] S302: For each group of two entities to be compared, determine the preset attribute similarity corresponding to the two entities to be compared in a preset attribute dimension.

[0074] Specifically, for each group of two entities to be compared, the computer device may determine the preset attribute similarity corresponding to the two entities to be compared on the preset attribute dimension. In one embodiment, the preset attribute dimension may be an attribute dimension that is identified as more important in practical experience.

[0075] S304: Determine a weighting coefficient configured corresponding to a preset attribute similarity.

[0076] Specifically, the computer device may determine the current actual scene according to the preset attribute similarity corresponding to the preset attribute dimension, and then dynamically adjust the weighting coefficient. The computer device may determine the current actual scene according to the current preset attribute similarity, and thus select a weighting coefficient that matches the scene. Among them, the weighting coefficients that match each scene may be set in advance, and the weights may be adjusted according to the current actual scene to increase the accuracy of determining whether the entities to be disambiguated are the same entity.

[0077] For example, taking the person entity as an example, when the entities to be disambiguated have no intersection in the school dimension, the weight coefficients corresponding to the department dimension and the enterprise dimension can be set to 0.5 respectively. When the entities to be disambiguated have no intersection in the enterprise dimension and the similarity of the preset attributes in the school dimension is greater than 40%, the weight coefficient corresponding to the school dimension can be set to 0.8; otherwise, the weight coefficient corresponding to the enterprise dimension is 0.8. When the entities to be disambiguated have no intersection in the department dimension and the similarity of the preset attributes in the school dimension is greater than 40%, the weight coefficient corresponding to the school dimension can be set to 0.8; otherwise, the weight coefficient corresponding to the department dimension is 0.8. When the entities to be disambiguated have no intersection in both the department dimension and the enterprise dimension, the weight coefficient corresponding to the school dimension can be set to 0.1. When the entities to be disambiguated have no intersection in the department dimension and the school dimension, the weight coefficient corresponding to the learning dimension can be set to 0.8. When the entities to be disambiguated have no intersection in both the enterprise dimension and the school dimension, the weight coefficient corresponding to the department dimension can be set to 0.8. When the entities to be disambiguated have intersections in all three dimensions, the weighting coefficient corresponding to the enterprise dimension can be set to 0.4, and the weighting coefficient corresponding to the school dimension can be set to 0.2. Among them, intersection means that there are the same characters; no intersection means that there are no same characters.

[0078] S306 , performing weighted summation on the similarities of the two entities to be compared in multiple attribute dimensions according to the weighting coefficients, to obtain a comprehensive similarity between the two entities to be compared.

[0079] Specifically, the computer device may perform weighted summation of the similarities of the two entities to be compared in multiple attribute dimensions according to the dynamically adjusted weighting coefficients, so as to obtain the comprehensive similarity between the two entities to be compared.

[0080] In the above embodiment, the weighting coefficient of the corresponding configuration can be dynamically adjusted based on the preset attribute similarities corresponding to the multiple entities to be disambiguated in the preset attribute dimension, so as to accurately determine the comprehensive similarity between the two entities to be compared.

[0081] In one embodiment, the entity disambiguation method also includes a data fusion step, which specifically includes: when multiple entities to be disambiguated are similar entities, respectively acquiring multi-source data corresponding to the entities; and fusing the multi-source data corresponding to the multiple entities to be disambiguated.

[0082] Specifically, when the multiple entities to be disambiguated are similar entities, it can be determined that the current multiple entities to be disambiguated point to the same entity. The computer device can respectively obtain multi-source data corresponding to the entities, and perform data fusion on the multi-source data corresponding to the multiple entities to be disambiguated.

[0083] In one embodiment, there are multiple methods for data fusion, which are not limited here. For example, for multi-source data under the same attribute dimension, the computer device may replace the old version of the data with the new version of the data. Alternatively, the computer device may replace the data with the low quality data with the data with the high quality data. Alternatively, the computer device may replace the data with the low quality data with the data with the high quality data according to the user rating.

[0084] In the above embodiment, data fusion is performed on multi-source data corresponding to similar entities, so as to expand the content corresponding to the entity, thereby making the content of the entity richer and more accurate.

[0085] In one embodiment, the entity disambiguation method also includes a training step of an information extraction model, which specifically includes: obtaining sample profile data; marking key sample attribute information in the sample profile data based on sample marker characters; inputting the sample profile data into the information extraction model, and outputting intermediate results through the information extraction model; adjusting model parameters based on the difference between the intermediate results and the sample marker characters until the training stop condition is met.

[0086] Specifically, in the process of training the information extraction model, the computer device may obtain the sample profile data in advance. The key sample attribute information in the sample profile data is marked according to the sample marking characters by manual or automatic marking. For example, when the sample profile data is sample resume data, the computer device may mark out the key sample attribute information such as the school name, academic level, major name, company name, and department name in the sample resume data. When marking the key sample attribute information, the computer device may pre-set the identification symbol "O" to identify invalid information; the identification symbol "B-ORG" is used to identify the school name, the identification symbol "I-ORG" is used to identify "academic level", etc. Distinguish valid information from invalid information, and use different identification symbols to mark the key sample attribute information corresponding to different attribute dimensions.

[0087] Furthermore, the computer device may input the sample profile data into the information extraction model to obtain an intermediate result, and adjust the model parameters according to the difference between the intermediate result and the sample tag character until the training stop condition is met. The intermediate result is the result output by the information extraction model after the sample profile data is input into the information extraction model during the training process. The training stop condition is the condition for ending the model training. The training stop condition may be reaching a preset number of iterations, or the output performance index of the information extraction model after adjusting the model parameters reaches a preset index.

[0088] Then, the trained information extraction model has the ability to automatically identify and label. After the information extraction model performs feature analysis on the input profile data, it outputs the profile data containing the identification symbol. The computer device can extract the key attribute information of the entity from the output profile data according to the identification symbol corresponding to each character.

[0089] In one embodiment, the computer device may compare the difference between the intermediate result and the sample marked character, thereby adjusting the model parameters of the information extraction model in the direction of reducing the difference. If the training stop condition is not met after adjusting the model parameters, the step of inputting the sample profile data into the information extraction model, outputting the intermediate result through the information extraction model and continuing the training, until the training stop condition is met, the training is terminated.

[0090] In the above embodiment, the information extraction model is trained by using sample profile data and sample marking characters that mark key sample attribute information, so that the information extraction model can learn the ability to identify key attribute information. During the model training process, by adjusting the model parameters, a model that can accurately extract key attribute information from the profile data can be trained as quickly as possible, thereby improving training efficiency.

[0091] It should be understood that although Figure 2-3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0092] In one embodiment, Figure 4 As shown, an entity disambiguation device 400 is provided, comprising: an acquisition module 401, an extraction module 402, a calculation module 403 and a determination module 404, wherein:

[0093] The acquisition module 401 is used to acquire profile data corresponding to multiple entities to be disambiguated; multiple entities correspond to the same entity name.

[0094] The extraction module 402 is used to input the profile data into a pre-trained information extraction model, and extract key attribute information corresponding to the multiple entities through the information extraction model.

[0095] The calculation module 403 is used to calculate the similarity between two entities to be compared in multiple attribute dimensions according to the key attribute information.

[0096] The calculation module 403 is further configured to perform weighted summation of the similarities of each group of two entities to be compared in multiple attribute dimensions, so as to obtain the comprehensive similarity between each group of two entities to be compared.

[0097] The determination module 404 is used to disambiguate the multiple entities to be disambiguated according to the comprehensive similarities.

[0098] In one embodiment, the acquisition module 401 is also used to acquire multi-source data to be processed; determine the entity name of the entity corresponding to the multi-source data; divide the multi-source data corresponding to the same entity name into the same group of multi-source data; respectively extract the introduction data in each group of multi-source data, and use the extracted introduction data as the introduction data corresponding to the multiple entities to be disambiguated.

[0099] In one embodiment, the calculation module 403 is also used to split the key attribute information corresponding to two entities to be compared among multiple entities into multiple characters in each attribute dimension to form a character set corresponding to the two entities to be compared and to each attribute dimension; select characters corresponding to both entities to be compared from the character set to form a character intersection set; determine the total set of characters corresponding to each attribute dimension based on the character set; calculate the character overlap between the character intersection set and the total set of characters in each attribute dimension; and use the character overlap corresponding to each attribute dimension as the similarity between the two entities to be compared in each attribute dimension.

[0100] In one embodiment, the calculation module 403 is also used to determine, for each group of two entities to be compared, the preset attribute similarity corresponding to the two entities to be compared in the preset attribute dimension; determine the weighting coefficient configured corresponding to the preset attribute similarity; and according to the weighting coefficient, perform weighted summation on the similarities of the two entities to be compared in multiple attribute dimensions to obtain the comprehensive similarity between the two entities to be compared.

[0101] In one embodiment, the entity disambiguation device 400 further includes a fusion module 405, wherein the acquisition module 401 is further used to respectively acquire multi-source data corresponding to the entities when the multiple entities to be disambiguated are similar entities. The fusion module 405 is used to perform data fusion on the multi-source data corresponding to the multiple entities to be disambiguated.

[0102] refer to Figure 5In one embodiment, the entity disambiguation device 400 also includes a model training module 406, which is used to obtain sample profile data; mark key sample attribute information in the sample profile data according to sample label characters; input the sample profile data into the information extraction model, and output the intermediate result through the information extraction model; adjust the model parameters according to the difference between the intermediate result and the sample label characters until the training stop condition is met.

[0103] The above-mentioned entity disambiguation device inputs the profile data corresponding to multiple entities with the same entity name into the information extraction model, and can quickly and accurately extract the key attribute information corresponding to each entity from the profile data. Therefore, when performing similarity calculation, a large amount of invalid data can be filtered out, and only the similarity of the two entities to be compared in multiple attribute dimensions needs to be calculated based on the valid key attribute information. For each group of two entities to be compared, the similarities of each group of two entities to be compared in multiple attribute dimensions are weighted and summed to obtain a comprehensive similarity that can more comprehensively reflect the similarity situation. In this way, based on the comprehensive similarity, it is possible to accurately determine whether the entities to be disambiguated point to the same entity, and unstructured data can be well used to clarify the correct direction of the entity, greatly improving the accuracy of disambiguating entities including unstructured data.

[0104] For the specific definition of the entity disambiguation device, please refer to the definition of the entity disambiguation method above, which will not be repeated here. Each module in the above-mentioned entity disambiguation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0105] In one embodiment, a computer device is provided. The computer device may be a terminal or a server. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an entity disambiguation method is implemented.

[0106] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the entity disambiguation method described above. The steps of the entity disambiguation method here may be the steps of the entity disambiguation method in the above-mentioned various embodiments.

[0108] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the entity disambiguation method described above. The steps of the entity disambiguation method here can be the steps of the entity disambiguation method in the above-mentioned various embodiments.

[0109] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An entity disambiguation method, characterized in that: The method comprises: Obtaining profile data corresponding to a plurality of entities to be disambiguated respectively; the plurality of entities correspond to the same entity name; Input the profile data into a pre-trained information extraction model, perform feature extraction analysis using the information extraction model to obtain identification symbols corresponding to each character in the profile data, and use the text corresponding to the specific identification symbol as the corresponding key attribute information; For two entities to be compared among the multiple entities, in each attribute dimension, the key attribute information corresponding to the two entities to be compared is respectively split into multiple characters to form a character set corresponding to the two entities to be compared and to each attribute dimension; Selecting characters corresponding to both entities to be compared from the character set to form a character intersection set; Determine, based on the character set, a total character set corresponding to each attribute dimension; In each attribute dimension, respectively calculating the character overlap degree between the character intersection set and the character total set; The character overlap degree corresponding to each attribute dimension is used as the similarity degree of the two characters to be compared in each attribute dimension; For each group of two entities to be compared, determining the preset attribute similarity of the two entities to be compared in a preset attribute dimension, determining the current actual scenario according to the preset attribute similarity, and selecting a weighting coefficient matching the actual scenario; According to the dynamically adjusted weighting coefficient, weighted summation is performed on the similarities of the two entities to be compared in multiple attribute dimensions to obtain a comprehensive similarity between the two entities to be compared; The multiple entities to be disambiguated are disambiguated according to the comprehensive similarity.

2. The method according to claim 1, characterized in that The obtaining of profile data corresponding to the plurality of entities to be disambiguated includes: Obtain multi-source data to be processed; Determine the entity name of the entity corresponding to the multi-source data; dividing the multi-source data corresponding to the same entity name into the same group of multi-source data; The brief introduction data in each group of multi-source data is extracted respectively, and the extracted brief introduction data is used as the brief introduction data corresponding to the multiple entities to be disambiguated respectively.

3. The method according to claim 1, characterized in that The method further comprises: When the multiple entities to be disambiguated are similar entities, respectively obtaining multi-source data corresponding to the entities; Data fusion is performed on multi-source data corresponding to the multiple entities to be disambiguated.

4. The method according to any one of claims 1 to 3, characterized in that The training steps of the information extraction model include: Get sample profile data; Marking key sample attribute information in the sample profile data according to the sample marking character; Inputting the sample profile data into an information extraction model, and outputting an intermediate result through the information extraction model; The model parameters are adjusted according to the difference between the intermediate result and the sample labeled characters until the training stop condition is met.

5. The method according to any one of claims 1 to 3, characterized in that The entity includes a person entity; the entity name includes a person name; the profile data includes resume data; and the key attribute information includes multiple ones of a school name, academic level, major name, company name, and department name.

6. An entity disambiguation device, characterized in that: The device comprises: An acquisition module, used to acquire profile data corresponding to a plurality of entities to be disambiguated; the plurality of entities correspond to the same entity name; An extraction module, used to input the profile data into a pre-trained information extraction model, obtain the identification symbol corresponding to each character in the profile data after performing feature extraction analysis through the information extraction model, and use the text corresponding to the specific identification symbol as the corresponding key attribute information; A calculation module is used for, for two entities to be compared among the multiple entities, splitting the key attribute information corresponding to the two entities to be compared into multiple characters in each attribute dimension to form a character set corresponding to the two entities to be compared and to each attribute dimension; selecting characters corresponding to both entities to be compared from the character set to form a character intersection set; determining a total character set corresponding to each attribute dimension according to the character set; calculating the character overlap between the character intersection set and the total character set in each attribute dimension; and using the character overlap corresponding to each attribute dimension as the similarity between the two entities to be compared in each attribute dimension; The calculation module is also used to determine, for each group of two entities to be compared, the preset attribute similarity of the two entities to be compared in a preset attribute dimension, determine the current actual scenario according to the preset attribute similarity, and select a weighting coefficient that matches the actual scenario; perform weighted summation on the similarities of the two entities to be compared in multiple attribute dimensions according to the dynamically adjusted weighting coefficient to obtain a comprehensive similarity between the two entities to be compared; A determination module is used to disambiguate the multiple entities to be disambiguated according to the comprehensive similarity.

7. The device according to claim 6, characterized in that The acquisition module is further used for: Obtain multi-source data to be processed; Determine the entity name of the entity corresponding to the multi-source data; dividing the multi-source data corresponding to the same entity name into the same group of multi-source data; The brief introduction data in each group of multi-source data is extracted respectively, and the extracted brief introduction data is used as the brief introduction data corresponding to the multiple entities to be disambiguated respectively.

8. The device according to claim 6, characterized in that The device also includes a fusion module, wherein: The acquisition module is further configured to respectively acquire multi-source data corresponding to the entities when the multiple entities to be disambiguated are similar entities; The fusion module is used to fuse the multi-source data corresponding to the multiple entities to be disambiguated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images