Entity disambiguation method, device and equipment based on rule screening and large model assistance and storage medium

Abstract

The invention discloses an entity disambiguation method, device and equipment based on rule screening and large model assistance and a storage medium, and aims at the actual situation that the throughput of a large model is limited, a rule screening mechanism is introduced, and candidate entities are preliminarily filtered through a lightweight algorithm; and then, a rule screening result is subjected to refined discrimination by using a large model, so that the overall disambiguation efficiency and accuracy are improved, and the problem of insufficient entity recognition efficiency and precision in a multi-language hybrid scene is solved. Cross-semantic uniformization processing of names, pinyin, translation names and mechanism fields is realized, and a stable and extensible basis is provided for academic talent portraits and intelligent recommendation.

Description

Technical Field

[0001] This invention relates to the field of entity disambiguation technology, and in particular to an entity disambiguation method, apparatus, device, and storage medium based on rule filtering and large model assistance. Background Technology

[0002] Named entity disambiguation is an important research direction in artificial intelligence semantic understanding, with wide applications in knowledge graph construction, semantic retrieval, recommender systems, and information extraction. The goal of named entity disambiguation is to accurately identify the real object referred to by an entity name in text when multiple entities with the same or similar names exist, thereby improving the accuracy and consistency of information recognition and knowledge reasoning.

[0003] Named entity ambiguity manifests primarily in two aspects: referential diversity and referential ambiguity. Referential diversity refers to the fact that the same entity may have multiple forms of expression, such as its full name, abbreviation, English name, pinyin form, mixed expression, or other aliases. Referential ambiguity refers to the fact that different entities may use the same or similar names. These characteristics are prevalent in real-world corpora, especially in scientific research, business settings, and cross-linguistic scenarios, where the mixing of Chinese, English, and pinyin is particularly prominent, making entity recognition and differentiation more complex.

[0004] Existing entity disambiguation methods mostly rely on keyword matching, string similarity, or rule-based templates to identify candidate entities by comparing their names with the input text. These methods struggle to handle scenarios involving a mix of Chinese and English, or the coexistence of pinyin and English abbreviations, as they suffer from inconsistencies arising from diverse expressions and language conversions. Furthermore, they fail to adequately utilize external entity information (such as organization, domain, time, and location), resulting in low disambiguation accuracy.

[0005] With the rapid development of artificial intelligence technologies such as Large Language Models (LLM), these models have demonstrated significant advantages in cross-linguistic feature representation and unified semantic modeling. Large models can identify semantic correspondences in mixed texts such as Chinese, English, and Pinyin through multilingual encoding capabilities, and achieve more accurate semantic differentiation by combining auxiliary entity information (such as institution names, research fields, and geographical locations). However, large models still suffer from high computational resource consumption and limited throughput during the inference phase, resulting in low efficiency when directly applied to large-scale text processing.

[0006] In view of the above problems, the existing technology lacks an entity name disambiguation method that can integrate the semantic recognition capabilities of large models with entity-related information while taking into account both recognition accuracy and processing efficiency. Summary of the Invention

[0007] Based on this, it is necessary to propose an entity disambiguation method, apparatus, device, and storage medium based on rule-based filtering and large model assistance to address the above problems.

[0008] An entity disambiguation method based on rule-based filtering and large model assistance, the method comprising:

[0009] Extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographic entries as original entities; multiple original entities constitute an original entity set;

[0010] The original entity set is standardized to obtain a standardized entity set;

[0011] The similarity is calculated for any two standardized entities in the standardized entity set, and the standardized entities are classified according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set.

[0012] Common features of standardized entities in suspected ambiguity sets are extracted and summarized to form prior constraints for entity disambiguation;

[0013] The prior constraints are encoded into the prompt templates respectively, and the generation control parameters in the reasoning stage are adjusted to form multiple prompt engineering models;

[0014] The optimal large model is selected from multiple prompting engineering models based on the validation set, which is a subset of the suspected ambiguity set;

[0015] The suspected ambiguity set is finally disambiguated by the optimal large model to obtain the unambiguous set, which contains the disambiguated entity names.

[0016] In one embodiment, the extraction of entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographical entries is used as the original entity; multiple original entities constitute an original entity set. include:

[0017] Obtain text information, and extract scholar information from the text information; the text information includes: the scientific research paper database of the scientific research project system, project application resumes, patent bibliographic information, scientific research awards and other data; the scholar information includes: undisambiguated entity names and entity attributes related to undisambiguated entity names; the undisambiguated entity names include: scholar names; the entity attributes include: original text data of information such as affiliated institutions, research directions, and geographical locations;

[0018] Undisambiguated entity names and entity attributes form a primitive entity; multiple primitive entities constitute a primitive entity set. ,in, The original entity represents the undisambiguated entity name and entity attributes, including the affiliated institution, research direction, geographical location, etc. The original entity is represented as follows:

[0019]

[0020] in, This represents the undisambiguated entity name of the i-th scholar. Indicates the institution to which the scholar belongs. Indicates the scholar's research direction. Indicates the scholar's geographical location.

[0021] In one embodiment, the original entity set Standardization yields a standardized entity set. ;

[0022] Based on the original entity set Undisambiguated entity names and entity attributes are formatted in a unified manner, including removing spaces, symbols and control characters, and performing operations such as unifying full-width and half-width characters, uppercase and lowercase, and traditional and simplified characters.

[0023] To address the issue of mixed Chinese, English, and Pinyin formats, a Chinese-English-Pinyin conversion tool is used to transcribe undisambiguated entity names from Chinese to Pinyin. The Chinese forms of affiliated institutions, research directions, and geographical locations are converted to English. Furthermore, the undisambiguated entity names in both English and Pinyin formats undergo standardization by removing symbols and text unrelated to the name, resulting in a standardized set of entities.

[0024]

[0025] in, For a standardized set of entities, It is a standardized entity.

[0026] In one embodiment, the standardized entity set The similarity between any two standardized entities is calculated, and the standardized entities are classified according to the similarity, i.e., a preset similarity threshold, to obtain a set of suspected ambiguities. And constitute the first unambiguous set include:

[0027] For standardized entity sets Any two standardized entities in , Calculate their overall similarity:

[0028]

[0029] in: The comprehensive similarity between any two standardized entities; For character-level similarity, the value obtained after editing the distance normalization is used; For statistical feature similarity, text similarity is calculated based on BM25; For semantic vector similarity, it is the cosine similarity between embedded vectors generated by a cross-language semantic model; For pronunciation normalization similarity, it is based on the distance of pinyin or phonetic code matching; The consistency score for structured information is calculated based on factors such as institution, research direction, and geographical location. For the weighting coefficients, satisfying ;

[0030] The overall similarity score for all values ​​greater than or equal to the similarity threshold. The corresponding standardized entities constitute a set of suspected ambiguities. ;

[0031] The overall similarity score for all values ​​less than the similarity threshold The corresponding standardized entities constitute the first unambiguous set. .

[0032] In one embodiment,

[0033] The prior constraints are expressed as follows:

[0034]

[0035] in, These are common characteristics of entities, and can be in text or table format; This represents the prediction function of a large model. Real labels labeled manually or semi-automatically; Let N be two standardized entities in the suspected ambiguity set; N is the suspected ambiguity set. The total number of standardized entities in the system;

[0036] The suggested engineering model is shown below:

[0037]

[0038] in, The optimal large model; To verify the set; ( ) for the verification set On the performance evaluation indicators; This represents the optimal filtering operation; m is the prompt engineering model, and M is the set of prompt engineering models consisting of multiple prompt engineering models.

[0039] In one embodiment, the optimal large model For the aforementioned set of suspected ambiguities The final disambiguation process yields an unambiguous set, which includes disambiguated entity names such as:

[0040]

[0041] in: To address suspected ambiguous sets The second unambiguous set after disambiguation ; 𝜉 is a set of suspected ambiguities; among which, the first unambiguous set Second unambiguous set Constituting a standardized entity set The corresponding unambiguous set, which contains disambiguated entity names.

[0042] In one embodiment,

[0043] The character-level similarity The expression is as follows:

[0044]

[0045] in, Character-level similarity; The minimum edit distance between two normalized entities, that is... Convert to The required number of insert, delete, and replace operations; and These represent the string lengths of the two standardized entities, respectively. (·) represents the maximum value operation;

[0046] The statistical feature similarity The expression is as follows:

[0047]

[0048] in, , Indicates a one-way correlation score; Indicates using words to match ;entity The set of unique terms extracted from all text fields, where t is a single word or token extracted from the entity. This indicates that term t exists within an entity, such as the cumulative result of matching entity name and entity attribute.

[0049] The semantic vector cosine similarity The expression is as follows:

[0050]

[0051] in, , ; The cosine similarity of the semantic vectors of two normalized entities is represented, with a value range of [−1, 1]. Represents a cross-language semantic encoding model; This represents the semantic embedding vector generated by the model;

[0052] The pronunciation normalization similarity The expression is as follows:

[0053]

[0054] in, , , respectively represent the sequence obtained by mapping standardized entities to phonetic codes; code(⋅) represents the process of mapping a name to a phonetic code sequence;

[0055] The structured information consistency score expression is as follows:

[0056]

[0057] in, Entity attributes Importance weights, satisfying ; Entity attributes The heterogeneity score function takes values ​​in the range [0,1]. Represents entity attributes; Represent entity attributes Consistency function, measuring and The similarity score for this attribute ranges from [0,1].

[0058] An entity disambiguation device based on rule-based filtering and large model assistance, the device comprising:

[0059] The extraction module is used to extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographical entries as raw entities; multiple raw entities constitute a raw entity set.

[0060] The standardization module is used to standardize the original entity set to obtain a standardized entity set.

[0061] The classification module is used to calculate the similarity between any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set.

[0062] The extraction module is used to extract common features of standardized entities in the suspected ambiguity set and summarize them into prior constraints for entity disambiguation.

[0063] The generation module is used to encode the prior constraints into the prompt templates respectively, and adjust the generation control parameters in the reasoning stage to form multiple prompt engineering models;

[0064] The selection module is used to select the optimal large model from multiple suggested engineering models based on a validation set, which is a subset of the suspected ambiguity set;

[0065] The disambiguation module is used to perform final disambiguation on the suspected ambiguity set using the optimal large model to obtain an unambiguous set, which contains the disambiguated entity names.

[0066] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0067] Extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographic entries as original entities; multiple original entities constitute an original entity set;

[0068] The original entity set is standardized to obtain a standardized entity set;

[0069] The similarity is calculated for any two standardized entities in the standardized entity set, and the standardized entities are classified according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set.

[0070] Common features of standardized entities in suspected ambiguity sets are extracted and summarized to form prior constraints for entity disambiguation;

[0071] The prior constraints are encoded into the prompt templates respectively, and the generation control parameters in the reasoning stage are adjusted to form multiple prompt engineering models;

[0072] The optimal large model is selected from multiple prompting engineering models based on the validation set, which is a subset of the suspected ambiguity set;

[0073] The suspected ambiguity set is finally disambiguated by the optimal large model to obtain the unambiguous set, which contains the disambiguated entity names.

[0074] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0075] Extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographic entries as original entities; multiple original entities constitute an original entity set;

[0076] The original entity set is standardized to obtain a standardized entity set;

[0077] The similarity is calculated for any two standardized entities in the standardized entity set, and the standardized entities are classified according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set.

[0078] Common features of standardized entities in suspected ambiguity sets are extracted and summarized to form prior constraints for entity disambiguation;

[0079] The prior constraints are encoded into the prompt templates respectively, and the generation control parameters in the reasoning stage are adjusted to form multiple prompt engineering models;

[0080] The optimal large model is selected from multiple prompting engineering models based on the validation set, which is a subset of the suspected ambiguity set;

[0081] The suspected ambiguity set is finally disambiguated by the optimal large model to obtain the unambiguous set, which contains the disambiguated entity names.

[0082] This invention addresses the limitation of large model throughput by introducing a rule-based filtering mechanism. A lightweight algorithm is used to initially filter candidate entities, and then a large model is used to refine the rule-based filtering results, thereby improving the overall disambiguation efficiency and accuracy. This solves the problem of insufficient entity recognition efficiency and accuracy in multilingual mixed scenarios. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] in:

[0085] Figure 1 This is an application environment diagram of an entity disambiguation method based on rule filtering and large model assistance in one embodiment;

[0086] Figure 2 This is a flowchart of an entity disambiguation method based on rule filtering and large model assistance in one embodiment;

[0087] Figure 3 This is a structural block diagram of an entity disambiguation device based on rule filtering and large model assistance in one embodiment;

[0088] Figure 4 This is a structural block diagram of a computer device in one embodiment. Detailed Implementation

[0089] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] To address the technical problems in the background art, this application provides an entity disambiguation method based on rule filtering and large model assistance.

[0091] Figure 1 This is a diagram illustrating the application environment of a rule-based filtering and large model-assisted entity disambiguation method in one embodiment. (Refer to...) Figure 1This rule-based filtering and large model-assisted entity disambiguation method is applied to a rule-based filtering and large model-assisted entity disambiguation system. The system includes a terminal 110 and a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal; a mobile terminal can be at least one of a mobile phone, tablet, or laptop. The server 120 can be a standalone server or a server cluster consisting of multiple servers. Terminal 110 is used to extract entity attributes with undisambiguated entity names or related to undisambiguated entity names from text or bibliographical items as original entities; multiple original entities constitute an original entity set; server 120 is used to standardize the original entity set to obtain a standardized entity set; perform similarity calculation on any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguity set and constitute a first unambiguous set; extract common entity features from the standardized entities in the suspected ambiguity set and summarize them to form prior constraints for entity disambiguation; encode the prior constraints into prompt templates respectively, and adjust the generation control parameters of the inference stage to form multiple prompt engineering models; select the optimal large model from multiple prompt engineering models based on the verification set, the verification set being a subset of the suspected ambiguity set; perform final disambiguation on the suspected ambiguity set through the optimal large model to obtain an unambiguous set, the unambiguous set containing disambiguated entity names.

[0092] like Figure 2 As shown, in one embodiment, a rule-based filtering and large model-assisted entity disambiguation method is provided. This method can be applied to both terminals and servers; this embodiment illustrates its application to a terminal. The rule-based filtering and large model-assisted entity disambiguation method specifically includes the following steps:

[0093] This invention is applied to personnel profiling and evaluation scenarios in scientific research project systems and academic headhunting firms. This embodiment uses entity recognition and talent profiling in a scientific research project system as an application background to illustrate the implementation process of the rule-based filtering and large model-assisted entity disambiguation method of this invention in the construction of academic talent databases and cross-language information integration.

[0094] S10: Extract entity attributes with undisambiguated entity names or related to undisambiguated entity names from the text or bibliographic items as the original entities; multiple original entities constitute the original entity set;

[0095] S20: Standardize the original entity set to obtain a standardized entity set;

[0096] S30: Calculate the similarity between any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set;

[0097] S40: Extract common features of standardized entities in the suspected ambiguity set and summarize them to form prior constraints for entity disambiguation;

[0098] S50: Encode the prior constraints into the prompt templates respectively, and adjust the generation control parameters in the reasoning stage to form multiple prompt engineering models; wherein, the generation control parameters include, but are not limited to, controlling the generation randomness and creativity (temperature), sampling probability (top-p, top-k), and penalty coefficient (frequency penalty, presence penalty).

[0099] S60: Select the optimal large model from multiple hint engineering models based on the validation set, wherein the validation set is a subset of the suspected ambiguity set;

[0100] S70: The suspected ambiguity set is finally disambiguated using the optimal large model to obtain an unambiguous set, which contains the disambiguated entity names.

[0101] In one embodiment, the extraction of entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographical entries is used as the original entity; multiple original entities constitute an original entity set. include:

[0102] The process involves acquiring text information and extracting scholar information from it. This text information includes: data from the research project system's research paper database, project application resumes, patent bibliographic information, research awards, etc. (For academic headhunting firms, it also includes data from university and research institute website announcements, etc.). Scholar information includes: undisambiguated entity names and entity attributes related to those names. Undisambiguated entity names include: scholar's name; entity attributes include: original text data containing information such as affiliated institution, research direction, and geographical location.

[0103] Undisambiguated entity names and entity attributes form a primitive entity; multiple primitive entities constitute a primitive entity set. ,in, The original entity represents the undisambiguated entity name and entity attributes, including the affiliated institution, research direction, geographical location, etc. The original entity is represented as follows:

[0104]

[0105] in, Represents the undisambiguated entity name (scholar's name) of the i-th scholar. Indicates the institution to which the scholar belongs. Indicates the scholar's research direction. Indicates the scholar's geographical location.

[0106] In one embodiment, the original entity set Standardization yields a standardized entity set. ;

[0107] Based on the original entity set Undisambiguated entity names and entity attributes are formatted in a unified manner, including removing spaces, symbols and control characters, and performing operations such as unifying full-width and half-width characters, uppercase and lowercase, and traditional and simplified characters.

[0108] To address the issue of mixed Chinese, English, and Pinyin formats, Chinese-English-Pinyin conversion tools (such as pypinyin, OpenCC, and Chinese-English dictionary lists) are used to transcribe undisambiguated entity names in Chinese form into undisambiguated entity names in Pinyin form. The Chinese forms of affiliated institutions, research directions, and geographical locations are all converted to English. Furthermore, the undisambiguated entity names in English or Pinyin form undergo standardization processing, removing symbols and text unrelated to the name, thus obtaining a standardized set of entities.

[0109]

[0110] in, For a standardized set of entities, It is a standardized entity.

[0111] In one embodiment, the standardized entity set The similarity between any two standardized entities is calculated, and the standardized entities are classified according to the similarity, i.e., a preset similarity threshold, to obtain a set of suspected ambiguities. And constitute the first unambiguous set include:

[0112] For standardized entity sets Any two standardized entities in , Calculate their overall similarity:

[0113]

[0114] in: The comprehensive similarity between any two standardized entities; For character-level similarity, the value obtained after editing the distance normalization is used; For statistical feature similarity, text similarity is calculated based on BM25; For semantic vector similarity, it is the cosine similarity between embedded vectors generated by a cross-language semantic model; For pronunciation normalization similarity, it is based on the distance of pinyin or phonetic code matching; The consistency score for structured information is calculated based on factors such as institution, research direction, and geographical location. For the weighting coefficients, satisfying .

[0115] The overall similarity score for all values ​​greater than or equal to the similarity threshold. The corresponding standardized entities constitute a set of suspected ambiguities. ;

[0116] The overall similarity score for all values ​​less than the similarity threshold The corresponding standardized entities constitute the first unambiguous set. .

[0117] In another embodiment, the standardized entity set can also be configured in the following manner. Any two standardized entities in , Calculate their overall similarity:

[0118]

[0119] in, The comprehensive similarity between any two standardized entities; For character-level similarity, the value obtained after editing the distance normalization is used; For semantic vector similarity, it is the cosine similarity between embedded vectors generated by a cross-language semantic model; and These are the new weighting coefficients, satisfying... .

[0120] In one embodiment,

[0121] The prior constraints are expressed as follows:

[0122]

[0123] in, These are common characteristics of entities, and can be in text or table format; This represents the prediction function of a large model. For real labels labeled manually or semi-automatically, the patterns learned by the model in this scenario include different order of signatures in different languages, such as "Zhang San" and "San Zhang", abbreviation restoration, such as "Zhang S.", and cases where the institution is in the same location but the name is different. Let N be two standardized entities in the suspected ambiguity set; N is the suspected ambiguity set. The total number of standardized entities in the system;

[0124] The template example in the prompt project is as follows:

[0125] Given a set of standardized entity records with multiple attributes related to people, different records may correspond to the same natural person entity due to language differences, writing conventions, abbreviations, or format noise, or they may correspond to different natural person entities. It is necessary to analyze the standardized entity record set based on the multi-attribute representation characteristics of the entities and determine a unique and stable disambiguation entity representation for each record.

[0126] To accomplish the above tasks, the large model should make judgments based on prior constraints derived from common features of similar entities during the inference process. These prior constraints are embodied in the following discriminative dimensions, including but not limited to:

[0127] (1) Identity identification dimension: Name and its different language forms, transliterations, abbreviations or variant expressions;

[0128] (2) Structural consistency dimension: name order, abbreviation structure, use of hyphens and their combinations;

[0129] (3) Semantic association dimension: Information on the organization, unit, affiliation, role or position related to the person entity;

[0130] (4) Language and format dimension: differences in language environment and format noise such as punctuation, spaces, and separators;

[0131] (5) Multi-attribute combination consistency dimension: a stable correspondence that occurs simultaneously on multiple attribute dimensions.

[0132] Based on the above-mentioned discrimination dimensions, for each standardized entity record in the entity record set, the corresponding disambiguation entity representation is output.

[0133] The output format requirements are as follows:

[0134] Standardized entity → Disambiguation entity

[0135] Each line contains only one set of entity mappings, and no explanatory text or additional descriptions are output except for the mappings.

[0136] The suggested engineering model is shown below:

[0137]

[0138] in, The optimal large model; To verify the set; ( ) for the verification set Performance evaluation metrics (such as F1, Accuracy, AUC, etc.) This represents the optimal filtering operation; m is the prompt engineering model, and M is the set of prompt engineering models consisting of multiple prompt engineering models;

[0139] In one embodiment, the optimal large model For the aforementioned set of suspected ambiguities The final disambiguation process yields an unambiguous set, which includes disambiguated entity names such as:

[0140]

[0141] in: To address suspected ambiguous sets The second unambiguous set after disambiguation ; 𝜉 is a set of suspected ambiguities; among which, the first unambiguous set Second unambiguous set Constituting a standardized entity set The corresponding unambiguous set, which contains disambiguated entity names.

[0142] In one embodiment,

[0143] The character-level similarity The expression is as follows:

[0144]

[0145] in, Character-level similarity; The minimum edit distance between two normalized entities, that is... Convert to The required number of insert, delete, and replace operations; and These represent the string lengths of the two standardized entities, respectively. (·) represents the maximum value operation;

[0146] The statistical feature similarity The expression is as follows:

[0147]

[0148] in, , Indicates a one-way correlation score; Indicates using words to match ;entity The set of unique terms extracted from all text fields, where t is a single word or token extracted from the entity. This indicates that term t exists within an entity, such as the cumulative result of matching entity name and entity attribute.

[0149] The semantic vector cosine similarity The expression is as follows:

[0150]

[0151] in, , ; The cosine similarity of the semantic vectors of two normalized entities is represented, with a value range of [−1, 1]. This refers to cross-language semantic encoding models, such as LaBSE, mUSE, XLM-R, BERT, etc. The semantic embedding vectors generated by the model are typically 128-dimensional, 512-dimensional, or 1024-dimensional. express The L2 norm;

[0152] The pronunciation normalization similarity The expression is as follows:

[0153]

[0154] in, , , respectively represent the sequence obtained by mapping standardized entities through phonetic codes, such as the first letter of Pinyin, the initial consonant string, the Soundex, etc.; code(⋅) represents the process of mapping a name to a phonetic code sequence, for example: Chinese characters → Pinyin first letter string (such as "Beijing University" → "bjdx"), Chinese characters → initial consonant-final vowel encoding, such as "bank" → "yh", English words → Soundex (such as "Robert" → "R163"); This represents the Levenshtein edit distance between phonetic code sequences;

[0155] The structured information consistency score expression is as follows:

[0156]

[0157] in, Entity attributes Importance weights, satisfying ; Entity attributes The heterogeneity score function takes values ​​in the range [0,1]. It represents entity attributes, such as the field, organization, and location; it can be expanded according to the scenario.

[0158] The consistency function representing entity attribute 𝑥 measures and The similarity score for this attribute ranges from [0,1]. The format can be customized based on the attribute type, such as: category matching: 1 for perfect match, 0 otherwise; geographical distance: calculated using the Haversine formula and then normalized; set-type attributes: calculated using Jaccard similarity. The importance weight of the entity attribute 𝑥 reflects its relative contribution to the overall structured consistency calculation; satisfying... ,and .

[0159] In academic headhunting systems, the method of this invention is used to construct globally unique identifiers for candidates, uniformly mapping records from different sources and languages ​​(such as papers, projects, and patents) to the same talent entity. This method achieves cross-semantic consistency processing for fields such as name, pinyin, translation, and institution, providing a stable and scalable foundation for academic talent profiling and intelligent recommendation.

[0160] This invention also provides an entity disambiguation device based on rule-based filtering and large model assistance, such as... Figure 3 As shown, the device includes:

[0161] Extraction module 10 is used to extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographical items as raw entities; multiple raw entities constitute a raw entity set.

[0162] Standardization module 20 is used to standardize the original entity set to obtain a standardized entity set;

[0163] The classification module 30 is used to calculate the similarity between any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set.

[0164] Extraction module 40 is used to extract common features of standardized entities in the suspected ambiguity set and summarize them to form prior constraints for entity disambiguation.

[0165] The generation module 50 is used to encode the prior constraints into the prompt templates respectively, and adjust the generation control parameters in the reasoning stage to form multiple prompt engineering models;

[0166] Selection module 60 is used to select the optimal large model from multiple prompt engineering models based on a validation set, wherein the validation set is a subset of the suspected ambiguity set;

[0167] The disambiguation module 70 is used to perform final disambiguation on the suspected ambiguity set through the optimal large model to obtain an unambiguous set, wherein the unambiguous set contains the disambiguated entity names.

[0168] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0169] S10: Extract entity attributes with undisambiguated entity names or related to undisambiguated entity names from the text or bibliographic items as the original entities; multiple original entities constitute the original entity set;

[0170] S20: Standardize the original entity set to obtain a standardized entity set;

[0171] S30: Calculate the similarity between any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set;

[0172] S40: Extract common features of standardized entities in the suspected ambiguity set and summarize them to form prior constraints for entity disambiguation;

[0173] S50: Encode the prior constraints into the prompt templates respectively, and adjust the generation control parameters in the reasoning stage to form multiple prompt engineering models;

[0174] S60: Select the optimal large model from multiple hint engineering models based on the validation set, wherein the validation set is a subset of the suspected ambiguity set;

[0175] S70: The suspected ambiguity set is finally disambiguated using the optimal large model to obtain an unambiguous set, which contains the disambiguated entity names.

[0176] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the following steps:

[0177] S10: Extract entity attributes with undisambiguated entity names or related to undisambiguated entity names from the text or bibliographic items as the original entities; multiple original entities constitute the original entity set;

[0178] S20: Standardize the original entity set to obtain a standardized entity set;

[0179] S30: Calculate the similarity between any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set;

[0180] S40: Extract common features of standardized entities in the suspected ambiguity set and summarize them to form prior constraints for entity disambiguation;

[0181] S50: Encode the prior constraints into the prompt templates respectively, and adjust the generation control parameters in the reasoning stage to form multiple prompt engineering models;

[0182] S60: Select the optimal large model from multiple hint engineering models based on the validation set, wherein the validation set is a subset of the suspected ambiguity set;

[0183] S70: The suspected ambiguity set is finally disambiguated using the optimal large model to obtain an unambiguous set, which contains the disambiguated entity names.

[0184] Figure 4 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 4 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a rule-based filtering and large model-assisted entity disambiguation method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to implement a rule-based filtering and large model-assisted entity disambiguation method. Those skilled in the art will understand that... Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0185] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0186] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0187] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for entity disambiguation based on rule-based filtering and large model assistance, characterized in that, The method includes: Extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographic entries as original entities; multiple original entities constitute an original entity set; The original entity set is standardized to obtain a standardized entity set; The similarity is calculated for any two standardized entities in the standardized entity set, and the standardized entities are classified according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set. Common features of standardized entities in suspected ambiguity sets are extracted and summarized to form prior constraints for entity disambiguation; The prior constraints are encoded into the prompt templates respectively, and the generation control parameters in the reasoning stage are adjusted to form multiple prompt engineering models; The optimal large model is selected from multiple prompting engineering models based on the validation set, which is a subset of the suspected ambiguity set; The suspected ambiguity set is finally disambiguated by the optimal large model to obtain the unambiguous set, which contains the disambiguated entity names.

2. The entity disambiguation method based on rule-based filtering and large model assistance according to claim 1, characterized in that, The process involves extracting entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographical entries as raw entities; multiple raw entities constitute a set of raw entities. include: Obtain text information, and extract scholar information from the text information; the text information includes: the scientific research paper database of the scientific research project system, project application resumes, patent bibliographic information, scientific research awards and other data; the scholar information includes: undisambiguated entity names and entity attributes related to undisambiguated entity names; the undisambiguated entity names include: scholar names; the entity attributes include: original text data of information such as affiliated institutions, research directions, and geographical locations; Undisambiguated entity names and entity attributes form a primitive entity; multiple primitive entities constitute a primitive entity set. ,in, The original entity represents the undisambiguated entity name and entity attributes, including the affiliated institution, research direction, geographical location, etc. The original entity is represented as follows: in, This represents the undisambiguated entity name of the i-th scholar. Indicates the institution to which the scholar belongs. Indicates the scholar's research direction. Indicates the scholar's geographical location.

3. The entity disambiguation method based on rule-based filtering and large model assistance according to claim 1, characterized in that, The original entity set Standardization yields a standardized entity set. ; Based on the original entity set Undisambiguated entity names and entity attributes are formatted in a unified manner, including removing spaces, symbols and control characters, and performing operations such as unifying full-width and half-width characters, uppercase and lowercase, and traditional and simplified characters. To address the issue of mixed Chinese, English, and Pinyin formats, a Chinese-English-Pinyin conversion tool is used to transcribe undisambiguated entity names from Chinese to Pinyin. The Chinese forms of affiliated institutions, research directions, and geographical locations are converted to English. Furthermore, the undisambiguated entity names in both English and Pinyin formats undergo standardization by removing symbols and text unrelated to the name, resulting in a standardized set of entities. in, For a standardized set of entities, It is a standardized entity.

4. The entity disambiguation method based on rule-based filtering and large model assistance according to claim 3, characterized in that, The standardized entity set The similarity between any two standardized entities is calculated, and the standardized entities are classified according to the similarity, i.e., a preset similarity threshold, to obtain a set of suspected ambiguities. And constitute the first unambiguous set include: For standardized entity sets Any two standardized entities in , Calculate their overall similarity: in: The comprehensive similarity between any two standardized entities; For character-level similarity, the value obtained after editing the distance normalization is used; For statistical feature similarity, text similarity is calculated based on BM25; For semantic vector similarity, it is the cosine similarity between embedded vectors generated by a cross-language semantic model; For pronunciation normalization similarity, it is based on the distance of pinyin or phonetic code matching; The consistency score for structured information is calculated based on factors such as institution, research direction, and geographical location. For the weighting coefficients, satisfying ; The overall similarity score for all values ​​greater than or equal to the similarity threshold. The corresponding standardized entities constitute a set of suspected ambiguities. ; The overall similarity score for all values ​​less than the similarity threshold The corresponding standardized entities constitute the first unambiguous set. .

5. The entity disambiguation method based on rule-based filtering and large model assistance according to claim 1, characterized in that, The prior constraints are expressed as follows: in, These are common characteristics of entities, and can be in text or table format; This represents the prediction function of a large model. Real labels labeled manually or semi-automatically; Let N be two standardized entities in the suspected ambiguity set; N is the suspected ambiguity set. The total number of standardized entities in the system; The suggested engineering model is shown below: in, The optimal large model; To verify the set; ( ) for the verification set On the performance evaluation indicators; This represents the optimal filtering operation; m is the prompt engineering model, and M is the set of prompt engineering models consisting of multiple prompt engineering models.

6. The entity disambiguation method based on rule-based filtering and large model assistance according to claim 5, characterized in that, The optimal large model For the aforementioned set of suspected ambiguities The final disambiguation process yields an unambiguous set, which includes disambiguated entity names such as: in: To address suspected ambiguous sets The second unambiguous set after disambiguation ; 𝜉 is a set of suspected ambiguities; among which, the first unambiguous set Second unambiguous set Constituting a standardized entity set The corresponding unambiguous set, which contains disambiguated entity names.

7. The entity disambiguation method based on rule-based filtering and large model assistance according to claim 4, characterized in that, The character-level similarity The expression is as follows: in, Character-level similarity; The minimum edit distance between two normalized entities, that is... Convert to The required number of insert, delete, and replace operations; and These represent the string lengths of the two standardized entities, respectively. (·) represents the maximum value operation; The statistical feature similarity The expression is as follows: in, , Indicates a one-way correlation score; Indicates using words to match ;entity The set of unique terms extracted from all text fields, where t is a single word or token extracted from the entity. This indicates that term t exists within an entity, such as the cumulative result of matching entity name and entity attribute. The semantic vector cosine similarity The expression is as follows: in, , ; The cosine similarity of the semantic vectors of two normalized entities is represented, with a value range of [−1, 1]. Represents a cross-language semantic encoding model; This represents the semantic embedding vector generated by the model; The pronunciation normalization similarity The expression is as follows: in, , , respectively represent the sequence obtained by mapping standardized entities to phonetic codes; code(⋅) represents the process of mapping a name to a phonetic code sequence; The structured information consistency score expression is as follows: in, For entity attributes Importance weights, satisfying ; For entity attributes The heterogeneity score function takes values ​​in the range [0,1]. Represents entity attributes; Represent entity attributes Consistency function, measuring and The similarity score for this attribute ranges from [0,1].

8. An entity disambiguation device based on rule-based filtering and large model assistance, characterized in that, The device includes: The extraction module is used to extract entity attributes containing undisambiguated entity names or related to undisambiguated entity names from text or bibliographical entries as raw entities; multiple raw entities constitute a raw entity set. The standardization module is used to standardize the original entity set to obtain a standardized entity set. The classification module is used to calculate the similarity between any two standardized entities in the standardized entity set, and classify the standardized entities according to the similarity and a preset similarity threshold to obtain a suspected ambiguous set and constitute a first unambiguous set. The extraction module is used to extract common features of standardized entities in the suspected ambiguity set and summarize them into prior constraints for entity disambiguation. The generation module is used to encode the prior constraints into the prompt templates respectively, and adjust the generation control parameters in the reasoning stage to form multiple prompt engineering models; The selection module is used to select the optimal large model from multiple suggested engineering models based on a validation set, which is a subset of the suspected ambiguity set; The disambiguation module is used to perform final disambiguation on the suspected ambiguity set using the optimal large model to obtain an unambiguous set, which contains the disambiguated entity names.

9. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 7.

Images