General entity extraction method, medium and equipment

Abstract

The invention relates to the technical field of entity extraction, in particular to a general entity extraction method, a medium and equipment, and aims to convert isolated entity names into structured refined names of attributes and names and provide richer semantic information for entities by adding corresponding semantic attributes to names of initial entities. The context corresponding to the initial entity which does not pass semantic uniqueness verification is expanded and iteratively updated to supplement more semantic details, so that the newly generated refined entity name has higher semantic distinction degree, the semantic conflict problem caused by insufficient information is solved, and the semantic uniqueness of the entity is improved. It is ensured that the finally output target entity name has strict semantic uniqueness, and inherent defects of an initial entity are thoroughly overcome; and the corresponding target entity extraction model is constructed and trained for each entity name set, so that the model only focuses on a certain type of entities with specific semantics, and the extraction effect for the entities with specific semantic attributes is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of entity extraction technology, and in particular to a general entity extraction method, medium and device. Background Technology

[0002] In the field of natural language processing, entity extraction, as a core task of information extraction, aims to identify and classify named entities, such as person names, place names, and organization names, from unstructured text. Traditional entity extraction methods, including rule-based, statistical machine learning, and deep learning approaches, typically rely on a predefined, flat set of entity types. However, existing technologies have an inherent drawback: the homogenization and semantic ambiguity of entity names. Specifically, existing methods classify entities solely based on their literal names, failing to effectively distinguish the deep semantic differences of the same name in different contexts. For example, in cross-domain business texts, the general type "name" may correspond to different types of people in different reports. Existing entity extraction models uniformly categorize all "names" with different semantic roles as "PER" (person), resulting in extraction results that, while literally correct, lose crucial business semantic information, failing to meet the precise requirements of downstream business systems such as case analysis and risk management for fine-grained semantic understanding of entities.

[0003] Therefore, how to solve the problems of entity name notification and semantic ambiguity in entity extraction tasks has become an urgent issue to be addressed. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention provides a general entity extraction method, which includes the following steps: S1. Based on the context scope corresponding to each initial entity extracted from the original text, extract the context text of each initial entity in the original text. There are issues of homogenization of entity names or semantic ambiguity among the initial entities.

[0005] S2, based on the context text corresponding to each initial entity, add corresponding semantic attributes to the name of each initial entity to obtain the refined entity name corresponding to each initial entity.

[0006] S3. Perform semantic uniqueness verification on all refined entity names, expand the context scope corresponding to the initial entity that fails the verification, and update the corresponding refined entity name according to the expanded context text until all the refined entity names corresponding to the initial entities pass the verification, and obtain the semantically unique target entity name corresponding to each initial entity. There is no problem of entity name homogenization or semantic ambiguity among the target entity names.

[0007] S4. Group the target entity names according to the semantic attributes corresponding to all the target entity names to obtain several sets of entity names.

[0008] S5: Build and train the corresponding initial entity extraction model for each entity name set, and obtain the target entity extraction model for each entity name set. The target entity extraction model is used to extract entities with specific semantic attributes from the text.

[0009] The present invention also provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the general entity extraction method described above.

[0010] The present invention also provides an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0011] This invention has at least the following beneficial effects: By extracting semantic attributes from the context text and adding corresponding semantic attributes to the name of each initial entity, isolated entity names are transformed into structured, refined names of attributes and names, providing richer semantic information for entities and thus providing a semantic foundation for solving the problems of homogenization and semantic ambiguity in entity names; by expanding and iteratively updating the context corresponding to initial entities that fail the semantic uniqueness check, more semantic details are added to the initial entities that fail the check, making the newly generated refined entity names have stronger semantic distinguishability, solving the semantic conflict problem caused by insufficient information, ensuring that the final output target entity name has strict semantic uniqueness, completely solving the inherent defects of the initial entities, and providing a guarantee for the accuracy of entity extraction results; by constructing and training a corresponding initial entity extraction model for each entity name set, a target entity extraction model corresponding to each entity name set is obtained, so that the model focuses only on entities of a certain type of specific semantics, greatly improving the extraction effect for entities with specific semantic attributes. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a general entity extraction method provided in Embodiment 1 of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is understood that, where appropriate, the terms used to distinguish similar objects can be interchanged so that the invention can also be implemented in other embodiments besides the illustrated or described embodiments. Furthermore, the terms "including," "having," and any variations are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0016] Example 1 This first embodiment provides a general entity extraction method, such as Figure 1 As shown, this general entity extraction method includes the following steps: S1. Based on the context scope corresponding to each initial entity extracted from the original text, extract the context text of each initial entity in the original text. There are issues of homogenization of entity names or semantic ambiguity among the initial entities.

[0017] The original text refers to the source text data from which entities are to be extracted, such as police case records, financial account opening information, and communication records. Its format can be plain text, structured documents, etc.

[0018] Initial entities are entities initially extracted from the original text using basic, general entity recognition models or methods, such as "name" and "location." Two core problems exist among these initial entities: first, homogenization of entity names, meaning that initial entities with different semantic meanings share the same or similar literal expressions; and second, semantic ambiguity, meaning that the literal name of the same initial entity can correspond to different semantic interpretations in different contexts of the original text. Therefore, it is impossible to effectively distinguish the deep semantic differences of the same initial entity's name in different contextual scenarios, and directly relying on initial entities cannot meet the precise requirements of downstream business systems such as case analysis and risk management for fine-grained semantic understanding of entities.

[0019] The context range is the specific text boundary in the original text that contains the initial entity semantic information. It is usually quantified by the character start index and character end index (e.g., [50, 200] represents a segment from the 50th to the 200th character of the original text). Its determination needs to consider factors such as entity location and text structure features to avoid introducing noise due to an excessively large range or missing key semantics due to an excessively small range. Contextual text is the specific text content extracted from the original text that falls within the context range.

[0020] The above approach, with the initial entity as the core and based on the context of the initial entity, extracts text fragments containing semantic association information of the entity from the original text, providing key semantic analysis basis for solving the problems of homogenization of entity names and semantic ambiguity among initial entities.

[0021] In one specific embodiment, S1 includes the following steps: S11. Based on the position coordinates of each initial entity in the original text and the structural features of the original text, determine the text extraction rules corresponding to each initial entity. The structural features include paragraph boundary markers and punctuation mark separators.

[0022] S12, based on the text truncation rules, obtain the context scope corresponding to each initial entity.

[0023] S13, extract the context text corresponding to each initial entity from the original text according to the context scope.

[0024] Specifically, using the initial entity's position coordinates as a reference point and combining them with the structural features of the original text, text truncation rules are formulated to adapt to the entity's position. This ensures that the subsequent truncation context covers the entity's semantic information without introducing irrelevant text noise. Specifically, the initial entity's position coordinates are used to accurately locate the initial entity's specific position in the original text, typically represented as "character start index - character end index". The structural features of the original text are objective identifiers naturally present in the original text, used to separate semantic units, including paragraph boundary markers and punctuation separators. These structural features can be identified and marked using text parsing tools (such as Python's spaCy and python-docx).

[0025] The text truncation rules are specific standards based on "entity location + structural features" used to determine the context range later. According to the text truncation rules, the context range corresponding to each initial entity is obtained to clarify which characters to truncate, avoiding over-truncation, under-truncation, or incorrect truncation. Based on the quantified context range, the corresponding context text is accurately extracted from the original text, providing semantic material for subsequently resolving issues of homogenized initial entity names and semantic ambiguity. Specifically, based on the character index of the context range, the operation of extracting corresponding content from the original text is performed (e.g., if the original text is the string raw_text, and the range is [20, 150], then raw_text[20:151] is truncated; because string slicing is left-closed and right-open, +1 is needed to ensure that the terminating index character is included).

[0026] As described above, by associating location coordinates with structural features, the text extraction rules are freed from the limitations of fixed windows, ensuring that the rules adapt to the actual position of entities in the text. At the same time, the objectivity of the rules can avoid subjective errors from different implementers, providing a guarantee for the consistency of subsequent context scope. By accurately extracting context text, the semantic information of entities is fully covered, and the format is neat, providing a reliable data foundation for distinguishing homogeneous entities and eliminating semantic ambiguity.

[0027] In one specific implementation, the context scope corresponding to each initial entity is obtained according to the text truncation rules, including at least one of the following methods: The first method of obtaining the context is as follows: For any initial entity, if the current initial entity is located inside a statement, then the statement containing the current initial entity, the M statements preceding it, and the M statements following it are taken as the context range corresponding to the current initial entity, where M is an integer greater than 0.

[0028] The second method of obtaining the context is as follows: For any initial entity, if the current initial entity is located at the beginning or end of a paragraph, then the paragraph containing the current initial entity is taken as the context scope corresponding to the current initial entity.

[0029] The third acquisition method: For any initial entity, if the original text contains domain identifier words related to the current initial entity, then the statement containing the domain identifier words is taken as the core sentence, and the initial sentence group range is taken as the core sentence.

[0030] Starting from the core sentence, select the first adjacent sentence in the preceding and following directions of the core sentence, and calculate the first semantic similarity between each adjacent sentence and the core sentence.

[0031] If the first semantic similarity is greater than or equal to the first preset similarity threshold, the corresponding adjacent sentences are included in the sentence group. Taking the corresponding adjacent sentences as the new starting point, the next adjacent sentences are selected in the corresponding preceding or following direction. The semantic similarity calculation and inclusion judgment are repeated until the first semantic similarity is less than the first preset similarity threshold, at which point the expansion in the corresponding preceding or following direction stops.

[0032] Integrate all statements that have been expanded in the preceding and following directions and included in the sentence group to form a sentence group containing domain identifiers. Use the sentence group containing domain identifiers as the context scope corresponding to the current initial entity.

[0033] The system determines the positional attributes of an entity by matching its coordinates with the preprocessed text structure. Specifically, if an entity's coordinates fall entirely between two punctuation marks and the sentence is neither the first nor the last sentence of a paragraph, the entity is considered to be within that sentence. Correspondingly, the system expands the initial entity's sentence by M adjacent sentences both forward and backward, forming a context that covers the entity's complete semantic relationships, balancing semantic completeness with conciseness. M is a positive integer, and its value can be dynamically adjusted based on text complexity. For example, M=1 when the sentence length is ≤50 characters, M=2 when 50 < sentence length ≤100 characters, and M=3 when the sentence length >100 characters, ensuring that the expansion covers the entity's key semantics. It should be noted that if there are insufficient sentences in the expansion direction (e.g., if the core sentence is the second sentence, expanding forward by M=2 will only capture the first sentence), the existing sentences will be used.

[0034] If the entity's coordinates fall within the start / end region corresponding to the paragraph boundary marker, the initial entity is determined to be located at the beginning or end of the paragraph. Correspondingly, the semantic information of the initial entity at the beginning or end of the paragraph is usually strongly related to the topic of the entire paragraph; simply extracting adjacent sentences may easily lose the paragraph-level topic semantics. Therefore, the entire paragraph containing the initial entity is used as the context scope to ensure coverage of the full topic semantics to which the entity belongs, avoiding topic fragmentation.

[0035] Domain identifiers are specialized terms that point to specific business domains and are stored in a pre-defined domain identifier database. For example, in the public security domain, these include terms like "involved in a case," "case handler," and "statement," while in the financial domain, they include terms like "account opening," "credit," and "clearing." These terms are used to locate the core semantic meaning of the domain to which an entity belongs. When the original text contains domain identifiers related to the initial entity, the sentences containing the domain identifiers and the related sentences forming the sentence group are the core semantic source of the entity's domain attributes and scenario attributes.

[0036] Therefore, this embodiment uses sentences containing domain identifiers as core sentences, expands them into sentence groups through semantic similarity, and uses these sentence groups as the context scope to ensure coverage of entity-related domain-specific semantics, thus solving the problem of cross-sentence domain semantic dispersion. Specifically, by matching domain identifiers in the domain identifier lexicon in the original text, domain identifiers related to the initial entity semantics are selected, and the sentences containing these identifiers are designated as core sentences. The character index range of the core sentence is included in the sentence group scope, with the initial sentence group scope being the index of the core sentence. Starting from the core sentence, the first adjacent sentence is selected (denoted as "preceding sentence 1"). Preceding sentence 1 and the core sentence are converted into semantic vectors, and their first semantic similarity is calculated. If the first semantic similarity is ≥ the first preset similarity threshold, the index of preceding sentence 1 is included in the sentence group scope, and "preceding sentence 2" is selected as the new starting point. The similarity calculation and inclusion determination between preceding sentence 2 and the core sentence are repeated until the first semantic similarity of a certain preceding sentence is < the first preset similarity threshold, at which point the preceding direction expansion stops. Correspondingly, the operation logic of the subsequent direction expansion is consistent with that of the preceding direction. Finally, the character indexes of all statements included after the pre- and post-order expansions are integrated to form a sentence group range containing domain identifier words, which serves as the context range of the initial entity. For example, the pre-order statement 1 [50,99] + core statement [100,200] + post-order statement 1 [201,250], after merging, has a range of [50,250].

[0037] Those skilled in the art will know that any semantic vector conversion method and similarity calculation method in the prior art falls within the protection scope of this invention, such as the Sentence-BERT model and the cosine similarity calculation method, which will not be elaborated here.

[0038] The specific value of the first preset similarity threshold can be set by the implementer according to the actual situation. For example, in this embodiment, the first preset similarity threshold is set to 0.6.

[0039] It should be noted that domain identifiers directly point to the core semantic attributes (domain, scenario) of an entity and have the highest priority; the thematic semantics of entities at the beginning and end of a paragraph depend on the complete paragraph and have the next highest priority; the semantics of entities within a sentence can be covered by adjacent M sentences and have the lowest priority. In other words, the third acquisition method has a higher priority than the second acquisition method, and the second acquisition method has a higher priority than the first acquisition method.

[0040] If the initial entity meets the applicable conditions of multiple acquisition methods (such as the entity being located in the first sentence of a paragraph and the text containing relevant domain identifiers), the method with higher priority will be used first to ensure that the context scope can cover the most critical semantic information.

[0041] The three acquisition methods described above target three typical entity locations and semantic scenarios: within a sentence, at the beginning and end of a paragraph, and in domain associations. They form a comprehensive and differentiated context scope determination scheme, avoiding the limitations of a single method. For example, a single sentence expansion cannot cover paragraph-level topics, and a single paragraph scope will lead to domain semantic redundancy. This ensures that the initial entities in different scenarios can obtain an appropriate and accurate context scope.

[0042] S2, based on the context text corresponding to each initial entity, add corresponding semantic attributes to the name of each initial entity to obtain the refined entity name corresponding to each initial entity.

[0043] The semantic attributes (role, domain, scenario) of the initial entity are hidden in its contextual text. Text semantic analysis is needed to transform this implicit information into explicit attribute labels. Specifically, the role attribute represents the identity, function, or role the initial entity assumes in a specific scenario, reflecting the entity's identity characteristics; the domain attribute represents the business domain to which the initial entity belongs, reflecting its domain affiliation, such as the public security domain; and the scenario attribute represents the specific business scenario in which the initial entity is located, reflecting the entity's context, such as a case investigation scenario.

[0044] By extracting semantic attributes from the context text and adding corresponding semantic attributes to the name of each initial entity, the refined entity name corresponding to each initial entity is obtained. This realizes the transformation of isolated entity names into structured refined names of attributes and names, providing richer semantic information for entities, and thus providing a semantic foundation for solving the problems of homogenization and semantic ambiguity of entity names.

[0045] In one specific embodiment, S2 includes the following steps: S21. Based on the context text corresponding to each initial entity, extract the role attributes, domain attributes, and scene attributes corresponding to each initial entity.

[0046] S22. According to the preset priority rules, non-empty semantic attribute values ​​are concatenated with predefined connectors to form attribute combination strings corresponding to each initial entity. The priority rules are: role attribute priority is higher than scene attribute priority, scene attribute priority is higher than domain attribute priority, and the absence of high priority attributes is not filled by low priority attributes.

[0047] S23, add the attribute combination string corresponding to each initial entity as a prefix / suffix to the name of the corresponding initial entity to form the refined entity name corresponding to each initial entity.

[0048] The role attribute extraction relies on a role keyword library in the context text (pre-set common role words in various fields, such as "account opener, payee, and loan officer" in the financial field). Through string matching or named entity recognition (NER) models, role keywords that are directly related to the initial entity are filtered in the context text. If there are no explicit role words in the text, they are deduced through the relationship between entities.

[0049] Domain attribute extraction relies on a domain identifier lexicon (preset domain-specific terms, such as "involved in a case, record, and investigation" in the public security field, and "account opening, credit, and clearing" in the financial field). By statistically analyzing the frequency of domain identifiers in the context text, the domain with the highest frequency is determined as the domain attribute of the initial entity. If the text contains multiple domain identifiers, the entity is filtered based on the correlation between the entity and the identifier.

[0050] Scene attribute extraction relies on a scene-keyword mapping table (presets the association between scenes and corresponding keywords, such as "case investigation scene" corresponding to "apprehending, querying, and evidence involved in the case", and "account opening scene" corresponding to "submitting materials, binding bank cards, and setting passwords"); by matching scene keywords in the context text, the corresponding scene attribute is determined according to the mapping table; if there are no explicit scene words in the text, it is deduced by combining domain attributes (such as "public security field" defaulting to matching "case investigation scene", and "financial field" defaulting to matching "account transaction scene").

[0051] Among them, the role attribute directly distinguishes the entity's identity and has the highest priority; the scene attribute supplements environmental information and has the next highest priority; the domain attribute defines the affiliation and has the lowest priority. They are concatenated in order of priority, while avoiding the logical confusion of filling the gaps with low-priority attributes when high-priority attributes are missing (e.g., if there is no role attribute, the scene attribute cannot be used to replace the role attribute), thus forming a string of attribute combinations with a unified structure.

[0052] Only extracted non-empty attributes are retained; empty attributes (such as no character attribute extracted) are skipped and do not occupy concatenation space. For example, if the attribute is "Role: Investigator, Scene: Case Investigation Scene, Domain: Public Security Domain", it is concatenated as "Investigator - Case Investigation Scene - Public Security Domain"; if the attribute is "Role: (empty), Scene: Account Opening Scene, Domain: Financial Domain", it is concatenated as "Account Opening Scene - Financial Domain" (empty character attribute is skipped).

[0053] Predefined connectors are used to separate different attributes. Unambiguous and easily recognizable symbols, such as "-" or "_", should be selected and kept consistent to avoid errors in subsequent model training or semantic verification due to different symbols.

[0054] When attribute combination strings are used as prefixes, users can quickly grasp the core semantics of an entity. When the initial entity name is short and the attribute combination string is long, a suffix can be chosen to avoid name redundancy caused by an excessively long prefix.

[0055] As described above, by transforming implicit semantics in the context text into explicit attribute labels, the problem of initial entities lacking semantic information is solved, providing core material for subsequent attribute combinations and name refinement. By prioritizing and using non-empty concatenation, the attribute combination string is ensured to highlight the core semantics (role attributes) without containing redundant empty attributes, thus avoiding attribute confusion. By binding attributes and names, richer semantic information is provided for entities, thereby providing a semantic foundation for solving the problems of homogenization and semantic ambiguity in entity names.

[0056] S3. Perform semantic uniqueness verification on all refined entity names, expand the context scope corresponding to the initial entity that fails the verification, and update the corresponding refined entity name according to the expanded context text until all the refined entity names corresponding to the initial entities pass the verification, and obtain the semantically unique target entity name corresponding to each initial entity. There is no problem of entity name homogenization or semantic ambiguity among the target entity names.

[0057] In one specific embodiment, S3 includes the following steps: S31, convert each refined entity name and its corresponding context text into a semantic vector.

[0058] S32, calculate the second semantic similarity between the semantic vectors corresponding to any two refined entity names.

[0059] S33, if the second semantic similarity is greater than or equal to the second preset similarity threshold, it is determined that the verification fails, and the corresponding two refined entity names are marked as entities to be optimized.

[0060] S34, expand the context scope of the entity to be optimized, and return to step S2 based on the expanded context text to obtain the new refined entity name corresponding to each entity to be optimized.

[0061] S35, if the second semantic similarity between all refined entity names is lower than the second preset similarity threshold, then it is determined that all the refined entity names corresponding to the initial entities have passed the verification, and the refined entity name corresponding to each initial entity when it passes the verification is determined as the corresponding target entity name; otherwise, return to step S31.

[0062] Semantic vectors are high-dimensional numerical vectors obtained by transforming text (refined entity names or contextual text) through deep learning models (such as Sentence-BERT, RoBERTa). They are typically 768-dimensional or 1024-dimensional. Each dimension of the vector represents the feature of the text in a certain semantic dimension, and the whole vector reflects the semantic connotation of the text.

[0063] The second semantic similarity is the cosine similarity between the semantic vectors of the entities corresponding to two refined entity names, used to measure the semantic closeness between them. The second preset similarity threshold is a critical value used to determine whether the semantics are unique. In this embodiment, a value of 0.7-0.8 is recommended, which can be adjusted according to the domain precision requirements.

[0064] The entities to be optimized are two entities in a pair whose second semantic similarity is greater than or equal to a second preset similarity threshold. Their refined names still exhibit semantic homogeneity or ambiguity. Semantic conflicts between the entities to be optimized often stem from insufficient contextual information (e.g., attributes extracted from limited statements are insufficient for differentiation). It is necessary to expand the contextual scope to supplement more semantic details, thereby obtaining richer semantic information and enhancing the semantic distinguishability between entities. Specifically, based on the expanded contextual scope, the contextual text is re-extracted from the original text. The new contextual text is input into step S2 to re-extract semantic attributes, generate new attribute combination strings, and generate new refined entity names. This yields the updated name of the entity to be optimized, which replaces the original name. The entity to be optimized is then de-marked and ready for the next round of verification.

[0065] After multiple rounds of verification and optimization, when the second semantic similarity of all entity pairs is lower than the second preset similarity threshold, it indicates that all entities have achieved semantic uniqueness, the process can be terminated, and the final refined entity name can be determined as the target entity name.

[0066] As described above, by expanding the context and iteratively updating, more semantic details are added to the entity to be optimized, so that the newly generated refined entity name has stronger semantic distinguishability and solves the semantic conflict problem caused by insufficient information. By iteratively verifying, the final output target entity name is ensured to have strict semantic uniqueness, which completely solves the inherent defects of the initial entity and provides a guarantee for the accuracy of the entity extraction results.

[0067] In one specific implementation, expanding the context scope of the entity to be optimized includes at least one of the following expansion methods: The first expansion method: For any initial entity, if the current initial entity is located inside a statement, then the N statements preceding and following the current initial entity in the context scope corresponding to the current initial entity are included in the new context scope corresponding to the current initial entity, where N is an integer greater than 0.

[0068] The second expansion method: For any initial entity, if the current initial entity is at the beginning or end of a paragraph, then the P paragraphs preceding and following the current initial entity in the context scope corresponding to the current initial entity are included in the new context scope corresponding to the current initial entity, where P is an integer greater than 0.

[0069] The third expansion method: For any initial entity, if the original text contains domain identifiers related to the current initial entity, then the first preset similarity threshold is subtracted from the preset value to obtain the new first preset similarity threshold.

[0070] Inclusion is determined based on the new first preset similarity threshold, and a new context scope corresponding to the current initial entity is obtained.

[0071] When the initial entity is located within a sentence (not at the beginning or end of a paragraph), the original context scope may lack key information to distinguish the entity's semantics due to insufficient sentence coverage (such as the entity's motivation or associated objects). In this case, the original context scope is expanded forward and backward by N adjacent sentences. This increases the sentence coverage to supplement semantic details and provides more sufficient contextual basis for subsequent updates to refine the entity name. N is an integer greater than 0, and a value of 1-2 is recommended (too small a value may not supplement enough information, while too large a value may introduce irrelevant sentence noise). The specific value can be adjusted according to the number of sentences in the original context scope. For example, if the original context scope contains 3 sentences, then N=1; if the original context scope contains 2 sentences, then N=2.

[0072] When the initial entity is located at the beginning or end of a paragraph, the original context (the entire paragraph) may only cover a single paragraph, missing cross-paragraph semantic connections (e.g., paragraph 1 mentions "Zhang San is involved in a case," paragraph 2 mentions "the amount Zhang San is involved in," but the original context does not include paragraph 2). Therefore, based on the original paragraph scope, P adjacent paragraphs are added forward and backward to supplement the complete information chain of the entity by covering cross-paragraph semantics, thus enhancing the semantic distinguishability of the entity. P is an integer greater than 0, and a value of 1 is recommended (values ​​≥2 may introduce irrelevant topic paragraphs, leading to semantic redundancy in the context), and it must be ensured that the expanded paragraphs are related to the topic of the original paragraph.

[0073] When the initial entity is related to a domain identifier, the original context (the sentence group expanded by the original first preset similarity threshold) may miss some semantically weak but crucial sentences that are important for distinguishing entities due to the threshold being too high. Therefore, by lowering the first preset similarity threshold, the coverage of the sentence group is expanded, incorporating the previously missed weakly related but crucial sentences into the context and supplementing the domain-specific semantic details of the entities. A preset value of 0.05-0.1 is recommended to avoid excessive reduction that could cause the sentence group's theme to deviate.

[0074] It should be noted that all expansion methods must ensure that the new range does not exceed the total character length of the original text to avoid truncating invalid text; the expansion range should be moderate, such as N≤2, P=1, and preset value≤0.1, to avoid excessive expansion introducing irrelevant semantic noise; after expansion, the semantic coherence of the new range should be verified by a topic consistency model (such as Latent Dirichlet Allocation) to remove sentences / paragraphs that deviate from the topic and ensure that the contextual semantics focus on the entity to be optimized.

[0075] As described above, by expanding with a fixed number of N statements, adjacent semantic information related to entities is supplemented without breaking the original statement-level semantic boundaries, thus solving the semantic differentiation problem caused by insufficient statement coverage; by expanding with a limited number of P paragraphs, cross-paragraph association information of entities is supplemented while controlling the redundancy of the scope, thus solving the problem of incomplete semantic information caused by single paragraph coverage; by appropriately lowering the similarity threshold, weakly related key statements that were originally omitted are incorporated into the context while controlling the risk of deviation from the topic of the sentence group, thus solving the problem of missing key semantic statements caused by the original threshold being too high.

[0076] S4. Group the target entity names according to the semantic attributes corresponding to all the target entity names to obtain several sets of entity names.

[0077] The target entity name has passed semantic uniqueness verification, and its associated semantic attributes are the core identifiers of entity semantic attribution. Therefore, based on the semantic attributes corresponding to the target entity name, entities with similar or identical semantic attribute features are grouped into the same set, realizing the aggregation of semantically similar entities and the separation of semantically different entities. This lays the foundation for subsequent training of a dedicated entity extraction model by set, enabling the entity extraction model to focus on entities with a certain type of semantic attribute and improve extraction accuracy.

[0078] Specifically, an entity name set is a collection of target entity names that have similar or identical semantic attributes. Entities within a set share a consistent core semantic affiliation, while entities across different sets exhibit significant semantic differences. Grouping is performed hierarchically from top to bottom, with priority decreasing to domain attributes, scenario attributes, and role attributes, ensuring consistency in the core semantic affiliation of entities within a set. This includes: First level (domain attribute): First, divide all target entities into major categories according to domain attributes (such as the sets of "public security", "finance", and "medical" fields) to ensure that entities within the same set belong to the same business domain.

[0079] The second level (scenario attributes): Under each major domain category, further subdivisions are made according to scenario attributes (e.g., under "public security domain", it is divided into "case investigation scenario" and "household registration scenario" sets) to ensure that entities within the same set are in the same business scenario.

[0080] The third level (role attributes): Under each scene subclass, entities are finally divided according to role attributes (e.g., under "Public Security Field - Case Investigation Scene", they are divided into "Case Handler", "Involved Person", and "Case Handlers"), ensuring that entities within the same set have the same role identity.

[0081] Specifically, each target entity name is structurally split, extracting its corresponding role attributes, scene attributes, and domain attributes. The domain attributes of all target entities are iterated over, creating a set with the "domain attribute" as the key (e.g., key = "Public Security Domain", value is a list of all entities under that domain). For each entity within a domain set, its scene attributes are iterated over, creating a subset under the domain set with the "scene attribute" as the key (e.g., a subset with key = "Case Investigation Scene" under "Public Security Domain"). For each entity within a scene subset, its role attributes are iterated over, creating a final set under the scene subset with the "role attribute" as the key (e.g., a final set with key = "Case Handler" under "Public Security Domain - Case Investigation Scene").

[0082] If the entity's domain attribute is empty, it is categorized into the "General Domain" set. If the entity's scene attribute is empty, it is categorized into the "General Scene" subset under that domain. If the entity's role attribute is empty, it is categorized into the final "General Role" set under that scene subset.

[0083] Each final collection is named according to the naming rule of "domain attribute - scene attribute - role attribute". The mapping relationship of "collection name - target entity name list" is stored in the form of a dictionary or database table for easy subsequent calling.

[0084] As described above, by using hierarchical attribute grouping operations, entities within a set share the core semantic attributes of domain, scene, and role. Each entity name set corresponds to a specific semantic entity, avoiding the mixing of entities from different domains / scenes. This provides a semantically focused sample basis for subsequent model training, thereby improving the model's accuracy in extracting specific types of entities and solving the problem of poor performance of general models in extracting subdivided entities.

[0085] S5: Build and train the corresponding initial entity extraction model for each entity name set, and obtain the target entity extraction model for each entity name set. The target entity extraction model is used to extract entities with specific semantic attributes from the text.

[0086] General entity extraction models (such as general BERT-NER and spaCy-NER) perform significantly worse than general entities (such as person names, place names, and organization names) when handling specific entities due to the inherent contradiction between the generalization of their training objectives and the specificity of individual entities. Therefore, this embodiment constructs and trains an initial entity extraction model for each set of entity names, thereby obtaining a target entity extraction model for each set of entity names, thus improving the extraction performance for entities with specific semantic attributes.

[0087] In one specific embodiment, S5 includes the following steps: S51. For any set of entity names, using the target entity name in the current set of entity names as the annotation type, collect the domain scene text data corresponding to the current set of entity names.

[0088] S52, perform entity annotation on the domain scene text data, wherein the annotation content includes the text quantity of each target entity name in the current entity name set in the domain scene text data.

[0089] S53, for any target entity name in the current entity name set, if the amount of text corresponding to the current target entity name is lower than the preset sample threshold, then supplementary text containing the current target entity name is generated through the preset large model. The semantic style of the supplementary text is consistent with the domain scene text data, and the sum of the amount of text corresponding to the current target entity name and the number of supplementary texts is not lower than the preset sample threshold.

[0090] S54 merges the domain scene text data with the supplementary text corresponding to all target entity names to form a dedicated training dataset corresponding to the current set of entity names.

[0091] S55, train the corresponding initial entity extraction model based on the dedicated training dataset corresponding to each entity name set to obtain the target entity extraction model corresponding to each entity name set.

[0092] The annotation type is the core semantic attribute identifier of all target entity names within the current entity name set (e.g., the annotation type for the set "Public Security Domain - Case Investigation Scene - Case Handler" is "Public Security - Case Investigation - Case Handler"), used to clarify the target of data collection. Domain scene text data is the original text that perfectly matches the domain attributes and scene attributes of the entity name set, and is the basic data source for model training.

[0093] Specifically, the data collection dimensions are defined by extracting "domain" and "scenario" keywords from the semantic attributes of the entity name set. Authoritative data channels matching the keywords are selected (e.g., data in the public security field comes from case disclosure databases and government information disclosure documents; data in the financial field comes from bank compliance documents and user account opening records). Domain scenario text data is obtained through keyword retrieval and manual initial screening. The collected text (e.g., DOC, TXT, PDF formats) is uniformly converted into plain text format to remove format noise (e.g., table and image annotations) to ensure that the data can be used for subsequent annotation.

[0094] Model training requires supervised data consisting of text and labeled text. The core of labeling is to identify which parts of the text belong to the target entities (i.e., entities within the set of entity names). Therefore, in the collected domain scenario text, all target entities belonging to the current set of entity names are labeled, and the text volume of each entity is counted. This provides a "text-entity" correspondence for subsequent sample supplementation and model training. Specifically, entity labeling is the operation of marking the specific location and corresponding type of the target entity in the text. It typically uses the "BIO labeling method" (B=entity start, I=entity middle, O=non-entity) or "entity boundary + type" labeling (e.g., in the text "Case handler Zhang San", the start / end index of "Zhang San" is marked, and the type is labeled "Public Security - Case Investigation - Case handler").

[0095] Text quantity refers to the number of valid text fragments in which a target entity name appears in the domain scene text data (e.g., if the entity "Zhang San" appears in 10 text fragments, and each fragment includes its context, then the text quantity is 10). It is used to measure the sufficiency of samples for that entity. If the text quantity of a target entity is lower than a preset sample threshold, the model will have low extraction accuracy for that entity due to insufficient learning samples. Therefore, supplementary text consistent with the style of the domain scene is generated using a preset large model to supplement the sample quantity and ensure that the training samples for each target entity can meet the model's learning needs, avoiding the problem of sample imbalance.

[0096] The preset sample threshold is the minimum number of samples required to ensure the model can effectively learn the features of a specific entity; a value of 50-100 is recommended and can be adjusted based on entity complexity. The preset large model is a pre-trained language model with text generation capabilities (such as Qwen2.5-3B-Instruct or Llama3-8B). It should be a model fine-tuned for the corresponding domain (e.g., "Public Security-Specific LLM" for the public security domain, and "Financial-Specific LLM" for the financial domain) to ensure the semantic style of the generated text matches the domain. The supplementary text is a text fragment generated by the large model that contains the target entity name and has a semantic style consistent with the domain context. It must include the entity and its context to ensure it can be used for model training.

[0097] The initial entity extraction model is the basic model architecture to be selected. It can be a mature entity extraction model (such as BERT-NER, RoBERTa-NER, SpanBERT), or a model with pre-trained weights similar to the target domain (such as "BERT-Public Security Pre-trained Model" for the public security domain and "BERT-Finance Pre-trained Model" for the financial domain) to reduce fine-tuning costs.

[0098] It should be noted that for the merged domain scenario text data and the supplementary text data corresponding to all target entity names, operations such as removing duplicate text, eliminating low-quality text, correcting annotation errors, and unifying format can be performed to obtain a dedicated training dataset. The initial entity extraction model can then be fine-tuned to adapt the model to the extraction requirements of specific entities, ultimately outputting a high-precision target entity extraction model that can accurately extract entities corresponding to a set of entity names.

[0099] As described above, targeted attribute matching ensures that the collected text data is highly consistent with the semantic environment of the entity name set, avoiding the problems of domain mixing and scene irrelevantness in general datasets. This provides an accurate data foundation for the model to learn the contextual features of specific semantic entities. Large-scale model generation and quality screening quickly supplement insufficient entity samples, avoiding extraction bias caused by missing entity samples. Simultaneously, supplementing text with consistent domain style ensures no irrelevant semantic noise is introduced, guaranteeing the overall quality of the training data. Fine-tuning training with a dedicated dataset allows the model to focus solely on entities of a specific semantic class, significantly improving the extraction accuracy for that class of entities. Furthermore, each model corresponds to a specific entity set, achieving precise adaptation of one model per entity class, solving the problem of general models being large but not precise and having poor extraction performance for specific entities.

[0100] In one specific embodiment, S55 includes the following steps: S551, for any set of entity names, divide the dedicated training dataset corresponding to the current set of entity names into a training set and a validation set according to a preset ratio.

[0101] S552 uses entity extraction accuracy, recall, and / or F1 score as evaluation metrics, and trains the initial entity extraction model corresponding to the current entity name set for a preset number of training rounds based on the training set.

[0102] S553 evaluates model performance using a validation set after completing a preset number of training rounds.

[0103] S554. If the numerical improvement ratio of model performance is lower than the preset ratio threshold within consecutive preset rounds, and the numerical values ​​of model performance are all higher than the preset performance threshold, then stop training and obtain the target entity extraction model corresponding to the current entity name set; otherwise, repeat step S552.

[0104] The recommended training set to validation set ratio is 8:2 or 7:3 (use 7:3 for smaller datasets to ensure more sufficient data; use 8:2 for larger datasets to improve training efficiency). The ratio must be kept constant to ensure consistent model training conditions across different entity name sets.

[0105] Precision P is the proportion of the results predicted by the model as the target entity that are actually the target entity; recall R is the proportion of all actual target entities that are successfully predicted by the model; the F1 score is the harmonic mean of precision and recall, and the formula is: F1 = 2 × P × R / (P + R), which is the core indicator for measuring the performance of entity extraction models.

[0106] The preset training epochs are the basic number of epochs for a single continuous training session, with 5-10 epochs recommended to observe the initial trend of model performance changes. If the performance does not meet the target after the initial epochs, continue to add more epochs for training.

[0107] Model training exhibits diminishing marginal returns, meaning performance improves rapidly in the initial stages but gradually slows down and eventually stagnates. Overtraining leads to overfitting. Therefore, a preset stopping condition should be used to determine whether to continue training. Training should be terminated promptly once the model performance meets the target, balancing model performance and training efficiency to avoid overfitting or ineffective training. The numerical improvement percentage is the relative improvement of the current performance metric (e.g., F1 score) compared to the previous epoch (e.g., (current F1 - previous F1) / previous F1). A recommended preset threshold of 5% is provided; an improvement of less than 5% is considered insignificant.

[0108] The preset performance threshold is the minimum performance requirement for the model to meet practical standards. It is recommended that the F1 value be ≥0.9, and it should be adjusted according to the business scenario: 0.95 for high-precision scenarios and 0.85 for general scenarios to ensure that the output model has practical application value.

[0109] As described above, by extracting semantic attributes from the context text and adding corresponding semantic attributes to the name of each initial entity, isolated entity names are transformed into structured, refined names containing attributes and names. This provides entities with richer semantic information, thus providing a semantic foundation for solving the problems of homogenization and semantic ambiguity in entity names. By expanding and iteratively updating the context corresponding to initial entities that fail semantic uniqueness verification, more semantic details are added to these entities, giving the newly generated refined entity names stronger semantic distinguishability. This solves the semantic conflict problem caused by insufficient information, ensuring that the final output target entity name has strict semantic uniqueness, completely resolving the inherent defects of the initial entities, and guaranteeing the accuracy of entity extraction results. By constructing and training a corresponding initial entity extraction model for each entity name set and obtaining the target entity extraction model for each entity name set, the model focuses only on entities with a specific semantic meaning, significantly improving the extraction effect for entities with specific semantic attributes.

[0110] Example 2 Embodiment 2 of the present invention provides a non-transitory computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the general entity extraction method provided in the above embodiment.

[0111] Example 3 Embodiment 3 of the present invention provides an electronic device, which includes a processor and the non-transitory computer-readable storage medium of Embodiment 2 of the present invention.

[0112] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A general entity extraction method, characterized in that, The general entity extraction method includes the following steps: S1, based on the context scope corresponding to each initial entity extracted from the original text, extract the context text of each initial entity in the original text, where there is a problem of homogenization of entity names or semantic ambiguity among the initial entities; S2, based on the context text corresponding to each initial entity, add corresponding semantic attributes to the name of each initial entity to obtain the refined entity name corresponding to each initial entity; S3, perform semantic uniqueness verification on all refined entity names, expand the context scope corresponding to the initial entity that fails the verification, update the corresponding refined entity name according to the expanded context text, until all the refined entity names corresponding to the initial entities pass the verification, and obtain the semantically unique target entity name corresponding to each initial entity. There is no problem of homogenization of entity names or semantic ambiguity among the target entity names. S4, group the target entity names according to the semantic attributes corresponding to all the target entity names to obtain a set of several entity names; S5, construct and train the corresponding initial entity extraction model for each entity name set, and obtain the target entity extraction model corresponding to each entity name set, wherein the target entity extraction model is used to extract entities with specific semantic attributes from the text.

2. The general entity extraction method according to claim 1, characterized in that, S1 includes the following steps: S11, Based on the position coordinates of each initial entity in the original text and the structural features of the original text, determine the text extraction rules corresponding to each initial entity, wherein the structural features include paragraph boundary markers and punctuation mark separators; S12, according to the text extraction rules, obtain the context scope corresponding to each initial entity; S13, according to the context scope, extract and extract the context text corresponding to each initial entity from the original text.

3. The general entity extraction method according to claim 2, characterized in that, The process of obtaining the context scope corresponding to each initial entity according to the text extraction rules includes at least one of the following methods: The first method of obtaining the context is as follows: For any initial entity, if the current initial entity is located inside a statement, then the statement containing the current initial entity, the M statements preceding it, and the M statements following it are taken as the context range corresponding to the current initial entity, where M is an integer greater than 0. The second method of obtaining the context is as follows: For any initial entity, if the current initial entity is located at the beginning or end of a paragraph, then the paragraph where the current initial entity is located is taken as the context scope corresponding to the current initial entity. The third acquisition method: For any initial entity, if the original text contains domain identifier words related to the current initial entity, then the statement containing the domain identifier words is taken as the core sentence, and the initial sentence group range is the core sentence; Starting from the core sentence, select the first adjacent sentence in the preceding and following directions of the core sentence respectively, and calculate the first semantic similarity between each adjacent sentence and the core sentence; If the first semantic similarity is greater than or equal to the first preset similarity threshold, the corresponding adjacent sentences are included in the sentence group range, and the next adjacent sentence is selected from the corresponding preceding or following direction as the new starting point. The semantic similarity calculation and inclusion determination are repeated until the first semantic similarity is less than the first preset similarity threshold, and the expansion to the corresponding preceding or following direction is stopped. Integrate all statements that have been expanded in the preceding and following directions and included in the sentence group to form a sentence group containing domain identifiers. Use the sentence group containing domain identifiers as the context scope corresponding to the current initial entity.

4. The general entity extraction method according to claim 3, characterized in that, S3 includes the following steps: S31, convert each refined entity name and its corresponding context text into a semantic vector; S32, calculate the second semantic similarity between the semantic vectors corresponding to any two refined entity names; S33, if the second semantic similarity is greater than or equal to the second preset similarity threshold, it is determined that the verification fails, and the corresponding two refined entity names are marked as entities to be optimized; S34, expand the context scope of the entity to be optimized, and return to step S2 based on the expanded context text to obtain the new refined entity name corresponding to each entity to be optimized; S35, if the second semantic similarity between all refined entity names is lower than the second preset similarity threshold, then it is determined that all the refined entity names corresponding to the initial entities have passed the verification, and the refined entity name corresponding to each initial entity when it passes the verification is determined as the corresponding target entity name; otherwise, return to step S31.

5. The general entity extraction method according to claim 4, characterized in that, Expanding the context scope of the entity to be optimized includes at least one of the following expansion methods: The first expansion method: For any initial entity, if the current initial entity is inside a statement, then the N statements preceding and following the current initial entity in the context scope corresponding to the current initial entity are included in the new context scope corresponding to the current initial entity, where N is an integer greater than 0. The second expansion method: For any initial entity, if the current initial entity is at the beginning or end of a paragraph, then the P paragraphs before and after the current initial entity's context scope are included in the new context scope corresponding to the current initial entity, where P is an integer greater than 0. The third expansion method: For any initial entity, if the original text contains domain identifier words related to the current initial entity, then the first preset similarity threshold is subtracted from the preset value to obtain the new first preset similarity threshold. Inclusion determination is performed based on the new first preset similarity threshold to obtain the new context range corresponding to the current initial entity.

6. The general entity extraction method according to claim 1, characterized in that, S5 includes the following steps: S51, For any set of entity names, using the target entity name in the current set of entity names as the annotation type, collect the domain scene text data corresponding to the current set of entity names; S52, perform entity annotation on the domain scene text data, wherein the annotation content includes the text amount of each target entity name in the current entity name set in the domain scene text data; S53, for any target entity name in the current entity name set, if the amount of text corresponding to the current target entity name is lower than the preset sample threshold, then supplementary text containing the current target entity name is generated through the preset large model, wherein the semantic style of the supplementary text is consistent with the domain scene text data, and the sum of the amount of text corresponding to the current target entity name and the number of supplementary texts is not lower than the preset sample threshold. S54, merge the domain scene text data with the supplementary text corresponding to all target entity names to form a dedicated training dataset corresponding to the current entity name set; S55, train the corresponding initial entity extraction model based on the dedicated training dataset corresponding to each entity name set to obtain the target entity extraction model corresponding to each entity name set.

7. The general entity extraction method according to claim 6, characterized in that, S55 includes the following steps: S551, for any set of entity names, divide the dedicated training dataset corresponding to the current set of entity names into a training set and a validation set according to a preset ratio; S552, using entity extraction accuracy, recall and / or F1 score as evaluation metrics, train the initial entity extraction model corresponding to the current entity name set for a preset number of training rounds based on the training set. S553, After completing the preset number of training rounds, the model performance is evaluated using the validation set; S554. If the numerical improvement ratio of model performance is lower than the preset ratio threshold within consecutive preset rounds, and the numerical values ​​of model performance are all higher than the preset performance threshold, then stop training and obtain the target entity extraction model corresponding to the current entity name set; otherwise, repeat step S552.

8. The general entity extraction method according to claim 1, characterized in that, S2 includes the following steps: S21, based on the context text corresponding to each initial entity, extract the role attributes, domain attributes, and scene attributes corresponding to each initial entity; S22, according to the preset priority rules, non-empty semantic attribute values ​​are concatenated with predefined connectors to form attribute combination strings corresponding to each initial entity. The priority rules are that the priority of role attributes is higher than that of scene attributes, the priority of scene attributes is higher than that of domain attributes, and the absence of high-priority attributes is not filled by low-priority attributes. S23, add the attribute combination string corresponding to each initial entity as a prefix / suffix to the name of the corresponding initial entity to form the refined entity name corresponding to each initial entity.

9. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the general entity extraction method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images