A document entity conflict detection method and apparatus

Abstract

The application discloses a document entity conflict detection method and device, the method comprises the following steps: performing entity recognition on a target document through an NLP model to determine entity mentions of the target document; comparing the entity mentions with each entity of a knowledge base entity pool to perform entity mention disambiguation; then performing attribute extraction on the entity mentions according to entity attribute types, entity format constraints and a sequence labeling model, and then performing entity attribute conflict detection on the target document; meanwhile, obtaining subject-object relationship pairs of the target document according to a preset subject-object relationship type and a triple model, and then performing entity relationship conflict detection on the subject-object relationship pairs according to a preset contradiction rule library of the entity mentions. The application simultaneously covers attribute value conflicts and complex relationship logical contradictions, supports conditional conflict fine processing, greatly improves the accuracy and efficiency of conflict detection, and improves the consistency of the knowledge base data.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method and device for detecting document entity conflicts. Background Technology

[0002] As enterprises increase their level of informatization, knowledge base systems are gradually becoming the core infrastructure for enterprise knowledge management. These systems contain a large number of heterogeneous documents from multiple sources, which may contain conflicting descriptions of the same entity. Therefore, enterprises need to regularly perform entity conflict checks on heterogeneous documents from multiple sources, or perform entity conflict checks on newly added documents to the knowledge base, in order to ensure data consistency.

[0003] Existing document entity conflict detection methods suffer from several drawbacks. Rule-based and structured data-based methods rely on manually defined rules, lack sufficient semantic understanding, and have poor scalability, leading to reduced consistency in the knowledge base. Simple NLP-based text comparison methods may overlook contextual relationships due to incomplete text paragraphs, resulting in high false positive rates and further reducing knowledge base consistency. Knowledge graph-based entity alignment techniques fail to analyze attribute-level conflicts, depend on graph completeness, and have high computational complexity, making it difficult to meet users' requirements for knowledge base data consistency. Summary of the Invention

[0004] This application provides a document entity conflict detection method and device to solve the problem of low document data consistency in enterprise knowledge bases.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] On the one hand, embodiments of this application provide a document entity conflict detection method, the method comprising:

[0007] Entity recognition is performed on the target document using a pre-trained NLP model to identify entity references. These entity references are then compared with each entity in the knowledge base entity pool. If an entity reference exists in the pool, it is disambiguated based on the cosine similarity between the reference and each entity to isolate disambiguated entity attributes. Attribute extraction is performed on the entity references based on preset entity attribute types, entity format constraints, and a pre-trained sequence labeling model to obtain the attribute extraction results for the target document. Numerical comparison and semantic reasoning are then performed on the attribute extraction results based on the unisolated entity attributes of the entity references to detect entity attribute conflicts in the target document. Simultaneously, subject-object relationship pairs are obtained based on preset subject-object relationship types and a pre-trained triplet model. Finally, entity relationship conflicts are detected on the subject-object relationship pairs based on a preset contradiction rule base for the entity references.

[0008] In one example, entity mentions are disambiguated based on their cosine similarity to each entity. Specifically, entities whose cosine similarity to an entity mention is less than or equal to a preset similarity threshold are disambiguated.

[0009] In one example, the method further includes: identifying entities with a cosine similarity greater than a preset similarity threshold to entity mentions as candidate entities; matching entity mention keywords with the corresponding attributes of candidate entities; linking the successfully matched target entities with the entity mentions; and adding an entity mention attribute ID for the unmatched target entities.

[0010] In one example, attribute extraction is performed on entity mentions based on preset entity attribute types, entity format constraints, and a pre-trained sequence labeling model to obtain the attribute extraction results of the target document. Specifically, this includes: when the entity mention is a structured description type, processing the entity mention using a pre-trained sequence labeling model to label the key-value pairs of the structured description as attribute extraction results; when the entity mention is an unstructured description type, identifying the basic attributes and conditional attributes of the entity mention using a pre-trained sequence labeling model, and labeling the basic attributes and the conditional attributes as attribute extraction results; when the entity mention is a numerical description type, matching the entity mention with preset entity format constraints using regular expressions, and determining the numerical value of the matched and converted entity mention as the attribute extraction result.

[0011] In one example, the subject-object relation pairs of the target document are obtained based on a preset subject-object relation type and a pre-trained triple model. Specifically, this includes: determining the input order of the corresponding statements of the preset subject-object relation type; inputting the corresponding statements of the subject-object relation type in the target document into the pre-trained triple model for single-sentence processing based on the input order, generating conditional relation statements for each subject-object relation type in the target document; and extracting conditional relations from the conditional relation statements to obtain subject-object relation pairs for each subject-object relation type in the target document.

[0012] In one example, based on the unisolated entity attributes mentioned by the entity, numerical comparison and semantic reasoning are performed on the attribute extraction results to detect entity attribute conflicts in the target document. Specifically, this includes: classifying the attribute extraction results to obtain numerical attribute results and textual attribute results; performing numerical comparison on the numerical attribute results based on the unisolated incompatible numerical relationships mentioned by the entity; if incompatible numerical values ​​exist in the numerical attribute results, it is determined that the target document has entity attribute conflicts; comparing the numerical attribute results with the unisolated error threshold mentioned by the entity; if the numerical attribute results are not within the unisolated error threshold, it is determined that the target document has entity attribute conflicts; and performing semantic reasoning on the textual attribute results based on the unisolated linguistic logical relationships mentioned by the entity; if two textual attribute descriptions in the textual attribute results contradict each other, it is determined that the target document has entity attribute conflicts.

[0013] In one example, entity relationship conflict detection is performed on the subject-object relationship pair based on a preset conflict rule base mentioned by the entity. Specifically, this includes: extracting the relationship establishment conditions of the subject-object relationship pair; performing overlap detection on the relationship establishment conditions according to the preset conflict rule base; if there is overlap in the relationship establishment conditions, it is determined that the target document has an entity relationship conflict; comparing the relationship type of the subject-object pair with the relationship types in the preset conflict rule base; if there is mutual exclusion, it is determined that the target document has an entity relationship conflict; and performing graph traversal on the subject-object relationship pair according to the preset conflict rule base to detect whether the subject-object relationship pair violates business rules; if it violates business rules, it is determined that the target document has an entity relationship conflict.

[0014] In one example, the method further includes: when the target document has entity conflicts, determining the credibility weight of the target document according to a preset credibility weight formula; the preset credibility weight formula is related to the credibility score, the current year, and the document year, and the credibility score is related to the document's publication location; comparing the credibility weight with a preset weight threshold, and when the credibility weight is less than the preset weight threshold, marking the target document as pending review and generating a first conflict report for the target document; when the credibility weight is greater than or equal to the preset weight threshold, generating a second conflict report for the target document; the first conflict report level is greater than the second conflict report level.

[0015] In one example, the credibility weight of the target document is determined according to a preset credibility weight formula, specifically including: determining the credibility score of the target document based on its publication source; determining the time decay factor of the target document based on the difference between the current year and the year of the target document; the larger the difference, the larger the time decay factor; and multiplying the credibility score of the target document by the time decay factor to obtain the credibility weight of the target document.

[0016] On the other hand, embodiments of this application provide a document entity conflict detection device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-mentioned document entity conflict detection methods.

[0017] On the other hand, embodiments of this application provide a non-volatile computer storage medium for document entity conflict detection, which stores computer-executable instructions capable of executing any of the above-mentioned document entity conflict detection methods.

[0018] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0019] This application provides both attribute value conflict detection and complex relational logic contradiction detection, supports refined processing of conditional conflicts, significantly improves the accuracy and efficiency of conflict detection, and enhances the consistency of knowledge base data. Secondly, this application also calculates the credibility score of a document based on the authority of its publication source and the document's time freshness, providing a clear level for conflict resolution strategies, greatly improving the credibility of the generated conflict report, and increasing the traceability of document conflicts. Attached Figure Description

[0020] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which:

[0021] Figure 1 A flowchart illustrating a document entity conflict detection method provided in an embodiment of this application;

[0022] Figure 2 A system architecture diagram of a document entity conflict detection method provided in this application embodiment;

[0023] Figure 3 A flowchart illustrating entity disambiguation and linking in a document entity conflict detection method provided in this application embodiment;

[0024] Figure 4 A flowchart of attribute extraction and detection for a document entity conflict detection method provided in this application embodiment;

[0025] Figure 5 A flowchart illustrating the relationship extraction and detection process of a document entity conflict detection method provided in this application embodiment;

[0026] Figure 6 This is a schematic diagram of the structure of a document entity conflict detection device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart illustrating a document entity conflict detection method provided in an embodiment of this application. This method can be applied to different business domains. Certain input parameters or intermediate results in this process can be manually adjusted to help improve accuracy.

[0030] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a controller as an example.

[0031] It should be noted in advance that:

[0032] In the patented technical solution, an entity mention refers to a fragment of description of a specific entity appearing in the text; it is a linguistic expression in natural language text that points to a real-world entity. It is the basic unit for entity recognition and processing, and needs to be associated with standardized entities in the knowledge base through subsequent disambiguation and linking processes.

[0033] Based on this Figure 1 The process may include the following steps:

[0034] S101: Entity recognition is performed on the target document using a pre-trained NLP model to determine the entity references in the target document.

[0035] It should be noted that in some embodiments of this application, entity recognition is achieved by fine-tuning a pre-trained RoBERTa-wwm-large model (an NLP model designed for Chinese) on domain documents (such as enterprise product manuals) to identify entity types (products, departments, parameters, etc.).

[0036] Furthermore, the model outputs entity mentions and their context (e.g., in "The parameters of product A (model: X-2023) are...", "product A" is identified as a product class entity).

[0037] S102: Compare the entity mention with each entity in the knowledge base entity pool. When the entity mention exists in the knowledge base entity pool, disambiguate the entity mention based on the cosine similarity between the entity mention and each entity to isolate the disambiguated entity attributes.

[0038] It should be noted that, in some embodiments of this application, after the entity references of the target document are determined, the entity references need to be disambiguated and linked.

[0039] First, the entity mention is compared with each entity in the knowledge base entity pool. When the entity mention exists in the knowledge base entity pool, a vector representation of the entity mention and each entity in the entity pool is generated based on Sentence-BERT, and the cosine similarity is calculated.

[0040] Secondly, after determining the cosine similarity, entities with a cosine similarity to entity mentions that is less than or equal to a preset similarity threshold (e.g., 0.85) are disambiguated; entities with a cosine similarity to entity mentions that is greater than the preset similarity threshold (e.g., 0.85) are identified as candidate entities; then, TF-IDF keywords (e.g., “model X-2023”, “version 2.0”) of the paragraph where the entity mentions are located are extracted and matched with the attributes of the candidate entities.

[0041] Finally, the successfully matched target entities are linked with the entity mentions, and the unmatched target entities are added with the attribute IDs of the entity mentions and recorded in the knowledge base.

[0042] S103: Based on the preset entity attribute types, entity format constraints, and pre-trained sequence labeling models, perform attribute extraction on the entity references to obtain the attribute extraction results of the target document.

[0043] It should be noted that, in some embodiments of this application, regarding the preset entity attribute types and entity format constraints: entity attribute types are predefined according to the needs of the enterprise knowledge base; for numerical attributes (such as date, version number), format constraints are defined (such as "YYYY-MM-DD" "vX.YZ"); the sequence labeling model adopts the SPANBERT model, and a domain-adapted sequence labeling task is constructed according to the needs of the enterprise knowledge base. The input is the entity mention and its context paragraph, and the output is the attribute slot filling result, that is, the attribute extraction result (such as the text fragment "January 2023" corresponding to "release date").

[0044] Furthermore, when the entity mention is a structured description type, the entity mention is processed by a pre-trained sequence labeling model, and the key-value pairs of the structured description are labeled as attribute extraction results.

[0045] When the entity mention is an unstructured description type, the basic attributes and conditional attributes of the entity mention are identified by a pre-trained sequence labeling model, and the basic attributes and conditional attributes are labeled as attribute extraction results, such as {"condition":"high temperature environment","functional support":"yes"}.

[0046] When the entity mention is a numerical description type, the entity mention is matched with the preset entity format constraints using regular expressions, and the numerical value of the matched and converted entity mention is determined as the attribute extraction result.

[0047] It should also be noted that after obtaining the attribute extraction results, attribute storage and tracing are performed: the extraction results are stored in the Neo4j graph database, recording the source document, position offset, and extraction confidence (model output probability) of the attribute values; the node structure is as follows:

[0048] cypher

[0049] (Entity:Product{id:"P001"})-[:HAS_ATTRIBUTE]->

[0050] (Attribute:{name:"Release Date",value:"2023-01",source_doc:"doc1.pdf",page:5}).

[0051] S104: Based on the unisolated entity attributes mentioned by the entity, perform numerical comparison and semantic reasoning on the attribute extraction results to detect entity attribute conflicts in the target document.

[0052] It should be noted that in some embodiments of this application, the detection of attribute conflicts is divided into numerical attribute conflict detection and textual attribute conflict detection.

[0053] First, the attribute extraction results are classified into numerical attribute results and text attribute results;

[0054] Secondly, based on the unisolated and incompatible numerical relationships mentioned by the entity (the physical implementation depends on the rule engine), the numerical attribute results are compared. If incompatible values ​​exist in the numerical attribute results (such as date uniqueness or version number increment), it is determined that the target document has an entity attribute conflict. For example, if document A declares "Product A release date is 2023-01" and document B declares it as "2023-03", then it is marked as a conflict.

[0055] Simultaneously, based on the unisolated error threshold mentioned by the entity, the numerical attribute result is compared with the unisolated error threshold. If the numerical attribute result is not within the unisolated error threshold, it is determined that the target document has an entity attribute conflict; for example, a threshold (e.g., ±5℃) is set for the allowable error value (such as temperature range). If the difference between the two values ​​exceeds the threshold, a conflict is marked.

[0056] Simultaneously, the RoBERTa-large-MNLI model is used to fine-tune enterprise domain text (such as product descriptions and user feedback). Based on the unisolated linguistic logical relationships mentioned by the entities (e.g., contradictions: "Supports function X" vs. "Does not support function X", implication: "Product A has a maximum temperature of 50℃" vs. "Product A has excellent high-temperature resistance"), semantic reasoning is performed on the text attribute results. If there is a contradiction between two text attribute descriptions in the text attribute results, it is determined that there is an entity attribute conflict in the target document.

[0057] One more point needs further explanation: Regarding the handling of compound conflicts, if two attribute values ​​are accompanied by different conditions (e.g., "Supports function X in high-temperature environments" vs. "Does not support function X in low-temperature environments"), they are considered non-conflicting, and additional condition comments are added. If an attribute has multiple values ​​within the same document (e.g., "Supports function X" and "Does not support function X"), it is marked as an internal contradiction, indicating an abnormality in document self-consistency.

[0058] S105: Based on the preset subject-object relationship type and the pre-trained triplet model, obtain the subject-object relationship pairs of the target document.

[0059] It should be noted that, in some embodiments of this application, the subject-object relationship types include, but are not limited to, the following:

[0060] Subordination: e.g., "Product A belongs to Department X" → (Product A, Subordination, Department X);

[0061] Dependency relationships: e.g., “Function X depends on component Y” → (Function X, Dependency, Component Y);

[0062] Compatibility relationships: e.g., "Product A is compatible with device B" → (Product A, compatible, device B);

[0063] Cause and effect relationship: such as "parameter exceeding the standard leads to failure" → (parameter exceeding the standard, leading to, failure);

[0064] Based on the preset subject-object relationship type, determine the input order of the corresponding statements of the subject-object relationship type (optional); input the corresponding statements of the subject-object relationship type contained in the target document into the pre-trained triplet model for single-sentence processing to generate the conditional relationship statements of the target document for each subject-object relationship type;

[0065] Conditional relation extraction is performed on the conditional relation statements to obtain the subject-object relation pairs for each subject-object relation type in the target document.

[0066] Furthermore, regarding the relational storage structure, it is based on graph database storage, recording relation type, source document, extraction confidence level, and timestamp.

[0067] cypher

[0068] (Subject:Entity{id:"Product A"})-[:RELATION{type:"Belonging",source_doc:"doc1.pdf"}]->(Object:Entity{id:"Department X"}).

[0069] S106: Based on the preset contradiction rule base mentioned by the entity, perform entity relationship conflict detection on the subject-object relationship pair.

[0070] It should be noted that in some embodiments of this application, the conditions for the establishment of the subject-object relationship are extracted, and the overlap of the conditions for the establishment of the relationship is detected according to a preset contradiction rule base. If there is an overlap of the conditions for the establishment of the relationship, it is determined that the target document has an entity relationship conflict. For example, the conditions for the establishment of the relationship (such as version number, time range) are extracted. If the conditions do not overlap, they are considered non-conflictual (Product A, compatible, device B)@condition{version:"V1.0"}, (Product A, incompatible, device B)@condition{version:"V2.0"} → no conflict.

[0071] The relationship type of the subject-object pair is compared with the relationship types in the preset contradiction rule base. If there are mutually exclusive relationships (such as "compatible" and "incompatible"), it is determined that the target document has an entity relationship conflict.

[0072] Based on a preset conflict rule base, a graph traversal is performed on the subject-object relationship pairs to detect whether the subject-object relationship pairs violate business rules. If they violate business rules, the target document is determined to have entity relationship conflicts.

[0073] Example rules:

[0074] if(A, membership, B)and(B, status, "disbanded"):

[0075] raise Conflict("Subject A belongs to the dissolved department B")

[0076] Path lookup:

[0077] cypher

[0078] MATCH(a:Entity{id:"Product A"})-[:Member]->(b:Entity),

[0079] (b)-[:status]->(c:Attribute{value:"disbanded"})RETURNa,b,c

[0080] Furthermore, when the target document has entity conflicts, the credibility weight of the target document is determined according to a preset credibility weight formula. The preset credibility weight formula is related to the credibility score, the current year, and the document year. The credibility score is related to the document's publication location (e.g., official technical document weight = 1.0, informal document weight = 0.3). Specifically: the credibility score of the target document is determined based on its publication location; the time decay factor of the target document is determined based on the difference between the current year and the target document year: 1.0 - 0.1 * (current year - document year); the larger the difference, the larger the time decay factor; the credibility score of the target document is multiplied by the time decay factor to obtain the credibility weight of the target document.

[0081] The credibility weight is compared with a preset weight threshold. When the credibility weight is less than the preset weight threshold (e.g., 0.2), the target document is marked as pending review, and a first conflict report of the target document is generated.

[0082] When the credibility weight is greater than or equal to a preset weight threshold, a second conflict report for the target document is generated; the first conflict report level is greater than the second conflict report level.

[0083] Specifically: Structured output of attribute conflicts:

[0084] {

[0085] "conflict_id":"C001",

[0086] "conflict_type":"Attribute conflict",

[0087] "entity_id":"P001",

[0088] "attribute":"Release Date",

[0089] "conflicting_values":[

[0090] {"value":"2023-01","source_doc":"doc1.pdf","weight":0.9},

[0091] {"value":"2023-03","source_doc":"doc2.docx","weight":0.7}

[0092] ],

[0093] "suggested_resolution":"Prefer the value of doc1.pdf (weight 0.9)"}

[0094] Structured output of relationship conflicts:

[0095] {

[0096] "conflict_type":"Relationship conflict",

[0097] "Main Body":"Product A",

[0098] "Relationship": "Subordinate",

[0099] "Object":"Department X",

[0100] "Conflict entry":[

[0101] {"value":"Department X","source_doc":"doc1.pdf","weight":0.9},

[0102] {"value":"Department Y","source_doc":"doc2.docx","weight":0.6}

[0103] ],

[0104] "Cause of Conflict": "Direct Conflict: Mutually Exclusive Subordinate Relationships",

[0105] "Suggested Solution": "Adopt doc1.pdf (weight 0.9)"

[0106] }

[0107] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S101 to S106 will be described sequentially, but this does not mean that steps S101 and S106 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S101 to S106 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S101 to S106 can be appropriately adjusted according to actual needs.

[0108] pass Figure 1This application provides both attribute value conflict detection and complex relational logic contradiction detection, supports refined processing of conditional conflicts, significantly improves the accuracy and efficiency of conflict detection, and enhances the consistency of knowledge base data. Secondly, this application also calculates the credibility score of a document based on the authority of its publication source and the freshness of the document, providing a clear level for conflict resolution strategies, greatly improving the credibility of the generated conflict report, and increasing the traceability of document conflicts.

[0109] Figure 2 This is a system architecture diagram of a document entity conflict detection method provided in an embodiment of this application.

[0110] exist Figure 2 The diagram shows the overall system architecture of the method presented in this application, presented in a modular format. This means the hardware implementation can also be modularized, including: an entity processing module for entity recognition, disambiguation, and standardized mapping; an attribute extraction module for extracting entity attributes and storing them as structured triples (entity ID, attribute type, attribute value, source document); an attribute conflict detection module for detecting attribute conflicts based on numerical comparison and semantic reasoning; a relation extraction module for structurally extracting text and outputting (subject, relation type, object); a relation conflict detection module for detecting direct contradictions based on a mutually exclusive relation rule base and logical contradictions based on relation establishment conditions; and a feedback and storage module for generating conflict reports and updating the knowledge base status (e.g., marking conflicting entities and prioritizing trusted data sources).

[0111] Figure 3 This is a flowchart illustrating entity disambiguation and linking in a document entity conflict detection method provided in this application embodiment.

[0112] exist Figure 3 The image shows the entity disambiguation and linking process of this application, including two key steps: cosine similarity comparison and entity matching.

[0113] Figure 4 This is a flowchart illustrating the attribute extraction and detection process of a document entity conflict detection method provided in an embodiment of this application.

[0114] exist Figure 4 The document demonstrates the main process of attribute extraction and detection in this application, including numerical entity conflict detection and judgment and textual entity conflict detection and judgment.

[0115] Figure 5 This is a flowchart illustrating the relationship extraction and detection process of a document entity conflict detection method provided in an embodiment of this application.

[0116] exist Figure 5The document demonstrates the main process of relation extraction and detection in this application, including three relation detection methods: conditional contradiction, direct contradiction, and logical contradiction, to determine whether there is a conflict in entity relations.

[0117] Figure 6 A schematic diagram of a document entity conflict detection device provided in this application embodiment includes:

[0118] At least one processor; and,

[0119] A memory that is communicatively connected to at least one processor; wherein,

[0120] A document entity conflict detection method is described in which the memory stores instructions that can be executed by at least one processor, such that the at least one processor is able to perform any of the above-mentioned actions.

[0121] Some embodiments of this application provide a non-volatile computer storage medium for document entity conflict detection, which stores computer-executable instructions capable of executing any of the above-described document entity conflict detection methods.

[0122] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0123] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0124] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0129] Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0130] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0132] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the technical principles of this application should fall within the protection scope of this application.

Claims

1. A document entity conflict detection method, characterized in that, The method includes: Entity recognition is performed on the target document using a pre-trained NLP model to determine the entity references in the target document; The entity mention is compared with each entity in the knowledge base entity pool. When the entity mention exists in the knowledge base entity pool, the entity mention is disambiguated according to the cosine similarity between the entity mention and each entity, so as to isolate the disambiguated entity attributes. Based on the preset entity attribute types, entity format constraints, and pre-trained sequence labeling models, attribute extraction is performed on the entity references to obtain the attribute extraction results of the target document. Based on the unisolated entity attributes mentioned by the entity, numerical comparison and semantic reasoning are performed on the attribute extraction results to detect entity attribute conflicts in the target document. Based on the preset subject-object relationship types and the pre-trained triple model, the subject-object relationship pairs of the target document are obtained; the subject-object relationship types include at least one of membership, dependency, compatibility, and causality. Based on the preset contradiction rule base mentioned by the entity, entity relationship conflict detection is performed on the subject-object relationship pair; The step of performing numerical comparison and semantic reasoning on the attribute extraction results based on the unisolated entity attributes mentioned by the entity, in order to detect entity attribute conflicts in the target document, specifically includes: The attribute extraction results are classified into numerical attribute results and text attribute results. Based on the non-isolated incompatible numerical relationships mentioned by the entity, the numerical attribute results are compared. If there are incompatible numerical values ​​in the numerical attribute results, it is determined that the target document has entity attribute conflicts. Based on the unisolation error threshold mentioned by the entity, the numerical attribute result is compared with the unisolation error threshold. If the numerical attribute result is not within the unisolation error threshold, it is determined that the target document has an entity attribute conflict. Based on the unisolated linguistic logical relationships mentioned by the entities, semantic reasoning is performed on the text-type attribute results. If there is a contradiction between two text attribute descriptions in the text-type attribute results, it is determined that the target document has entity attribute conflicts.

2. The method according to claim 1, characterized in that, The disambiguation of entity mentions based on their cosine similarity to each entity specifically includes: Disambiguate entities whose cosine similarity to entity mentions is less than or equal to a preset similarity threshold.

3. The method according to claim 2, characterized in that, The method further includes: Entities whose cosine similarity to entity mentions is greater than a preset similarity threshold are identified as candidate entities; Match the keywords mentioned by the entity with the corresponding attributes of the candidate entity; Link the successfully matched target entity to the entity mention, and add the attribute ID of the entity mention to the unmatched target entity.

4. The method according to claim 1, characterized in that, The step of extracting attributes from entity references based on preset entity attribute types, entity format constraints, and a pre-trained sequence labeling model to obtain the attribute extraction results of the target document specifically includes: When the entity mention is a structured description type, the entity mention is processed by a pre-trained sequence labeling model, and the key-value pairs of the structured description are labeled as attribute extraction results; When the entity mentions are of the unstructured description type, the basic attributes and conditional attributes of the entity mentions are identified by a pre-trained sequence labeling model, and the basic attributes and conditional attributes are labeled as attribute extraction results. When the entity mention is a numerical description type, the entity mention is matched with the preset entity format constraints using regular expressions, and the numerical value of the matched and converted entity mention is determined as the attribute extraction result.

5. The method according to claim 1, characterized in that, The step of obtaining the subject-object relation pairs of the target document based on the preset subject-object relation type and the pre-trained triplet model specifically includes: Based on the preset subject-object relationship type, determine the input order of the corresponding statements for the subject-object relationship type; Based on the input order, the corresponding sentences containing the subject-object relation type in the target document are input into the pre-trained triplet model for single-sentence processing to generate conditional relation sentences for each subject-object relation type in the target document; Conditional relation extraction is performed on the conditional relation statements to obtain the subject-object relation pairs for each subject-object relation type in the target document.

6. The method according to claim 1, characterized in that, The step of detecting entity relationship conflicts in the subject-object relationship pair based on the preset contradiction rule base mentioned by the entity specifically includes: Extract the conditions for the establishment of the subject-object relationship pair, and perform overlap detection on the conditions for the establishment of the relationship according to the preset contradiction rule library. If there is an overlap of the conditions for the establishment of the relationship, it is determined that the target document has an entity relationship conflict. The relationship type of the subject-object pair is compared with the relationship types in the preset contradiction rule base. If there is mutual exclusion of relationships, it is determined that the target document has entity relationship conflict. Based on a preset conflict rule base, a graph traversal is performed on the subject-object relationship pairs to detect whether the subject-object relationship pairs violate business rules. If they violate business rules, the target document is determined to have entity relationship conflicts.

7. The method according to claim 1, characterized in that, The method further includes: When the target document has entity conflicts, the credibility weight of the target document is determined according to a preset credibility weight formula; the preset credibility weight formula is related to the credibility score, the current year, and the document year, and the credibility score is related to the document's publication location; The credibility weight is compared with a preset weight threshold. When the credibility weight is less than the preset weight threshold, the target document is marked as pending review, and a first conflict report of the target document is generated. When the credibility weight is greater than or equal to a preset weight threshold, a second conflict report for the target document is generated; the first conflict report level is greater than the second conflict report level.

8. The method according to claim 7, characterized in that, The step of determining the credibility weight of the target document according to a preset credibility weight formula specifically includes: Determine the credibility score of the target document based on its publication source; The time decay factor of the target document is determined based on the difference between the current year and the target document year; the larger the difference, the larger the time decay factor. The credibility weight of the target document is obtained by multiplying its credibility score by the time decay factor.

9. A document entity conflict detection device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a document entity conflict detection method according to any one of claims 1-8.

Images