Text content conflict detection method and system for multi-source heterogeneous knowledge base and storage medium

Abstract

The invention discloses a multi-source heterogeneous knowledge base-oriented text content conflict detection method and system and a storage medium, and the method comprises the steps: obtaining a to-be-detected text slice and a stock text slice in a knowledge base, and carrying out the multi-stage filtering of an initial conflict detection object set based on different preset screening and filtering rules; inputting the conflict candidate object set into a trained conflict perception reasoning model, and outputting all conflict text units forming conflicts between the stock text slices and the to-be-detected text slices; meanwhile, sentence level segmentation is conducted on the conflict candidate object set based on a context sensing rule, old sentences which have the highest similarity with the new sentences and have the similarity higher than a set threshold value are obtained, and potential conflict pairs are formed by the old sentences and the corresponding new sentences; and calculating a matching rate between the conflict text unit list and the potential conflict pair in sequence, determining a final conflict pair according to the matching rate, and outputting and recording a conflict pair detection result containing positioning information. The problems that effective means for processing detection objects are lacked, and it is difficult to efficiently screen and optimize a to-be-detected text set are solved; the problem that semantic conflicts are difficult to accurately recognize and locate to specific conflict texts due to insufficient effectiveness of conflict detection and location is solved, the conflict detection accuracy is improved, and deep semantic conflict recognition and location are realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, and storage medium for detecting text content conflicts in multi-source heterogeneous knowledge bases. Background Technology

[0002] Knowledge bases are the core carriers for storing, organizing, and managing structured and unstructured knowledge, and are widely used in scenarios such as intelligent question answering, decision support, and information retrieval. With the rapid growth of multi-source heterogeneous data, large knowledge bases often integrate text content from different channels and formats. Conflict detection, as a key technology to ensure knowledge consistency, directly affects the practical value of knowledge bases.

[0003] Among existing knowledge base conflict detection technologies, knowledge graph-based conflict detection techniques extract entities, entity attributes, and relationships between entities through structured processing of text and construct a knowledge graph. Based on pre-defined logical rules for entity attributes within the graph, the consistency of entity attributes and the rationality of relationships corresponding to text in the knowledge base are verified, thereby identifying conflicting content. Text vectorization-based conflict detection techniques, on the other hand, convert text into high-dimensional vectors using pre-trained language models or traditional text feature engineering. The similarity between texts is measured by calculating metrics such as cosine similarity and Euclidean distance, and then combined with a pre-defined similarity threshold, batches of text are quickly screened to identify text pairs with low similarity that may indicate conflict.

[0004] However, existing technologies still suffer from several shortcomings when dealing with batch text conflict detection in multi-source heterogeneous knowledge bases. They lack effective means for processing the detection objects, making it difficult to efficiently screen and optimize the set of texts to be detected. Furthermore, the effectiveness of conflict detection and localization is insufficient, making it difficult to accurately identify semantic conflicts and locate specific conflicting texts. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention discloses a text content conflict detection method for multi-source heterogeneous knowledge bases, comprising the following steps:

[0006] S1. Establish an interactive link with the data storage carrier, obtain the text slice to be detected and the existing text slices in the knowledge base, and construct an initial set of conflict detection objects;

[0007] S2. Based on different preset filtering rules, the initial conflict detection object set is filtered in multiple levels to obtain a conflict candidate object set.

[0008] S3. Conflict detection is performed on the set of conflict candidate objects using a dual-thread conflict detection method; Thread 1 outputs conflicting text units in the existing slice set through a conflict-aware inference model; Thread 2 uses a text segmentation processor based on context-aware rules to perform sentence-level segmentation on the set of conflict candidate objects and outputs potential conflict pairs at the sentence level.

[0009] S4. After segmenting and cleaning the content of the conflicting text units using a preset text extraction strategy, a list of conflicting text units is obtained. The list of conflicting text units is then sequentially matched with the potential conflict pairs to calculate the matching rate. The final conflict pair is determined based on the matching rate, and a conflict pair detection result containing location information is output.

[0010] S5. Update the conflict status flag based on the conflict detection results and save the conflict record.

[0011] Preferably, in step S4, the location information includes new sentence index information, old sentence index information, and similarity information for the corresponding conflict pair.

[0012] Preferably, step S3 includes:

[0013] The set of conflict candidates is segmented at the sentence level based on context-aware rules, which include setting symbols to prevent segmentation, merging incorrectly segmented sentence fragments based on closing symbols, and filtering invalid sentences shorter than a set number of characters.

[0014] An improved sentence-level Levenstein algorithm is applied to the segmented sentences, and corresponding weights are configured based on character type. The old sentences in the text slices of the conflict candidate set that have the highest similarity to the new sentences in the text slices to be detected and whose similarity is higher than a set threshold are selected, and then formed potential conflict pairs with the corresponding new sentences.

[0015] Preferably, the conflict-aware reasoning model is configured to identify common descriptive subjects, detect conflict types at the numerical, state, and attribute levels, and output all conflicting text units that constitute a conflict between the text slices in the conflict candidate object set and the text slices to be detected.

[0016] Preferably, step S3 further includes:

[0017] If the conflict-aware reasoning model fails to detect any conflicting text units in the conflict candidate object set after identification, and the potential conflict pair analysis process has not yet been completed, then the potential conflict pair analysis process is terminated.

[0018] Preferably, after segmenting and cleaning the content of the conflicting text units using a preset text extraction strategy to obtain a list of conflicting text units, the list of conflicting text units is sequentially compared with the potential conflict pairs to calculate the matching rate, and the final conflict pair is determined based on the matching rate. The result of the conflict pair detection, including location information, is then output, specifically including:

[0019] The conflict candidate set composed of the potential conflict pairs is filtered. If the longest common substring of the new sentence and the old sentence of the potential conflict pair is themselves, the corresponding potential conflict pair is deleted from the conflict candidate set.

[0020] The content of the conflicting text units is segmented and extracted using regular expressions, and irrelevant characters are removed. An improved longest common substring algorithm is then used to calculate the matching rate Rm between the resulting list of conflicting text units and each potential conflict pair in the candidate conflict set.

[0021]

[0022] in, The length of the conflicting text string. Let be the weight of the i-th matching substring. The weight is the length of the substring, and different matching weights are assigned based on preset key semantic units;

[0023] The conflict pair with the highest matching rate in the conflict candidate set is selected as the final conflict pair detected this time, and the conflict pair detection result containing location information is output. The location information includes the new sentence index information, old sentence index information and similarity information of the corresponding conflict pair.

[0024] Preferably, selecting the conflict pair with the highest matching rate in the conflict candidate set as the final conflict pair detected this time, and outputting the conflict pair detection result containing location information, further includes:

[0025] The conflict pair with the highest matching rate in the conflict candidate set is selected. It is then determined whether the highest matching rate is greater than the corresponding dynamic threshold for matching rate. If it is greater, the conflict pair with the highest matching rate is taken as the final conflict pair detected this time; otherwise, it is not taken as the final conflict pair detected this time. The dynamic threshold for matching rate is determined based on the length of the conflict text, the distribution density of key semantic units in the text, and the structural similarity between the old and new sentences.

[0026] Preferably, the dynamic threshold for the matching rate is determined by the joint adjustment of the semantic density factor (SDF) and the structural similarity factor (SSF). for:

[0027] ;

[0028] ;

[0029] ;

[0030] in The total length of the conflicting text. The number of key semantic units identified. For semantic unit weights, Let j be the length of the j-th semantic unit; The difference in character distribution entropy between the conflicting text and the candidate sentence is obtained by performing character distribution statistics on the conflicting text and the candidate sentence respectively. The preset basic matching threshold, and These are the preset adjustment coefficients.

[0031] This invention also discloses a text content conflict detection system for multi-source heterogeneous knowledge bases, comprising:

[0032] The data interaction module is used to establish an interaction link with the data storage carrier, obtain the text slice to be detected and the existing text slices in the knowledge base, and construct an initial set of conflict detection objects.

[0033] A multi-level filtering module is used to perform multi-level filtering on the initial conflict detection object set based on different preset filtering rules to obtain a conflict candidate object set.

[0034] The output processing module is used to perform conflict detection on the set of conflict candidate objects through a dual-thread conflict detection method. Thread 1 outputs conflicting text units in the existing slice set through a conflict-aware inference model. Thread 2 uses a text segmentation processor based on context-aware rules to perform sentence-level segmentation on the set of conflict candidate objects and outputs potential conflict pairs at the sentence level.

[0035] The segmentation and localization module is used to segment and clean the content of the conflicting text units using a preset text extraction strategy to obtain a list of conflicting text units, calculate the matching rate between the list of conflicting text units and the potential conflicting pairs in turn, determine the final conflicting pair based on the matching rate, and output the conflicting pair detection result containing localization information.

[0036] The update record module updates the conflict status flag based on the conflict detection results and saves the conflict record.

[0037] The present invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0038] This invention discloses a method, system, and storage medium for text content conflict detection in multi-source heterogeneous knowledge bases. The method constructs an initial conflict detection object set by acquiring a text slice to be detected and existing text slices in the knowledge base. Based on preset filtering rules, the initial conflict detection object set is filtered at multiple levels to obtain a conflict candidate object set. This conflict candidate object set is input into a trained conflict-aware inference model, which outputs all conflicting text units that conflict with the existing text slice and the text slice to be detected. Simultaneously, the conflict candidate object set is segmented at the sentence level based on context-aware rules to obtain old sentences with the highest similarity to new sentences and whose similarity exceeds a set threshold, forming potential conflict pairs with the corresponding new sentences. A list of conflicting text units is obtained by segmenting and cleaning the content of the conflicting text units using a preset text extraction strategy. The matching rate of the conflicting text unit list is calculated sequentially with the potential conflict pairs, and the final conflict pair is determined based on the matching rate. The conflict pair detection result, including location information, is output. The conflict status flag is updated based on the conflict pair detection result, and the conflict record is saved. This method automatically skips obviously non-conflicting text pairs by using multiple different filtering rules, significantly reducing unnecessary detection attempts. Multi-threaded execution supports high-throughput processing, making it suitable for large-scale knowledge bases, effectively reducing waiting time and improving response speed. Based on sentence-level segmentation and large language model text understanding, combined with sentence-level similarity matching calculation, it achieves deep semantic conflict identification, effectively avoiding the omission of key conflict points.

[0039] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0041] Figure 1 This is a schematic diagram illustrating the specific process of a text content conflict detection method for multi-source heterogeneous knowledge bases disclosed in an embodiment of the present invention.

[0042] Figure 2 This is a schematic diagram of the specific process of step S104 disclosed in an embodiment of the present invention.

[0043] Figure 3 This is a schematic diagram of a conflict detection process disclosed in an embodiment of the present invention.

[0044] Figure 4 This is a schematic diagram of the specific process of step S3 disclosed in an embodiment of the present invention.

[0045] Figure 5 This is a schematic diagram of the structure of a text content conflict detection system for multi-source heterogeneous knowledge bases disclosed in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0047] Unless otherwise defined, the technical or scientific terms used herein should be understood in their ordinary sense by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms as used in this embodiment do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one.

[0048] In this embodiment, as Figure 1 As shown, a text content conflict detection method for multi-source heterogeneous knowledge bases is disclosed, which may include the following steps.

[0049] Step S1: Establish an interaction link with the data storage carrier, obtain the text slice to be detected and the existing text slices in the knowledge base, and construct an initial set of conflict detection objects.

[0050] Specifically, the data storage medium can be a structured database used to store knowledge base text data. An interaction link is established with this database through the SQLAlchemyORM framework to achieve structured access to the document table, tag table, text slice table, and conflict record table within the database. When a unique identifier of the text slice to be detected, such as the slice ID, is received, the content of the text slice to be detected and its associated metadata, such as creation time and the document identifier, are queried through this interaction link. At the same time, all existing text slices in the text slice table of the database whose creation time is earlier than the text slice to be detected are retrieved to form a set of existing text slices. The logical consistency of the initial detection objects is ensured by time range constraints to avoid interference from existing slices in the future time dimension to the current detection process. The text slice to be detected is associated and combined with the filtered existing text slices to construct an initial conflict detection object set containing the pairing of the text slice to be detected and existing slices, providing a basic data carrier for subsequent steps.

[0051] Step S2: Perform multi-level filtering on the initial conflict detection object set based on different preset filtering rules to obtain a conflict candidate object set.

[0052] In this step, the initial set of conflict detection objects is filtered layer by layer based on candidate screening elements and different preset screening rules. Objects with no possibility of conflict are eliminated to reduce the subsequent computational load, resulting in a set of conflict candidate objects. The candidate screening elements include conflict state information, text association features, and text similarity measurement methods. The different preset screening rules are configured as a multi-level filtering mechanism, specifically including progressive steps of tag domain filtering, conflict state filtering, and coarse-grained similarity filtering.

[0053] Specifically, the multi-level filtering mechanism includes: The first level is tag domain filtering, implemented based on a tag intersection indexing mechanism: semantic domain matching is performed on the associated tags of the slice to be detected and the existing slices. Only object pairs whose tags overlap in semantic domain are retained. For example, if the label of the slice to be detected is "judicial statistics" and the label of the existing slice is "judicial human resources," both belong to the same semantic domain of the judicial field and are retained. At the same time, cross-domain irrelevant object pairs are excluded, such as medical data slices and judicial statistics slices, thus narrowing the detection scope from a semantic category perspective. The second level is conflict state filtering, performed based on the conflict state markers in the conflict record table: conflict state markers include five preset states: not detected, detected without conflict, detected with conflict but unresolved, resolved, and detection failed. Only object pairs with the state of not detected or detection failed are selected. Incremental detection is achieved through state markers, avoiding repeated processing of objects that have already undergone effective detection. The third level is coarse-grained similarity filtering, using a slice-level Levenstein algorithm to calculate the edit distance between the slice to be detected and the existing slice in the object pair, and converting this distance into a similarity ratio. The calculation formula is:

[0054] ;

[0055] Where Distance(A,B) is the edit distance between strings A and B, that is, the number of operations required to transform string A into B through character insertion, deletion, and replacement. Max(len(A),len(B)) is the maximum length of the two strings; if Levenshtein is 1, it indicates a complete match, that is, the two strings are the same; if Levenshtein is 0, it indicates a complete mismatch, that is, the two strings are completely different; at the same time, a very low similarity threshold is set, such as <0.05. When the similarity ratio of the object pair is lower than this threshold, it is judged as obviously non-conflicting and is removed; after the above three-level filtering, the remaining object pairs that meet the three elements of overlapping label domains, undetected / failed detection state, and similarity higher than the threshold constitute the conflict candidate object set.

[0056] Step S3: Conflict detection is performed on the set of conflict candidate objects using a dual-thread conflict detection method; Thread 1 outputs conflicting text units in the existing slice set through a conflict-aware inference model; Thread 2 uses a context-aware rule-based text segmentation processor to perform sentence-level segmentation on the set of conflict candidate objects and outputs potential conflict pairs at the sentence level.

[0057] In this embodiment, the conflict candidate set can be input into a pre-trained conflict-aware inference model, which outputs all conflicting text units that constitute a conflict between the existing text slice and the text slice to be detected. Simultaneously, the conflict candidate set is segmented at the sentence level based on context-aware rules to obtain the old sentences of the existing text slice in the conflict candidate set that have the highest similarity to the new sentences in the text slice to be detected within the conflict candidate set and whose similarity is higher than a set threshold. These old sentences are then paired with the corresponding new sentences to form potential conflict pairs.

[0058] In this step, detection operations can be performed on the set of conflict candidate objects from both semantic and syntactic matching perspectives. A dual-threaded parallel architecture is adopted to identify conflicting text units and generate potential conflict pairs. The dual-threaded parallel architecture includes Thread 1 and Thread 2.

[0059] Thread 1 is used for semantic-level conflict determination. It inputs the set of conflict candidate objects into a conflict-aware inference model trained on the general-purpose large language model Qwen3 through a large-model conflict determiner. This model employs conflict context-aware prompt templates and structured output constraints. The inference process strictly follows multi-level decision logic and outputs all conflicting text units in the existing slices that conflict with the slice to be detected. The training and inference process of this model is as follows:

[0060] Step S101: Extract historical text slice pairs from multi-source heterogeneous knowledge bases, construct training samples by combining manual annotation and rule-based selection, and introduce domain experts to quantify the conflict intensity of conflict text pairs.

[0061] Specifically, historical text slice pairs are extracted from a multi-source heterogeneous knowledge base. Training samples are constructed through a combination of manual annotation and rule-based selection. The constructed training sample set covers four predefined conflict types: numerical conflict, temporal conflict, state conflict, and attribute conflict, ensuring comprehensive coverage and representativeness of conflict features across multiple scenarios. Each training sample integrates new slice text, old slice text, contextual labels, and structured conflict labels. The structured conflict labels contain conflict type and conflict text location information, providing fine-grained semantic features and location annotation support for the model. Simultaneously, domain experts are introduced to quantify the conflict intensity of conflict text pairs, setting the scoring dimension range to 1 to 5 points, thereby achieving measurable and structured representation of conflict intensity. This standardized quantified scoring result is further used as a weighted adjustment factor for the loss function during model training. By assigning differentiated weights to conflict samples of different intensities, the model focuses on learning high-priority conflict samples during training, improving the model's perception and fitting ability regarding conflict intensity dimensions.

[0062] Step S102: Configure the conflict context-aware prompt template to construct the model input paradigm. The model input paradigm is defined within the conflict context-aware prompt template to limit the structured output constraints, context information integration, and conflict type boundaries.

[0063] Specifically, when setting structured output constraints within the conflict context-aware prompt template, the design must be based on the subsequent processing requirements of the model's inference results. Fixed JSON format output constraints must be defined, explicitly requiring that the inference results generated by the model adhere to a preset field structure, such as including key fields like conflict presence identifier, conflict type, conflict text fragment, and confidence level. Simultaneously, the data type and format specifications of each field must be enforced through template rules; for example, the conflict type field can only accept preset enumerated values, and the confidence level field must retain two decimal places. This constraint ensures the consistency and parsability of the model's output results, thereby preventing data processing anomalies in subsequent conflict localization and record storage due to inconsistent output formats.

[0064] The context information integration involves adding contextual aids and hints. Document tags and slice positions are embedded in the conflict context-aware hint template, along with contextual information such as document tags and slice positions. This enhances the model's understanding of conflict semantics and integrates it as a fixed input into the template. The document tag field must be associated with the semantic domain classification information of the text slice, providing the model with a basic semantic background of the text's domain. The slice position field must include the start and end line numbers of the text slice in the original document, as well as the chapter identifier, clarifying the spatial location of the text slice in the knowledge base. The embedding of both types of fields must use a standardized key-value pair format to ensure that the model can directly parse and associate them with the corresponding text slice content, thereby providing data support for the model to understand the text semantic scenario and eliminate cross-domain irrelevant conflict interference.

[0065] Configure a conflict type limitation mechanism in the conflict context-aware prompt template to define the boundaries of conflict types. Clearly define the model's conflict type recognition boundaries by pre-setting selectable output ranges. Within the template, explicitly limit the model to output only four predefined conflict types: numerical conflict, temporal conflict, state conflict, and attribute conflict. Use rules to restrict the model from outputting irrelevant semantic conflict types outside these four categories. Simultaneously, add definitional explanations of conflict types to the template, such as numerical conflict referring to inconsistent numerical representations of the same indicator in the text, to ensure the model accurately understands the semantic boundaries of each conflict type. This prevents the model from outputting conflict type results unrelated to the detection target due to semantic generalization.

[0066] Step S103: Optimize model parameters using a multi-task collaborative training framework and integrate parallel optimization task modules during model training. The parallel optimization task modules include: a conflict association subject consistency determination module, a conflict type semantic classification module, a conflict text unit localization module, and a conflict text unit localization module.

[0067] Specifically, the conflict existence determination module calculates entity alignment and semantic similarity to output a binary classification result indicating whether the old and new text slices point to the same descriptive subject. The loss value of this module is... A binary cross-entropy algorithm is used to calculate the deviation between the subject determination result and the labeled true value. The conflict type identification module only activates when the subjects are consistent and conflict exists. Based on four predefined conflict labels (numerical, time, state, and attribute), a multi-classifier outputs the probability distribution of conflict types, and the module loss value is calculated. A multi-class cross-entropy algorithm is used to optimize the accuracy of type recognition. The conflict text localization module uses the BIO annotation system and a sequence labeling model to delineate the boundaries of conflict segments in the text slice, outputting the start and end position indices of the conflicting units in the original text. The module loss value is also calculated. The sequence labeling cross-entropy algorithm is used to calculate and minimize the difference between the localization boundary and the labeling results; the conflict intensity prediction module outputs the intensity prediction value based on the conflict intensity quantification score labeled by domain experts, and the module loss value. The mean squared error algorithm is used to reduce the error between the predicted value and the domain expert score.

[0068] During training, a multi-task loss function is used, as shown in the following formula:

[0069]

[0070] in, The loss for conflict existence judgment is calculated using binary cross-entropy. To calculate the conflict type identification loss using multi-class cross-entropy, The collision location identification loss is calculated using sequence label cross-entropy. This represents the predicted loss based on the mean square error of the conflict intensity. , , and The task weight coefficient is determined by optimization of the validation set.

[0071] In step S104, the model strictly follows the multi-level conflict determination logic to make inferences and determinations, identify common descriptive subjects, detect conflict types at the numerical, state, and attribute levels, and output all semantic units in the existing slice collection that conflict with the new slice.

[0072] Specifically, such as Figure 2As shown, the multi-stage conflict determination logic includes: subject determination stage, conflict detection stage, type confirmation stage, and unit output stage. The first stage is the subject determination stage, where the model extracts core entities such as person names, indicator names, and event names from new and existing slices and calculates entity similarity, combining this with contextual relevance to determine if they describe the same subject. If the determination result is that they are not the same subject, the subsequent reasoning for that slice pair is terminated and marked as conflict-free; if the determination result is that they are the same subject, the next stage of determination is initiated. The second stage is the conflict detection stage, which, for text slice pairs with the same subject, performs conflict detection from three dimensions: numerical, state, and attribute. The numerical dimension extracts numerical data from the text using regular expressions and compares the numerical differences of the same indicator in the new and existing slices. The state dimension identifies the subject's state description through semantic role labeling, such as "completed" and "not started," and determines whether there is a contradiction. The attribute dimension detects inconsistencies in attribute values ​​through attribute extraction. If any dimension has a contradiction, it is determined to be a conflict, and the next stage of processing is initiated. If all dimensions... If all dimensions are consistent, the system is marked as conflict-free and the reasoning process terminates. The third stage is the type confirmation stage. Based on the conflict dimensions detected in the second stage, the conflicts are mapped to four predefined conflict types: numerical dimension conflicts are classified as numerical conflicts, state dimension conflicts are classified as state conflicts, attribute dimension conflicts are classified as attribute conflicts, and time dimension conflicts are detected synchronously in the second stage and classified as time conflicts. At the same time, the key features of the conflict dimensions, such as the numerical difference and the state contradiction point, are recorded. The fourth stage is the unit output stage. Based on the conflict dimensions detected in the second stage, the specific semantic units that generate conflicts in the text slices are located, and the position index of the unit in the new slice and the existing slice is output. These are then integrated into a structured set of conflict semantic units. Finally, all semantic units in the set that conflict with the new slice are output in JSON format, including key fields such as conflict type, conflict unit content, position index, and conflict dimension, providing basic data for subsequent conflict pair matching.

[0073] Thread 2 is used for filtering potential conflict pairs at the grammatical level. The text segmentation processor performs sentence-level segmentation on the conflict candidate object set based on context-aware rules, and obtains the old sentences of the text slices in the memory size of the conflict candidate object set that have the highest similarity to the new sentences in the text slices to be detected within the conflict candidate object set and whose similarity is higher than a set threshold. These old sentences are then combined with the corresponding new sentences to form potential conflict pairs.

[0074] Specifically, a text segmentation processor is used to perform sentence-level segmentation on the conflict candidate set, and the segmentation process deeply integrates context-aware rules to ensure sentence segmentation accuracy. The text segmentation processor uses a preset set of terminating punctuation marks as the basic segmentation delimiters, traversing the character sequence of each text slice in the conflict candidate set. When a matching terminating punctuation mark is detected, it is initially marked as a sentence breakpoint and an initial sentence list is generated. An improved sentence-level Levenstein algorithm is used to calculate the similarity between the new sentence set and all sentence pairs in the old sentence set, and the old sentence with the highest similarity to the new sentence is obtained. This algorithm improves the matching sensitivity of key information by introducing a character weight mechanism. The calculation formula of the improved sentence-level Levenstein algorithm is defined as follows:

[0075] ;

[0076] in, and These are the characters before and after the operation, respectively. The basic cost of insertion, deletion, and replacement operations is uniformly set to 1. The character type weighting factor is set as follows: if the character type is ordinary text, it is set to 1; if the character type is proper nouns such as personal names or place names, it is set to 1.3.

[0077] In this embodiment, as Figure 4 As shown, step S3 can specifically include the following:

[0078] Step S201: Segment the conflict candidate object set at the sentence level based on context-aware rules. The context-aware rules include setting symbols not to be segmented, merging incorrectly segmented sentence segments according to closing symbols, and filtering invalid sentences shorter than a set number of characters.

[0079] Specifically, multi-level optimization processing can be performed on the initial sentence list based on context-aware rules. This multi-level optimization includes: number format protection optimization, quotation mark closure and merging optimization, and invalid sentence filtering optimization. Specifically, number format protection optimization identifies numbers containing periods (.) in the text, such as the date 2023.12.31, marking these periods as non-delimiters and canceling the initial sentence breakpoints at the corresponding positions to prevent incorrect number segmentation. Quotation mark closure and merging optimization uses a stack structure to track the opening and closing states of quotation marks in the text. When an initial sentence is detected to begin with a left quotation mark but not a right quotation mark, and the next adjacent sentence ends with a right quotation mark, it is determined to be a sentence enclosed in quotation marks across breakpoints, and a sentence merging operation is performed to restore sentence integrity. Invalid sentence filtering optimization counts the number of valid characters in each sentence after the first two optimization steps, excluding spaces, tabs, and other whitespace characters. When the number of valid characters is less than a preset threshold, it is determined to be an invalid sentence, such as isolated punctuation marks, and a filtering and deletion operation is performed. Through the above multi-level optimization, we obtain a new set of sentences corresponding to the text slice to be detected and an old set of sentences corresponding to the existing text slice. The sentences in both sets have semantic integrity and format standardization.

[0080] Step S202: Apply the improved sentence-level Levenstein algorithm to the segmented sentences and configure corresponding weights based on character type to filter out the old sentences in the text slices of the conflict candidate set that have the highest similarity to the new sentences in the text slices to be detected within the conflict candidate set and whose similarity is higher than a set threshold, and form potential conflict pairs with the corresponding new sentences.

[0081] An improved sentence-level Levenstein algorithm is applied to the segmented sentences, and the edit distance is calculated by combining the corresponding weights configured according to the character type. The edit distance is used to determine the similarity of sentence pairs. Each old sentence pair with the highest similarity to the sentence unit in the new slice and whose similarity is lower than a set threshold is identified as a "potential conflict pair" of the new sentence and enters the subsequent conflict localization process. Specifically, for each new sentence in the new sentence set, the similarity data of all its corresponding old sentences are extracted. The old sentences are sorted from high to low similarity, and the old sentence at the top of the sort is selected as the optimal matching old sentence of the new sentence. The similarity of the optimal matching old sentence is compared with a preset similarity threshold. This threshold is determined based on the performance tuning of the validation set and can be dynamically adjusted according to the text type of the knowledge base. If the similarity is higher than the set threshold, it indicates that the new sentence and the optimal matching old sentence have a high semantic correlation and a possibility of conflict, and they are determined to form a potential conflict pair. If the similarity is lower than the set threshold, it is determined that the semantic correlation between the two is too low and there is no possibility of conflict, and a potential conflict pair is not constructed. Finally, all new sentences that meet the conditions and the best matching old sentence pairs are integrated to form a structured set of potential conflict pairs. Each potential conflict pair in the set needs to be associated with the corresponding sentence source information, which includes the text slice ID to be detected and the existing text slice ID.

[0082] In this embodiment, the conflict-aware reasoning model is configured to identify common descriptive subjects, detect conflict types at the numerical, state, and attribute levels, and output all conflicting text units that constitute a conflict between the text slices in the conflict candidate object set and the text slices to be detected.

[0083] Specifically, the model performs cross-slice common description subject identification on existing text slices and text slices to be detected in the conflict candidate object set, anchors entity descriptions pointing to the same real object, and establishes cross-slice subject associations. On this basis, differentiated conflict type detection is deployed for three core conflict levels: numerical, state, and attribute. At the numerical level, numerical normalization is achieved through regular expression matching and semantic parsing rules, and the numerical descriptions of common subjects are compared and the numerical range is identified by combining predefined numerical constraint rules. At the state level, relying on a pre-constructed state transition knowledge graph, the state transition of common subjects is verified by graph traversal algorithm to check whether it conforms to the preset legality rules and identify state conflicts. At the attribute level, subject, attribute, and value triples are extracted based on sequence labeling and relation extraction models, and the conflict of the same attribute value is determined by combining the attribute constraint library. Finally, all conflicting text units that constitute a conflict between existing text slices and text slices to be detected are accurately located and output. Each unit contains structured fields such as a unique identifier of the conflicting subject, a conflict type label, and a corresponding text fragment.

[0084] In this embodiment, step S3 may also include the following:

[0085] If the conflict-aware reasoning model fails to detect any conflicting text units in the conflict candidate object set after identification, and the potential conflict pair analysis process has not yet been completed, then the potential conflict pair analysis process is terminated.

[0086] Specifically, this step can adopt a dual-threaded parallel architecture. Thread 1 is the conflict detection process of the conflict-aware inference model, responsible for performing conflict determination on the existing text slices and the text slices to be detected in the conflict candidate set, and outputting the detection result of whether there are conflicting text units. Thread 2 is the potential conflict pair analysis process, which undertakes sub-tasks such as sentence segmentation of the text segmentation processor and similarity calculation of the improved Levenstein algorithm of the conflict candidate filter, and is used to generate potential conflict pairs composed of new sentences and old sentences. Thread 1 and Thread 2 run synchronously, and are scheduled and managed by a thread synchronization control module, which dynamically coordinates the running state of the threads. If Thread 1, which uses the language model to determine conflict, completes first and determines that there is no conflict in the corresponding text slice combination, then the execution of Thread 2, which is used for text segmentation and conflict candidate set generation, is terminated.

[0087] Step S4: After segmenting and cleaning the content of the conflicting text units using a preset text extraction strategy, a list of conflicting text units is obtained. The list of conflicting text units is then sequentially matched with the potential conflicting pairs to calculate the matching rate. The final conflicting pair is determined based on the matching rate, and a conflicting pair detection result containing location information is output. The location information includes the new sentence index information, the old sentence index information, and the similarity information of the corresponding conflicting pair.

[0088] Specifically, based on a preset text extraction strategy and using regular expressions, the content of conflict text units output by the conflict-aware inference model is segmented and extracted. After capturing core semantic segments, redundant spaces, special symbols, and meaningless segments are removed, generating a list of conflict text units with associated conflict type labels. The preset text extraction strategy includes preset regular rules and preset types and weights for key semantic unit extraction. The regular rules are applied to the regular expression calculation process. An improved longest common substring algorithm is called, and numbers, times, and proper nouns are predefined as key semantic units to be extracted first, with fixed weights assigned to them. Each unit in the list is sequentially matched with the potential conflict pairs to calculate the matching rate. The potential conflict pairs with the highest matching rate are selected as the final conflict pairs. The conflict locator integrates the location information, including the index of the new sentence in the slice to be detected, the index of the old sentence in the existing slice, and sentence-level weighted similarity, and encapsulates it into a conflict pair detection result in a structured format and outputs it, providing data support for subsequent conflict records and status updates.

[0089] In this embodiment, step S4 may specifically include the following:

[0090] Step S41: Filter the conflict candidate set composed of the potential conflict pairs. If the longest common substring of the new sentence and the old sentence of the potential conflict pair is themselves, then delete the corresponding potential conflict pair from the conflict candidate set.

[0091] The maximum common substring length of the new sentence and the old sentence in each potential conflict pair is calculated using a string substring matching algorithm. If the calculation result shows that the maximum common substring length of the new sentence and the old sentence is equal to the length of the new sentence itself (i.e., the new sentence is a complete substring of the old sentence), or equal to the length of the old sentence itself (i.e., the old sentence is a complete substring of the new sentence), then the potential conflict pair is determined to be an invalid potential conflict pair. Such conflict pairs only have differences in expression length and no semantic contradiction. The candidate set is further updated, and such invalid potential conflict pairs are removed from the conflict candidate set to ensure that all potential conflict pairs in the remaining candidate set have a semantic difference basis, thereby reducing invalid data interference for subsequent matching rate calculations.

[0092] Step S42: Use regular expressions to segment and extract the content of the conflicting text units, and clean up irrelevant characters; use an improved longest common substring algorithm to calculate the matching rate Rm between the obtained list of conflicting text units and each potential conflict pair in the conflict candidate set.

[0093]

[0094] in, The length of the conflicting text string. Let be the weight of the i-th matching substring. The weight is the length of the substring, and different matching weights are assigned based on preset key semantic units;

[0095] Step S42 specifically includes structured processing of conflicting text units and calculation of matching rate. The conflicting text unit processing uses regular expressions to segment and extract the content of each conflicting text unit, capturing the core semantic fragments of each unit. Simultaneously, irrelevant characters are cleaned, redundant spaces, special symbols, and meaningless short characters are removed from the fragments, generating a semantically complete and uniformly formatted list of conflicting text units. Each unit in the list is associated with its corresponding conflict type tag. The matching rate calculation uses an improved longest common substring algorithm, sequentially matching each unit in the conflicting text unit list with each potential conflict pair in the conflict candidate set. That is, the conflicting unit is matched with both the new and old sentences of the potential conflict pair. Differential weights are assigned to different types of substrings according to preset rules, with higher matching weights given to key semantic units such as numbers, time, and proper nouns, improving the positioning accuracy of the conflicting text. A higher matching rate indicates a stronger semantic correlation between the conflicting text unit and the potential conflict pair.

[0096] Step S43: Select the conflict pair with the highest matching rate in the conflict candidate set as the final conflict pair detected this time, and output the conflict pair detection result containing location information. The location information includes the new sentence index information, old sentence index information and similarity information of the corresponding conflict pair.

[0097] Specifically, the matching rate of each pair of sentences in the conflict candidate set is calculated based on the improved longest common substring algorithm. And the matching rate for all candidate conflict pairs The conflict pairs are sorted in descending order, and the highest-ranking pair with the highest matching rate is selected. This conflict pair represents the sentence pair in the candidate set that has the strongest semantic association and the highest structural similarity with the conflicting text extracted by the model. The location information includes the new sentence index information, the old sentence index information, and the similarity information for the corresponding conflict pair. Specifically, the new sentence index information is the position number of the conflicting sentence in the sentence sequence of the new slice after it has been segmented by the text segmentation processor. For example, if the new slice results in 10 sentences and the conflicting sentence is the 3rd sentence, the index is 3. The old sentence index information is the position number of the conflicting sentence in the sentence sequence of the old slice after it has been segmented using the same segmentation rules. For example, if the old slice results in 8 sentences and the conflicting sentence is the 5th sentence, the index is 5. The similarity information is the matching rate of the conflict pair. The specific numerical value is used to quantify the similarity of conflicting texts, providing a reference for subsequent conflict priority determination.

[0098] In this embodiment, step S43 may further include: selecting the conflict pair with the highest matching rate in the conflict candidate set, determining whether the highest matching rate is greater than the corresponding dynamic matching rate threshold, and if it is greater, then taking the conflict pair with the highest matching rate as the final conflict pair detected this time; otherwise, not taking it as the final conflict pair detected this time; the dynamic matching rate threshold is determined based on the conflict text length, the distribution density of key semantic units in the text, and the structural similarity between the old and new sentences.

[0099] Specifically, from the output set of conflict candidates, the conflict pair with the highest matching rate is selected, and the highest matching rate of that conflict pair is obtained. The system records the highest matching rate of the conflict pair; it calculates the dynamic threshold of the matching rate corresponding to the conflict pair, which is determined based on three factors: the length of the conflicting text, i.e., the total number of characters of the conflicting text units extracted from the model output through regular expressions; the distribution density of key semantic units in the text, quantified by the semantic density factor SDF; and the structural similarity between the old and new sentences, quantified by the structural similarity factor SSF. The highest matching rate is compared with the calculated dynamic threshold of the matching rate. If the highest matching rate is greater than the dynamic threshold of the matching rate, the conflict pair is determined to be the final conflict pair detected in this test. If the highest matching rate is less than or equal to the dynamic threshold of the matching rate, the conflict pair is determined to be a pseudo-conflict and is not included in the final conflict pair results detected in this test.

[0100] In this embodiment, the dynamic threshold for the matching rate is determined by the semantic density factor (SDF) and the structural similarity factor (SSF). for:

[0101] ;

[0102] ;

[0103] ;

[0104] in The total length of the conflicting text. The number of key semantic units identified. For semantic unit weights, Let j be the length of the j-th semantic unit; The difference in character distribution entropy between the conflicting text and the candidate sentence is obtained by performing character distribution statistics on the conflicting text and the candidate sentence respectively. The preset basic matching threshold, and These are the preset adjustment coefficients.

[0105] Specifically, a dual evaluation based on text semantic density and structural similarity is proposed to achieve intelligent control of the matching rate threshold. A two-dimensional dynamic threshold adjustment algorithm is then developed: the first dimension is the semantic density factor (SDF), calculated using the following formula:

[0106] A higher SDF indicates denser text semantics, so the matching threshold should be lowered to improve recognition sensitivity; a lower SDF indicates sparser semantics, so the matching threshold should be increased to prevent false matches.

[0107] Dimension two is the structural similarity factor (SSF). The SSF first performs character distribution statistics on the conflicting text and the candidate sentence respectively, and obtains the difference in character distribution entropy between the two. And combined with the aforementioned improved matching rate The final SDF value is obtained using the following formula:

[0108]

[0109] A higher SSF indicates greater structural similarity, requiring a lower matching threshold; a lower SSF indicates greater structural difference, requiring a higher matching threshold to prevent false positives. Combining the above two-dimensional factors, the dynamic threshold calculation formula is as follows:

[0110]

[0111] In another embodiment, to address the issue of large language model service calls being susceptible to network fluctuations or server-side limitations, this method integrates an intelligent API rate controller. The controller employs a sliding window strategy to monitor the request volume per unit time in real time. When the request approaches a preset threshold, it automatically calculates the remaining window time and introduces a controllable sleep mode to smooth request traffic. Simultaneously, an exponential backoff retry mechanism is implemented to handle abnormal server responses: after the initial failure, a 1-second delay is applied before retrying, and the delay time for each subsequent retry increases exponentially, as shown in the following formula: Where T is the delay time, is the initial delay time, and n is the number of retries.

[0112] Step S5: Update the conflict status flag and save the conflict record based on the conflict detection results.

[0113] Specifically, based on the conflict detection results, three types of structured update operations are performed on the knowledge base associated data: First, the conflict status flag of the text slice to be detected is updated, and its status is set to "not detected" or "in detection" in the text slice table to clarify its detection completion status and avoid duplicate processing; Second, for the final conflict pair confirmed in step S4, a standardized five-tuple structured record is written into the conflict record table. This five-tuple contains the text slice IDs of both conflicting parties, the existing text slice IDs, the index information of the corresponding conflicting sentences, the sentence-level similarity score, and the initial resolution status, which is set to "unresolved" by default, to achieve structured retention of conflict information; Third, the sentence segmentation cache data of the existing text slices is updated, and the sentence set and position index after segmentation are cached in a dedicated cache table of the database to avoid repeatedly calling the text segmentation processor to perform sentence segmentation operations during subsequent conflict detection, thereby reducing computational redundancy.

[0114] The data update process utilizes the SQLAlchemy framework to implement database session management and atomic commit mechanisms. This mechanism is designed based on the ACID properties of database transactions: the transaction isolation level is configured as READ COMMITTED, ensuring that in multi-threaded concurrent conflict detection scenarios, each session can only read committed transaction data, avoiding detection bias caused by reading uncommitted intermediate state data; it supports unified transaction encapsulation and commit for multi-table operations on document tables, text slice tables, tag tables, and conflict record tables, guaranteeing the atomicity of cross-table data updates; if an exception occurs during transaction execution, such as database network interruption, data write failure, or model call anomaly, the system automatically triggers a transaction rollback mechanism, restoring all executed update operations to the state before the transaction started, preventing data inconsistency; simultaneously, detailed logs are recorded for the execution time, operation objects, modified content, and execution results of each transaction operation, stored in a dedicated transaction log table, providing data support for subsequent conflict tracing audits and problem investigation.

[0115] After a transaction is committed, a conflict propagation graph mechanism is introduced to achieve dynamic consistency maintenance and incremental conflict detection of the knowledge base. Each text slice is mapped to a node in the graph. If two text slices are found to be in conflict, a directed edge is constructed between the corresponding nodes to mark the conflict association. When the resolution status of an existing conflict in the conflict record table is updated, or a new text slice is written to the knowledge base, the system uses the conflict node or new slice node corresponding to the status update as the starting point and traverses the conflict propagation graph structure using a breadth-first search algorithm to identify all associated nodes that are path-connected to the starting node. The text slices corresponding to these associated nodes together constitute the affected slice set. An incremental conflict detection process is initiated for the affected slice set, performing conflict detection only on slices within the set rather than full detection. This effectively avoids the waste of global detection resources caused by local data changes, ensuring the real-time performance and effectiveness of conflict management in a large-scale knowledge base environment, while preventing the destruction of global consistency of the knowledge base caused by local modifications.

[0116] In another embodiment, such as Figure 3As shown, the text content conflict detection process for multi-source heterogeneous knowledge bases specifically includes: filtering the old slice collection of the knowledge base based on slice edit distance and associated tags, wherein the slice edit distance is calculated based on the sentence-level Levenstein algorithm; performing multi-level filtering on new slices to be detected based on conflict status flags, discarding slices that do not meet the conditions to obtain filtered detection data; the filtered detection data is processed in parallel in two paths: one path is input to the large model conflict judge, if no conflict is determined, the detection process ends, if a conflict is determined, the conflict text is extracted by the conflict text extractor; the other path is input to the text sentence processor for sentence segmentation and the conflict candidate filter filters the conflict candidate objects; the results of both paths are input to the conflict locator to complete the conflict location and change the conflict status flag, and submit the detection results.

[0117] The text content conflict detection method for multi-source heterogeneous knowledge bases disclosed in the above embodiments constructs an initial conflict detection object set by acquiring the text slice to be detected and the existing text slices in the knowledge base; performs multi-level filtering on the initial conflict detection object set based on different preset filtering rules to obtain a conflict candidate object set; inputs the conflict candidate object set into a trained conflict-aware inference model, and outputs all conflicting text units that constitute a conflict between the existing text slice and the text slice to be detected; simultaneously, the conflict candidate object set is segmented at the sentence level based on context-aware rules to obtain old sentences with the highest similarity to new sentences and whose similarity is higher than a set threshold, and forms potential conflict pairs with the corresponding new sentences; after segmenting and cleaning the content of the conflicting text units using a preset text extraction strategy, a conflicting text unit list is obtained; the conflicting text unit list is sequentially matched with the potential conflict pairs to calculate the matching rate, and the final conflict pair is determined based on the matching rate, outputting the conflict pair detection result containing location information; and the conflict status flag is updated and the conflict record is saved based on the conflict pair detection result. It solves the problems of lack of effective means to process the detection objects, making it difficult to efficiently screen and optimize the set of texts to be detected; and the insufficient effectiveness of conflict detection and localization, making it difficult to accurately identify semantic conflicts and locate specific conflicting texts. It improves the accuracy of conflict detection and realizes deep semantic conflict identification and localization.

[0118] like Figure 5As shown, another embodiment discloses a text content conflict detection system for multi-source heterogeneous knowledge bases, including: a data interaction module 1, a multi-level filtering module 2, a processing output module 3, a segmentation and localization module 4, and an update record module 5. The data interaction module establishes an interaction link with the data storage carrier, acquires the text slice to be detected and the existing text slices in the knowledge base, and constructs an initial conflict detection object set. The multi-level filtering module performs multi-level filtering on the initial conflict detection object set based on preset different filtering rules to obtain a conflict candidate object set. The processing output module inputs the conflict candidate object set into a trained conflict-aware inference model, outputs all conflicting text units that constitute a conflict between the existing text slice and the text slice to be detected; simultaneously, it performs sentence-level segmentation on the conflict candidate object set based on context-aware rules, acquires the old sentences of the existing text slices in the conflict candidate object set that have the highest similarity to the new sentences in the text slice to be detected within the conflict candidate object set and whose similarity is higher than a set threshold, and forms potential conflict pairs with the corresponding new sentences. The segmentation and localization module uses a preset text extraction strategy to segment and clean the content of the conflicting text units to obtain a list of conflicting text units. The list of conflicting text units is then sequentially compared with the potential conflicting pairs to calculate the matching rate, and the final conflicting pair is determined based on the matching rate. The module outputs a conflicting pair detection result containing localization information, including the new sentence index information, old sentence index information, and similarity information of the corresponding conflicting pair. The update and record module is used to update the conflict status flag and save the conflict record based on the conflicting pair detection result.

[0119] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the schematic diagram generation system for multimodal data disclosed in the embodiments, since it corresponds to the schematic diagram generation method for multimodal data disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to the method section.

[0120] In other embodiments, a server is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the text content conflict detection method for multi-source heterogeneous knowledge bases as described in the above embodiments.

[0121] If the aforementioned text content conflict detection system for multi-source heterogeneous knowledge bases is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various embodiments of the text content conflict detection method for multi-source heterogeneous knowledge bases. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0123] In summary, the above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be covered by the present invention.

Claims

1. A method for detecting text content conflicts in a multi-source heterogeneous knowledge base, characterized in that, The method comprises the following steps: S1, establishing an interactive link with a data storage carrier, obtaining a to-be-detected text slice and an inventory text slice in a knowledge base, and constructing an initial conflict detection object set; S2, performing multi-level filtering on the initial conflict detection object set based on preset different screening filtering rules to obtain a conflict candidate object set; S3, performing conflict detection on the conflict candidate object set through a double-thread conflict detection mode; Thread one outputs a conflict text unit in the existing slice set through a conflict awareness reasoning model; thread two performs sentence-level segmentation on the conflict candidate object set through a text sentence processing device based on a context awareness rule, and outputs a potential conflict pair at a sentence level; S4, obtaining a conflict text unit list after segmenting and cleaning the content of the conflict text unit through a preset text extraction strategy, calculating a matching rate of the conflict text unit list with the potential conflict pair in sequence, and determining a final conflict pair according to the matching rate, and outputting a conflict pair detection result containing positioning information; S5, updating a conflict state flag and saving a conflict record according to the conflict pair detection result.

2. The method for detecting text content conflicts of multi-source heterogeneous knowledge base according to claim 1, characterized in that, In step S4, the positioning information includes new sentence index information, old sentence index information and similarity information of the corresponding conflict pair.

3. The method for detecting text content conflicts of multi-source heterogeneous knowledge base according to claim 1, characterized in that, The step S3 comprises: performing sentence-level segmentation on the conflict candidate object set based on a context awareness rule, wherein the context awareness rule comprises setting a symbol not to be segmented, merging a sentence fragment that is incorrectly segmented according to a closed symbol, and filtering an invalid sentence that is shorter than a set character number; using an improved sentence-level Levenshtein algorithm on the segmented sentence and combining character types to configure corresponding weights, screening an old sentence of the inventory text slice in the conflict candidate object set that has the highest similarity with a new sentence in the to-be-detected text slice in the conflict candidate object set and is higher than a set threshold, and forming a potential conflict pair with the corresponding new sentence.

4. The text content conflict detection method for a multi-source heterogeneous knowledge base according to claim 3, characterized in that: the conflict awareness reasoning model is configured to output all conflict text units in which the inventory text slice in the conflict candidate object set and the to-be-detected text slice constitute a conflict by identifying a common description subject, and further detecting conflict types at a numerical, state and attribute level.

5. The method for detecting text content conflicts of multi-source heterogeneous knowledge base according to claim 4, characterized in that, The step S3 further comprises: if the conflict awareness reasoning model does not detect a conflict text unit in the conflict candidate object set after identification, and the potential conflict pair analysis process is not yet completed, the potential conflict pair analysis process is aborted.

6. The method for detecting text content conflicts of multi-source heterogeneous knowledge base according to claim 5, characterized in that, After segmenting and cleaning the content of the conflict text unit through a preset text extraction strategy, a conflict text unit list is obtained, a matching rate of the conflict text unit list with the potential conflict pair is calculated in sequence, and a final conflict pair is determined according to the matching rate, and a conflict pair detection result containing positioning information is output, which specifically comprises: screening a conflict candidate set composed of the potential conflict pair, and if the maximum common substring of the new sentence and the old sentence of the potential conflict pair is the two themselves, the corresponding potential conflict pair is deleted from the conflict candidate set; The conflict text unit content is segmented and extracted using a regular expression, and irrelevant characters are cleaned; the improved longest common substring algorithm is used to calculate the matching rate Rm between the conflict text unit list obtained after processing and each potential conflict pair in the conflict candidate set: ； wherein, is the length of the conflict text string, is the weight of the ith matching substring, is the length of the substring, and the weight is assigned different matching weights according to preset key semantic units. The conflict pair with the highest matching rate in the conflict candidate set is selected as the final conflict pair detected this time, and a conflict pair detection result containing positioning information is output, including new sentence index information, old sentence index information, and similarity information of the corresponding conflict pair.

7. The method for detecting text content conflicts of multi-source heterogeneous knowledge base according to claim 6, characterized in that, The conflict pair with the highest matching rate in the conflict candidate set is selected as the final conflict pair detected this time, and a conflict pair detection result containing positioning information is output, including new sentence index information, old sentence index information, and similarity information of the corresponding conflict pair. The conflict pair with the highest matching rate in the conflict candidate set is selected, and it is determined whether the highest matching rate is greater than the corresponding matching rate dynamic threshold. If it is greater, the conflict pair with the highest matching rate is taken as the final conflict pair detected this time, otherwise it is not taken as the final conflict pair detected this time. The matching rate dynamic threshold is determined based on the conflict text length, the distribution density of key semantic units in the text, and the structural similarity between the new and old sentences.

8. The text content conflict detection method for multi-source heterogeneous knowledge base according to claim 7, characterized in that: The matching rate dynamic threshold is determined by jointly regulating the semantic density factor SDF and the structural similarity factor SSF, and the matching rate dynamic threshold is: ； ； ； wherein is the total length of the conflict text, is the number of identified key semantic units, is the semantic unit weight, is the length of the jth semantic unit; is the difference between the character distribution entropy of the conflict text and the candidate sentence obtained by respectively performing character distribution statistics on the conflict text and the candidate sentence; is a preset basic matching threshold, and are respectively preset adjustment coefficients.

9. A text content conflict detection system for multi-source heterogeneous knowledge bases, characterized in that, It comprises: A data interaction module for establishing an interaction link with a data storage carrier, obtaining a text slice to be detected and a stock text slice in the knowledge base, and constructing an initial conflict detection object set; A multi-level filtering module for performing multi-level filtering on the initial conflict detection object set based on different preset filtering rules to obtain a conflict candidate object set; A processing output module for performing conflict detection on the conflict candidate object set through a double-thread conflict detection method; Thread one outputs conflict text units in the existing slice set through a conflict awareness reasoning model, and thread two uses a text sentence processor based on context awareness rules to perform sentence-level segmentation on the conflict candidate object set to output sentence-level potential conflict pairs; A segmentation positioning module for segmenting and cleaning the conflict text unit content using a preset text extraction strategy to obtain a conflict text unit list, and calculating the matching rate between the conflict text unit list and the potential conflict pairs in sequence and determining the final conflict pair according to the matching rate, and outputting a conflict pair detection result containing positioning information; An update record module for updating a conflict state flag and saving a conflict record according to the conflict pair detection result.

10. A computer-readable storage medium, the computer-readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the method of any one of claims 1-8.

Images