Multi-source data consistency management methods, devices, computer equipment and storage media

Abstract

This invention relates to the field of data management technology, specifically to a method, apparatus, computer equipment, and storage medium for multi-source data consistency management. The method includes: acquiring data records from at least two source systems to generate change events; converting the change events into evidence records containing original values ​​and contextual information based on a semantic rule registration library, and generating a standardized value view; performing entity parsing based on the standardized value view to obtain a set of candidate entity matching relationships, and outputting a credible candidate set through semantic equivalence transformation stability testing and evidence conflict measurement; performing constraint consistency inference based on the credible candidate set and the semantic rule registration library to generate a consistency state snapshot and a correction action plan; generating a synchronous execution rule set based on the correction action plan and performing a pre-rehearsal acceptance test; executing correction and re-collecting verification when successful, and rolling back when unsuccessful. This invention improves the traceability of cross-source consistency determination and the verifiability of correction execution, and reduces the risk of erroneous corrections.

Description

Technical Field

[0001] This invention relates to the field of data management technology, specifically to a method, apparatus, computer equipment, and storage medium for managing multi-source data consistency. Background Technology

[0002] In enterprise digital operations, key master data such as customers, materials, assets, and work orders are often scattered across multiple business systems, forming a multi-source factual system. Due to inconsistencies in field semantics, value formats, coding systems, and update rhythms, the same business entity is prone to duplication, conflict, and drift across different systems. This can then impact statistical accounting, risk control, compliance auditing, and cross-departmental collaboration, resulting in distorted indicators, process rework, and decision-making biases. Existing consistency management largely relies on static comparison tables or manual rule verification, which struggles to maintain a traceable chain of evidence and verifiable corrective measures in scenarios with frequent changes. Furthermore, corrective actions often lack a closed loop of pre-rehearsal and re-verification, leading to erroneous write-backs and cascading effects, resulting in continuously escalating governance costs. Summary of the Invention

[0003] This invention provides a method, apparatus, computer equipment, and storage medium for managing multi-source data consistency, which can at least solve the problems of multiple versions of the same entity, inconsistent semantic definitions, and lack of verifiable closed loops for correction and write-back in multi-source systems.

[0004] In a first aspect, the present invention provides a method for managing multi-source data consistency, the method comprising: Acquire data records from at least two source systems and generate change events. Based on the semantic rule registration library, convert the change events into evidence records. The evidence records include the original values ​​and context information, and generate a standardized value view from the evidence records. Based on the standardized value view, entity parsing is performed to obtain a set of candidate entity matching relationships. Then, the set of candidate entity matching relationships is subjected to semantic equivalence transformation stability test and evidence conflict measurement to output a set of credible candidates. Based on the trusted candidate set and semantic rule registration library, perform constraint consistency inference to generate a consistent state snapshot and corrective action plan; Based on the corrective action plan, a set of rules for synchronous execution is generated and a pre-rehearsal acceptance is performed. If the pre-rehearsal acceptance is passed, the corrective action is executed and the data is collected and verified again. If the pre-rehearsal acceptance is not passed or the data collection and verification fails, a rollback is executed.

[0005] In one possible implementation, converting change events into evidence records based on a semantic rule registry includes: writing a source system identifier, a source object identifier, a semantic field identifier, an event time, and a collection time into the evidence record, where the source system identifier indicates the source system to which the data record belongs, the source object identifier indicates the object corresponding to the data record, and the semantic field identifier indicates the semantics of the field corresponding to the original value; writing a data quality marker into the evidence record based on data format validation and value range validation, whereby the data quality marker indicates the validation result; and generating a standardized value view from the evidence record includes: standardizing the original value in combination with contextual information according to the standardization rules of the semantic rule registry to generate a standardized value view, and writing a standardization rule version identifier into the standardized value view.

[0006] In one possible implementation, entity resolution based on a standardized value view includes: generating a retrieval key based on the standardized value view, wherein the retrieval key is generated by combining standardized value views corresponding to at least two semantic fields; retrieving a set of candidate master entity identifiers in a master data repository based on the retrieval key, wherein the master data repository is used to store master entity identifiers and standardized value view indexes associated with master entity identifiers, wherein master entity identifiers are used to represent cross-source unified entities; and generating a set of candidate entity matching relationships based on the source object identifier and the set of candidate master entity identifiers.

[0007] In one possible implementation, the semantic equivalence transformation stability test includes: generating a transformed standardized value view from the standardized value view based on the semantic equivalence transformation rules in the semantic rule registry, wherein the semantic equivalence transformation rules are used to transform the standardized value view without changing the semantics of the fields; calculating a first similarity score based on the standardized value view within the candidate entity matching relationship set, and calculating a second similarity score based on the transformed standardized value view; determining a stability index based on the difference between the first similarity score and the second similarity score, and comparing the stability index with a preset threshold to eliminate candidate entity matching relationships in the candidate entity matching relationship set whose stability index exceeds the preset threshold; the evidence conflict measurement includes: generating a conflict count based on the mutual exclusion constraint rules in the semantic rule registry, wherein the mutual exclusion constraint rules are used to describe incompatible value combinations, and comparing the conflict count with a preset threshold to eliminate candidate entity matching relationships in the candidate entity matching relationship set whose conflict count exceeds the preset threshold.

[0008] In one possible implementation, constraint consistency inference includes: determining a set of consistency constraints based on a semantic rule registration library, the set of consistency constraints including uniqueness constraints and referential integrity constraints; determining a subset of trusted candidate sets that satisfy the set of consistency constraints from a set of trusted candidate sets to generate a consistency state snapshot; and generating a correction action plan based on the consistency state snapshot and the data records of the source system, the correction action plan including the scope of the correction object, the field to be modified, the target value, and the execution preconditions, the scope of the correction object being used to indicate the scope of data records for which correction actions need to be performed.

[0009] In one possible implementation, the pre-rehearsal acceptance includes: applying a set of synchronous execution rules and a corrective action plan in a simulated execution environment to generate a snapshot of the consistent state after the pre-rehearsal, and isolating the data records of the simulated execution environment and the source system; performing constraint consistency inference based on the snapshot of the consistent state after the pre-rehearsal to verify that the set of consistency constraints is satisfied, and verifying that the execution preconditions are satisfied; and determining that the pre-rehearsal acceptance is passed if the set of consistency constraints is satisfied and the execution preconditions are satisfied.

[0010] In one possible implementation, the re-acquisition verification includes: in response to the execution of a correction action, acquiring data records from the source system based on the scope of the correction object and generating post-execution change events; converting the post-execution change events into post-execution evidence records according to the semantic rule registry, and generating a post-execution standardized value view from the post-execution evidence records; performing entity parsing based on the post-execution standardized value view, and performing semantic equivalence transformation stability testing and evidence conflict measurement on the entity parsing results to output a post-execution credible candidate set; performing constraint consistency inference based on the post-execution credible candidate set and the semantic rule registry to verify that the consistency constraint set is satisfied; and performing a rollback if the consistency constraint set is not satisfied, the rollback including revoking the correction action or performing a compensatory correction action.

[0011] In a second aspect, the present invention provides a multi-source data consistency management device for implementing a multi-source data consistency management method. The device includes: an acquisition module, an evidence construction module, an entity parsing module, an inference module, and an execution module. The acquisition module is used to acquire data records from at least two source systems and generate change events; The evidence construction module is used to convert change events into evidence records based on the semantic rule registration library. The evidence records include the original values ​​and context information, and are used to generate a standardized value view from the evidence records. The entity parsing module is used to perform entity parsing based on the standardized value view to obtain a set of candidate entity matching relationships, and to perform semantic equivalence transformation stability test and evidence conflict measurement on the set of candidate entity matching relationships to output a set of credible candidates; The inference module is used to perform constraint consistency inference based on the trusted candidate set and the semantic rule registration library to generate a consistent state snapshot and a corrective action plan; The execution module is used to generate a set of synchronous execution rules based on the corrective action plan and to perform a pre-rehearsal acceptance. It is also used to execute corrective actions and re-collect and verify when the pre-rehearsal acceptance is passed, and to execute a rollback when the pre-rehearsal acceptance is not passed or the re-collection and verification fails.

[0012] Thirdly, the present invention provides a computer device, which includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, the computer device performs a multi-source data consistency management method.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a multi-source data consistency management method.

[0014] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: By constructing evidence records and generating standardized value views driven by a semantic rule registration library, we have achieved unified cross-source data standards and a traceable evidence chain. By combining entity parsing with semantic equivalence transformation stability testing and evidence conflict measurement, we have achieved robust screening of candidate matching relationships and reduced false matching. By inferring consistency constraint set constraints to generate consistency state snapshots and correction action plans, we have achieved calculable determination of correction targets and controllable scope of action. By synchronously executing rule sets and conducting pre-rehearsal acceptance and re-collection verification closed loops, we have achieved verifiability before correction write-back, reviewability after write-back, and rollback capability in case of failure to suppress cascading risks. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the execution flow of the method of the present invention; Figure 2 This is a diagram showing the similarity score distribution and reliable screening effect in a specific embodiment of the present invention; Figure 3 This is a graph showing the variation of the number of constraint violations in the correction loop in a specific embodiment of the present invention; Figure 4 This is a statistical chart of pre-review acceptance and re-collection verification in a specific embodiment of the present invention; Figure 5 This is a structural block diagram of the system of the present invention. Detailed Implementation

[0016] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0019] Consistency management refers to establishing a unified semantic interpretation, unified value expression, and unified constraint judgment for the same business entity and its key attributes under the condition of multiple data sources existing in parallel. This enables data from different systems to be compared, verified, and corrected within the same judgment framework, and to maintain traceability and reproducibility after correction. This management process typically includes three key tasks: First, establishing an executable rule system around field semantics, enabling original records to be converted into a unified representation; second, establishing cross-source associations around entity identity, enabling records from different sources to be merged into a unified entity and form candidate matches; and third, establishing inference and verification mechanisms around consistency constraints, ensuring that the merging results and correction actions have a verifiable correctness boundary. Based on the above basic requirements of consistency management, this invention proposes a closed-loop consistency management method oriented towards change events. Through evidence representation, robust candidate relationship screening, constraint inference and pre-rehearsal acceptance, and then collection verification and rollback mechanisms, it achieves a controllable closed loop of cross-source consistency from "judgment" to "execution".

[0020] like Figure 1 As shown, a multi-source data consistency management method includes: Acquire data records from at least two source systems and generate change events. Based on the semantic rule registration library, convert the change events into evidence records. The evidence records include the original values ​​and context information, and generate a standardized value view from the evidence records. In one embodiment, the system accesses data record streams from at least two source systems. Data records are received separately for each source system, with field parsing and basic deduplication performed upon receipt. For each newly added, updated, or deleted data record, a change event is generated. The change event carries at least the identifier of the data record that triggered the change, the change type, and the change time. Subsequently, the system reads the semantic rule registry and converts the change event into an evidence record. The evidence record aggregates the original value and contextual information, providing traceable source information for subsequent processing. Based on the evidence record, the system generates a standardized value view according to the standardized rules in the semantic rule registry. The standardized value view is output with unified field semantics and a unified value expression format, providing input for subsequent entity parsing and consistency inference.

[0021] Converting change events into evidence records based on the semantic rule registry includes: writing source system identifier, source object identifier, semantic field identifier, event time, and collection time into the evidence record. The source system identifier indicates the source system to which the data record belongs, the source object identifier indicates the corresponding object, and the semantic field identifier indicates the semantics of the field corresponding to the original value. Based on data format validation and value range validation, data quality markers are written into the evidence record to indicate the validation results. Generating a standardized value view from the evidence record includes: standardizing the original value and combining it with contextual information according to the standardization rules of the semantic rule registry to generate a standardized value view, and writing a standardization rule version identifier into the standardized value view.

[0022] In one embodiment, to clarify the source and processing boundaries of evidence records, after a change event enters the evidence construction process, a source system identifier, a source object identifier, a semantic field identifier, an event time, and a collection time are written into the evidence record. The source system identifier indicates the source system to which the data record belongs, the source object identifier indicates the corresponding object, and the semantic field identifier indicates the semantic meaning of the field corresponding to the original value. The event time is taken from the change time carried by the change event, and the collection time is taken from the local time when the system receives the data record, maintaining a consistent time format. The evidence record is written with the original value and context information. The context information includes at least the business type, field unit, field enumeration meaning, source channel type, and collection batch number, used to constrain the interpretation scope of subsequent standardized processing. To avoid redundancy caused by repeated reporting of the same source object within a short period, a deduplication fingerprint is written into the evidence record. The deduplication fingerprint is generated by combining the source system identifier, source object identifier, semantic field identifier, and the original value, and is used for deduplication judgment in evidence storage.

[0023] In one embodiment, after the evidence record is generated, data quality writing is performed. Data quality writing does not change the original value, but only adds a data quality marker. The data quality marker is generated by data format verification and value range verification. Data format verification checks whether the data type and encoding form of the original value meet the format constraints corresponding to the semantic field identifier. Value range verification checks whether the parsed value of the original value falls within a preset range. The data quality marker includes at least the format verification result and the value range verification result, and includes a failure reason code to distinguish between null values, format errors, out-of-bounds errors, and unparsable values. For data quality markers that fail verification, the evidence record still retains the original value and context information, and uses the verification result as an input constraint for subsequent processing to avoid breaking the chain of evidence.

[0024] In one embodiment, the generation of the standardized value view relies on standardized rules from a semantic rule registry. Standardized rules are configured according to semantic field identifiers and, in conjunction with context information, complete unit conversion and expression unification. Standardization processing converts the original values, combined with context information, into standardized values ​​with unified expressions, including unified units, unified precision, and unified enumeration encoding. Numerical fields undergo unit conversion and precision processing, date / time fields undergo time zone alignment and format normalization, and enumeration fields undergo alias mapping and canonical enumeration value mapping. The standardized value view is written with semantic field identifiers and standardized values, and also with a standardized rule version identifier. The standardized rule version identifier is taken from the effective version in the semantic rule registry and is used to reproduce the standardization logic during subsequent playback; when the semantic rule registry undergoes a version switch, newly generated standardized value views use the new version identifier, while existing standardized value views retain the original version identifier to maintain traceability and recalculation.

[0025] Based on the standardized value view, entity parsing is performed to obtain a set of candidate entity matching relationships. Then, the set of candidate entity matching relationships is subjected to semantic equivalence transformation stability test and evidence conflict measurement to output a set of credible candidates. In one embodiment, entity resolution is performed based on a standardized value view. For the same source object identifier, entity resolution retrieves possible corresponding master entity identifiers in the master data repository, forming a candidate master entity identifier set, and constructing a candidate entity matching relationship set accordingly. Subsequently, semantic equivalence transformation stability tests and evidence conflict measures are performed on the candidate entity matching relationship set to eliminate unstable or highly conflicting candidate entity matching relationships, resulting in a reliable candidate set, which serves as input for subsequent consistency inference.

[0026] Entity parsing based on standardized value views includes: generating a retrieval key based on the standardized value view, wherein the retrieval key is generated by combining standardized value views corresponding to at least two semantic fields; retrieving a set of candidate master entity identifiers in the master data repository based on the retrieval key, wherein the master data repository is used to store master entity identifiers and standardized value view indexes associated with master entity identifiers, and master entity identifiers are used to represent cross-source unified entities; and generating a set of candidate entity matching relationships based on the source object identifier and the set of candidate master entity identifiers.

[0027] In one embodiment, to make the retrieval boundaries of entity parsing clearer and to ensure that entity parsing can be stably implemented in engineering, the entity parsing process introduces a retrieval key and a master data repository index structure. The retrieval key is generated by combining standardized value views corresponding to at least two semantic fields. The combination method is defined by the semantic rule registry. The combination content preferentially selects fields with consistent and highly distinguishable semantics across different source systems, such as a combination of a unified coding field and a name field, or a combination of a unified coding field and a contact information field.

[0028] When generating search keys, standardized values ​​are first extracted from the standardized value view according to semantic field identifiers, and then search keys are constructed according to the concatenation order and delimiter rules given by the semantic rule registration library. When the standardized value view is missing a standardized value corresponding to a certain semantic field identifier, a missing placeholder is written into the search key, and the missing placeholder is also included in the search scope control to avoid the expansion of mismatches caused by missing values.

[0029] The master data repository stores master entity identifiers and associated normalized value view indexes, which represent cross-source unified entities. The normalized value view index includes at least an inverted index from the search key to the master entity identifier, and a forward index from the master entity identifier to the normalized values ​​corresponding to key semantic field identifiers. The inverted index is used to quickly locate the set of candidate master entity identifiers, while the forward index is used to quickly retrieve the normalized value view information from the master entity side during subsequent similarity calculations. During retrieval, the search key generated from the data record corresponding to the source object identifier is used as the query key to retrieve the set of candidate master entity identifiers from the inverted index. When the number of hits in the inverted index exceeds a preset limit, convergence is performed according to the field priority and matching leniency rules in the semantic rule registration library. For example, first, exact consistency is required for coded fields, then the normalization matching of name fields is relaxed, until the number of hits falls within a manageable range.

[0030] Subsequently, a candidate entity matching relationship set is generated based on the source object identifier and the candidate main entity identifier set. Each relationship in the candidate entity matching relationship set contains at least the pairing information of the source object identifier and the main entity identifier, and retains field-level matching evidence for subsequent evaluation, such as whether the normalized values ​​corresponding to each semantic field identifier are consistent, whether they are empty, and whether missing placeholders are triggered. Through the above structure, the input of entity parsing comes from the normalized value view, and the output is the candidate entity matching relationship set. Furthermore, the size of the candidate set is constrained by the search key and indexing strategy, which facilitates stable operation under different data scales.

[0031] The semantic equivalence transformation stability test includes: generating a transformed standardized value view based on the semantic equivalence transformation rules in the semantic rule registry, whereby the semantic equivalence transformation rules are used to transform the standardized value view without changing the semantics of the fields; calculating a first similarity score based on the standardized value view within the candidate entity matching relationship set, and calculating a second similarity score based on the transformed standardized value view; determining a stability index based on the difference between the first and second similarity scores, and comparing the stability index with a preset threshold to eliminate candidate entity matching relationships in the candidate entity matching relationship set whose stability index exceeds the preset threshold; and the evidence conflict measurement includes: generating a conflict count based on the mutual exclusion constraint rules in the semantic rule registry, whereby the mutual exclusion constraint rules are used to describe incompatible value combinations, and comparing the conflict count with a preset threshold to eliminate candidate entity matching relationships in the candidate entity matching relationship set whose conflict count exceeds the preset threshold.

[0032] In one embodiment, to ensure that the selection rules for the credible candidate set have verifiable criteria and to reduce the randomness caused by relying solely on a single similarity calculation, the candidate entity matching relationship set undergoes a semantic equivalence transformation stability test and an evidence conflict measurement before output. The semantic equivalence transformation stability test generates a standardized value view after transformation based on the semantic equivalence transformation rules in the semantic rule registry.

[0033] Semantic equivalence transformation rules are used to transform standardized value views without changing the semantics of the fields. Transformation methods include, but are not limited to, equivalent unit conversion and equivalent precision normalization for numeric fields, case normalization and whitespace character normalization for text fields, equivalent time zone alignment and format normalization for date and time fields, and alias normalization and normalized enumeration value mapping for enumeration fields. Stability testing calculates the first similarity score and the second similarity score within the candidate entity matching relationship set.

[0034] The first similarity score is calculated based on the standardized value view, and the second similarity score is calculated based on the transformed standardized value view. The similarity score is calculated using semantic field identifiers as the alignment unit. The standardized values ​​of the source object identifier and the main entity identifier are compared field by field to obtain field-level consistency and difference markers. These are then aggregated according to the field weights configured in the semantic rule registration library to obtain the similarity score. Field weights reflect the ability of different semantic field identifiers to distinguish entity consistency; coded fields have higher weights than descriptive fields, and the weight of fields corresponding to missing placeholder markers is automatically reduced to avoid excessive penalties caused by missing markers. A stability index is determined based on the difference between the first and second similarity scores. The stability index characterizes the sensitivity of candidate entity matching relationships to semantic equivalence transformations. When the stability index exceeds a preset threshold, it indicates that the similarity of the candidate entity matching relationship fluctuates too much under equivalence transformations, posing a risk of oversensitivity to the expression form. Therefore, this candidate entity matching relationship is removed from the candidate entity matching relationship set.

[0035] The evidence conflict metric is used to further filter candidate entity matching relationships from the perspective of "semantic incompatibility of fields". The evidence conflict metric generates conflict counts based on the mutual exclusion constraint rules in the semantic rule registry. Mutual exclusion constraint rules are used to describe incompatible value combinations. The granularity of the rules can cover incompatible values ​​within the same semantic field identifier, or incompatible combinations between multiple semantic field identifiers, such as the combination constraint of a status field and a time field, or the combination constraint of a category field and a code field.

[0036] The conflict count is generated as follows: for each candidate entity matching relationship, the mutual exclusion constraint rule is checked one by one to see if it has been triggered; for each triggered mutual exclusion constraint rule, the conflict count is incremented by one, and the set of semantic field identifiers and the combination of values ​​for triggering are recorded for subsequent traceability. The conflict count is compared with a preset threshold, and candidate entity matching relationships whose conflict count exceeds the preset threshold are removed. After completing the stability test and evidence conflict measurement, the remaining candidate entity matching relationships form a trusted candidate set. The trusted candidate set retains the source object identifier, main entity identifier, similarity score, stability index, conflict count, and trigger record summary, providing input and traceability basis for subsequent consistency inference.

[0037] Based on the trusted candidate set and semantic rule registration library, perform constraint consistency inference to generate a consistent state snapshot and corrective action plan; In one embodiment, the system takes a set of trusted candidates as input and performs constraint consistency inference in conjunction with a semantic rule registry. Constraint consistency inference uses "cross-source unified entity" as the basic unit, treating the relationship between source object identifiers and main entity identifiers in the trusted candidate set as candidate associations, and filtering for simultaneously valid association combinations within the constraint framework defined in the semantic rule registry. After filtering, a consistency state snapshot is generated, representing the correspondence and state of data records from each source system under a unified semantic caliber at the current moment. Subsequently, the system performs a difference analysis between the consistency state snapshot and the data records of the source systems, outputting a correction action plan. The correction action plan clarifies the scope of data records requiring correction, the semantics of the fields to be modified, and the target values, and provides preconditions for execution, serving as input for subsequent pre-rehearsal acceptance and execution phases.

[0038] Constraint consistency inference includes: determining a set of consistency constraints based on a semantic rule registration library, which includes uniqueness constraints and referential integrity constraints; determining a subset of trusted candidate sets that satisfy the set of consistency constraints from a set of trusted candidates to generate a consistency state snapshot; and generating a correction action plan based on the consistency state snapshot and the data records of the source system, which includes the scope of the correction object, the fields to be modified, the target values, and the preconditions for execution. The scope of the correction object is used to indicate the range of data records for which correction actions need to be performed.

[0039] In one embodiment, to make the boundaries of constraint consistency inference clearer and ensure that the corrective action plan has an executable generation path, constraint consistency inference first determines the set of consistency constraints by a semantic rule registry. The set of consistency constraints includes uniqueness constraints and referential integrity constraints. Uniqueness constraints are used to limit that a source object identifier under the same source system identifier corresponds to only one main entity identifier at any given time, or to limit that the same main entity identifier accepts only one source object identifier that satisfies a specific combination of semantic field identifiers at any given time, thereby avoiding entity drift caused by one-to-many or many-to-one relationships.

[0040] Referential integrity constraints are used to limit the closure of cross-object associations. For example, when a data record contains an association field pointing to another object, that association field must point to an existing and valid primary entity identifier under a unified semantic definition, and the key semantic field identifiers of the associated objects must satisfy preset existence conditions. For ease of engineering implementation, consistency constraint sets are stored in the form of rule items. Each rule item must at least include the constraint type, constraint scope, the set of participating semantic field identifiers, and violation judgment conditions. The constraint scope is used to limit the effective range of the constraint, which can be limited by business type, data record category, or source system identifier to avoid triggering constraints on irrelevant data.

[0041] In one embodiment, a subset of the trusted candidate set is determined from the trusted candidate set based on a set of consistency constraints. This process can adopt a step-by-step screening method: first, group the trusted candidate set according to the uniqueness constraint. The grouping key can be generated by combining the source system identifier and the source object identifier, or by combining the main entity identifier and the critical semantic field identifier. Within each group, preferentially retain the candidate entity matching relationships with higher similarity scores, smaller stability indicators, and lower conflict counts, and when there are ties, make a ruling according to the field priorities defined in the semantic rule registry, for example, preferentially satisfy the candidate entity matching relationships with consistent coding fields. After completing the uniqueness screening, an intermediate set is obtained.

[0042] Subsequently, apply the referential integrity constraint to the intermediate set. The check of the referential integrity constraint takes the associated field in the data record as the entry point, maps the associated field to the corresponding semantic field identifier, and determines whether there is an available main entity identifier for the associated target according to the intermediate set; if the associated target is missing or the critical semantic field identifier of the associated target does not meet the existence condition, it will cause the candidate entity matching relationship not to meet the referential integrity constraint.

[0043] For the candidate entity matching relationships that do not meet the referential integrity constraint, they can be excluded or downgraded according to the processing strategy configured in the semantic rule registry, for example, preferentially exclude when there are alternative candidates, and mark them as associated with pending rectification and retain them for subsequent plan generation when there are no alternative candidates. Finally, a subset of the trusted candidate set that meets the set of consistency constraints is determined, and a consistency status snapshot is generated based on this subset. The consistency status snapshot at least includes: the main entity identifier and its associated list of source object identifiers, the data record version information corresponding to each source object identifier, the unified value view of the critical semantic field identifier, and the summary of the verification results of the set of consistency constraints, which is used to support subsequent difference analysis and traceability verification.

[0044] In one embodiment, the rectification action plan is generated based on the consistency status snapshot and the data records of the source system. The difference analysis first determines the scope of the rectification object. The scope of the rectification object is described by a combination of the source system identifier, the source object identifier, and the semantic field identifier, and is used to indicate the range of data records for which rectification actions need to be performed. Subsequently, compare the data records of the source system with the unified value view in the consistency status snapshot field by field for the scope of the rectification object, and identify the fields to be modified and the target values. The fields to be modified are represented by semantic field identifiers to avoid ambiguity caused by differences in field names in different source systems. The target values are taken from the unified value view in the consistency status snapshot and maintain the same expression form as the Figure 1 standardized value view.

[0045] The corrective action plan also includes execution preconditions. These preconditions indicate the verification conditions before executing the corrective action, including but not limited to: unchanged data record versions, data quality markers meeting preset requirements, the existence of associated objects and satisfaction of referential integrity constraints, and the scope of the corrective object not being concurrently locked. The preconditions are written in the form of decidable condition items, each including condition type, condition parameters, and determination method, facilitating consistency verification before subsequent pre-rehearsal acceptance and actual execution. Through this process, constraint consistency inference outputs a consistency state snapshot and the corrective action plan, and the corrective action plan forms a continuous data connection with subsequent synchronous execution rule sets, pre-rehearsal acceptance, and re-collection verification.

[0046] Based on the corrective action plan, a set of rules for synchronous execution is generated and a pre-rehearsal acceptance is performed. If the pre-rehearsal acceptance is passed, the corrective action is executed and the data is collected and verified again. If the pre-rehearsal acceptance is not passed or the data collection and verification fails, a rollback is executed.

[0047] In one embodiment, after receiving the corrective action plan, the system generates a set of synchronous execution rules and performs a pre-rehearsal acceptance test before the actual write-back. The synchronous execution rules set constrains the scope, execution order, and pre-verification of the corrective actions, ensuring that the corrective actions are implemented uniformly across multiple source systems. After the pre-rehearsal acceptance test passes, the system executes the corrective actions according to the synchronous execution rules set and triggers a re-data collection verification after execution to confirm that the corrected data records satisfy the consistency constraint set under a unified semantic definition. If the pre-rehearsal acceptance test fails, the system does not execute the corrective actions and enters rollback processing; if the re-data collection verification fails, the system performs a rollback to restore the state before execution or uses compensation methods to eliminate inconsistencies.

[0048] The pre-rehearsal acceptance includes: applying the synchronous execution rule set and corrective action plan in the simulated execution environment, generating a snapshot of the consistent state after the pre-rehearsal, and isolating the data records of the simulated execution environment and the source system; performing constraint consistency inference based on the snapshot of the consistent state after the pre-rehearsal to verify that the consistency constraint set is satisfied, and verifying that the execution preconditions are satisfied; and determining that the pre-rehearsal acceptance is passed if the consistency constraint set is satisfied and the execution preconditions are satisfied.

[0049] In one embodiment, to ensure a verifiable decision path for the pre-rehearsal acceptance, the pre-rehearsal acceptance applies a synchronous execution rule set and corrective action plan in the simulated execution environment to generate a snapshot of the consistent state after the pre-rehearsal. Data records in the simulated execution environment and the source system are isolated, using methods including data snapshot copying and write interception. Data snapshot copying copies data records within the scope of the corrective action to the simulated execution environment, while write interception prevents the simulated execution environment from performing actual write-backs to the source system. When applying the synchronous execution rule set and corrective action plan, the execution order and grouping boundaries are first determined according to the synchronous execution rule set. The execution order handles cases where dependencies exist within the same corrective action scope, such as processing the referenced object before processing the referencing object. The grouping boundaries divide the corrective action scope into multiple execution batches, each batch applying corrective actions independently within the simulated execution environment to reduce interference during the pre-rehearsal process and facilitate the identification of failure causes. When applying corrective actions, each field to be modified is written according to the target value, and the value before writing is retained for subsequent rollback determination.

[0050] In one embodiment, after generating a post-rehearsal consistency state snapshot, the system performs constraint consistency inference based on the snapshot to verify that the consistency constraint set is satisfied. The verification process utilizes the uniqueness and referential integrity constraints of the consistency constraint set: for uniqueness constraints, it checks whether the same source object identifier in the post-rehearsal consistency state snapshot corresponds to multiple main entity identifiers, and whether the same main entity identifier is occupied mutually exclusively by multiple source object identifiers within a preset scope; for referential integrity constraints, it checks whether the main entity identifier pointed to by the associated field in the post-rehearsal consistency state snapshot exists and is valid, and checks whether the key semantic field identifiers of the associated objects satisfy the existence condition. Simultaneously, the system verifies that the execution preconditions are met. The verification of the execution preconditions includes data record version consistency verification and data quality mark verification. Data record version consistency verification confirms that the data record version within the scope of the correction object is consistent with the version used when generating the correction action plan, avoiding distortion of the pre-rehearsal results due to concurrent changes. Data quality mark verification confirms that the evidence record corresponding to the field to be written has not been marked as formatting error or out of bounds, avoiding writing obviously unreliable values ​​back to the source system. If the set of consistency constraints is satisfied and the preconditions for execution are met, the pre-rehearsal acceptance is deemed successful; otherwise, the pre-rehearsal acceptance is deemed unsuccessful, and a set of reasons for failure is output. The set of reasons for failure includes at least the triggered constraint type, the triggering semantic field identifier, and the associated source object identifier, which are used to subsequently select rollback or adjust the corrective action plan.

[0051] The re-acquisition verification includes: in response to the execution of corrective actions, acquiring data records from the source system based on the scope of the corrective object and generating post-execution change events; converting the post-execution change events into post-execution evidence records according to the semantic rule registry, and generating a post-execution standardized value view from the post-execution evidence records; performing entity parsing based on the post-execution standardized value view, and performing semantic equivalence transformation stability tests and evidence conflict measures on the entity parsing results to output a post-execution credible candidate set; performing constraint consistency inference based on the post-execution credible candidate set and the semantic rule registry to verify that the consistency constraint set is satisfied, and performing a rollback if the consistency constraint set is not satisfied. The rollback includes revoking the corrective action or performing a compensatory corrective action.

[0052] In one embodiment, to ensure a closed-loop verification path for the effect of the corrective action, re-data collection and verification are triggered after the corrective action is executed. In response to the corrective action, the system retrieves data records from the source system based on the scope of the corrective object and generates post-execution change events. Each post-execution change event carries at least the source system identifier, source object identifier, change type, and change time, and points to the modified data records within the scope of the corrective object. The system converts the post-execution change events into post-execution evidence records according to a semantic rule registration library, and generates a post-execution standardized value view from these evidence records. The post-execution evidence records continue the field structure of evidence records, including original values ​​and contextual information, and include the event time, data collection time, and data quality markers, enabling post-execution data to be aligned and compared with pre-execution data under the same semantic caliber.

[0053] In one embodiment, the system performs entity parsing based on the post-execution standardized value view to obtain a post-execution candidate entity matching relationship set. It then performs a semantic equivalence transformation stability test and evidence conflict measurement on this set to output a post-execution trusted candidate set. This process confirms that the correction write-back did not introduce new entity drift or new field semantic conflicts. Subsequently, the system performs constraint consistency inference based on the post-execution trusted candidate set and the semantic rule registry to verify that the consistency constraint set is satisfied. Verification uses a consistency state snapshot as the output target: when the consistency constraint set is satisfied, the system saves the post-execution consistency state snapshot as the latest snapshot and marks the correction action plan as completed; when the consistency constraint set is not satisfied, the system determines that the re-collection verification has failed and performs a rollback. Rollback includes reversing the correction action or performing a compensatory correction action. Reversing the correction action is used to restore the fields to be modified within the scope of the correction object to their values ​​before the correction execution. The restored values ​​are taken from the write-before value record saved during the execution phase. Compensation and correction actions are used to resolve inconsistencies by appending writes when previous values ​​cannot be directly restored or when chained dependencies exist. For example, missing fields in related objects are first added, and then the reference relationships are restored. After the rollback is completed, the system regenerates change events and enters a re-collection and verification process until the consistency constraint set is satisfied or the preset rollback limit is reached.

[0054] In one specific embodiment, customer master data from three business systems of a certain group are used as the object. The source systems include the Customer Relationship Management System (CRM), the Enterprise Resource Planning (ERP) System, and the Work Order System. Incremental data within the same natural day is selected for playback: 12,430 customer data records in the CRM System, 11,980 in the ERP System, and 8,210 in the Work Order System; a total of 850 change events occurred on that day, including additions, updates, and deletions. The changes mainly focus on fields such as customer name, mobile phone number, last six digits of ID card, fragments of account opening address, and customer unified code, and there are differences in format, aliases, and organizational structure between the different systems.

[0055] After a change event is processed, evidence records are constructed according to the semantic rules registration library. For each change, the evidence record includes the source system identifier, source object identifier, semantic field identifier, event time, and collection time, while retaining the original values ​​and context information. The context information includes field units, enumeration meanings, source channels, business types, and collection batch numbers, used to constrain subsequent standardized interpretations. On that day, 850 change events were broken down into 2720 evidence records, with an average of 3.2 semantic field identifiers per change. Typical examples are as follows (to avoid disclosure, only necessary field formats are retained): In the Customer Relationship Management System, the original mobile phone number for the source object identifier CRM-021583 is “138-0013-8000”; in the Enterprise Resource Planning System, the original mobile phone number for the source object identifier ERP-009771 is “13800138000”; the customer names are “Zhang San” and “Zhang San” respectively; and the customer unified codes are “CUST-000871” and “000871” respectively. Evidence records are written into data format validation and value range validation results to form data quality markers. When out-of-bounds or unresolvable values ​​are encountered, the original values ​​are retained, but the validation results are used as input for subsequent constraints. Subsequently, standardized value views are generated according to the standardization rules of the semantic rule registration library: mobile phone numbers are uniformly converted to 11-digit strings; customer codes are uniformly converted to fixed-length codes with prefixes; customer names undergo whitespace character normalization and alias mapping; the last six digits of identification documents are uniformly converted to numeric strings; and each standardized value view is written with a standardization rule version identifier. Evidence records and standardized value views are traceable at the field level, facilitating playback, recalculation, and corrective acceptance.

[0056] During the entity resolution phase, a search key is generated using a normalized value view. The search key is generated by combining the normalized values ​​corresponding to at least two semantic fields. In this example, "Customer Uniform Code + Mobile Number" is preferred as the search key. Missing data markers are used for retrieval boundary control to avoid candidate explosion caused by missing data. The master data repository stores the master entity identifier and its normalized value view index. The master entity identifier represents a unified entity across sources. On that day, entity resolution generated 2954 candidate entity matches, averaging 3.47 matches per change event. To facilitate review, a field-level weighted consistency similarity score is calculated, and the stability of semantic equivalence transformations and the evidence conflict metric are incorporated into the screening process. The expressions are as follows:

[0057]

[0058] in, The similarity score for candidate entity matching relationships is used to characterize the weighted consistency between the normalized value view corresponding to the source object identifier and the normalized value view corresponding to the candidate main entity identifier in the semantic field dimension. The number of semantic fields involved in the matching; For the first Field weights of semantic fields; For the first The consistency indicator for each semantic field, set to 1 for consistency and 0 for inconsistency; The similarity score is calculated based on the standardized value view; The similarity score is calculated based on the transformed standardized value view; This is a stability metric used to characterize the change in similarity score before and after semantic equivalence transformation. For example, for a combination of five fields: "customer unified code, mobile phone number, last six digits of ID card, customer name, and fragment of account opening address," the weights are set to 0.35, 0.25, 0.20, 0.10, and 0.10 respectively. The score is calculated when the first four fields are identical but the address is inconsistent. After semantic equivalence transformation (name alias mapping and case normalization), the score is: Stability indicators The score was retained within the threshold of 0.12. Conversely, the score of a candidate relation fluctuated from 0.72 to 0.54. If the threshold is exceeded, the candidate is deemed sensitive to the form of expression and is directly eliminated. The evidence conflict metric generates a conflict count based on mutually exclusive constraint rules. For example, "customer status is cancelled" and "there are valid work orders in the last 7 days" are incompatible combinations, triggering a one-to-one conflict count. When the conflict count exceeds the threshold, the candidate relationship is eliminated. After comprehensive screening, a set of 1578 credible candidates is output, accounting for 53.4% ​​of the total candidates; the average similarity score of the credible set is 0.829, and the overall candidate average is 0.668. Figure 2 The similarity distribution before and after filtering is presented. The credible set is clearly concentrated in the high-score interval, while low-score candidates are suppressed by stability and conflict rules. Manual sampling and verification of 120 change events shows that 103 events were correctly selected by directly choosing the highest score before filtering, and 115 events were correctly selected after filtering, indicating a significant decrease in the probability of triggering error correction.

[0059] In the constraint consistency inference phase, the semantic rule registry provides a set of consistency constraints, including uniqueness constraints and referential integrity constraints. Uniqueness constraints restrict that the same source object can only correspond to one main entity at a time, or that the same main entity cannot be mutually exclusive within a specified scope. Referential integrity constraints restrict that customer references in work order records must point to an existing and valid main entity and satisfy the existence condition of key fields. Inference selects a subset of constraints from the trusted candidate set to generate a consistency state snapshot, and generates a corrective action plan based on this. The corrective action plan includes the scope of the corrective object, the fields to be modified, the target value, and the execution preconditions. In this example, the generated corrective action plan covers 206 corrective object scopes, involving 184 field-level corrective actions, 27 reference relationship repairs, and 61 code standardization write-backs. Before correction, constraint checks were performed on the consistency state snapshot, revealing 214 uniqueness constraint violations and 98 referential integrity violations; after plan generation, these decreased to 53 and 21 respectively at the snapshot level, indicating that the candidate combinations hypothetically satisfy most constraints.

[0060] Pre-execution rehearsal and acceptance: In a simulated execution environment isolated from the source system's data records, the synchronous execution rule set and corrective action plan are applied to generate a snapshot of the consistency status after the rehearsal. Constraint consistency inference is then performed again, while verifying the preconditions (unchanged data record versions, data quality markers meeting requirements, and corrective objects not being concurrently locked, etc.). In this example, the rehearsal acceptance pass rate was 92.0%, corresponding to 190 out of 206 corrective object ranges passing. The 16 that failed were mainly due to version inconsistencies and missing referenced objects. After passing the rehearsal acceptance, corrective actions are executed, triggering re-collection and verification: Source system data records are re-retrieved according to the corrective object range to generate post-execution change events, which are converted into post-execution evidence records and post-execution standardized value views. Entity resolution, stability testing, evidence conflict measurement, and constraint consistency inference are repeated to verify that the consistency constraint set is satisfied. The re-collection verification pass rate was 97.0%, with a rollback trigger rate of 4.0%. Rollback includes undoing corrective actions or performing compensatory corrective actions: in undoable scenarios, the previous value is directly restored; in scenarios with chained dependencies, the key fields of the referenced object are filled in first, and then the reference relationship is restored. Figure 3 The decrease in the number of violations in the closed loop is shown: uniqueness constraint violations decreased from 214 to 5, and citation integrity constraint violations decreased from 98 to 1. Moreover, the decrease occurred within the continuous chain of "credible candidate screening - constraint inference - pre-rehearsal acceptance - re-collection verification", which facilitates the explanation of the contribution of each stage. Figure 4 The statistical results of the pre-review acceptance and re-collection verification are presented. The rollback ratio remains at a low level, indicating that the corrective action plan and the synchronous execution rule set can be stably implemented in most scenarios.

[0061] like Figure 5 As shown, a multi-source data consistency management device is used to implement a multi-source data consistency management method. The device includes: an acquisition module, an evidence construction module, an entity parsing module, an inference module, and an execution module. The acquisition module is used to acquire data records from at least two source systems and generate change events. The acquisition module includes a network interface controller, a bus interface, and a buffer storage unit in hardware, which is used to establish communication links with at least two source systems and receive data records. The received data records are modified by a timestamp unit and an event triggering unit, and a change event containing change type and time information is generated. The change event is then written into the event queue buffer.

[0062] The evidence construction module is used to convert change events into evidence records based on the semantic rule registration library. The evidence records include original values ​​and context information, and are used to generate a standardized value view from the evidence records. The evidence construction module includes a processor execution unit, a rule cache unit, and a high-speed key-value storage unit in hardware, which is used to load the rule index of the semantic rule registration library; extract fields and encapsulate the change events in a structured way to generate evidence records and write them to the evidence storage; and perform standardization processing on the original values ​​and context information through the standardization calculation unit to generate a standardized value view and write the rule version identifier into the storage.

[0063] The entity parsing module is used to perform entity parsing based on the standardized value view to obtain a set of candidate entity matching relationships, and to perform semantic equivalence transformation stability test and evidence conflict measurement on the set of candidate entity matching relationships to output a set of credible candidates; The inference module performs constraint consistency inference based on a trusted candidate set and a semantic rule registry, generating a consistency state snapshot and a corrective action plan. The entity parsing module, in hardware, includes an index retrieval acceleration unit, an in-memory database cache, and a vector or hash calculation unit. It generates retrieval keys based on a standardized value view and accesses the master data repository index; outputs a set of candidate entity matching relationships; and performs semantic equivalence transformation, similarity score calculation, stability testing, and conflict counting through a parallel computing unit to filter and obtain a trusted candidate set, which is then written to the candidate result cache. The inference module, in hardware, includes a rule reasoning execution unit, a constraint verification acceleration unit, and a state snapshot storage unit. It loads a consistency constraint set and performs constraint consistency inference on the trusted candidate set; generates a consistency state snapshot and writes it to snapshot storage; and generates a corrective action plan based on the difference analysis calculation unit, writing the scope of the corrective object, the fields to be modified, and the target value to the plan storage area.

[0064] The execution module is used to generate a synchronous execution rule set based on the corrective action plan and perform pre-rehearsal acceptance. It is also used to execute corrective actions and re-collect and verify data when the pre-rehearsal acceptance is passed, and to perform rollback when the pre-rehearsal acceptance fails or the re-collection and verification fails. The execution module includes isolated execution environment resources, a transaction control unit, and a rollback log storage unit. It is used to generate a synchronous execution rule set based on the corrective action plan; perform pre-rehearsal acceptance in an isolated simulated execution environment and output the acceptance results; when acceptance is passed, it issues a corrective write-back operation to the source system through the transaction control unit and records the rollback log; after correction, it re-acquires data records through the acquisition interface to complete the re-collection and verification; when the pre-rehearsal fails or the re-collection and verification fails, it triggers a reversal write-back or a compensatory write-back based on the rollback log to complete the rollback.

[0065] A computer device includes a processor and a memory, the memory storing a computer program that, when executed by the processor, causes the computer device to perform a multi-source data consistency management method.

[0066] In one exemplary embodiment, a computer device is provided. The computer device includes a processor, a memory, and a bus, with the processor and memory connected via the bus. The computer device may further include an input / output interface and a communication interface. The input / output interface is used for information exchange between the processor and external devices, and the communication interface is used for communication with at least two source systems via wired or wireless means. The memory is used to store computer programs and data related to multi-source data consistency management, including a semantic rule registry, evidence records, standardized value views, consistency state snapshots, corrective action plans, and synchronized execution rule sets. The processor executes a computer program in memory to enable the computer device to perform a multi-source data consistency management method, specifically including: acquiring data records from at least two source systems and generating change events; converting change events into evidence records based on a semantic rule registry and generating a standardized value view from the evidence records; performing entity parsing based on the standardized value view to obtain a set of candidate entity matching relationships, and performing semantic equivalence transformation stability checks and evidence conflict measurements on the set of candidate entity matching relationships to output a set of credible candidates; performing constraint consistency inference based on the set of credible candidates and the semantic rule registry to generate a consistency state snapshot and a correction action plan; generating a set of synchronous execution rules based on the correction action plan and performing a pre-rehearsal acceptance; executing correction actions and re-collecting verification when the pre-rehearsal acceptance is passed; and executing rollback when the pre-rehearsal acceptance is failed or the re-collection verification fails.

[0067] A computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for managing multi-source data consistency.

[0068] In one exemplary embodiment, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program or computer-executable instructions, which, when loaded and executed by a processor, enable a computer device to implement a multi-source data consistency management method. The computer-readable storage medium can be a non-transitory computer-readable storage medium, specifically including a read-only memory, random access memory, flash memory, solid-state drive, disk storage, optical disk storage, or other storage media capable of storing program instructions.

[0069] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0070] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for managing multi-source data consistency, characterized in that, The method includes: Data records from at least two source systems are acquired and change events are generated. The change events are then converted into evidence records based on a semantic rule registration library. The evidence records include original values ​​and contextual information. A standardized value view is generated from the evidence records. Based on the standardized value view, entity parsing is performed to obtain a set of candidate entity matching relationships. Then, the set of candidate entity matching relationships is subjected to semantic equivalence transformation stability test and evidence conflict measurement to output a set of credible candidates. Based on the trusted candidate set and the semantic rule registration library, a consistency state snapshot and a correction action plan are generated by performing constraint consistency inference. Based on the corrective action plan, a set of synchronous execution rules is generated and a pre-rehearsal acceptance is performed. If the pre-rehearsal acceptance is passed, the corrective action is executed and the data is collected and verified again. If the pre-rehearsal acceptance is not passed or the data collection and verification fails, a rollback is executed.

2. The method according to claim 1, characterized in that, The step of converting the change event into the evidence record according to the semantic rule registration library includes: The evidence record is written with a source system identifier, a source object identifier, a semantic field identifier, an event time, and a collection time. The source system identifier is used to indicate the source system to which the data record belongs, the source object identifier is used to indicate the object to which the data record corresponds, and the semantic field identifier is used to indicate the semantic field to which the original value corresponds. Based on data format verification and value range verification, a data quality marker is written to the evidence record. The data quality marker is used to indicate the verification result. The step of generating the standardized value view from the evidence record includes: standardizing the original value in combination with the context information according to the standardization rules of the semantic rule registration library to generate the standardized value view, and writing a standardization rule version identifier into the standardized value view.

3. The method according to claim 2, characterized in that, The entity parsing based on the standardized value view includes: A search key is generated based on the standardized value view, wherein the search key is generated by combining the standardized value views corresponding to at least two of the semantic fields; A set of candidate master entity identifiers is retrieved in the master data repository based on the search key. The master data repository is used to store the master entity identifiers and the normalized value view index associated with the master entity identifiers. The master entity identifiers are used to represent cross-source unified entities. The candidate entity matching relationship set is generated based on the source object identifier and the candidate main entity identifier set.

4. The method according to claim 1, characterized in that, The stability test of the semantic equivalence transformation includes: Based on the semantic equivalence transformation rules in the semantic rule registration library, a transformed standardized value view is generated from the standardized value view. The semantic equivalence transformation rules are used to transform the standardized value view without changing the semantics of the fields. Within the set of candidate entity matching relationships, a first similarity score is calculated based on the standardized value view, and a second similarity score is calculated based on the transformed standardized value view. A stability index is determined based on the difference between the first similarity score and the second similarity score, and the stability index is compared with a preset threshold to eliminate candidate entity matching relationships in the candidate entity matching relationship set whose stability index exceeds the preset threshold. The evidence conflict measurement includes: generating a conflict count based on the mutual exclusion constraint rules of the semantic rule registration library, wherein the mutual exclusion constraint rules are used to describe incompatible value combinations, and comparing the conflict count with a preset threshold to eliminate candidate entity matching relationships in the candidate entity matching relationship set whose conflict count exceeds the preset threshold.

5. The method according to claim 1, characterized in that, The constraint consistency inference includes: Based on the semantic rule registration library, a set of consistency constraints is determined, which includes uniqueness constraints and referential integrity constraints. A subset of trusted candidates that satisfies the set of consistency constraints is determined from the set of trusted candidates to generate the consistency state snapshot; The corrective action plan is generated based on the consistency state snapshot and the data records of the source system. The corrective action plan includes the scope of the corrective object, the fields to be modified, the target values, and the execution preconditions. The scope of the corrective object is used to indicate the range of data records for which corrective actions need to be performed.

6. The method according to claim 5, characterized in that, The pre-rehearsal acceptance includes: In the simulated execution environment, the synchronous execution rule set and the corrective action plan are applied to generate a snapshot of the consistent state after the pre-rehearsal, and the data records of the simulated execution environment and the source system are isolated. Based on the pre-simulation consistent state snapshot, the constraint consistency inference is performed to verify that the consistency constraint set is satisfied, and to verify that the execution preconditions are satisfied; The pre-rehearsal acceptance is determined to be successful if the set of consistency constraints is satisfied and the execution preconditions are satisfied.

7. The method according to claim 5, characterized in that, The re-collection verification includes: In response to the execution of a corrective action, data records of the source system are obtained based on the scope of the corrective action and a post-execution change event is generated; Based on the semantic rule registration library, the post-execution change event is converted into a post-execution evidence record, and a post-execution standardized value view is generated from the post-execution evidence record; The entity parsing is performed based on the post-execution standardized value view, and the semantic equivalence transformation stability test and the evidence conflict metric are performed on the entity parsing results to output a post-execution credible candidate set; Based on the post-execution trusted candidate set and the semantic rule registration library, the constraint consistency inference is performed to verify that the consistency constraint set is satisfied. If the consistency constraint set is not satisfied, the rollback is performed, and the rollback includes revoking the correction action or performing a compensation correction action.

8. A multi-source data consistency management device, used to implement the multi-source data consistency management method according to any one of claims 1-7, characterized in that, The device includes: an acquisition module, an evidence construction module, an entity parsing module, an inference module, and an execution module; The acquisition module is used to acquire data records from at least two source systems and generate change events; The evidence construction module is used to convert the change event into an evidence record based on the semantic rule registration library. The evidence record includes the original value and context information, and is used to generate a standardized value view from the evidence record. The entity parsing module is used to perform entity parsing based on the standardized value view to obtain a set of candidate entity matching relationships, and to perform semantic equivalence transformation stability test and evidence conflict measurement on the set of candidate entity matching relationships to output a set of credible candidates. The inference module is used to perform constraint consistency inference based on the trusted candidate set and the semantic rule registration library to generate a consistency state snapshot and a correction action plan; The execution module is used to generate a set of synchronous execution rules based on the corrective action plan and perform a pre-rehearsal acceptance, and to execute corrective actions and re-collect verification when the pre-rehearsal acceptance is passed, and to execute rollback when the pre-rehearsal acceptance is not passed or the re-collection verification fails.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, which, when executed by the processor, causes the computer device to perform the multi-source data consistency management method as described in claim 1.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the multi-source data consistency management method as described in claim 1.

Images