Multi-source heterogeneous data conflict resolution method and device, electronic equipment and storage medium

Abstract

The application provides a multi-source heterogeneous data conflict resolution method and device, electronic equipment and a storage medium. The method comprises the following steps: mapping multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder to generate an observation feature tensor; performing probability mapping on each attribute dimension through a credibility measurement function to jointly generate a coupled probability fingerprint; constructing an information entropy evolution equation; combining the information entropy evolution equation with a weight matrix through an energy functional to construct a nonlinear dynamic evolution system; driving the system through a gradient flow to perform nonlinear self-evolution; calculating a Lyapunov index and outputting a conflict resolution stability judgment result; when the judgment result is convergence, generating unique data based on maximum a posteriori probability fusion; when the judgment result is divergence, introducing external logical constraints to correct the observation feature tensor and reinitializing until convergence. The application improves the accuracy, stability and efficiency of multi-source heterogeneous data conflict resolution.

Description

Technical Field

[0001] This invention relates to the field of data management technology, specifically to methods, apparatus, electronic devices, and storage media for resolving conflicts in multi-source heterogeneous data. Background Technology

[0002] In modern institutional management (such as universities and enterprises), there are usually multiple independently operating information systems, such as human resources management systems, academic affairs management systems, and scientific research management systems. Each of these systems maintains master data related to personnel. However, due to differences in data collection standards, update delays, and field definitions, records of the same person in different systems often exhibit semantic ambiguity, inconsistent attributes, or even logical conflicts.

[0003] Existing technologies for handling conflicts in multi-source heterogeneous master data primarily employ rule-based matching methods (such as field comparison and keyword alignment) or statistical weighting methods (such as voting mechanisms and confidence scores). However, these methods have the following drawbacks: First, traditional conflict resolution methods often use static comparison approaches, simply matching, weighting, or voting on attribute values ​​from multiple sources. This fails to dynamically monitor and evaluate the stability of the data alignment process. When conflicts are severe or logical paradoxes exist, the system cannot quantify the degree of conflict divergence, easily getting trapped in local optima or failing to converge. Second, existing technologies often focus only on differences in a single attribute during conflict assessment, failing to fully consider the differences in importance between different attributes, leading to inaccurate conflict assessment results. Furthermore, existing technologies rely on manual intervention for conflict judgment and correction. Once the conflict resolution process becomes highly chaotic (severe conflict), timely and effective repair and reinitialization are impossible, severely impacting the efficiency and reliability of data governance.

[0004] In summary, existing multi-source heterogeneous data conflict resolution technologies suffer from insufficient accuracy, stability, and efficiency, failing to meet the practical needs of current multi-source heterogeneous data governance. Therefore, there is an urgent need for a technical solution that can overcome these shortcomings and achieve dynamic, accurate, and stable multi-source heterogeneous data conflict resolution. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, apparatus, electronic device and storage medium for resolving conflicts in multi-source heterogeneous data, so as to solve the technical problem that the existing technology is insufficient in terms of accuracy, stability and efficiency of conflict resolution, and cannot meet the actual needs of current multi-source heterogeneous data governance.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a method for resolving conflicts in multi-source heterogeneous data, comprising: Multi-source heterogeneous data is mapped to a high-dimensional feature space by a pre-trained semantic encoder to generate observation feature tensors; Based on the observed feature tensor, a probability mapping is performed on each attribute dimension through a confidence metric function to jointly generate a coupled probability fingerprint. Based on the coupled probability fingerprint, an information entropy evolution equation is constructed. The information entropy evolution equation is combined with the weight matrix through energy functional to construct a nonlinear dynamic evolution system. The nonlinear dynamic evolution system is driven by gradient flow to perform nonlinear self-evolution. During the evolution process, the Lyapunov exponent is calculated, and the result of conflict resolution stability judgment is output. When the stability determination result of the conflict resolution is converged, unique data is generated based on the maximum a posteriori probability fusion; when the stability determination result of the conflict resolution is divergent, external logical constraints are introduced to correct the observed feature tensor and reinitialize it until convergence.

[0007] In one possible implementation, the step of mapping multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder to generate an observation feature tensor further includes: calculating the semantic similarity between multi-source heterogeneous data using an alignment operator with position encoding.

[0008] In one possible implementation, the step of generating a coupled probabilistic fingerprint by performing probability mapping on each attribute dimension based on the observed feature tensor using a confidence metric function includes: Extract the feature values ​​of each data point in the observed feature tensor for each attribute dimension; By using a preset credibility metric function, the feature values ​​of each attribute dimension are quantified and transformed into credibility probability values ​​for the corresponding dimension. The credibility probability values ​​of all attribute dimensions are jointly calculated and integrated to obtain the coupled probability fingerprint.

[0009] In one possible implementation, the construction of an information entropy evolution equation based on the coupled probability fingerprint, the combination of the information entropy evolution equation with a weight matrix through an energy functional to construct a nonlinear dynamic evolution system, and the nonlinear dynamic evolution system being driven by gradient flow to undergo nonlinear self-evolution, includes: Using the coupling probability fingerprint as the input to the information entropy evolution equation at the initial time, the information entropy evolution equation is constructed; A weight matrix is ​​introduced to constrain the evolution direction, and the global conflict energy level is calculated using the matrix trace; A nonlinear dynamic evolution system is constructed by coupling the information entropy in the information entropy evolution equation with the global conflict energy level calculated by the weight matrix through energy functional theory. The state of a nonlinear dynamic system evolves along the negative gradient direction of the energy functional by driving gradient flow.

[0010] In one possible implementation, the calculation of the Lyapunov index and the output of the conflict resolution stability determination result include: During the evolution of the nonlinear dynamic evolution system, the Lyapunov exponent is calculated to quantify the sensitivity of the nonlinear dynamic evolution system to disturbances; If the Lyapunov exponent is less than zero, the nonlinear dynamic evolution system is determined to be in a stable convergent state; if the Lyapunov exponent is greater than zero, the nonlinear dynamic evolution system is determined to be in a divergent chaotic state.

[0011] In one possible implementation, the step of introducing external logical constraints to modify the observed feature tensor and reinitialize it includes: Hard constraint tensors from independent data sources are extracted. The original observation feature tensors and hard constraint tensors are then weighted and fused using linear operators and confidence decay coefficients to generate the corrected observation feature tensors. Reinitialize based on the corrected observation feature tensor until convergence.

[0012] In one possible implementation, the confidence decay coefficient is dynamically determined by the global average value of the coupled probability fingerprint. The higher the degree of chaos in the nonlinear dynamic evolution system, the closer the confidence decay coefficient is to zero, causing the nonlinear dynamic evolution system to turn to an evolution path driven by hard constraint tensors.

[0013] On the other hand, the present invention also provides a multi-source heterogeneous data conflict resolution device, comprising: The mapping module is used to map multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder, generating observation feature tensors. The probabilistic fingerprint generation module is used to generate coupled probabilistic fingerprints by performing probability mapping on each attribute dimension based on the observed feature tensor through a confidence metric function. The stability assessment module is used to construct an information entropy evolution equation based on the coupling probability fingerprint, combine the information entropy evolution equation with the weight matrix through energy functional to construct a nonlinear dynamic evolution system, drive the nonlinear dynamic evolution system to perform nonlinear self-evolution through gradient flow, calculate the Lyapunov exponent during the evolution process, and output the conflict resolution stability judgment result. The decision execution module is used to generate unique data based on maximum a posteriori probability fusion when the conflict resolution stability determination result is converged; and to introduce external logical constraints to correct the observed feature tensor and reinitialize it when the conflict resolution stability determination result is divergent, until convergence.

[0014] In a second aspect, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the multi-source heterogeneous data conflict resolution method described in any of the above implementations.

[0015] Thirdly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in the multi-source heterogeneous data conflict resolution method described in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: The multi-source heterogeneous data conflict resolution method provided by this invention first maps multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder, generating an observation feature tensor. Based on the observation feature tensor, a confidence metric function is used to perform probability mapping on each attribute dimension, jointly generating a coupled probability fingerprint, transforming the high-dimensional semantic features into fingerprint probabilities that can characterize entity consistency. Based on the coupled probability fingerprint, an information entropy evolution equation is constructed. An energy functional is used to combine the information entropy evolution equation with a weight matrix to construct a nonlinear dynamic evolution system. Gradient flow drives the nonlinear dynamic evolution system to undergo nonlinear self-evolution, transforming the original... Static, discrete data conflicts are transformed into a continuous, evolvable dynamic system, enabling real-time monitoring of the divergence or convergence trends of conflicts during alignment. This solves the problem that existing technologies cannot dynamically assess the stability of conflict resolution. Furthermore, by calculating the Lyapunov exponent and outputting the conflict resolution stability judgment result, when the judgment result is convergent, unique data is generated based on maximum a posteriori probability fusion. When the judgment result is divergent, external logical constraints are introduced to correct the observed feature tensor and reinitialize it until convergence. This avoids subjective errors introduced by human intervention and can efficiently handle large-scale, multi-type heterogeneous data conflicts, significantly improving the efficiency of data governance. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of an embodiment of the multi-source heterogeneous data conflict resolution method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 4 For the present invention Figure 1 Another embodiment of the process diagram of S103; Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 6 A schematic diagram of an embodiment of the multi-source heterogeneous data conflict resolution device provided by the present invention; Figure 7 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] Before demonstrating the embodiments, the following terms will be explained.

[0024] Observation feature tensor: refers to the multi-dimensional, multi-channel tensor structure formed after mapping multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder. Its dimensions include data sample dimension, attribute feature dimension and semantic representation dimension, which can completely preserve the semantic information and attribute features of the original data.

[0025] Coupled probabilistic fingerprint: refers to the set of probability vectors that can uniquely represent the data credibility distribution after the credibility quantification of the feature values ​​of each attribute dimension in the observation feature tensor is performed and jointly calculated. It can be understood as the "credibility identity" of the data and is used for stability assessment of subsequent conflict resolution.

[0026] This invention provides a method, apparatus, electronic device, and storage medium for resolving conflicts in multi-source heterogeneous data, which will be described below.

[0027] Figure 1 This is a flowchart illustrating an embodiment of the multi-source heterogeneous data conflict resolution method provided by the present invention. The execution entity of the method is a server, computer, or data platform system with data processing capabilities. The method runs in a master data management platform, an integrated personnel system, or a data governance engine. This invention is mainly applied in universities, research institutes, and large enterprises, and is particularly suitable for scenarios with multi-source heterogeneous business systems such as personnel, academic affairs, scientific research, and office work, where personnel master data is collected in a decentralized manner, updated asynchronously, and prone to attribute conflicts and semantic inconsistencies. It can achieve automatic alignment of personnel entities, dynamic conflict assessment, automatic generation of authoritative data, and autonomous repair of severe conflicts. Figure 1 As shown, methods for resolving conflicts in multi-source heterogeneous data include: S101. Multi-source heterogeneous data is mapped to a high-dimensional feature space through a pre-trained semantic encoder to generate observation feature tensors.

[0028] It should be noted that the pre-trained semantic encoder can employ a deep neural network based on a bidirectional Transformer architecture (such as BERT, RoBERTa, or Sentence-BERT specifically optimized for entity alignment). The pre-trained semantic encoder, through a hierarchical design, achieves efficient semantic feature extraction from unstructured human records. Its specific structure includes the following three core functional levels, each working collaboratively to transform discrete text into dense feature vectors: Input embedding layer: This layer stores raw, unstructured personnel records. The input text string (including but not limited to names, titles, affiliated units, and job information) undergoes discrete symbol parsing and vectorization. Specifically, the input text string is first decomposed into several token sequences, and then a positional encoding mechanism is introduced to compensate for the Transformer architecture's inability to capture sequence order information. Finally, the discrete token symbols are transformed into low-dimensional dense vectors, laying the foundation for subsequent deep extraction of semantic features.

[0029] Multi-head self-attention layer: This is the core functional layer of the pre-trained semantic encoder, whose core function is to capture the deep semantic dependencies between different attributes within personnel records. This layer uses multiple parallel attention heads to perform multi-dimensional attention calculations on the low-dimensional dense vector output from the input embedding layer. This allows for the accurate identification of semantically related attribute information (e.g., effectively identifying the high semantic correlation between titles such as "professor" and "doctoral supervisor," or "associate professor" and "master's supervisor"), thereby achieving comprehensive and in-depth mining of the semantic information in personnel records. The computational logic for each attention head is as follows:

[0030] in, They are obtained from the input sequences through linear transformations. Using the scaling factor, the outputs of multiple heads are concatenated and then linearly transformed to obtain a context-enhanced sequence feature tensor.

[0031] Feature Convergence Layer: This layer aggregates and normalizes the sequence feature tensors output by the multi-head self-attention layer. Specifically, it uses CLS token aggregation or mean pooling to compress and integrate the sequence feature tensors, eliminating the impact of differences in sequence length. Subsequently, a linear projection layer is used for dimension mapping, ultimately outputting a semantic feature vector with a fixed dimension of d. The preferred value of d is 768 or 1024 to ensure the uniformity and computability of the output feature vector.

[0032] The process of mapping personnel records to a high-dimensional feature space and generating observation feature tensors through the pre-trained semantic encoder is essentially a nonlinear manifold embedding process. Its core purpose is to utilize the redundancy characteristics of high-dimensional space to eliminate heterogeneous data noise, providing a computable mathematical basis for subsequent conflict resolution. The specific implementation steps are as follows: First, the pre-trained semantic encoder utilizes the massive amount of linguistic knowledge and semantic association patterns learned during the pre-training phase to process each personnel record. This is mapped to the coordinates of a point in a high-dimensional feature space. In this high-dimensional feature space, the distance between any two points directly represents the semantic similarity between the corresponding two personnel records: for example, personnel records from the "Personnel Management System". Records of the same person from the "Academic Affairs Management System" Even if there are slight differences in the literal descriptions of the two (such as abbreviations of names or abbreviations of company names), they will still be mapped to adjacent regions in this high-dimensional space, thereby achieving accurate semantic matching.

[0033] Next, for each of the m personnel records from M heterogeneous source systems, semantic feature extraction is performed using a pre-trained semantic encoder: for each personnel record... After processing by a pre-trained semantic encoder, the result is obtained ,in, It is a d-dimensional real vector (where d is a fixed dimension of the output of the feature convergence layer, preferably 768 or 1024). This is a pre-trained semantic encoder. All m d-dimensional real vectors are stacked row-wise to form the final observation feature tensor, which fully preserves the semantic feature information recorded by personnel from all heterogeneous source systems. This provides data support for subsequent conflict resolution and handling.

[0034] The semantic features of personnel records are mapped to a high-dimensional feature space of d dimensions (typically 768 or 1024). The redundancy of this high-dimensional space is utilized to eliminate noise interference in heterogeneous data. In this high-dimensional space, conflicting information in personnel records from heterogeneous systems (e.g., different ways of writing the same person's birth date or inconsistent job title descriptions in different systems) will manifest as an offset in the corresponding semantic feature vector along a specific dimension. This quantifiable offset feature provides a clear and computable mathematical basis for subsequent probabilistic fingerprint calculations, ensuring the accuracy and efficiency of conflict resolution.

[0035] S102. Based on the observed feature tensor, perform probability mapping on each attribute dimension through a confidence metric function to jointly generate a coupled probability fingerprint.

[0036] S103. Based on the coupled probability fingerprint, construct the information entropy evolution equation, combine the information entropy evolution equation with the weight matrix through the energy functional, construct a nonlinear dynamic evolution system, drive the nonlinear dynamic evolution system to perform nonlinear self-evolution through gradient flow, calculate the Lyapunov exponent during the evolution process, and output the conflict resolution stability judgment result.

[0037] S104. When the stability determination result of the conflict resolution is converged, unique data is generated based on the maximum a posteriori probability fusion; when the stability determination result of the conflict resolution is divergent, external logic constraints are introduced to correct the observed feature tensor and reinitialize it until convergence.

[0038] In summary, the multi-source heterogeneous data conflict resolution method provided by this invention first maps multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder, generating an observation feature tensor. Based on the observation feature tensor, a confidence metric function is used to perform probability mapping on each attribute dimension, jointly generating a coupled probability fingerprint, transforming the high-dimensional semantic features into fingerprint probabilities that can characterize entity consistency. Based on the coupled probability fingerprint, an information entropy evolution equation is constructed. An energy functional is used to combine the information entropy evolution equation with a weight matrix to construct a nonlinear dynamic evolution system. Gradient flow drives the nonlinear dynamic evolution system to undergo nonlinear self-evolution, transforming the originally static... Discrete data conflicts are transformed into a continuous, evolvable dynamic system, which can monitor the divergence or convergence trend of conflicts in real time during the alignment process, solving the problem that existing technologies cannot dynamically evaluate the stability of conflict resolution. Furthermore, by calculating the Lyapunov exponent and outputting the conflict resolution stability judgment result, when the judgment result is convergent, unique data is generated based on the maximum a posteriori probability fusion; when the judgment result is divergent, external logical constraints are introduced to correct the observed feature tensor and reinitialize it until convergence. This not only avoids the subjective error introduced by human intervention, but also efficiently handles large-scale, multi-type heterogeneous data conflicts, greatly improving the efficiency of data governance.

[0039] In some embodiments of the present invention, step S101, which maps multi-source heterogeneous data to a high-dimensional feature space by a pre-trained semantic encoder and generates an observation feature tensor, further includes: calculating the semantic similarity between multi-source heterogeneous data using an alignment operator with position encoding.

[0040] The alignment operator expression is as follows:

[0041] , Records for personnel from different systems, , These are personnel feature vectors from different systems. This is a cosine similarity operator with positional encoding.

[0042] It's important to note that semantic encoders based on the Transformer architecture lack sequence position awareness and are insensitive to the order of attribute sequences. Relying solely on content features for comparison can easily misclassify records with similar attribute content but different order as highly similar, such as treating "name + title" and "title + name" as the same semantic expression, leading to entity alignment errors. Introducing positional encoding endows the model with the ability to perceive attribute order, allowing feature vectors to carry both content and positional information, better aligning with the structured attribute arrangement patterns of personnel master data. Furthermore, different business systems such as personnel, education, and research may differ in the order and position of attributes for the same type of personnel. Positional encoding can identify the relative or absolute position of each attribute in a record, enabling tensor alignment operators to... When calculating similarity, features that are semantically consistent and geographically close can be prioritized for matching, thereby improving the accuracy of cross-system entity alignment.

[0043] It should also be noted that in the similarity calculation process, the final input feature vector v is obtained by fusing the attribute content feature c and the positional encoding feature p, which can usually be expressed as: v = c + p. Correspondingly, the cosine similarity calculation can be expressed as:

[0044] In summary, the similarity score is determined by both content similarity and location similarity. A high similarity score can only be obtained when the attribute content of two personnel records is similar and the corresponding attribute locations are close.

[0045] In this way, location coding can significantly enhance the matching accuracy of structured records and avoid misaligning attribute features with different semantic locations and different business meanings. For example, it can prevent the unit field in the personnel system from being mismatched with the project name field in the scientific research system, thereby improving the reliability of subsequent conflict assessment and data fusion.

[0046] In some embodiments of the present invention, such as Figure 2 As shown, step S102, based on the observed feature tensor, performs probability mapping on each attribute dimension through a confidence metric function to jointly generate a coupled probability fingerprint, including: S201. Extract the feature values ​​of each data point in the observed feature tensor for each attribute dimension; S202. Using a preset credibility measurement function, the feature values ​​of each attribute dimension are quantified and converted into credibility probability values ​​for the corresponding dimension. S203. Jointly calculate the confidence probability values ​​of all attribute dimensions and integrate them to obtain the coupled probability fingerprint.

[0047] It should be noted that the specific formula for the credibility measurement function is as follows:

[0048] in, From the observation feature tensor The Personnel record in the first Observations on dimensional attributes; Assign authority weights to data sources by pre-setting corresponding weight coefficients based on the authority level of each business system (for example, the weight of the human resources system is higher than that of the academic affairs system and the scientific research system, to ensure that the credibility of authoritative data sources is higher). , The first The empirical mean and variance of the dimensional attribute across all records are used to quantify the current observation. The degree to which the attribute deviates from the mainstream description; the smaller the deviation, the higher the credibility. To update the frequency decay function, which reflects the impact of data update timeliness on reliability, the frequency decay function is used. The formula for the interval between the last update time and the current time is usually: , ( (This is the decay coefficient, which can be adjusted according to the actual business scenario). The shorter the update interval, the smaller the decay and the higher the reliability.

[0049] Based on the credibility metric function, the credibility probability values ​​of all attribute dimensions are jointly calculated and integrated to obtain the coupled probability fingerprint:

[0050] Coupling probability fingerprint As a steady-state initial value input for subsequent conflict dynamics assessment, its numerical value directly characterizes the consistency and credibility of the multi-source system data in describing the same person entity at the initial moment. It provides a basis for the construction of subsequent conflict evolution models and the calculation of Lyapunov indices, ensuring the accuracy and rationality of conflict assessment.

[0051] This invention embodiment uses the observation feature tensor This transforms the originally fragmented heterogeneous system records into numerical representations in a unified dimensional space, utilizing the observation feature tensor. The numerical distribution of each dimension is mapped probabilistically using a confidence metric function, thereby transforming high-dimensional semantic features into fingerprint probabilities that can characterize entity consistency. Specifically, this involves tensor observation feature tensors. elements in Directly used as the input variable for the confidence metric function, it achieves a seamless transition from semantic features to statistical confidence, providing a reliable initial input for subsequent conflict dynamics assessment.

[0052] In some embodiments of the present invention, such as Figure 3 As shown, step S103, which involves constructing an information entropy evolution equation based on the coupled probability fingerprint, combining the information entropy evolution equation with the weight matrix through an energy functional, constructing a nonlinear dynamic evolution system, and driving the nonlinear dynamic evolution system to undergo nonlinear self-evolution through gradient flow, includes: S301. Using the coupled probability fingerprint as the input to the information entropy evolution equation at the initial moment, construct the information entropy evolution equation; S302. Introduce a weight matrix to constrain the evolution direction, and calculate the global conflict energy level through the matrix trace; S303. By coupling the information entropy in the information entropy evolution equation with the global conflict energy level calculated by the weight matrix through energy functional theory, a nonlinear dynamic evolution system is constructed. S304. The state of the nonlinear dynamic system evolves along the negative gradient direction of the energy functional by driving the gradient flow.

[0053] The specific formula for the information entropy evolution equation is as follows:

[0054] in, For the first Dimensional attributes in the first The information entropy value of the iteration step size; For attributes The corresponding probability distribution value.

[0055] To highlight the crucial role of key attributes in conflict assessment, this invention introduces an attribute importance weight matrix W, and calculates the global conflict energy level through matrix trace operations. The alignment process is transformed into a nonlinear dynamic evolution system, with the specific expression as follows:

[0056] in, For matrix trace operations, The attribute importance weight matrix is ​​preset according to the attribute business priority. For example, the weight of key attributes such as ID number and employee number is much higher than that of non-key attributes such as hobbies and remarks. This is the transpose of the observation feature tensor. Global conflict energy level. Weight matrix used to quantify the overall conflict intensity of multi-source data Its core function is to constrain the evolution direction of the feature space and to influence the observed feature tensor. By imposing manifold constraints on the evolutionary trajectory, conflicts of key attributes are prioritized for resolution, thereby enhancing the rationality and relevance of conflict assessment.

[0057] It should be noted that the weight matrix With information entropy Through synergistic effects, the multi-source data alignment process is transformed into a nonlinear dynamic evolution system. The specific modeling logic is as follows: By constructing an energy functional to couple the information entropy in the information entropy evolution equation with the global conflict energy level calculated from the weight matrix, a nonlinear dynamic evolution system is constructed, specifically expressed as follows:

[0058] in, , For weighting coefficients, the energy functional The magnitude of the value directly reflects the overall conflict state of the system.

[0059] Nonlinear dynamic evolution systems perform state transitions through gradient flow, minimizing the energy functional and driving the system towards a steady state. The differential equations for the state transitions are as follows:

[0060] Due to information entropy The differential equation contains logarithmic terms, and the weight matrix constraints contain quadratic terms (or higher-order terms). This shows typical nonlinear characteristics, indicating that the multi-source data alignment process is not a simple linear weighted fusion, but a nonlinear self-evolutionary process that fully considers the importance of attributes and statistical consistency and can autonomously adjust its evolutionary trajectory. It can more accurately simulate the dynamic changes in conflict resolution.

[0061] It should be understood that, in order to quantify the divergence strength of the aligned trajectory and determine the system stability, this invention calculates the Lyapunov exponent. To achieve stability monitoring, the specific formula is as follows:

[0062] Among them, evolution operators This refers to changing the system state from time 1 to 2. Mapping to time The state transition mapping, i.e. Evolutionary operators derivative , which represents the Jacobian matrix or local rate of change of the mapping in the current state, and is used to quantify the sensitivity of the system to disturbances.

[0063] In some embodiments of the present invention, such as Figure 4 As shown, step S103, which involves calculating the Lyapunov exponent and outputting the conflict resolution stability determination result, includes: S401. During the evolution of the nonlinear dynamic evolution system, the Lyapunov exponent is calculated to quantify the sensitivity of the nonlinear dynamic evolution system to disturbances. S402. If the Lyapunov exponent is less than zero, the nonlinear dynamic evolution system is determined to be in a stable convergent state; if the Lyapunov exponent is greater than zero, the nonlinear dynamic evolution system is determined to be in a divergent chaotic state.

[0064] It should be noted that if the Lyapunov exponent is less than zero, the system is considered to be in a stable convergent state, and unique data is generated based on the fusion of maximum a posteriori probabilities. The formula is expressed as follows:

[0065] in, For observation feature tensors The posterior probability distribution is calculated. By selecting the record combination with the highest posterior probability, this invention can automatically integrate the attribute information with the highest credibility and most consistent logic from various systems to generate a unique personnel master data record with high authority, ensuring the accuracy and immutability of data governance.

[0066] In some embodiments of the present invention, such as Figure 5 As shown, step S104, which involves introducing external logical constraints to modify the observed feature tensor and reinitializing it, includes: S501. Extract the hard constraint tensor from the independent data source, and use a linear operator to weight and fuse the original observation feature tensor and the hard constraint tensor with the confidence decay coefficient to generate the corrected observation feature tensor. S502. Reinitialize based on the corrected observation feature tensor until convergence.

[0067] It should be noted that the hard constraint tensor includes, but is not limited to, research contract trajectories, teaching schedule logs, and attendance trajectories. These data originate from real business physical records and possess irrefutable factual attributes, serving as the core basis for correcting fuzzy semantic features. The original observed feature tensor and the hard constraint tensor are then connected using linear operators. Weighted fusion is performed to generate the corrected observation feature tensor, expressed by the following formula: =

[0068] in, For linear operators, This is the confidence decay coefficient, with a value range of [0, 1]. This is a homogeneous mapping matrix used to linearly map trajectory data in physical space (such as attendance time and spatial location) to a high-dimensional semantic feature space (dimension d), thereby connecting it with the observed feature tensor. It has the same computational dimension, thus achieving the unification of heterogeneous data dimensions.

[0069] It should be understood that the observed feature tensor is reinitialized based on the corrected tensor until convergence.

[0070] In some embodiments of the present invention, the confidence attenuation coefficient The higher the degree of chaos in the nonlinear dynamic evolution system, the closer the confidence decay coefficient is to zero, which is dynamically determined by the global average value of the coupling probability fingerprint, thus causing the nonlinear dynamic evolution system to turn to an evolution path driven by hard constraint tensors.

[0071] To better implement the multi-source heterogeneous data conflict resolution method in the embodiments of the present invention, based on the multi-source heterogeneous data conflict resolution method, correspondingly, as follows: Figure 6 As shown, this embodiment of the invention also provides a multi-source heterogeneous data conflict resolution device, the multi-source heterogeneous data conflict resolution device 600 comprising: The mapping module 601 is used to map multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder to generate an observation feature tensor. The probabilistic fingerprint generation module 602 is used to generate a coupled probabilistic fingerprint by performing probability mapping on each attribute dimension based on the observed feature tensor and through a confidence measurement function. The stability assessment module 603 is used to construct an information entropy evolution equation based on the coupling probability fingerprint, combine the information entropy evolution equation with the weight matrix through energy functional to construct a nonlinear dynamic evolution system, drive the nonlinear dynamic evolution system to perform nonlinear self-evolution through gradient flow, calculate the Lyapunov exponent during the evolution process, and output the conflict resolution stability judgment result. The decision execution module 604 is used to generate unique data based on maximum a posteriori probability fusion when the conflict resolution stability determination result is converged; and to introduce external logical constraints to correct the observed feature tensor and reinitialize it when the conflict resolution stability determination result is divergent, until convergence.

[0072] The multi-source heterogeneous data conflict resolution device 600 provided in the above embodiments can realize the technical solutions described in the above embodiments of the multi-source heterogeneous data conflict resolution method. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the multi-source heterogeneous data conflict resolution method, which will not be repeated here.

[0073] like Figure 7As shown, the present invention also provides an electronic device 700. The electronic device 700 includes a processor 701 and a memory 702. Figure 7 Only some components of the electronic device 700 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0074] In some embodiments, processor 701 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 702 or process data, such as the multi-source heterogeneous data conflict resolution method of the present invention.

[0075] In some embodiments, processor 701 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 701 may be local or remote. In some embodiments, processor 701 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0076] In some embodiments, memory 702 may be an internal storage unit of electronic device 700, such as a hard disk or memory of electronic device 700. In other embodiments, memory 702 may also be an external storage device of electronic device 700, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 700.

[0077] Furthermore, the memory 702 may include both internal storage units of the electronic device 700 and external storage devices. The memory 702 is used to store application software and various types of data installed on the electronic device 700.

[0078] In one embodiment, when the processor 701 executes the multi-source heterogeneous data conflict resolution program in the memory 702, the following steps can be implemented: Multi-source heterogeneous data is mapped to a high-dimensional feature space by a pre-trained semantic encoder to generate observation feature tensors; Based on the observed feature tensor, a probability mapping is performed on each attribute dimension through a confidence metric function to jointly generate a coupled probability fingerprint. Based on the coupled probability fingerprint, an information entropy evolution equation is constructed. The information entropy evolution equation is combined with the weight matrix through energy functional to construct a nonlinear dynamic evolution system. The nonlinear dynamic evolution system is driven by gradient flow to perform nonlinear self-evolution. During the evolution process, the Lyapunov exponent is calculated, and the result of conflict resolution stability judgment is output. When the stability determination result of the conflict resolution is converged, unique data is generated based on the maximum a posteriori probability fusion; when the stability determination result of the conflict resolution is divergent, external logical constraints are introduced to correct the observed feature tensor and reinitialize it until convergence.

[0079] It should be understood that when the processor 701 executes the multi-source heterogeneous data conflict resolution program in the memory 702, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0080] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the multi-source heterogeneous data conflict resolution methods provided in the above-described method embodiments.

[0081] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0082] The foregoing has provided a detailed description of the multi-source heterogeneous data conflict resolution method, apparatus, electronic device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A multi-source heterogeneous data conflict resolution method, characterized in that, include: Multi-source heterogeneous data is mapped to a high-dimensional feature space by a pre-trained semantic encoder to generate observation feature tensors; Based on the observed feature tensor, a probability mapping is performed on each attribute dimension through a confidence metric function to jointly generate a coupled probability fingerprint. Based on the coupled probability fingerprint, an information entropy evolution equation is constructed. The information entropy evolution equation is combined with the weight matrix through energy functional to construct a nonlinear dynamic evolution system. The nonlinear dynamic evolution system is driven by gradient flow to perform nonlinear self-evolution. During the evolution process, the Lyapunov exponent is calculated, and the result of conflict resolution stability judgment is output. When the conflict resolution stability determination result is converged, unique data is generated based on the maximum a posteriori probability fusion. When the stability determination result of the conflict resolution is divergent, external logic constraints are introduced to correct the observed feature tensor and reinitialize it until convergence.

2. The multi-source heterogeneous data conflict resolution method of claim 1, wherein, The step of mapping multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder to generate an observation feature tensor also includes: calculating the semantic similarity between multi-source heterogeneous data using an alignment operator with position encoding.

3. The multi-source heterogeneous data conflict resolution method of claim 2, wherein, The step of generating a coupled probabilistic fingerprint based on the observed feature tensor, through a confidence metric function, performs probability mapping on each attribute dimension, including: Extract the feature values ​​of each data point in the observed feature tensor for each attribute dimension; By using a preset credibility metric function, the feature values ​​of each attribute dimension are quantified and transformed into credibility probability values ​​for the corresponding dimension. The credibility probability values ​​of all attribute dimensions are jointly calculated and integrated to obtain the coupled probability fingerprint.

4. The multi-source heterogeneous data conflict resolution method of claim 1, wherein, The process involves constructing an information entropy evolution equation based on the coupled probability fingerprint, combining the information entropy evolution equation with a weight matrix using an energy functional, constructing a nonlinear dynamic evolution system, and driving the nonlinear dynamic evolution system to undergo nonlinear self-evolution through gradient flow, including: Using the coupling probability fingerprint as the input to the information entropy evolution equation at the initial time, the information entropy evolution equation is constructed; A weight matrix is ​​introduced to constrain the evolution direction, and the global conflict energy level is calculated using the matrix trace; A nonlinear dynamic evolution system is constructed by coupling the information entropy in the information entropy evolution equation with the global conflict energy level calculated by the weight matrix through energy functional theory. The state of a nonlinear dynamic system evolves along the negative gradient direction of the energy functional by driving gradient flow.

5. The multi-source heterogeneous data conflict resolution method of claim 4, wherein, The calculation of the Lyapunov index and the output of the conflict resolution stability determination result include: During the evolution of the nonlinear dynamic evolution system, the Lyapunov exponent is calculated to quantify the sensitivity of the nonlinear dynamic evolution system to disturbances; If the Lyapunov exponent is less than zero, the nonlinear dynamic evolution system is determined to be in a stable convergent state; if the Lyapunov exponent is greater than zero, the nonlinear dynamic evolution system is determined to be in a divergent chaotic state.

6. The multi-source heterogeneous data conflict resolution method of claim 1, wherein, The method of introducing external logical constraints to modify the observed feature tensor and reinitialize it includes: Hard constraint tensors from independent data sources are extracted. The original observation feature tensors and hard constraint tensors are then weighted and fused using linear operators and confidence decay coefficients to generate the corrected observation feature tensors. Reinitialize based on the corrected observation feature tensor until convergence.

7. The multi-source heterogeneous data conflict resolution method of claim 6, wherein, The confidence decay coefficient is dynamically determined by the global average value of the coupled probability fingerprint. The higher the degree of chaos in the nonlinear dynamic evolution system, the closer the confidence decay coefficient is to zero, causing the nonlinear dynamic evolution system to turn to an evolution path driven by hard constraint tensors.

8. A multi-source heterogeneous data conflict resolution apparatus, characterized in that, include: The mapping module is used to map multi-source heterogeneous data to a high-dimensional feature space through a pre-trained semantic encoder, generating observation feature tensors. The probabilistic fingerprint generation module is used to generate coupled probabilistic fingerprints by performing probability mapping on each attribute dimension based on the observed feature tensor through a confidence metric function. The stability assessment module is used to construct an information entropy evolution equation based on the coupling probability fingerprint, combine the information entropy evolution equation with the weight matrix through energy functional to construct a nonlinear dynamic evolution system, drive the nonlinear dynamic evolution system to perform nonlinear self-evolution through gradient flow, calculate the Lyapunov exponent during the evolution process, and output the conflict resolution stability judgment result. The decision execution module is used to generate unique data based on maximum a posteriori probability fusion when the conflict resolution stability determination result is converged; and to introduce external logical constraints to correct the observed feature tensor and reinitialize it when the conflict resolution stability determination result is divergent, until convergence.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the multi-source heterogeneous data conflict resolution method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the multi-source heterogeneous data conflict resolution method according to any one of claims 1 to 7.

Images