A method and system for integrating multi-source heterogeneous data

Abstract

This application relates to the technical field of data integration, providing a method and system for integrating multi-source heterogeneous data, addressing the challenge of effectively coordinating data security control and business application flexibility during cross-domain data integration. The method includes: collecting multi-source heterogeneous data from different security domains and associating them with trust level identifiers and digital certificates; encrypting and transmitting the multi-source heterogeneous data according to the trust level identifiers to generate encrypted transmission data; verifying and decrypting the encrypted transmission data using Public Key Infrastructure (PKI) based on the digital certificate in the target security domain to obtain plaintext data, and then using key derivation functions to hierarchically store the plaintext data to obtain hierarchical data; employing a data integration algorithm, calling a preset master data management specification and semantic mapping table to perform entity merging and attribute alignment on the hierarchical data, generating trusted data assets. This application enables secure transmission, trusted processing, and intelligent integration of multi-source heterogeneous data.

Description

Technical Field

[0001] This application relates to the technical field of data integration, and in particular to a method and system for integrating multi-source heterogeneous data. Background Technology

[0002] With the deepening of enterprise digital transformation and the normalization of cross-organizational collaboration, the integration of multi-source heterogeneous data from different business systems, departments and even external partners has become the key to mining data value and supporting intelligent decision-making. It has broad application prospects in areas such as large group management, supply chain collaboration and government data sharing.

[0003] Currently, common data integration methods typically rely on a unified data access platform, which collects data through standardized data interfaces or middleware, and then cleans, transforms, and performs correlation analysis on the aggregated data in a centralized data storage system to achieve centralized data management and business applications.

[0004] However, in scenarios involving different security control areas and where the data itself has different sensitivity levels or business importance, the above methods focus on processing data format and content, but lack a refined security control mechanism that starts from the data source, runs through the entire process of transmission, storage and integration, and is linked to the data business attributes. This makes it difficult for the integration process to take into account the confidentiality and integrity requirements of the data and the flexibility of subsequent integrated applications, thereby affecting the reliability of data assets and business availability. Summary of the Invention

[0005] The purpose of this application is to provide a data integration method and system for multi-source heterogeneous data, so as to solve the problem that data security control and business application flexibility are difficult to effectively coordinate in the process of cross-domain data integration in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a data integration method for multi-source heterogeneous data, comprising: Collect multi-source heterogeneous data from at least two different security domains, and associate each piece of multi-source heterogeneous data with the trust level identifier and digital certificate of the security domain to which it belongs; Based on the trust level identifier, the hardware encryption module is invoked through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data, thereby generating encrypted transmission data; In the target security domain, based on the digital certificate, the encrypted transmission data is verified and decrypted using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then stored hierarchically using a key derivation function to obtain hierarchical data. A differentiated data integration algorithm is used to call the preset master data management specifications and preset semantic mapping table to perform entity merging and attribute alignment on the hierarchical data, thereby generating trusted data assets.

[0007] Optionally, in the target security domain, based on the digital certificate, the encrypted transmission data is verified and decrypted using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then hierarchically stored using a key derivation function to obtain hierarchical data, including: Within the target security domain, the certificate revocation list and verification points of the Public Key Infrastructure (PKI) are used to perform compliance verification of the digital certificate's status and policy, and the verification result is obtained. Once the verification result indicates that the verification is successful, the encrypted transmission data is decrypted using the private key corresponding to the digital certificate to obtain plaintext data. The data entity and data feature code are parsed from the plaintext data; The key derivation function is invoked to dynamically generate a one-time pad derived key based on the trust level identifier and the session random number extracted during the decryption process. Using the derived key, the data feature code is decrypted to obtain the level feature, and the level feature is fused with the data entity to obtain fused data with a privacy label; Based on the trust level identifier and the privacy tag, the fused data is stored in the corresponding level storage area, and storage-level encryption and access control policy binding are performed during storage to obtain hierarchical data with different security isolation and access permissions.

[0008] Optionally, the use of a Public Key Infrastructure (PKI) certificate revocation list and verification points to perform compliance verification of the digital certificate's status and policy, obtaining verification results, includes: The system invokes the online certificate status protocol service of the Public Key Infrastructure (PKI) and verifies the digital certificate through the certificate revocation list to obtain the certificate revocation status. Access the policy information of the Public Key Infrastructure (PKI) to obtain the certificate policy corresponding to the trust level identifier. The certificate policy includes at least: the set of allowed certificate authorities, key usage constraints, and certificate validity period threshold. Based on the trust level identifier and the certificate policy, generate policy constraints; The issuer identifier, key usage extension, and validity period of the digital certificate are compared and verified with the set of allowed certificate authorities, key usage constraints, and certificate validity period threshold in the policy constraints to obtain the policy determination result. A verification result is generated by combining the certificate revocation status with the policy determination result.

[0009] Optionally, the step of employing a differentiated data integration algorithm, calling a preset master data management specification and a preset semantic mapping table to perform entity merging and attribute alignment on the hierarchical data, and generating trusted data assets, includes: Based on the trust level identifier corresponding to each data subset in the hierarchical data, a differentiated processing path is assigned to each data subset through the strategy routing module of the data integration algorithm; A preset semantic mapping table is invoked, and level-aware semantic mapping is performed on the corresponding data subset under the processing path to obtain semantic data; Based on the preset master data management specifications, entity association and evidence synthesis processing are performed on the semantic data through the entity alignment model of the data integration algorithm to obtain the associated entity set; Based on the trust level identifier associated with the attributes of each entity in the associated entity set, attribute values ​​are selected and synthesized through the arbitration rules preset in the data integration algorithm to generate fused entity records. All fused entity records are then integrated to obtain trusted data assets.

[0010] Optionally, the step of generating a fused entity record by selecting and synthesizing attribute values ​​based on the trust level identifier associated with the attributes of each entity in the associated entity set, using a preset arbitration rule in the data integration algorithm, includes: Extract multiple candidate attribute values ​​corresponding to each entity attribute in the associated entity set; By using the arbitration rules in the data integration algorithm, and based on the trust level identifier associated with each candidate attribute value, a level constraint is generated between each candidate attribute value. The level constraints are input into the arbitration engine in the data integration algorithm, and the arbitration logic is executed by the arbitration engine to generate the target attribute values ​​of each entity attribute. All target attribute values ​​belonging to the same entity attribute are assembled to obtain a fused entity record of each entity attribute.

[0011] Optionally, the step of calling the hardware encryption module through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data based on the trust level identifier, and generating encrypted transmission data, includes: Based on the trust level identifier, configure differentiated transmission security strategies for the multi-source heterogeneous data; Based on the aforementioned transmission security strategy, a corresponding logical transmission channel is established for the multi-source heterogeneous data through a secure data exchange gateway; On the logical transmission channel, the multi-source heterogeneous data is encrypted in real time by a hardware encryption module to obtain encrypted data. The encrypted data is hierarchically packaged to obtain ciphertext data units, and an integrity check code is applied to the ciphertext data units to obtain encrypted transmission data.

[0012] Optionally, the step of performing real-time encryption on the multi-source heterogeneous data via a hardware encryption module on the logical transmission channel to obtain encrypted data includes: Obtain the channel encryption parameters bound to the logical transmission channel; Based on the channel encryption parameters, a channel session key is generated by a hardware encryption module; Using the channel session key, the multi-source heterogeneous data is encrypted and converted to generate encrypted data blocks; Channel identification information is added to the encrypted data block to obtain encrypted data.

[0013] Secondly, this application provides a data integration system for multi-source heterogeneous data, comprising: The data acquisition module is used to acquire multi-source heterogeneous data from at least two different security domains, and associate the trust level identifier and digital certificate of the security domain to which each piece of multi-source heterogeneous data belongs; The encryption module is used to call the hardware encryption module through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data according to the trust level identifier, and generate encrypted transmission data. The decryption module is used to perform certificate verification and decryption of the encrypted transmission data in the target security domain based on the digital certificate and using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then stored hierarchically using a key derivation function to obtain hierarchical data. The alignment module is used to employ differentiated data integration algorithms, call preset master data management specifications and preset semantic mapping tables to perform entity merging and attribute alignment on the hierarchical data, and generate trusted data assets.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of a data integration method for multi-source heterogeneous data as described in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of a data integration method for multi-source heterogeneous data as described in the first aspect above.

[0016] The data integration method for multi-source heterogeneous data provided in this application has the following beneficial effects: This application first marks the trustworthiness and identity of data from different security domains by associating trust level identifiers and digital certificates, thus laying the foundation for subsequent differentiated processing; then, based on the identifier, it calls the secure data exchange gateway and hardware encryption module to perform differentiated encrypted transmission, which can ensure that data of different importance levels receive matching security protection during transmission, thereby generating secure encrypted transmission data; Next, the digital certificate is verified and decrypted using Public Key Infrastructure (PKI) in the target domain, which confirms the authenticity and integrity of the data source, thus obtaining trusted plaintext data. Subsequently, the plaintext data is hierarchically stored using key derivation functions, which enables data to be classified, isolated and protected according to security levels, resulting in structured hierarchical data. Finally, a differentiated integration algorithm is used, and the master data management specification and semantic mapping table are called to merge entities and align attributes of the hierarchical data. This enables intelligent data fusion under the premise of ensuring security policies, thereby generating high-quality trusted data assets.

[0017] Furthermore, this application ensures high trustworthiness of the decryption operation by performing dual verification, including revocation status and level policies, on the digital certificate in the target security domain. Then, after decryption, a one-time pad derived key is dynamically generated based on the trust level identifier and temporary random number, and the embedded data feature code is decrypted and fused to achieve data content enhancement and privacy labeling. Finally, according to the level and privacy label, the fused data is stored in the corresponding storage area and bound with fine-grained access control, so that the output hierarchical data has inherent security isolation and differentiated usage permissions, thereby providing a high-quality data foundation with both security and usability for subsequent integration. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a data integration method for multi-source heterogeneous data provided in an embodiment of this application; Figure 2 A schematic diagram illustrating a specific implementation of a data integration method for multi-source heterogeneous data provided in this application embodiment; Figure 3 A schematic diagram of the structure of a data integration system for multi-source heterogeneous data provided in this application embodiment; Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] In cross-departmental and cross-organizational enterprise data integration scenarios, the convergence and utilization of multi-source heterogeneous data is the foundation for supporting business collaboration and intelligent decision-making. However, when data comes from independent domains with different security policies and control levels, existing integration methods adopt a unified access, transmission and processing process, lacking differentiated security mechanisms that run through the entire data lifecycle and are adapted to the importance of the data itself. This leads to the risk of leakage of highly sensitive data during transmission and storage, and data of different credibility may also interfere with each other during mixed processing, ultimately affecting the overall reliability and business availability of the integrated data assets.

[0021] To address this, this application proposes a data integration method for multi-source heterogeneous data. The core idea is to assign trust level identifiers and digital certificates to data from different sources, and dynamically select matching secure transmission channels and encryption strategies based on these levels. Then, in the target domain, the data source is verified and decrypted using public key infrastructure, followed by hierarchical security storage based on the levels. Finally, differentiated integration rules and semantic mappings are invoked to achieve entity alignment and attribute fusion of the data while ensuring secure isolation. This method can achieve precise matching of security control strength and data business attributes throughout the entire data flow process across security domains, thereby improving the consistency and trustworthiness of the integrated data while ensuring the security of sensitive data. This effectively overcomes the limitation of existing technologies where security and availability are difficult to coordinate.

[0022] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The core of this application is to provide a data integration method for multi-source heterogeneous data, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: S101. Collect multi-source heterogeneous data from at least two different security domains, and associate the trust level identifier and digital certificate of the security domain to which each piece of multi-source heterogeneous data belongs.

[0024] Among them, a security domain refers to a logically independent data environment with a unified security control strategy, such as the networks of different departments within a company or the systems of external partners; multi-source heterogeneous data refers to raw business data with different formats and structures originating from these different security domains, which may include tables in databases, streaming records uploaded by sensors, and document files, etc. A trust level identifier is a predefined classification label for each piece of data based on the security domain level of the data source, the sensitivity of the data, and the business importance. It is used to identify and distinguish the trustworthiness and security level of data throughout its entire lifecycle. For example, in this application, the trust level identifier can be divided into three levels: Level 1: Applicable to extremely sensitive data involving core trade secrets, whose source domain has the highest level of security control and requires the strictest encryption and access control strategies; Level 2: Applicable to sensitive internal enterprise data, such as financial information and human resource data, whose source domain has relatively high security control and requires strong encryption and access control strategies; Level 3: Applicable to low-sensitivity business data such as public data, whose source domain has basic security control and can adopt conventional encryption and access control strategies. A digital certificate is an electronic credential issued by an authoritative certification authority. It can be understood as a data identity card, used to uniquely and securely prove the source of the data on the network and ensure that it is not tampered with during transmission.

[0025] In step S101, firstly, various types of raw data that need to be integrated are automatically collected from the specified business systems and data sources through data interfaces or collection agents deployed in each security domain, thereby obtaining an unprocessed multi-source heterogeneous data set.

[0026] Then, based on the preset rules of the security domain to which each piece of multi-source heterogeneous data belongs, a corresponding trust level identifier is automatically attached to it, which is equivalent to labeling the data with a tag indicating its importance or confidentiality level.

[0027] At the same time, a digital certificate issued by the security domain certification center is applied for or bound to each piece of multi-source heterogeneous data. This certificate is associated with and encapsulated with the data itself and its level label, and together they constitute the complete identity credential and security attribute statement when the data is sent from the source, and provide the foundation for subsequent differentiated secure transmission and trusted verification.

[0028] S102. Based on the trust level identifier, the hardware encryption module is invoked through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data, thereby generating encrypted transmission data.

[0029] In one specific implementation, step S102 includes: Step 1021: Based on the trust level identifier, configure differentiated transmission security policies for the multi-source heterogeneous data.

[0030] Among them, the transmission security strategy refers to a set of security rules and configurations predefined for the data transmission process. It clearly stipulates the specific technical parameters such as the encryption algorithm, key strength, and whether replay protection is required for data with different security requirements.

[0031] In step 1021, the system maintains a policy mapping table that maps different trust level identifiers to specific cryptographic requirements. Then, the trust level identifier attached to each piece of multi-source heterogeneous data is read and used as the key to query the policy mapping table, thereby automatically and accurately obtaining the complete set of security configurations preset for that level of data.

[0032] For example, a database record from S101, identified as Level 1 trusted, is configured with a policy of "Algorithm=SM4, Mode=CBC, Key Length=256 bits" after querying the policy mapping table. Meanwhile, another log file identified as Level 3 trusted is configured with a policy of "Algorithm=AES, Mode=CBC, Key Length=128 bits".

[0033] Step 1022: Based on the transmission security policy, establish a corresponding logical transmission channel for the multi-source heterogeneous data through a secure data exchange gateway.

[0034] Among them, a secure data exchange gateway refers to a dedicated device deployed at the network boundary to establish a controlled data exchange path between different security domains; a logical transmission channel refers to a logically independent virtual data transmission pipeline created on a physical network through software-defined technology.

[0035] In step 1022, the secure data exchange gateway receives the data and its bound security policy from step 1021. Then, the security data exchange gateway's policy engine parses the content of the security policy and dynamically schedules its internal network resources according to the security level strength defined therein to create or allocate corresponding logical transmission channels, thereby isolating data of different security levels on the logical transmission path.

[0036] For example, a secure data exchange gateway creates a separate logical channel for database records carrying a Level 1 trusted policy. Simultaneously, a shared logical channel was allocated for log file data carrying the Level 3 Trust Policy. .

[0037] Step 1023: On the logical transmission channel, the multi-source heterogeneous data is encrypted in real time by a hardware encryption module to obtain encrypted data.

[0038] Step 1023 may specifically include the following steps: Step a1: Obtain the channel encryption parameters bound to the logical transmission channel.

[0039] Among them, channel encryption parameters refer to cryptographic configuration information bound to a specific logical transmission channel, such as the specified encryption algorithm and working mode.

[0040] In step a1, specific parameters such as the preset encryption algorithm and working mode are extracted from the configuration information of the established logical channel to serve as inputs for subsequent key generation and encryption execution. For example, for The parameters to be bound to it are: Algorithm=SM4, Mode=CBC.

[0041] Step a2: Based on the channel encryption parameters, generate a channel session key through a hardware encryption module.

[0042] Among them, the hardware encryption module refers to the physical security chip used to perform high-strength cryptographic operations; the channel session key refers to the temporary key dynamically generated for this transmission.

[0043] In step a2, the hardware encryption module receives the channel encryption parameters and, within its internal secure environment, generates a random number seed using a true random number generator. Based on the algorithm type specified in the parameters, it generates a completely new encryption key, usable only for the current channel session, through a key derivation process. For example, the hardware encryption module... The parameters were used to generate a 256-bit random key K. A .

[0044] Step a3: Using the channel session key, encrypt and convert the multi-source heterogeneous data to generate an encrypted data block.

[0045] In step a3, the hardware encryption module uses the session key generated for this channel to perform real-time block encryption on the data byte stream flowing through the channel according to the specified algorithm and operating mode, and converts each input plaintext data block into a corresponding ciphertext data block. For example, for the plaintext P database record uploaded by ChA... A1 Using key K A Encryption is performed using the SM4-CBC algorithm, with the initialization vector set to IV. The encryption process can be represented as C. A1 =Enc SM4 (K A P A1 ⊕IV), to obtain ciphertext block C A1 , among which, Enc SM4 ⊕ represents the SM4 encryption algorithm, and ⊕ represents the XOR operation.

[0046] Step a4: Add channel identification information to the encrypted data block to obtain encrypted data.

[0047] The channel identifier is a short label added to the header of a data block to identify its source channel.

[0048] In step a4, a header containing the unique ID of the logical channel from which each encrypted data block is generated is added before the encrypted data block, thereby forming an encrypted data unit with a clear path attribution.

[0049] For example, for ciphertext block C A1 Add header information [CID: A] to form the final encrypted data unit E. A =[CID:A∣C A1 ].

[0050] Step 1024: Perform hierarchical encapsulation on the encrypted data to obtain ciphertext data units, and apply an integrity check code to the ciphertext data units to obtain encrypted transmission data.

[0051] Among them, hierarchical encapsulation refers to adding metadata containing level information to the outer layer of encrypted data packets according to the trust level identifier of the data; integrity check code refers to a value calculated by a hash function, which is used to verify whether the data has been tampered with.

[0052] In step 1024, an encapsulation header containing the original trust level identifier of the encrypted data unit is first added to it; then a cryptographic hash value is calculated for the entire encapsulated data unit as an integrity check code and appended to the end of the data, thereby forming the final encrypted transmission data that is ready to be sent and has both identity identification and tamper-proof capabilities.

[0053] For example, for the encryption unit E from first-level trusted data A Add the encapsulation header [Level: 1] to get the U A =[Level:1∣E A Next, calculate its SM3 hash value H. A =Hash SM3 (U A Finally, encrypted transmission data T is generated. A =[U A |H A The receiver can then recalculate the new hash value. =Hash SM3 (U A And verify Is it equal to H? A To verify the integrity of the data.

[0054] This application achieves differentiated security protection for data of varying importance during transmission by dynamically and precisely matching the business trust level of data with the security policies, channel resources, and encryption strength of the transport layer. At the same time, it ensures high performance and high security of the encryption process through hardware encryption modules and ensures the immutability of data after transmission through integrity verification, thus laying a solid security foundation for subsequent trusted verification and decryption.

[0055] S103. In the target security domain, based on the digital certificate, the encrypted transmission data is verified and decrypted using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then stored hierarchically using a key derivation function to obtain hierarchical data.

[0056] Public Key Infrastructure (PKI) refers to a trust framework that manages the entire process of digital certificates from issuance to expiration.

[0057] In one specific implementation, such as Figure 2 As shown, step S103 includes: Step 1031: In the target security domain, the digital certificate is verified for compliance with status and policy using the certificate revocation list and verification points of the Public Key Infrastructure (PKI) to obtain the verification result.

[0058] Verification points refer to network interfaces that can provide policy information query services.

[0059] Step 1031 may specifically include the following steps: Step b1: Call the online certificate status protocol service of Public Key Infrastructure (PKI) and check the digital certificate through the certificate revocation list to obtain the certificate revocation status.

[0060] The Certificate Revocation List is a list published periodically by a Certificate Authority that lists all expired or revoked certificate serial numbers. The Online Certificate Status Protocol (OSP) service is a service that allows users to check the validity of a specific certificate in real time. It complements the Certificate Revocation List and provides a faster way to check the status.

[0061] In step b1, a query request is sent to the designated online certificate status protocol service responder. The request explicitly carries the unique serial number of the digital certificate to be verified. The responder then queries its maintained real-time revocation database and returns the current status response of the online certificate, such as normal, revoked, or unknown. This response is the certificate revocation status.

[0062] For example, if a query request for serial number SN001 is sent to the online certificate status protocol service and a normal response is received, then the certificate revocation status is normal.

[0063] Step b2: Access the policy information of the Public Key Infrastructure (PKI) to obtain the certificate policy corresponding to the trust level identifier. The certificate policy includes at least: the set of allowed certificate authorities, key usage constraints, and certificate validity period threshold.

[0064] In step b2, a query is initiated to the policy verification point in the public key infrastructure (PKI) based on the trust level identifier carried in the data. Subsequently, the verification point retrieves and returns the certificate policy pre-configured for this level based on the identifier. This certificate policy explicitly specifies the acceptable range of certificate authorities, restrictions on the fields for the use of certificate keys, and the minimum remaining validity period that the certificate must have.

[0065] For example, based on the secondary trusted identifier, the corresponding certificate policy is obtained from the policy verification point in the Public Key Infrastructure (PKI): the allowed set of issuing authorities is {Issuing Authority A, Issuing Authority B}; the key usage must include "data encryption"; and the remaining validity period of the certificate must be greater than 30 days.

[0066] Step b3: Generate policy constraints based on the trust level identifier and the certificate policy.

[0067] In step b3, the obtained certificate policy is parsed, and each provision is transformed into a specific constraint expression that can be used to determine whether it is true or false. This expression serves as a policy constraint condition and is used for subsequent automated comparison with the actual attributes of the certificate.

[0068] For example, the obtained certificate policy can be transformed into three policy constraints: the certificate issuing authority is {issuing authority A, issuing authority B}; the certificate's key usage extension includes "data encryption"; and the certificate's remaining validity period (Tremain) is greater than 30 days.

[0069] Step b4: Compare and verify the issuer identifier, key usage extension, and validity period range of the digital certificate with the set of allowed certificate authorities, key usage constraints, and certificate validity period threshold in the policy constraints to obtain the policy determination result.

[0070] In step b4, firstly, the actual issuer identifier, key usage extension list, and certificate validity period are extracted from the digital certificate to be verified. Then, these actual attribute values ​​are compared one by one with each policy constraint generated in the previous step. Each comparison is a logical judgment; the overall policy judgment result is passed only when all comparison items are judged as "true"; otherwise, it fails.

[0071] For example, from the digital certificate with serial number SN001, the following can be extracted: the issuer identifier is "Issuing Authority A"; the key usage includes "digital signature" and "data encryption"; the certificate validity period is remaining Tremain=200 days; then this part is compared with each policy constraint generated in the previous step: "Issuing Authority A" belongs to Issuing Authority A in {Issuing Authority A, Issuing Authority B}, which is correct; the key usage includes "data encryption", which is correct; 200>30, which is correct. Therefore, all constraints are satisfied, and the policy decision result is passed.

[0072] Step b5: Combine the certificate revocation status with the policy determination result to generate a verification result.

[0073] In step b5, the certificate revocation status from step b1 and the policy judgment result from step b4 are received as inputs. The system will only generate a final verification result of "pass" if the certificate revocation status is normal and the policy judgment result is "pass". If either condition is not met, the final verification result will be "fail". For example, the final verification result of "pass" is generated by combining the normal certificate revocation status and the "pass" policy judgment result.

[0074] Step 1032: When the verification result indicates that the verification is successful, the encrypted transmission data is decrypted using the private key corresponding to the digital certificate to obtain plaintext data.

[0075] The private key is a password generated in pairs with the public key in the digital certificate and kept secret only by the certificate holder, specifically for decryption.

[0076] In step 1032, the private key paired with the verified digital certificate is retrieved from the secure key storage only if the verification result is successful. Then, using the private key and following the parameters such as the type of symmetric encryption algorithm, working mode, and initialization vector used in step S102 for encrypted data transmission, the received encrypted data transmission is decrypted in reverse. This process first verifies and strips the integrity check code and hierarchical encapsulation header outside the data packet, then decrypts the core ciphertext payload, and finally restores the structured original plaintext data.

[0077] For example, since the verification result is successful, the system obtains the private key Kpri that is paired with certificate SN001; then Kpri is used to decrypt the encrypted transmitted data, and its algorithm is the same as that of S102, thereby obtaining the plaintext data Dplain.

[0078] Step 1033: Parse the data entity and data feature code from the plaintext data.

[0079] Among them, the data entity refers to the main part of the decrypted data, which contains the core business information; the data signature is a section of specially encrypted data following the data entity, which contains additional description information added by the source system.

[0080] In step 1033, according to the encapsulation format specification negotiated and agreed with the data sender in advance, the plaintext data obtained by decryption is parsed. Specifically, it means that the parser identifies specific delimiters or length fields, and accurately divides the complete data stream into two parts: the first part is the structured business information, that is, the data entity; the second part is an encrypted and non-transparent binary data block, that is, the data signature. The two are separated for subsequent different processing flows.

[0081] For example, parsing the format of the plaintext data Dplain to obtain the JSON format data entity Entity={company: Company A, value: 100} and the binary data signature Feature.

[0082] Step 1034: Call the key derivation function to dynamically generate a one-time pad derived key based on the trust level identifier and the session random number extracted during the decryption process.

[0083] Among them, the key derivation function is a standard algorithm that can securely generate encryption keys from initial secret information, such as passwords or random numbers. It should be noted that the key derivation function can be implemented using standard cryptographic key derivation algorithms. The focus of this application is on the composition of its input and the application scenario, and the specific implementation process of the function can refer to relevant technical standards, which will not be elaborated here; The session random number is a random number specifically generated for this communication during the encrypted transmission process in step S102, which is different for each transmission; the one-time pad means that the generated key is only used once and becomes invalid after this use, greatly improving security.

[0084] In step 1034, first, extract the unique "session random number" generated by the sender in step S102 from the metadata of the current transmission session or the decrypted header information; at the same time, obtain the trust level identifier of the current data item; then use these two as input materials and call the preset key derivation function. This function performs a series of cryptographic hashing and mixing operations, and finally outputs a key with a fixed length and high cryptographic strength, that is, the "derived key". This key is only bound to this specific data transmission session and business level, implementing the "one-time pad".

[0085] For example, extract the session random number R = 0x5f3a and the trust level identifier L = second-level trusted, and then call the key derivation function to generate the derived key Kder = KDF(L, R).

[0086] Step 1035: Decrypt the data feature code using the derived key to obtain the level feature, and fuse the level feature with the data entity to obtain fused data with a privacy label.

[0087] Among them, graded features refer to the readable information obtained after decrypting the data feature code, which usually describes the security or business attributes of the data; privacy labels are a common type of graded features, used to indicate the privacy protection level of the data, such as public, internal, and secret; fusion refers to adding the decrypted graded feature information as a new field to the original data entity.

[0088] In step 1035, the derived key generated in the previous step is used, and the data feature code is decrypted using an algorithm that matches the encryption of the source domain. After successful decryption, structured hierarchical feature information is obtained, which may include fields such as data sensitivity and processing permission indication. Subsequently, the system extracts key attributes, such as privacy tags, from it, and then uses these attributes as new key-value pairs and merges them into the previously parsed data entity object, thereby generating a richer "fused data" object that carries the security context of the source domain.

[0089] For example, using the derived key Kder to decrypt the data feature code yields the hierarchy feature {privacyLabel: internally sensitive}, which is then fused with the data entity to obtain the fused data FusedData. company: Company A, value: 100, privacyLabel: Internal Sensitive}.

[0090] Step 1036: Based on the trusted level identifier and the privacy tag, store the fused data in the corresponding level storage area, and perform storage-level encryption and access control policy binding during storage to obtain hierarchical data with different security isolation and access permissions.

[0091] Among them, graded storage areas refer to pre-divided and isolated storage spaces based on different data security levels; storage-level encryption refers to encrypting the data again when it is finally saved to the hard drive or database to ensure the security of static data; access control policies refer to the rules that specify who can access the data under what conditions and how; and hierarchical data refers to data that has completed all security processing and rule binding and is finally stored in the corresponding security area.

[0092] In step 1036, predefined storage routing rules are queried based on the trust level identifier and privacy tag attributes embedded in the fused data object to determine the target level storage area where it should be persistently stored. Then, before the data is written to the storage area, it is encrypted at the storage level using a key specific to that storage area to ensure the confidentiality of the data in a static storage state. At the same time, based on the data's level and tag combination, a matching access control policy is retrieved from the policy library and associated with it. This policy clarifies the access permission rules for this data. Finally, this encrypted data record bound to the access policy is stored in the designated storage area, becoming hierarchical data subject to security isolation and access control.

[0093] It should be noted that storage-level encryption can be implemented using technologies such as transparent data encryption or field-level encryption; access control policy binding can be implemented by associating policy identifiers in the metadata of database records or by using cryptographic schemes such as attribute-based encryption. This application emphasizes the association between this operation and data level and privacy label. For specific technology selection and implementation, please refer to relevant technologies, which will not be elaborated here.

[0094] For example, based on the trusted level identifier (Level 2 Trusted), privacy label, and internal sensitivity in the fused data FusedData, it is routed to the preset Level 2 storage area. Before writing, FusedData is encrypted at the storage level using the dedicated key Kstore of Level 2 storage area, resulting in the encrypted result EncData. At the same time, based on the above attributes, the access control policy "Policy02: Only department head can read" is matched. After binding EncData with "Policy02", it is stored in Level 2 storage area, and this record becomes the final hierarchical data.

[0095] This application ensures the credibility of data sources through rigorous certificate verification and policy compliance checks. By utilizing dynamic key derivation and hierarchical storage mechanisms, it can achieve secure decryption, content enhancement, and controlled storage of data in the target domain, ultimately forming hierarchical data with secure isolation and clear permissions.

[0096] S104. Using a differentiated data integration algorithm, the system calls the preset master data management specifications and the preset semantic mapping table to perform entity merging and attribute alignment on the hierarchical data, thereby generating trusted data assets.

[0097] Among them, data integration algorithms refer to a set of programmed computer instructions used to merge, transform, and unify data from different sources according to specific rules.

[0098] In one specific implementation, step S104 includes: Step 1041: Based on the trust level identifier corresponding to each data subset in the hierarchical data, a differentiated processing path is assigned to each data subset through the strategy routing module of the data integration algorithm.

[0099] Among them, the strategy routing module is a functional component in the data integration algorithm. It assigns different subsequent processing flows to different types of data according to predefined business rules.

[0100] In step 1041, the policy routing module receives the hierarchical data and its associated trust level identifier. Then, the policy routing module maintains a routing configuration table, which defines the processing pipelines corresponding to different trust levels. The policy routing module then queries this configuration table based on the level identifier of each piece of data or subset of data and assigns it to the corresponding processing path. For example, high-trust data may be assigned to a path that includes stricter verification and more complex fusion logic, while low-trust data may be assigned to a path that focuses more on basic alignment and simple verification.

[0101] For example, suppose there are two sets of hierarchical data: data A is identified as Level 1 and data B is identified as Level 3. The policy routing module then assigns data A to processing path A according to the configuration, which will subsequently use the alignment model; and assigns data B to processing path B, which uses the fast alignment algorithm.

[0102] Step 1042: Call the preset semantic mapping table and perform level-aware semantic mapping on the corresponding data subset under the processing path to obtain semantic data.

[0103] Among them, the preset semantic mapping table refers to a predefined data dictionary or set of rules that specifies how to map fields from different data sources with different names but the same meaning to a standardized name and definition. It should be noted that the construction and maintenance of semantic mapping tables are common techniques in the field of data governance. They can include manually defined mapping rules, ontology-based automated mapping, or mapping relationships learned from historical alignment results. The focus of this application is to introduce hierarchy awareness in the mapping process. The specific construction method of the mapping table can be referred to relevant technologies, which will not be elaborated here. Level awareness refers to the process of performing semantic mapping that considers not only the correspondence between field names, but also the confidence level of the data itself. This may involve using slightly different mapping rules or confidence levels for data of different levels.

[0104] In step 1042, a preset semantic mapping table is invoked under the assigned processing path. This mapping table may contain multiple mapping rules, each of which specifies the source field name, target field name, mapping conditions, and an optional confidence weight related to the data level. Then, each field of the input data subset is traversed, and a matching rule is searched in the mapping table according to its name and the confidence level of the data it belongs to. The field is then renamed to a standard field name. Finally, after all fields are mapped, semantic data with a unified structure is obtained.

[0105] For example, for data A on processing path A, its original field CoName is mapped to the target field CompanyName according to the mapping table. After the mapping, semantic data with the CompanyName field is generated.

[0106] Step 1043: Based on the preset master data management specifications, the semantic data is processed by entity association and evidence synthesis through the entity alignment model of the data integration algorithm to obtain the associated entity set.

[0107] Among them, the preset master data management specification refers to the data standard defined by the enterprise for core business entities, which includes unique identification rules, key attribute definitions and data models; the entity alignment model refers to an algorithm or rule set used to determine whether two or more data records point to the same entity in the real world; entity association refers to the process of identifying and linking different data records that point to the same entity. Evidence synthesis refers to the process of integrating multiple pieces of evidence from different data sources and with different levels of confidence during the alignment process, and calculating a comprehensive confidence level through methods such as weighting, voting, or probability models to support or refute the judgment that these records point to the same entity.

[0108] In step 1043, firstly, the entity alignment model performs pairwise comparisons or clustering analysis on all records in the semantic data based on the entity identification code or a set of key attributes defined in the master data management specification, and generates a preliminary judgment to generate multiple pieces of evidence, such as text similarity scores, attribute consistency Boolean values, and the credibility level weight of the data source. Then, through a preset synthesis logic and formula, the multiple pieces of evidence are combined to calculate a comprehensive matching confidence level representing the reliability of the association between records. When the confidence level exceeds a preset threshold, it is determined that these records point to the same entity and are merged into a set of associated entities. This set contains all candidate records pointing to the same entity and their comprehensive confidence levels.

[0109] It should be noted that the entity alignment model can be implemented using existing entity parsing techniques. The core of this application lies in introducing the credibility level as a weighting factor for alignment evidence. The specific implementation details of the model can be found in relevant technologies, and will not be elaborated here.

[0110] For example, consider two semantic data records: record T1 comes from data M at level 1, and record T2 comes from data N at level 3. They are then compared using an entity alignment model, generating evidence: company name similarity score Sn = 0.95, registered address consistency Ba = 1 (indicating consistency), and an average confidence level weight Wavg of 0.8. A pre-defined weighted composite formula J = α × Sn + β × Ba + γ × Wavg is then used to calculate the overall confidence level, where α, β, and γ are pre-defined weight coefficients. Assuming α = 0.5, β = 0.3, and γ = 0.2, then J = 0.5 × 0.95 + 0.3 × 1 + 0.2 × 0.8 = 0.935. A judgment threshold θ = 0.85 is then set. Since J > θ, records T1 and T2 are determined to point to the same entity, forming an associated entity set ES1 = {R1, R2} with a confidence level of 0.935.

[0111] Step 1044: Based on the trust level identifier associated with the attributes of each entity in the associated entity set, select and synthesize attribute values ​​through the preset arbitration rules in the data integration algorithm to generate fused entity records, integrate all the fused entity records, and obtain trusted data assets.

[0112] Among them, each entity attribute refers to the various characteristic fields that describe an entity. For example, for a company entity, it includes registered capital, establishment date, industry, etc. The preset arbitration rules refer to a set of predefined decision logic used to decide which value to adopt or how to generate a composite value when there are multiple different candidate values ​​for the same attribute of the same entity. These rules usually consider the credibility level of the data from which the candidate values ​​are sourced.

[0113] Step 1044 may specifically include the following steps: Step c1: Extract multiple candidate attribute values ​​corresponding to each entity attribute in the associated entity set.

[0114] In step c1, for a given set of associated entities and a specified entity attribute, each data record in the set is traversed and the value of the attribute is read. All the read values ​​are then gathered into a list, which is the list of multiple candidate attribute values ​​for the current entity.

[0115] For example, extract a list of multiple candidate attribute values ​​for the registered capital attribute from the associated entity set ES1.

[0116] Step c2: Using the arbitration rules in the data integration algorithm, generate level constraints between the candidate attribute values ​​based on the trust level identifier associated with each candidate attribute value.

[0117] In step c2, the arbitration rule module in the data integration algorithm receives a list of candidate values ​​and the credibility level identifiers of their respective source records. Then, the arbitration rule engine generates explicit level constraints for the relationship between these candidate values ​​according to a preset strategy. For example, the rule may generate a weight constraint, that is, assign a weight coefficient to each candidate value, which is a monotonically increasing function of its source credibility level.

[0118] For example, the arbitration rules stipulate that the first-level credibility weight is W1=1.0 and the third-level credibility weight is W3=0.6. Based on this, level constraints are generated for candidate values: the weight of the value "10 million yuan" is wa=1.0; the weight of the value "12 million yuan" is wb=0.6.

[0119] Step c3: Input the level constraints into the arbitration engine in the data integration algorithm, execute the arbitration logic through the arbitration engine, and generate the target attribute values ​​of each entity attribute.

[0120] It should be noted that the arbitration rules and engine can be implemented using existing data fusion technologies such as priority rules, weighted averages, confidence level voting, or more complex decision-making models. The innovation of this application lies in using the confidence level identifier as a key input parameter for arbitration. The specific implementation of its arbitration logic can be referred to relevant technologies, which will not be elaborated here.

[0121] In step c3, after receiving the candidate value list and level constraints, the arbitration engine implements specific arbitration logic internally, such as prioritizing the highest weight value. Then, the engine performs calculations based on the constraints and finally outputs a definite target attribute value.

[0122] For example, the arbitration engine adopts the "highest weight priority" logic, and according to the weight constraints of the input, such as wa=1.0 and wb=0.6, the engine selects "10 million yuan" with higher weight as the target attribute value Vtarget.

[0123] Step c4: Assemble all target attribute values ​​belonging to the same entity attribute to obtain a fused entity record of each entity attribute.

[0124] In step c4, after determining the target attribute values ​​for all attributes within a set of related entities, these values ​​are assembled according to a predetermined data model structure to form a complete, conflict-free data record object, i.e., a merged entity record.

[0125] This application integrates scattered, heterogeneous, and potentially conflicting hierarchical data into high-quality, non-duplicative, and attribute-authoritative trusted data assets through a grade-driven differentiated processing path, precise semantic mapping, entity alignment based on multi-evidence synthesis, and intelligent arbitration based on trust level. This provides a solid and reliable data foundation for upper-layer business applications.

[0126] Figure 3 This is a schematic diagram illustrating a specific implementation of a data integration system for multi-source heterogeneous data provided in this application. (Refer to...) Figure 3 The system may include: The acquisition module 31 is used to acquire multi-source heterogeneous data from at least two different security domains, and associate the trust level identifier and digital certificate of the security domain to which each piece of multi-source heterogeneous data belongs.

[0127] The encryption module 32 is used to call the hardware encryption module through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data according to the trust level identifier, and generate encrypted transmission data.

[0128] The decryption module 33 is used to perform certificate verification and decryption of the encrypted transmission data in the target security domain based on the digital certificate and using Public Key Infrastructure (PKI) to obtain plaintext data, and to perform hierarchical storage of the plaintext data using key derivation functions to obtain hierarchical data.

[0129] Alignment module 34 is used to employ differentiated data integration algorithms, call preset master data management specifications and preset semantic mapping tables to perform entity merging and attribute alignment on the hierarchical data, and generate trusted data assets.

[0130] The data integration system for multi-source heterogeneous data in this application is used to implement the aforementioned data integration method for multi-source heterogeneous data. Therefore, the specific implementation of the data integration system for multi-source heterogeneous data can be found in the embodiment section of the data integration method for multi-source heterogeneous data above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0131] like Figure 4 As shown, this application also provides an electronic device, including: a memory 41 for storing a computer program; and a processor 42 for executing the computer program to implement the steps of the data integration method for multi-source heterogeneous data described above.

[0132] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for integrating multi-source heterogeneous data.

[0133] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0134] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the data integration method for multi-source heterogeneous data.

[0135] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0136] The data integration method and system for multi-source heterogeneous data provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for integrating multi-source heterogeneous data, characterized in that, include: Collect multi-source heterogeneous data from at least two different security domains, and associate each piece of multi-source heterogeneous data with the trust level identifier and digital certificate of the security domain to which it belongs; Based on the trust level identifier, the hardware encryption module is invoked through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data, thereby generating encrypted transmission data; In the target security domain, based on the digital certificate, the encrypted transmission data is verified and decrypted using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then stored hierarchically using a key derivation function to obtain hierarchical data. A differentiated data integration algorithm is used to call the preset master data management specifications and preset semantic mapping table to perform entity merging and attribute alignment on the hierarchical data, thereby generating trusted data assets; The process employs a differentiated data integration algorithm, invoking a preset master data management specification and a preset semantic mapping table to perform entity merging and attribute alignment on the hierarchical data, generating trusted data assets, including: Based on the trust level identifier corresponding to each data subset in the hierarchical data, a differentiated processing path is assigned to each data subset through the strategy routing module of the data integration algorithm; A preset semantic mapping table is invoked, and level-aware semantic mapping is performed on the corresponding data subset under the processing path to obtain semantic data; Based on the preset master data management specifications, entity association and evidence synthesis processing are performed on the semantic data through the entity alignment model of the data integration algorithm to obtain the associated entity set; Based on the trust level identifier associated with the attributes of each entity in the associated entity set, attribute values ​​are selected and synthesized through the arbitration rules preset in the data integration algorithm to generate fused entity records. All fused entity records are then integrated to obtain trusted data assets.

2. The data integration method for multi-source heterogeneous data according to claim 1, characterized in that, Within the target security domain, based on the digital certificate, the encrypted transmission data is verified and decrypted using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then hierarchically stored using a key derivation function to obtain hierarchical data, including: Within the target security domain, the certificate revocation list and verification points of the Public Key Infrastructure (PKI) are used to perform compliance verification of the digital certificate's status and policy, and the verification result is obtained. Once the verification result indicates that the verification is successful, the encrypted transmission data is decrypted using the private key corresponding to the digital certificate to obtain plaintext data. The data entity and data feature code are parsed from the plaintext data; The key derivation function is invoked to dynamically generate a one-time pad derived key based on the trust level identifier and the session random number extracted during the decryption process. Using the derived key, the data feature code is decrypted to obtain the level feature, and the level feature is fused with the data entity to obtain fused data with a privacy label; Based on the trust level identifier and the privacy tag, the fused data is stored in the corresponding level storage area, and storage-level encryption and access control policy binding are performed during storage to obtain hierarchical data with different security isolation and access permissions.

3. The data integration method for multi-source heterogeneous data according to claim 2, characterized in that, The certificate revocation list and verification points using Public Key Infrastructure (PKI) are used to perform compliance verification of the digital certificate's status and policy, obtaining verification results, including: The system invokes the online certificate status protocol service of the Public Key Infrastructure (PKI) and verifies the digital certificate through the certificate revocation list to obtain the certificate revocation status. Access the policy information of the Public Key Infrastructure (PKI) to obtain the certificate policy corresponding to the trust level identifier. The certificate policy includes at least: the set of allowed certificate authorities, key usage constraints, and certificate validity period threshold. Based on the trust level identifier and the certificate policy, generate policy constraints; The issuer identifier, key usage extension, and validity period of the digital certificate are compared and verified with the set of allowed certificate authorities, key usage constraints, and certificate validity period threshold in the policy constraints to obtain the policy determination result. A verification result is generated by combining the certificate revocation status with the policy determination result.

4. The data integration method for multi-source heterogeneous data according to claim 1, characterized in that, The step of generating a fused entity record by selecting and synthesizing attribute values ​​based on the trust level identifier associated with the attributes of each entity in the associated entity set, using a preset arbitration rule in the data integration algorithm, includes: Extract multiple candidate attribute values ​​corresponding to each entity attribute in the associated entity set; By using the arbitration rules in the data integration algorithm, and based on the trust level identifier associated with each candidate attribute value, a level constraint is generated between each candidate attribute value. The level constraints are input into the arbitration engine in the data integration algorithm, and the arbitration logic is executed by the arbitration engine to generate the target attribute values ​​of each entity attribute. All target attribute values ​​belonging to the same entity attribute are assembled to obtain a fused entity record of each entity attribute.

5. The data integration method for multi-source heterogeneous data according to claim 1, characterized in that, The step of calling the hardware encryption module through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data based on the trust level identifier, and generating encrypted transmission data, includes: Based on the trust level identifier, configure differentiated transmission security strategies for the multi-source heterogeneous data; Based on the aforementioned transmission security strategy, a corresponding logical transmission channel is established for the multi-source heterogeneous data through a secure data exchange gateway; On the logical transmission channel, the multi-source heterogeneous data is encrypted in real time by a hardware encryption module to obtain encrypted data. The encrypted data is hierarchically packaged to obtain ciphertext data units, and an integrity check code is applied to the ciphertext data units to obtain encrypted transmission data.

6. The data integration method for multi-source heterogeneous data according to claim 5, characterized in that, On the logical transmission channel, the multi-source heterogeneous data is subjected to real-time encryption via a hardware encryption module to obtain encrypted data, including: Obtain the channel encryption parameters bound to the logical transmission channel; Based on the channel encryption parameters, a channel session key is generated by a hardware encryption module; Using the channel session key, the multi-source heterogeneous data is encrypted and converted to generate encrypted data blocks; Channel identification information is added to the encrypted data block to obtain encrypted data.

7. A data integration system for multi-source heterogeneous data, characterized in that, A data integration method for multi-source heterogeneous data as described in any one of claims 1 to 6 includes: The data acquisition module is used to acquire multi-source heterogeneous data from at least two different security domains, and associate the trust level identifier and digital certificate of the security domain to which each piece of multi-source heterogeneous data belongs; The encryption module is used to call the hardware encryption module through the corresponding secure data exchange gateway to perform differentiated encrypted transmission of the multi-source heterogeneous data according to the trust level identifier, and generate encrypted transmission data. The decryption module is used to perform certificate verification and decryption of the encrypted transmission data in the target security domain based on the digital certificate and using Public Key Infrastructure (PKI) to obtain plaintext data. The plaintext data is then stored hierarchically using a key derivation function to obtain hierarchical data. The alignment module is used to employ differentiated data integration algorithms, call preset master data management specifications and preset semantic mapping tables to perform entity merging and attribute alignment on the hierarchical data, and generate trusted data assets.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data integration method for multi-source heterogeneous data as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the data integration method for multi-source heterogeneous data as described in any one of claims 1 to 6.

Images