A data governance method based on a hybrid architecture

Abstract

The application discloses a kind of data governance methods based on hybrid architecture, for including perception layer, knowledge layer, decision layer, feedback layer hybrid architecture, method includes: deployment hybrid architecture, load pre-training model, to judge data drift according to the distribution variation of data, trigger knowledge layer update;Based on algorithm constraint model parameter update, limit parameter update amplitude, dynamically update the node attribute of knowledge graph in database;Extract the feature vector of text, time series, graph data, get three-order tensor and decompose projection to uniform feature space, through the fusion Paillier homomorphic encryption and differential privacy under federated learning framework, carry out safe aggregation, to generate adversarial network reconfiguration data distribution.

Description

Technical Field

[0001] This invention belongs to the field of data governance technology, and specifically relates to a data governance method based on a hybrid architecture. Background Technology

[0002] With the rapid development of big data technology, enterprises face increasingly complex data sources, exhibiting multimodal (text, time-series, graph data) and heterogeneous (structured, semi-structured, unstructured) data characteristics. Traditional data governance technologies mainly rely on predefined rule bases (such as Apache Atlas) or static models (such as ETL tools), which are difficult to adapt to the needs of dynamically changing data distributions. Especially in highly sensitive industries such as finance, healthcare, and manufacturing, data governance must simultaneously meet stringent requirements for dynamic adaptability, cross-modal correlation analysis, and a privacy-utility balance.

[0003] In existing technologies, data governance systems based on static rules or pre-trained models (such as traditional ETL tools) suffer from the following problems: high rule maintenance costs and slow response to concept drift. Static rule bases require regular manual updates, and maintenance costs increase exponentially with the amount of data.

[0004] Furthermore, existing technologies face several bottlenecks in processing multi-source heterogeneous data: the heterogeneity of feature space remains unresolved; text, time-series, and graph data require divide-and-conquer processing; and cross-modal correlation analysis is inefficient. A unified representation algorithm is lacking, and there is a lack of methods to project heterogeneous features into a unified space, making it difficult to generate cross-modal governance strategies.

[0005] Furthermore, existing privacy protection technologies (such as fully homomorphic encryption and differential privacy) suffer from the following drawbacks: severe loss of utility, with traditional encryption technologies (such as fully homomorphic encryption) leading to high data utility loss; and a disconnect between privacy protection and governance processes, with individual privacy protection technologies (such as differential privacy) not being deeply integrated with governance processes, making it difficult to balance security and usability.

[0006] In summary, there is an urgent need to develop a data governance method based on a hybrid architecture to address the problems of insufficient adaptability to dynamic environments, low efficiency of multimodal governance, and the incompatibility between privacy protection and data utility in existing technologies. Summary of the Invention

[0007] This invention provides a data governance method based on a hybrid architecture to address the problems of insufficient adaptability to dynamic environments, low efficiency of multimodal governance, and the incompatibility between privacy protection and data utility in existing technologies.

[0008] The technical solution adopted in this invention is as follows:

[0009] A data governance method based on a hybrid architecture is used for a hybrid architecture including a perception layer, a knowledge layer, a decision layer, and a feedback layer. The method includes:

[0010] Deploy a hybrid architecture, load pre-trained models, and determine data drift based on changes in data distribution to trigger knowledge layer updates;

[0011] Based on algorithmic constraints, the model parameter updates are limited, and the knowledge graph node attributes in the database are dynamically updated.

[0012] Feature vectors are extracted from text, time series, and graph data to obtain third-order tensors, which are then decomposed and projected onto a unified feature space. Through a federated learning framework, Paillier homomorphic encryption and differential privacy are fused together to perform secure aggregation, thereby generating an adversarial network to reconstruct the data distribution.

[0013] The data governance method based on a hybrid architecture proposed in this invention also has the following additional technical features:

[0014] The algorithm constraint model is specifically as follows:

[0015] Calculate model parameters θ i Fisher information matrix F θi ;

[0016] Through the loss function,

[0017]

[0018] Execute parameter updates, where λ is the preset learning rate threshold.

[0019] Feature vectors are extracted from text, time series, and graph data to obtain third-order tensors, which are then decomposed and projected onto a unified feature space. Specifically:

[0020] Based on textual data, a 768-dimensional semantic vector is generated using the RoBERTa model; based on temporal data, a 128-dimensional state vector is extracted using the LSTM-Attention model; and based on graph data, a 256-dimensional node embedding vector is generated using GraphSAINT sampling.

[0021] A third-order tensor is constructed and projected onto a unified feature space using Tucker decomposition.

[0022]

[0023] Where W1, W2, and W3 are decomposition matrices, and X fused This is the fused feature vector.

[0024] The federated learning framework is specifically as follows:

[0025] Determine sensitive and non-sensitive fields.

[0026] For sensitive fields, 2048-bit Paillier homomorphic encryption is used;

[0027] For non-sensitive fields, Laplace differential privacy is used, where anonymity k=10 and diversity l=3;

[0028] The model gradient is segmented and aggregated based on the secret sharing technique.

[0029] Generative adversarial networks, specifically:

[0030] Train the generative adversarial network to learn the distribution of the original data;

[0031] Reconstructed data X is generated through noise injection. reconstructed =G(Z), where Z follows a standard normal distribution;

[0032] Compare the PSNR values ​​of the original data and the reconstructed data. Stop training when PSNR ≥ 32.6dB.

[0033] The parameter update is performed as follows:

[0034] Parameter updates are stopped when the number of iterations is ≥1000 or the rate of change of Q value is ≤0.01%.

[0035] Execution parameter updates also include:

[0036] New relationships between entities are inferred using the TransHR model, generating an incremental knowledge graph ΔKG.

[0037] Update the node attribute tags in the ontology repository;

[0038] Verify the semantic consistency between ΔKG and existing knowledge graphs.

[0039] This invention also provides a data governance system based on a hybrid architecture, comprising the following modules:

[0040] Initialize the module, deploy the hybrid architecture, and load the pre-trained model;

[0041] The dynamic perception module monitors the distribution changes of data in real time, identifies data drift, and triggers knowledge layer updates.

[0042] The elastic continuous learning module constrains model parameter updates based on algorithms, limits the magnitude of parameter updates, and dynamically updates the attributes of knowledge graph nodes in the database.

[0043] The multimodal joint governance module extracts feature vectors from text, time series, and graph data, obtains third-order tensors, and decomposes and projects them onto a unified feature space.

[0044] The privacy computing and utility compensation module integrates Paillier homomorphic encryption and differential privacy within a federated learning framework to perform secure aggregation, thereby generating an adversarial network to reconstruct the data distribution.

[0045] This invention further provides a storage medium,

[0046] The storage medium stores a computer program, which, when executed, implements the steps of the data governance method based on a hybrid architecture.

[0047] The present invention also provides a processing apparatus, comprising:

[0048] Memory, used to store computer programs;

[0049] A processor is used to implement the steps of the hybrid architecture-based data governance method when executing the computer program.

[0050] Due to the adoption of the above technical solution, the beneficial effects achieved by this invention are as follows:

[0051] 1. In this invention, the perception layer monitors changes in data distribution in real time, detects data drift, and triggers updates to the knowledge layer. Algorithm constraints limit the update magnitude of key parameters to prevent catastrophic forgetting. This solution, through a dynamic update mechanism, reduces the decay rate, improves dynamic adaptability, and eliminates the need for manual maintenance of a static rule base, thus reducing rule maintenance costs.

[0052] Feature vectors are extracted from text, time series, and graph data to obtain third-order tensors, which are then decomposed and projected onto a unified feature space. This scheme supports joint governance of multiple data types, improves cross-modal governance efficiency, unifies the feature space mapping algorithm, and promotes improved data annotation accuracy and efficiency.

[0053] This scheme employs a federated learning framework, integrating Paillier homomorphic encryption and differential privacy for secure aggregation to generate an adversarial network that reconstructs the data distribution. Through GAN reconstruction, this approach improves data utility retention and reduces the risk of privacy breaches. Furthermore, the secure aggregation (secret-sharing technique) within the federated learning framework optimizes computational latency.

[0054] This invention addresses the problems of insufficient dynamic adaptability, lack of multimodal governance capabilities, and the contradiction between privacy and utility in existing data governance by using a hybrid architecture and EWC algorithm, tensor fusion, federated learning, and GAN. Attached Figure Description

[0055] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0056] Figure 1 This is a flowchart illustrating a data governance method based on a hybrid architecture according to one embodiment of the present invention. Detailed Implementation

[0057] To more clearly illustrate the overall concept of the present invention, a detailed description will be provided below with reference to the accompanying drawings and examples.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0059] like Figure 1 As shown, a data governance method based on a hybrid architecture is used for a hybrid architecture including a perception layer, a knowledge layer, a decision layer, and a feedback layer. The method includes:

[0060] S100: Deploy a hybrid architecture, load pre-trained models, and determine data drift based on changes in data distribution to trigger knowledge layer updates.

[0061] The core objective of this step is to achieve real-time monitoring and dynamic response to changes in data distribution through hybrid architecture deployment and pre-trained model loading, thereby solving the problems of insufficient adaptability of static models, high cost of manual rule maintenance, and delayed response to data drift in traditional data governance.

[0062] The hybrid architecture deployment, employing a four-layer collaborative architecture (perception layer → knowledge layer → decision layer → feedback layer), constructs a closed-loop governance framework of "data-knowledge-decision-execution," ensuring the independence and collaboration of functions at each layer. The perception layer is responsible for real-time collection of multi-source heterogeneous data (such as structured and unstructured data) and preliminary processing based on pre-trained models.

[0063] Loading pre-trained models (such as RoBERTa, LSTM-Attention, and GraphSAINT) provides the foundation for data drift detection and knowledge updating. The Kolmogorov-Smirnov test is introduced as a core algorithm, which uses statistical methods to quantify changes in data distribution (p-value < 0.05) to achieve automated identification of data drift.

[0064] To achieve dynamic response, the data drift trigger condition is that when a significant change in data distribution is detected (p-value < 0.05), the knowledge layer update process is automatically triggered, avoiding delays caused by manual intervention. After triggering, the knowledge layer is updated in real time through the collaboration between the Dynamic Ontology Library and the TransHR model, enabling real-time updates of knowledge graph node attributes.

[0065] By combining the Kolmogorov-Smirnov test with a pre-trained model, the quarterly performance degradation rate was reduced, significantly enhancing the system's responsiveness to data drift. An automated triggering mechanism replaced manual rule base maintenance, reducing maintenance costs from exponential growth (C = 1.8N). 1.5 The rate of change decreases to linear growth (C≈0.5N). Data drift detection and knowledge layer update processes are completed within seconds, meeting real-time governance requirements.

[0066] This step addresses three core issues in traditional data governance through hybrid architecture deployment and pre-trained model loading: insufficient dynamic adaptability, high manual maintenance costs, and delayed response. Algorithm-driven data drift detection and knowledge updates reduce manual intervention and improve governance efficiency. A second-level response mechanism meets the stringent requirements of highly sensitive industries, providing real-time assurance. Reduced maintenance costs and resource consumption align with the economic needs of enterprises undergoing digital transformation.

[0067] S200: Based on algorithm constraints, model parameter updates are limited, and knowledge graph node attributes in the database are dynamically updated.

[0068] The core objective of this step is to address the problems of catastrophic model forgetting and lagging static updates of knowledge graphs in traditional data governance by using algorithmic constraints on model parameter updates and dynamic updates of knowledge graph node attributes. This prevents the model from losing old knowledge due to drastic parameter updates during incremental training, ensuring the model's continuous learning ability. Dynamically updating node attributes ensures that the knowledge graph reflects the latest business logic and synchronizes knowledge graph and data changes in real time.

[0069] Based on the Elastic Weight Consolidation (EWC) algorithm, the importance of model parameters to old tasks is quantified by calculating the Fisher information matrix, thus limiting the update magnitude of key parameters. An elastic constraint term is added to the loss function to ensure that the update magnitude of important parameters is limited during training for new tasks, preventing the overwriting of old knowledge.

[0070] The knowledge graph node attributes are dynamically updated, and the TransHR model infers new entity relationships. Entity and relationship extraction involves identifying new entities (e.g., "new energy vehicles") and relationships (e.g., "related policies and regulations") from incremental data, generating an incremental knowledge graph ΔKG. The ΔKG is evaluated through intra-chain consistency (node ​​attribute logical closure) and cross-chain consistency (verification of associations with other knowledge bases) to ensure the integrity of the updated knowledge graph.

[0071] In addition, during the database update process, constraint strategies are implemented, including primary key constraints to ensure the uniqueness of new nodes (e.g., using "policy number" as the primary key), foreign key constraints to maintain relationships between entities (e.g., the foreign key mapping between "new energy vehicles" and "subsidy policy"), and default constraints to set default values ​​for new attributes (e.g., "risk level = low") to avoid null values.

[0072] The EWC algorithm reduces the quarterly performance degradation rate of the model. Key parameter updates are controlled, accuracy retention for older tasks is improved, and catastrophic forgetting is mitigated. The TransHR model adds entity relationships for inference, shortening knowledge graph update time and improving update efficiency. Database constraint strategies (primary keys, foreign keys, default values) prevent node attribute conflicts or missing values, ensuring logical closure of the knowledge graph and improving data consistency.

[0073] S300: Extract feature vectors from text, time series, and graph data, obtain third-order tensors, decompose and project them onto a unified feature space, and perform secure aggregation by fusing Paillier homomorphic encryption and differential privacy under a federated learning framework to generate an adversarial network to reconstruct the data distribution.

[0074] The core objective of this step is to address three major challenges in traditional data governance: low efficiency in processing cross-modal heterogeneous data, high risk of privacy breaches, and uneven data distribution. This is achieved through multimodal feature fusion and privacy-preserving secure aggregation. A unified feature space mapping eliminates the heterogeneity of text, time-series, and graph data, enabling joint cross-modal analysis. A privacy-utility balance is achieved by integrating Paillier homomorphic encryption and differential privacy within a federated learning framework, protecting data privacy while preserving data utility. Finally, data distribution optimization is achieved by reconstructing the data distribution using generative adversarial networks (GANs), improving model training quality and generalization ability.

[0075] Multimodal feature extraction and tensor modeling, text feature extraction, temporal feature extraction, graph data feature extraction, the above feature vectors are combined into a three-dimensional tensor (text × temporal × graph), and projected to a unified feature space (256 dimensions) through Tucker decomposition or CP decomposition.

[0076] Secure aggregation within the federated learning framework, including Paillier homomorphic encryption, allows each participant to upload encrypted local model parameters (with a 2048-bit key), enabling the server to complete global model aggregation without decryption. Secret sharing technology reduces computational latency and optimizes computational efficiency.

[0077] Differential privacy fusion adds Laplacian noise (ε = 0.7) during model parameter updates via a noise injection mechanism, satisfying k-anonymity (k = 10) and l-diversity (l = 3) requirements. The sensitivity of local model updates is limited (Clipping threshold set to 0.1), reducing the impact of noise on model performance and achieving gradient clipping.

[0078] Generative Adversarial Networks (GANs) are used to reconstruct data distribution. Based on the ResNet-50 architecture, they learn the characteristics of real data distribution and design a discriminator. High-quality synthetic data is generated using the WaveNet architecture, and a generator is designed. The objective function is optimized using Wasserstein GAN to minimize the distribution difference between the generated and real data, achieving joint training.

[0079] A unified feature space mapping is implemented, eliminating heterogeneity through Tucker decomposition and supporting joint governance of multiple data types, thus improving efficiency. The hybrid model (BERT-LSTM-GraphSAGE) enhances annotation accuracy. A privacy-utility balance is achieved through a GAN compensation mechanism, improving utility retention and reducing the risk of privacy breaches. Data distribution optimization is achieved through Wasserstein GAN, which reduces the distributional difference (Wasserstein distance) between generated and real data, significantly improving model robustness. Furthermore, secure aggregation latency optimization is implemented within the federated learning framework.

[0080] This step addresses the problems of low efficiency in cross-modal heterogeneous data processing, high risk of privacy leakage, and uneven data distribution in traditional data governance by using multimodal feature fusion, secure aggregation through federated learning, and GAN data distribution reconstruction.

[0081] As a preferred embodiment of the present invention, the algorithm constraint model is specifically as follows:

[0082] Calculate model parameters θ i Fisher information matrix F θi ;

[0083] Through the loss function,

[0084]

[0085] Execute parameter updates, where λ is the preset learning rate threshold.

[0086] When continuously learning new tasks, the model parameters are limited to the degree of deviation from the key knowledge of previous tasks by regularization constraints based on the Fisher information matrix, thus solving the catastrophic forgetting problem (i.e. the model overfits to the new task, causing a significant drop in the performance of the old task) and achieving a balanced learning of knowledge from both new and old tasks.

[0087] Calculate the Fisher information matrix and, under the data distribution of previous tasks (tasks that have already been learned), statistically analyze the model parameters θ. i The sensitivity to the loss function of the old task is used to obtain the Fisher information matrix F. θi Fisher information is essentially a measure of parameter importance; if F... θi A larger value indicates that θ i The impact on the loss of old tasks is significant (i.e., this parameter is key knowledge of old tasks); conversely, the impact is smaller.

[0088] Construct the EWC loss function, combining the new task loss with a regularization penalty term, as shown in the formula:

[0089]

[0090] Where Lnew is the loss for the new task (such as classification loss, regression loss, etc., driving the model to learn the new task); λ is the preset learning rate threshold (controlling the strength of regularization and balancing the importance of new and old tasks); θ i old These are the parameter values ​​after the previous task training was completed (the knowledge baseline of the old task); For the weighted penalty term, if the parameter θ i Important to the old task (F) θi If the value is large, then it deviates from the old value θ. i old The cost is higher, while the cost is lower, allowing for greater updates. (L) EWC To optimize the objective, the parameter θ is adjusted through backpropagation. While fitting the new task, the update of key parameters is constrained by a regularization term to avoid disrupting the knowledge structure of the old task.

[0091] Balancing knowledge retention between new and old tasks ensures stable output across multiple tasks (e.g., learning to "recognize cats" first, then "recognize dogs," and still accurately recognizing cats), improving multi-task performance; enhancing generalization ability avoids overfitting to new tasks and maintains adaptability to the distribution of data for old tasks (e.g., knowledge retention is more critical when data for old tasks is scarce); and allowing adjustable constraint strength by adjusting the learning emphasis between new and old tasks through λ (e.g., increasing λ when old tasks are more important, and decreasing λ when new tasks are more critical), adapting to different scenario requirements.

[0092] As one embodiment of this implementation, the parameter update is performed as follows:

[0093] Parameter updates are stopped when the number of iterations is ≥1000 or the rate of change of Q value is ≤0.01%.

[0094] This embodiment aims to address two core issues in federated learning: low model training efficiency and overfitting risk, by setting parameter update termination conditions (iteration count ≥ 1000 or Q-value change rate ≤ 0.01%). Training is stopped promptly after model convergence to avoid redundant computation and optimize computational resource utilization. A Q-value change rate threshold (≤ 0.01%) is used to determine whether the model has entered a convergence plateau, preventing performance degradation due to continued training and thus preventing overfitting.

[0095] When the number of training iterations reaches 1000, parameter updates are forcibly terminated. Most federated learning tasks can converge within 1000 iterations.

[0096] Calculate the rate of change of Q (ΔQ / Q) over two consecutive iterations. If the rate of change is ≤0.01%, training is terminated. The Q value can be represented as a model performance metric (such as accuracy or loss value) or an objective function value (such as Pareto front convergence in multi-objective optimization).

[0097] Specifically, record the current iteration number (Counter = 0). Update the Q-value (e.g., accuracy or loss value) after each iteration. Calculate the rate of change of the Q-value:

[0098]

[0099] If ΔQ ≤ 0.01% or Counter ≥ 1000, training terminates. The final model is then output, and the global model parameters at the point where the termination condition is met are saved.

[0100] Early termination of training after model convergence saves computational resources and improves resource efficiency. The Q-value change rate threshold (≤0.01%) can effectively identify the model's convergence plateau, preventing performance degradation due to continued training and avoiding overfitting. Through dual termination conditions (number of iterations + Q-value change rate), it ensures that the model can still converge stably under complex data distributions, thus guaranteeing model performance.

[0101] This embodiment solves the core problems of resource waste and model overfitting in federated learning by using dual termination conditions (iteration count ≥ 1000 times or Q-value change rate ≤ 0.01%).

[0102] As another embodiment of this implementation, the parameter update process further includes:

[0103] New relationships between entities are inferred using the TransHR model, generating an incremental knowledge graph ΔKG.

[0104] Update the node attribute tags in the ontology repository;

[0105] Verify the semantic consistency between ΔKG and existing knowledge graphs.

[0106] This embodiment uses the TransHR model to infer new entity relationships, update ontology node attribute labels, and perform semantic consistency checks. It aims to solve the problems of insufficient relationship reasoning, lagging attribute labels, and semantic conflict risks in the dynamic updating of knowledge graphs.

[0107] TransHR model inference adds entity relations. TransHR inherits from the TransR model and adds triples (h,r,t) through a dual projection mechanism (head entity and tail entity are mapped to relation subspaces respectively).

[0108] l hr =l h M rh , l tr =l t M rt , l hr +l r ≈l tr ,

[0109] Among them l h l t M is an entity vector. rh M rt Let l be the projection matrix of the relational subspace. r It is a relation vector.

[0110] Input a new entity pair (h,t), predict relation r using the TransHR model, and generate triples (h,r,t) in ΔKG.

[0111] Update node attribute labels in the ontology, extract attributes from ΔKG (e.g., "driving range" and "subsidy policy" for "new energy vehicles"). Standardize labels by mapping attribute values ​​to the predefined label system in the ontology (e.g., classifying "driving range" as "performance indicators"). Implement dynamic maintenance by updating the ontology through primary key constraints (ensuring entity uniqueness) and default constraints (setting default values ​​for new attributes).

[0112] Verify the semantic consistency between ΔKG and the existing knowledge graph to maintain intra-chain consistency. Verify whether newly added triples in ΔKG satisfy logical closure (e.g., whether "h→r→t" conforms to existing rules). For cross-chain consistency, use semantic matching techniques (e.g., entity-relation similarity calculation) to detect whether there are conflicts between ΔKG and the existing knowledge graph.

[0113] This embodiment addresses the challenges of dynamic knowledge graph maintenance by using the TransHR model to infer new relationships, dynamically update attribute tags, and perform semantic consistency checks. Specifically, it leverages the TransHR model to handle complex relationships, enhancing dynamic expansion capabilities and resolving insufficient relationship reasoning. Real-time updates to the ontology ensure data synchronization with definitions, mitigating attribute tag lag. Furthermore, automated verification tools ensure logical consistency of the knowledge graph, addressing the risk of semantic conflicts.

[0114] In a preferred embodiment of the present invention, feature vectors of text, time series, and graph data are extracted to obtain a third-order tensor, which is then decomposed and projected onto a unified feature space. Specifically:

[0115] Based on textual data, a 768-dimensional semantic vector is generated using the RoBERTa model; based on temporal data, a 128-dimensional state vector is extracted using the LSTM-Attention model; and based on graph data, a 256-dimensional node embedding vector is generated using GraphSAINT sampling.

[0116] A third-order tensor is constructed and projected onto a unified feature space using Tucker decomposition.

[0117]

[0118] Where W1, W2, and W3 are decomposition matrices, and X fused This is the fused feature vector.

[0119] The core objective of this implementation is to achieve feature fusion of heterogeneous multi-source data (text, time series, graph data), and to uniformly map feature vectors from different modalities to the same feature space, providing a consistent feature representation for subsequent cross-modal analysis (such as joint modeling, prediction, or classification).

[0120] For modal feature extraction, the text data is encoded using the pre-trained language model RoBERTa to generate 768-dimensional semantic vectors. Leveraging RoBERTa's powerful semantic understanding capabilities in natural language processing tasks, the text's contextual dependencies and deep semantic information are captured.

[0121] The time-series data is processed using an LSTM-Attention model to extract a 128-dimensional state vector. LSTM captures temporal dependencies (such as the long-term trend of a time series), while the Attention mechanism focuses on key time segments, balancing the global trend of the time series with local details.

[0122] Graph data is used to generate 256-dimensional node embedding vectors using the GraphSAINT graph neural network sampling algorithm. GraphSAINT alleviates the computational burden of graph neural networks through sampling, while preserving the local structure and neighbor relationships of nodes, making it suitable for feature learning on large-scale graph data.

[0123] A third-order tensor is constructed to organize the feature vectors of three modalities (768 dimensions for text, 128 dimensions for temporal data, and 256 dimensions for graph data) into a single third-order tensor, with each dimension corresponding to the feature space of a data modality. The multidimensional structure of the tensor is naturally adapted to multimodal data, preserving high-order correlations between modalities and avoiding the loss of feature interaction information caused by traditional vector concatenation.

[0124] Tucker decomposition projects the third-order tensor onto a unified feature space. The formula for this projection is:

[0125]

[0126] Tucker decomposition is a classic tensor decomposition method that decomposes the original tensor into a core tensor (T) and projection matrices (W1, W2, W3) corresponding to three modes. The projection matrices map the high-dimensional features of each mode to a unified low-dimensional space, achieving alignment and compression of heterogeneous features. The fused feature vector X fused It retains the common information and key features of the three modalities, and reduces the dimensionality, making it easier for subsequent models to process uniformly.

[0127] By fusing complementary information from three modalities (textual semantics, temporal dynamics, and structural relationships), the feature vectors more comprehensively characterize the essential features of the samples, improving the accuracy of subsequent tasks (such as classification and prediction). Tucker decomposition compresses high-dimensional heterogeneous features into a unified low-dimensional space, reducing the input dimension of subsequent models and improving computational efficiency.

[0128] Tensor structures preserve the intermodal relationships (such as the correspondence between text and graph structures, and the relationship between temporal changes and node states), avoiding the loss of interactive information caused by traditional feature concatenation and improving the expressive power of features. A unified feature space enables the model to learn multimodal patterns simultaneously, has a certain robustness to missing modalities (such as some samples lacking text data), and is adaptable to cross-modal tasks in complex scenarios.

[0129] This implementation method achieves deep alignment and compression of heterogeneous data at the feature level through modal professional extraction and tensor decomposition fusion, providing an efficient and comprehensive feature representation foundation for subsequent cross-modal intelligent analysis.

[0130] As a preferred embodiment of the present invention, the federated learning framework is specifically as follows:

[0131] Determine sensitive and non-sensitive fields.

[0132] For sensitive fields, 2048-bit Paillier homomorphic encryption is used;

[0133] For non-sensitive fields, Laplace differential privacy is used, where anonymity k=10 and diversity l=3;

[0134] The model gradient is segmented and aggregated based on the secret sharing technique.

[0135] The core objective of this implementation method is to achieve differentiated protection of sensitive and non-sensitive data in federated learning through a multi-layered privacy protection mechanism, while taking into account both model training efficiency and privacy security requirements.

[0136] For highly sensitive fields (such as medical records and financial transactions), 2048-bit Paillier homomorphic encryption is used to ensure that data is encrypted throughout transmission and computation, preventing direct leakage. For less sensitive fields (such as user behavior statistics), Laplacian differential privacy (ε = 0.7, k-anonymity k = 10, l-diversity l = 3) is employed, injecting noise to obfuscate individual data features and reduce the risk of privacy inference. Based on secret sharing technology, model gradients are fragmented and aggregated to avoid a central server obtaining complete gradient information and prevent gradient backpropagation attacks.

[0137] The fields are categorized into sensitive and non-sensitive fields. Sensitive fields are data that directly or indirectly identify an individual (such as ID card numbers or medical diagnosis results). Non-sensitive fields are statistical data unrelated to identity (such as user click frequency or number of product purchases).

[0138] Data can be automatically categorized using a rule engine based on predefined rules (such as regular expression matching and field type judgment). Furthermore, for complex scenarios (such as mixed data), manual annotation can be introduced to assist in ensuring classification accuracy.

[0139] Paillier homomorphic encryption processes sensitive fields, generating a 2048-bit Paillier public-private key pair to encrypt the values ​​of sensitive fields, ensuring that the data remains ciphertext throughout the transmission and aggregation process.

[0140] The central server performs addition operations (such as summation or averaging) on ​​the ciphertext fields, completing the global model update without decryption. If the ciphertext fields are E(x1), E(x2), ..., E(x... n If the aggregation result is:

[0141] E(x agg )=E(x1)+E(x2)+…+E(x n ).

[0142] A plaintext aggregation result x is obtained after decryption agg =x1+x2+…+x n .

[0143] In addition, Laplace differential privacy processes non-sensitive fields, adds noise to the gradient or statistical value of non-sensitive fields through the Laplace mechanism. The parameters are set as follows: privacy budget ε=0.7, k-anonymity k=10 (at least 10 similar records), and l-diversity l=3 (there are at least 3 types of sensitive value distribution).

[0144] k-anonymity ensures that each group of data contains at least k records through generalization (e.g., age intervalization) or suppression (e.g., deleting partial fields). l-diversity introduces at least l different sensitive values (e.g., disease types) in each group to reduce the possibility of privacy inference.

[0145] Secret sharing technology implements gradient slice aggregation. First, gradient segmentation is performed: the local model gradient is split into n parts (n≥3), and only m parts (m<n) are uploaded to the central server. Second, slice aggregation is performed: the central server performs an addition operation on the m slices, and at least t slices (t≤m) need to be collected to recover the global gradient.

[0146] In this way, even if some slices are stolen, an attacker cannot restore the complete gradient. The number of slices m can be adjusted dynamically (e.g., reducing the number of uploaded slices when network bandwidth is insufficient), which reduces communication overhead and achieves optimization of computation and communication overhead.

[0147] Paillier encryption prevents attackers from directly obtaining plaintext data; the combination of differential privacy with k-anonymity / l-diversity reduces the risk of privacy inference; secret sharing technology ensures that the central server cannot obtain the complete gradient, which improves the strength of privacy protection.

[0148] Homomorphic encryption optimization: a 2048-bit key length achieves a balance between security and computational efficiency; differential privacy compensation: after Laplace noise injection, the model accuracy decreases less, and the model utility retention is high.

[0149] The present embodiment solves the problems of insufficient privacy protection, privacy-utility contradiction, and high computation and communication overhead in federated learning through the combination of sensitive field encryption, non-sensitive field differential privacy, and gradient slice aggregation.

[0150] As a preferred embodiment under the present invention, the generative adversarial network is specifically:

[0151] Train a generative adversarial network to learn the distribution of original data;

[0152] Generate reconstructed data X through noise injection reconstructed =G(Z), where Z follows a standard normal distribution;

[0153] Compare the PSNR values ​​of the original data and the reconstructed data. Stop training when PSNR ≥ 32.6dB.

[0154] The core objective of this implementation is to achieve efficient reconstruction of original data using Generative Adversarial Networks (GANs), and to control the training process through objective quantitative metrics (PSNR) to ensure that the generative model accurately learns the data distribution and achieves the preset reconstruction quality. This leverages the generative capabilities of GANs to restore the features of the original data while avoiding overtraining or underfitting.

[0155] Generative Adversarial Networks (GANs) are trained to learn the distribution of the original data. Using standard GAN frameworks (such as DCGAN and WGAN), the generator (G) and discriminator (D) are trained adversarially, enabling G to learn the latent distribution patterns of the original data (such as images and signals). During training, G maps random noise to pseudo-data, and D distinguishes the differences between the pseudo-data and the real data. Backpropagation optimizes the parameters of both, ultimately making the data generated by G approximate the real data distribution.

[0156] By leveraging the unsupervised learning capabilities of GANs, complex data features (such as image textures and edges) can be captured without requiring a large amount of labeled data, providing a foundational model for subsequent reconstruction.

[0157] Reconstructed data X is generated through noise injection. reconstructed = G(Z), where Z follows a standard normal distribution. The input is random noise Z ~ N(0, I) following a standard normal distribution, fed into the trained generator G, and the output is reconstructed data X. reconstructed Here, the dimension of Z must be consistent with the input dimension of the generator during training.

[0158] The trained signal G is used to convert noise into data, mapping the random signal to the original data space to complete the reconstruction task. The selection of a standard normal distribution ensures the stability and repeatability of the input noise, avoiding reconstruction bias caused by uneven noise distribution.

[0159] Compare the PSNR values ​​of the original data and the reconstructed data. Stop training when PSNR ≥ 32.6 dB. Calculate the original data X. original With reconstructing data X reconstructed The peak signal-to-noise ratio (PSNR) is given by the formula:

[0160]

[0161] MAX I The maximum pixel value of the data (e.g., 255 for an 8-bit image), and MSE is the mean squared error. Training termination is triggered when PSNR reaches or exceeds 32.6 dB.

[0162] PSNR, as an objective image quality assessment metric, quantifies the similarity between reconstructed and original data (the higher the value, the smaller the reconstruction error). The 32.6dB threshold is set based on the accuracy requirements of specific application scenarios to ensure that the reconstruction results meet practical standards (typically, when PSNR > 30dB, the human eye cannot perceive a significant difference).

[0163] After learning the data distribution, GANs generate reconstructed data that highly restores the original features (such as image edges and textures), improving reconstruction accuracy and outperforming traditional interpolation or filtering methods. A 32.6dB threshold avoids ineffective training iterations, reduces computational resource consumption (such as GPU time), and prevents underfitting (insufficient reconstruction quality) or overfitting (overfitting training data noise), thus optimizing training efficiency. Standard normally distributed noise input ensures the method's versatility and adaptability to multimodal data reconstruction tasks, including images, signals, and text.

[0164] This invention also provides a data governance system based on a hybrid architecture, comprising the following modules:

[0165] Initialize the module, deploy the hybrid architecture, and load the pre-trained model;

[0166] The dynamic perception module monitors the distribution changes of data in real time, identifies data drift, and triggers knowledge layer updates.

[0167] The elastic continuous learning module constrains model parameter updates based on algorithms, limits the magnitude of parameter updates, and dynamically updates the attributes of knowledge graph nodes in the database.

[0168] The multimodal joint governance module extracts feature vectors from text, time series, and graph data, obtains third-order tensors, and decomposes and projects them onto a unified feature space.

[0169] The privacy computing and utility compensation module integrates Paillier homomorphic encryption and differential privacy within a federated learning framework to perform secure aggregation, thereby generating an adversarial network to reconstruct the data distribution.

[0170] Therefore, the data governance methods based on hybrid architecture can achieve any effect, which will not be elaborated here.

[0171] This invention further provides a storage medium,

[0172] The storage medium stores a computer program, which, when executed, implements the steps of the data governance method based on a hybrid architecture.

[0173] Therefore, the data governance methods based on hybrid architecture can achieve any effect, which will not be elaborated here.

[0174] The present invention also provides a processing apparatus, comprising:

[0175] Memory, used to store computer programs;

[0176] A processor is used to implement the steps of the hybrid architecture-based data governance method when executing the computer program.

[0177] Therefore, the data governance methods based on hybrid architecture can achieve any effect, which will not be elaborated here.

[0178] For any parts not mentioned in this invention, existing technologies can be used or referenced.

[0179] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0180] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A data governance method based on a hybrid architecture, characterized in that, For a hybrid architecture including a perception layer, a knowledge layer, a decision layer, and a feedback layer, the method includes: Deploy a hybrid architecture, load pre-trained models, and determine data drift based on changes in data distribution to trigger knowledge layer updates; Based on algorithmic constraints, the model parameter updates are limited, and the knowledge graph node attributes in the database are dynamically updated. Feature vectors are extracted from text, time series, and graph data to obtain third-order tensors, which are then decomposed and projected onto a unified feature space. Through a federated learning framework, Paillier homomorphic encryption and differential privacy are fused together to perform secure aggregation, thereby generating an adversarial network to reconstruct the data distribution. Based on textual data, a 768-dimensional semantic vector is generated using the RoBERTa model; based on temporal data, a 128-dimensional state vector is extracted using the LSTM-Attention model; and based on graph data, a 256-dimensional node embedding vector is generated using GraphSAINT sampling. A third-order tensor is constructed and projected onto a unified feature space using Tucker decomposition. ， Where W1, W2, and W3 are decomposition matrices, and X fused The fused feature vector; The federated learning framework is specifically as follows: Determine sensitive and non-sensitive fields. For sensitive fields, 2048-bit Paillier homomorphic encryption is used; For non-sensitive fields, Laplace differential privacy is used, where anonymity k=10 and diversity l=3; The model gradient is segmented and aggregated based on the secret sharing technique.

2. The data governance method based on a hybrid architecture according to claim 1, characterized in that, The algorithm constraint model is specifically as follows: Calculate model parameters θ i Fisher information matrix F θi ; Through the loss function, ， Execute parameter updates, where λ is the preset learning rate threshold.

3. The data governance method based on a hybrid architecture according to claim 1, characterized in that, Generative adversarial networks, specifically: Train the generative adversarial network to learn the distribution of the original data; Reconstructed data is generated through noise injection. , where Z follows a standard normal distribution; Compare the PSNR values ​​of the original data and the reconstructed data. Stop training when PSNR ≥ 32.6dB.

4. The data governance method based on a hybrid architecture according to claim 2, characterized in that, The parameters are updated as follows: Parameter updates are stopped when the number of iterations is ≥1000 or the rate of change of Q value is ≤0.01%.

5. A data governance method based on a hybrid architecture according to claim 2, characterized in that, Execution parameter updates also include: New relationships between entities are inferred using the TransHR model, generating an incremental knowledge graph ΔKG. Update the node attribute tags in the ontology repository; Verify the semantic consistency between ΔKG and existing knowledge graphs.

6. A data governance system based on a hybrid architecture, characterized in that, Includes the following modules: Initialize the module, deploy the hybrid architecture, and load the pre-trained model; The dynamic perception module monitors the distribution changes of data in real time, identifies data drift, and triggers knowledge layer updates. The elastic continuous learning module constrains model parameter updates based on algorithms, limits the magnitude of parameter updates, and dynamically updates the attributes of knowledge graph nodes in the database. The multimodal joint governance module extracts feature vectors from text, time series, and graph data, obtains third-order tensors, and decomposes and projects them onto a unified feature space. Based on textual data, a 768-dimensional semantic vector is generated using the RoBERTa model; based on temporal data, a 128-dimensional state vector is extracted using the LSTM-Attention model; and based on graph data, a 256-dimensional node embedding vector is generated using GraphSAINT sampling. A third-order tensor is constructed and projected onto a unified feature space using Tucker decomposition. ， Where W1, W2, and W3 are decomposition matrices, and X fused The fused feature vector; The privacy computing and utility compensation module integrates Paillier homomorphic encryption and differential privacy under the federated learning framework to perform secure aggregation, thereby generating an adversarial network to reconstruct the data distribution. The federated learning framework is specifically as follows: Determine sensitive and non-sensitive fields. For sensitive fields, 2048-bit Paillier homomorphic encryption is used; For non-sensitive fields, Laplace differential privacy is used, where anonymity k=10 and diversity l=3; The model gradient is segmented and aggregated based on the secret sharing technique.

7. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed, implements the steps of the data governance method based on a hybrid architecture as described in any one of claims 1 to 5.

8. A processing apparatus, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data governance method based on a hybrid architecture as described in any one of claims 1 to 5.

Images