Automatic data governance strategy generation method and system based on reinforcement learning

Abstract

The present invention relates to the field of machine learning technology, specifically to an automated data governance strategy generation method and system based on reinforcement learning, the method comprising: collecting multiple types of data from multiple data sources, and constructing a data quality assessment index system for different data types; constructing a data governance strategy optimization model based on a deep deterministic policy gradient algorithm, iteratively updating and optimizing the data governance strategy through data state feedback and reward function drive as well as the constraints of a rule engine and an expert system; introducing a federated learning mechanism to perform multi-node collaborative training on the constructed data governance strategy optimization model; designing a data governance strategy execution mechanism based on a stream processing framework; introducing federated distillation technology to compress the deep reinforcement learning model into a lightweight version through distillation, and deploying it with a data processing operator. The method provided in this application can automatically generate the optimal data governance strategy and improve the intelligence level and efficiency of data governance.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and specifically to a method and system for generating automated data governance strategies based on reinforcement learning. Background Art

[0002] In the current big data era, data is characterized by huge volume, diverse types, and scattered sources. Faced with such a complex data environment, traditional data governance methods are difficult to adapt to rapidly changing data forms and data processing needs.

[0003] Data quality issues are becoming increasingly prominent. Massive heterogeneous data sources are constantly emerging, and quality issues such as data missing, data redundancy, and data inconsistency are emerging in an endless stream, seriously affecting the availability of data and the accuracy of analysis results. Manually formulating data quality rules and thresholds has poor flexibility and is difficult to cope with the frequently changing data environment.

[0004] The efficiency of data governance needs to be improved. Traditional data governance is highly dependent on expert experience, and the governance process lacks automation. Data governance knowledge is difficult to reuse. Different data users have accumulated a lot of data governance experience, but lack an effective sharing mechanism. Data governance knowledge is scattered across various nodes and cannot achieve collaborative optimization. Distributed machine learning technologies such as federated learning are urgently needed to achieve federal sharing and evolution of data governance knowledge while protecting data privacy. The traditional data governance paradigm can no longer meet the needs of data management in the big data era.

[0005] In view of this, this application proposes an automated data governance strategy generation method and system based on reinforcement learning. Summary of the Invention

[0006] To achieve the above objectives, the present invention provides a method and system for generating an automated data governance strategy based on reinforcement learning. The specific technical solutions are as follows:

[0007] An automated data governance policy generation method based on reinforcement learning, including:

[0008] Collect multiple types of data from multiple data sources and build a data quality assessment indicator system for different data types;

[0009] Build a data governance policy optimization model based on a deep deterministic policy gradient algorithm, and iteratively update and optimize the data governance strategy through data state feedback and reward function driving;

[0010] Introducing a federated learning mechanism to conduct multi-node collaborative training on the data governance strategy optimization model built based on the DDPG algorithm;

[0011] Design a data governance policy execution engine based on a stream processing framework, build data processing operators, and execute data governance policies through data processing operators;

[0012] Federated distillation technology is introduced to compress the deep reinforcement learning model trained by federated learning into a lightweight version through distillation and deploy it together with data processing operators.

[0013] Preferably, the multi-type data includes structured data and unstructured data;

[0014] Construct a data quality assessment indicator system, set quantitative indicators from the dimensions of completeness, accuracy, consistency, timeliness and interpretability of the collected structured data; for the collected unstructured data, additional indicators of structural integrity, semantic accuracy and relevance are added.

[0015] Preferably, a data governance policy optimization model based on a deep deterministic policy gradient algorithm is constructed;

[0016] Define the state space of the deep deterministic policy gradient algorithm, using the set of data quality assessment indicators as the state space; use data governance operations as the action space, where each action represents a governance operation performed on the dataset;

[0017] Design a reward function for a deep deterministic policy gradient algorithm to evaluate the gain obtained from the state after executing an action.

[0018] Preferably, a policy network based on a deep deterministic policy gradient algorithm is constructed, including the design of an actor network and a critic network;

[0019] The Actor network generates data governance operations based on the state; the Critic network is used to evaluate the value of the state-action pair;

[0020] Train the deep deterministic policy gradient algorithm model, driven by data state feedback and reward function, and constrained by the rule engine and expert system, to continuously update the parameters of the Actor network and Critic network and optimize the data governance strategy.

[0021] Preferably, a federated learning mechanism is introduced to perform multi-node collaborative training on the deep deterministic policy gradient algorithm;

[0022] During the training process of federated learning, each node independently performs local model training and regularly aggregates and updates parameters with the central server.

[0023] Preferably, the local model of federated learning is trained. Each node, based on the local dataset, updates the local actor network parameters and critic network parameters using a deep deterministic policy gradient algorithm by performing data governance operations and observing rewards and state transitions, thereby optimizing the local policy model.

[0024] At the end of each federated learning round, each node uploads the local Actor network parameters and Critic network parameters to the central server; the central server aggregates the collected parameters to generate global Actor network parameters and global Critic network parameters;

[0025] The central server distributes the aggregated global parameters to each node, and each node uses the global parameters to update the local model.

[0026] Preferably, a stream processing framework for data governance is constructed, and data processing operators are built within the framework;

[0027] The constructed data processing operators include: data cleaning operators, data conversion operators, data aggregation operators and data connection operators. The data cleaning operators include: outlier processing operators, missing value processing operators and data standardization operators; the data conversion operators include: data type conversion operators, data format conversion operators and data encoding conversion operators; the data aggregation operators include group aggregation operators and window aggregation operators; the data connection operators include: inner connection operators and outer connection operators.

[0028] Preferably, a federated distillation mechanism is established to compress the complex deep reinforcement learning model trained by federated learning into a lightweight version;

[0029] The deep reinforcement learning model trained by federated learning is used as the teacher model; a lightweight student model is used to learn knowledge from the teacher model;

[0030] In a federated learning environment, the student model is distilled and trained. Each participating node uses local data and the teacher model for distillation to update the parameters of the student model. At the same time, through the federated averaging algorithm, the parameters of the student model of each node are regularly aggregated to obtain a global student model.

[0031] Preferably, the lightweight student model is integrated with the data processing operator as a data governance policy execution mechanism;

[0032] The lightweight student model generates the optimal data governance operation based on the data status, and the data processing operator performs data governance operations on the collected data.

[0033] An automated data governance policy generation system based on reinforcement learning, which is used to implement the automated data governance policy generation method based on reinforcement learning, including: a data acquisition module, a policy generation module, a policy model optimization module, a data governance module, and a federated distillation deployment module;

[0034] The data acquisition module collects multiple types of data from multiple data sources and builds a data quality assessment indicator system for different data types;

[0035] The strategy generation module is used to build a data governance strategy optimization model based on a deep deterministic policy gradient algorithm, and iteratively updates and optimizes the data governance strategy through data state feedback and reward function driving.

[0036] The policy model optimization module introduces a federated learning mechanism to perform multi-node collaborative training on the data governance policy optimization model built based on the DDPG algorithm;

[0037] The data governance module is used to design a data governance policy execution engine based on a stream processing framework, build data processing operators, and execute data governance policies through data processing operators;

[0038] The federated distillation deployment module introduces federated distillation technology to compress the deep reinforcement learning model trained by federated learning into a lightweight version through distillation, and deploys it together with the data processing operator.

[0039] Beneficial effects of this application: This application comprehensively collects multi-source heterogeneous data, sets data quality assessment indicators from multiple dimensions based on the characteristics of structured data and unstructured data, provides a quality assessment basis for subsequent data governance, and helps to quantify the effectiveness of data governance.

[0040] This application uses the DDPG algorithm in deep reinforcement learning. Through data quality status feedback and reward function design, the model can autonomously learn and optimize data governance strategies, continuously improve the effectiveness and intelligence level of data governance, and reduce dependence on expert experience.

[0041] This application uses federated learning to enable multiple data nodes to participate in collaborative training. While protecting data privacy, it uses distributed data to improve the generalization performance of the data governance strategy optimization model, while realizing the federal sharing and integration of data governance knowledge and expanding the scope of reuse of data governance experience.

[0042] This application builds a data governance execution engine based on the stream processing framework, which can process massive data in real time and efficiently, build a variety of data processing operators, and improve data governance efficiency by flexibly orchestrating data processing operators to execute data governance strategies.

[0043] This application uses federated distillation technology to compress the trained deep reinforcement learning model to obtain a lightweight model with fewer parameters and faster inference speed, reducing model storage and computing overhead; by integrating the lightweight model with data processing operators, efficient execution of data governance strategies is achieved, reducing deployment difficulty and hardware costs, and improving the availability of data governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Flowchart of the method for generating automated data governance strategies based on reinforcement learning provided by the present invention;

[0045] Figure 2 The multi-source heterogeneous data collection and quality assessment flow chart provided by the present invention;

[0046] Figure 3 The flowchart of the data governance strategy optimization model based on the DDPG algorithm provided by the present invention;

[0047] Figure 4 A flowchart of multi-node strategy collaborative training in federated learning provided by the present invention;

[0048] Figure 5 A flow chart of the data governance strategy mechanism based on stream processing provided by the present invention;

[0049] Figure 6 A flow chart for the deployment of the federated distillation and lightweight strategy model provided by the present invention;

[0050] Figure 7 This is a structural diagram of the reinforcement learning-based automated data governance strategy generation system provided by the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0054] Example 1

[0055] Reference Figures 1 to 6 , which is the first embodiment of the present invention, provides an automated data governance strategy generation method based on reinforcement learning.

[0056] Step 1: Collect multiple types of data from multiple data sources and build a data quality assessment indicator system for different data types. Figure 2 , which is the flow chart of multi-source heterogeneous data collection and quality assessment in this step.

[0057] The multi-type data includes structured data and unstructured data; illustratively, the structured data collection step includes: collecting structured data from sources such as relational databases (such as MySQL and Oracle) and data warehouses through ETL tools or writing data collection scripts; during the structured data collection process, data cleaning operations such as data type conversion, field selection, and null value processing are required to ensure that the data format is consistent and the fields are complete.

[0058] Exemplarily, the unstructured data collection steps include: for unstructured data such as text, images, audio and video, first, it is necessary to determine the type of information to be extracted (such as text topics, keywords, image objects and voice keywords) according to business needs; then use natural language processing, computer vision or speech recognition AI technology to extract features and extract information from the data; you can use pre-trained deep learning models (such as BERT or ResNet) and fine-tune them according to specific fields to improve the accuracy of information extraction.

[0059] It should be noted that the following points need to be noted during the data collection stage: Desensitization: Desensitize the collected data to shield sensitive personal information, business secrets, etc. to ensure data security and compliance; Metadata extraction: Extract metadata information of the data during the collection process, including data source, collection time, data format and field meaning, to lay the foundation for subsequent data quality assessment and analysis; Incremental collection: Develop an incremental collection strategy based on the frequency of data updates, and regularly synchronize newly generated or updated data from the data source to ensure the timeliness of the data.

[0060] Through multi-source heterogeneous data collection, all types of data inside and outside the enterprise can be fully obtained, providing a sufficient data foundation for data governance; at the same time, operations such as data cleaning, desensitization, and metadata extraction during the collection process can improve data availability and security, laying a good data foundation for subsequent data governance.

[0061] Build a data quality assessment indicator system, set quantitative indicators from the dimensions of completeness, accuracy, consistency, timeliness and interpretability of collected structured data, comprehensively evaluate the quality levels of different types of data, and add additional indicators of structural integrity, semantic accuracy and relevance for collected unstructured data.

[0062] The integrity includes: setting an upper threshold for the field missing rate, when the field missing rate When the threshold is exceeded, the data integrity is considered to be substandard. is the number of records where the field is missing, is the total number of records.

[0063] The accuracy includes: for fields whose values ​​are enumeration types, setting an enumeration value whitelist and calculating the enumeration value accuracy ,in The number of records whose values ​​are in the whitelist. For numeric fields, set a value range. Values ​​outside the range are considered inaccurate.

[0064] The consistency includes: calculating the hash value for each data, comparing the hash values ​​to identify duplicate records, and calculating the data duplication rate. ,in The number of duplicate records.

[0065] The timeliness includes: setting data timeliness thresholds according to data service requirements , when data timeliness When the threshold is exceeded, the data is considered untimely; is the current time, The data update time.

[0066] The interpretability includes: setting a data description completeness standard and requiring metadata information to be complete. Calculating the data description completeness ,in is the number of fields that contain the complete description, is the total number of fields.

[0067] For unstructured data, metrics are also needed to evaluate structural completeness, semantic accuracy, and relevance.

[0068] The structural integrity includes: setting a key tag set, calculating the tag integrity rate ,in is the number of records containing all key tags.

[0069] The semantic accuracy includes: manually sampling some data, judging whether the key information is extracted accurately, and calculating the key information accuracy rate. ,in The accurate number of records for key information, The total number of records containing key information.

[0070] The relevance includes: calculating the similarity between the data and the target topic based on TF-IDF or Word2Vec method, and evaluating the data relevance .

[0071] By constructing a set of quality assessment indicators for structured and unstructured data, a unified framework can be used to evaluate the quality of different data types, while also taking into account the specific characteristics of unstructured data. This quality assessment indicator system covers key dimensions of data quality, including completeness, accuracy, consistency, timeliness, and interpretability, as well as the structural integrity, semantic accuracy, and relevance of unstructured data, enabling a comprehensive assessment of data quality.

[0072] The quantitative definitions of evaluation indicators are clear, facilitating automated evaluation. Based on the evaluation results, data quality bottlenecks can be identified, targeted data governance strategies can be formulated, data governance work can be guided, and the effectiveness of data governance practices can be improved.

[0073] This step begins with the collection of multi-source, heterogeneous data. Through desensitization, cleansing, metadata extraction, and other processes during the data collection phase, it ensures data security, availability, and manageability, laying a solid data foundation for data governance. This step also builds on this foundation by constructing a set of structured and unstructured data evaluation indicators, forming a unified data quality assessment framework. This indicator system is both comprehensive and targeted, covering key dimensions of data quality. It can accurately assess the quality status of different types of data, providing an actionable basis and direction for data governance.

[0074] Step 2: Build a data governance policy optimization model based on the deep deterministic policy gradient algorithm, and iteratively update and optimize the data governance strategy through data state feedback and reward function drive. Figure 3 , which is the flow chart of building a data governance strategy optimization model based on the DDPG algorithm in this step.

[0075] Define the state space of the DDPG algorithm and use the set of data quality assessment indicators as the state space , each state Indicates the quality assessment results of the current dataset, that is, the scores of various evaluation indicators.

[0076] For example, when the collected data is structured data, the state Expressed as: ;in, is the field missing rate, is the enumeration value accuracy, is the numerical range accuracy, is the data repetition rate, For data timeliness, Describes the completeness of the data.

[0077] When the collected data is unstructured, the status It can be expressed as: ;in, is the key tag completeness rate, is the accuracy of key information, For data relevance.

[0078] Define the action space of the DDPG algorithm and use data governance operations as the action space , each action Represents one or more governance operations performed on a dataset, such as data cleansing, data completion, and data standardization.

[0079] action It can be expressed as: ;in, For the Governance operations, ; For example, Indicates filling in missing values. Indicates correction of outliers, etc.

[0080] Design a reward function for the DDPG algorithm, and design a reward function that integrates data quality improvement and resource consumption , used to evaluate the execution action After the state from Transfer to The income obtained.

[0081] It should be noted that factors that need to be considered when constructing a reward function include: data quality: the degree of improvement of various data quality assessment indicators in the state after executing the action, such as reduced missing rate and improved accuracy; resource consumption: the computing resources and time cost consumed by executing the action.

[0082] For example, the reward function is designed as:

[0083]

[0084] in, For the The weight of each data quality assessment indicator, is the total number of data quality assessment indicators, For the The improvement value of an indicator, that is, the difference in the score of the indicator before and after the action is performed; To perform an action Resource consumption, which may include computing resources and time costs, is the weight coefficient of resource consumption, Set the resource sensitivity based on the task; for example, define If the data governance action is not sensitive to resource consumption: The value ranges from 0.01 to 0.1. If the data governance action being performed is sensitive to resource consumption, then The value ranges from 0.1 to 1.

[0085] Design the policy network of the DDPG algorithm and design the Actor network based on the DDPG algorithm. and Critic Network ;Actor network is used to Generate Data Governance Actions , the Critic network is used to evaluate the state-action pair value.

[0086] Actor network can be implemented using deep neural network, with input as state , the output is action For example, the Actor network can be designed as an MLP (Multi-layer Perceptron) structure containing several fully connected layers: ,in It is the set of learnable parameters of the Actor network, including all weights and bias terms in the neural network.

[0087] The critic network can be implemented using a deep neural network, with the input being the state and actions , the output is a state-action pair The value of For example, the critic network can be designed as an MLP structure containing several fully connected layers: ,in It is a set of learnable parameters of the Critic network, which is used to adjust the mapping relationship from state s and action a to Q value in the neural network structure.

[0088] A rule engine and expert system are introduced to constrain and optimize the data governance policy generated by the DDPG algorithm. The specific plan is as follows:

[0089] Build a data governance rule base based on domain knowledge, and formalize data quality requirements and business constraints into executable rules, such as "the missing rate cannot exceed 5%" and "the income field must be greater than 0". Summarize the experience of data governance experts as rule templates to form an expert system knowledge base. The knowledge base contains typical governance models for various data problems, such as "interpolation is required to fill in continuous fields with missing values ​​exceeding 10%".

[0090] Design a rule engine to match and verify the data governance actions generated by the DDPG algorithm with the rule base. The rule engine blocks or replaces actions that violate rule constraints to ensure that the strategy meets data governance requirements.

[0091] The expert system evaluates governance actions that pass rule matching and compares the actions with typical governance models in the knowledge base. For actions that deviate significantly from best practices, the expert system adjusts the data status feedback of the DDPG algorithm model.

[0092] Train the DDPG algorithm model, driven by data state feedback and reward functions, continuously update the parameters of the Actor network and Critic network, and optimize the data governance strategy.

[0093] In each training step, according to the current state , generating governance operations through the Actor network ; Perform an action After that, get the new state and rewards ;use Sample data, update the parameters of the Critic network through the gradient descent algorithm , minimize the timing difference (TD) error:

[0094]

[0095] in, is the discount factor, For the experience replay buffer.

[0096] Using the gradient information of the Critic network, the parameters of the Actor network are updated through the policy gradient algorithm. , maximize the expected return:

[0097]

[0098] The trained Actor network is the optimized data governance strategy model, which can be used according to the current state of the data set. , automatically generating optimal governance operations .

[0099] During DDPG algorithm training, the training strategy is dynamically adjusted based on feedback from the rule engine and expert system. This includes adding rule matches and deviations from best practices to training samples. During experience replay, samples with no constraint violations and minimal deviations are prioritized to accelerate model convergence. The rule base and expert system knowledge base are dynamically adjusted based on different data domains and business scenarios, ensuring that policy constraints are updated synchronously with actual needs.

[0100] When rule constraints change, the Actor network and Critic network are fine-tuned to enable the policy network to quickly adapt to the new rule requirements; an active learning mechanism is introduced to proactively trigger expert review for governance actions where rule matching is uncertain or deviates from best practices, and the policy network is reinforced with feedback.

[0101] The data governance strategy optimization model built based on the DDPG algorithm can continuously learn and optimize through data state feedback and reward functions, and adaptively generate optimal governance strategies for different data states; it also forms a policy optimization mechanism that combines data-driven and knowledge-driven by introducing rule engines and expert systems.

[0102] This step utilizes deep reinforcement learning technology to construct a data governance policy optimization model, enabling the automatic generation and dynamic optimization of data governance policies. Using data quality assessment indicators as the state space and governance operations as the action space, combined with a quantified reward function, this model enables intelligent optimization and adaptive adjustment of governance policies. This data governance policy optimization model significantly improves the intelligence and automation of data governance while also addressing governance costs, laying the foundation for subsequent multi-node collaborative optimization and real-time governance execution.

[0103] Step 3: Introduce the federated learning mechanism to conduct multi-node collaborative training on the data governance strategy optimization model built based on the DDPG algorithm. Figure 4 , which is the multi-node strategy collaborative training flow chart in federated learning in this step.

[0104] The federated learning mechanism is introduced. During the data governance process, the data owned by different nodes (such as different departments and branches) may be different and local. If the strategy optimization model is trained based on the data of a single node, it may be difficult to learn the globally optimal governance strategy. In order to achieve multi-node collaborative optimization, the federated learning mechanism is introduced to improve the DDPG algorithm.

[0105] Specifically, there is Participating nodes, each node Have a local dataset , based on the constructed DDPG model, at each node Initialize local Actor network parameters and Critic network parameters .

[0106] During the training process of federated learning, each node independently performs local model training and regularly aggregates and updates parameters with the central server.

[0107] Training the local model for federated learning, each node Based on local dataset , by performing data governance operations and observing rewards and state transfers, the DDPG algorithm is used to update the local Actor network parameters and Critic network parameters , optimize the local policy model; the local model training process is the same as the single-node training process in step 2.

[0108] In each federated learning round At the end of each time, each node will set the local Actor network parameters and Critic network parameters Upload to the central server; the central server aggregates the collected parameters and generates global Actor network parameters and global critic network parameters ;

[0109] The aggregation method can be weighted average: ;in, and Representation node and nodes The number of local dataset samples.

[0110] The central server aggregates the global parameters and Distributed to each node, each node updates the local model with the global parameters and starts a new round of local training.

[0111] Repeat the steps of the federated learning process, and through multiple rounds of federated learning, make the local strategy model of each node gradually approach the global optimum, achieving the goal of multi-node collaborative optimization.

[0112] In order to further improve the efficiency and performance of federated learning, optimization strategies can be adopted in the federated learning process, including adaptive aggregation frequency, differential privacy protection, model compression, and incentive mechanism design.

[0113] Adaptive aggregation frequency involves dynamically adjusting the frequency of parameter aggregation based on the differences in data distribution between nodes and the speed of model convergence. When data distribution between nodes differs significantly or convergence is slow, the aggregation frequency can be increased; otherwise, the frequency can be decreased to reduce communication overhead.

[0114] The differential privacy protection includes: introducing a differential privacy mechanism in the parameter aggregation process to prevent malicious nodes from inferring sensitive information of other nodes through parameters while ensuring the utility of aggregated parameters; adding random noise to the uploaded local parameters to achieve -Differential privacy protection.

[0115] The model compression includes: in order to reduce the communication overhead during the parameter aggregation process, local model parameters can be compressed, such as pruning, quantization and sparsification techniques; the compressed model parameters can significantly reduce the communication volume during aggregation and distribution, thereby improving the efficiency of federated learning.

[0116] The incentive mechanism design includes: A rational inter-node incentive mechanism is designed to encourage each node to actively participate in federated learning and provide high-quality local model updates. The incentive mechanism can provide corresponding incentives based on the node's contribution (such as data quality and model performance improvement), such as resource allocation and revenue sharing, to form a healthy collaborative ecosystem.

[0117] By introducing the federated learning mechanism, multi-node collaborative training of the data governance policy optimization model is achieved; federated learning allows each node to share each other's knowledge and experience through parameter aggregation and updating while protecting data privacy, thereby learning a more comprehensive and robust policy model; compared with single-node training, federated learning can make full use of the data resources of different nodes and improve the generalization performance of the policy model; federated learning also reduces the transmission and sharing of original data through local training and parameter aggregation, achieving collaborative optimization while protecting data security and privacy.

[0118] This step introduces the federated learning mechanism to conduct multi-node collaborative training on the data governance policy optimization model built based on the DDPG algorithm. Through local training and parameter aggregation, while protecting data privacy, it fully utilizes the data resources of different nodes to learn a more comprehensive and robust policy model. This step also adopts a variety of optimization strategies to further improve the efficiency, security and operability of federated learning. The introduction of federated learning enables the data governance policy optimization model to adapt to the distributed data environment, realize multi-node collaborative optimization, and improve the intelligence and automation level of data governance.

[0119] Step 4: Design a data governance policy execution engine based on the stream processing framework, build data processing operators, and execute data governance policies through data processing operators. Figure 5 , a flow chart of the data governance strategy mechanism based on stream processing is constructed for this step.

[0120] Build a stream processing framework for real-time data governance. The stream processing framework can provide rich data processing operators, flexible window mechanism, reliable state management and high scalability, and can also meet the low latency and high throughput requirements of real-time data governance.

[0121] Construct data processing operators and build a set of general data processing operators in the stream processing framework to execute data governance strategies; according to the common needs of data governance, the constructed data processing operators include: data cleaning operators, data conversion operators, data aggregation operators and data connection operators, the data cleaning operators include: outlier processing operators, missing value processing operators and data standardization operators; the data conversion operators include: data type conversion operators, data format conversion operators and data encoding conversion operators; the data aggregation operators include group aggregation operators and window aggregation operators; the data connection operators include: inner join operators and outer join operators.

[0122] The outlier processing operator is used to detect and process outliers in data, and may use statistical methods (such as Z-score) or machine learning methods (such as isolation forest).

[0123] The missing value processing operator is used to process missing values ​​in data, and methods such as deletion and filling (such as mean, median, KNN) can be used.

[0124] The data normalization operator is used to perform normalization processing on the data, and methods such as minimum-maximum normalization and Z-score normalization can be used.

[0125] For example, the outlier processing operator uses the Z-score method to calculate outliers. Suppose a numerical attribute in the data stream is , whose mean is , the standard deviation is , the outlier threshold is , then the outlier processing operator can be expressed as: ,in, Indicates the original value to be input as the output value.

[0126] The data type conversion operator is used to perform data type conversion, such as converting a string to a value, or converting a date and time to a timestamp.

[0127] The data format conversion operator is used to convert the data format, such as JSON to CSV, XML to JSON, etc.

[0128] The data encoding conversion operator is used to perform encoding conversion on data, such as UTF-8 to GBK, ASCII to Unicode, etc.

[0129] For example, the data type conversion operator converts a string into a value. Suppose the string in the data stream , the target type is a numeric value, then the data type conversion operator can be expressed as: , represents a numerical conversion function, A set of all legal strings that can be successfully converted to numeric values.

[0130] The group aggregation operator is used to group data according to a specified key and perform aggregation operations on the data in each group, such as sum, average, maximum, minimum, etc.

[0131] The window aggregation operator is used to perform aggregation operations on data according to a specified window (such as a rolling window, a sliding window, or a session window).

[0132] The inner join operator is used to perform an inner join operation on two data streams according to a specified key, and only retain records with matching keys.

[0133] The outer join operator is used to perform an outer join operation on two data streams according to a specified key, retaining records with matching and non-matching keys, and filling non-matching records with NULL values.

[0134] Convert the data governance strategy into an execution plan based on data processing operators. According to the data processing steps and rules defined in the strategy, select the corresponding data processing operators, set the operator parameters and execution order, and form a complete data governance execution plan.

[0135] For example, assuming the data governance strategy is to deduplicate data, the following execution plan can be used: (a) Use the group aggregation operator to group the data by a specified key (such as a primary key or unique key). (b) For each group, only the first record is retained, and the remaining records are discarded.

[0136] Design a data governance policy execution engine based on the stream processing framework, combine data processing operators into a data processing flow, and process data streams in real time.

[0137] According to the execution plan of the data governance strategy, the Data Stream API of the stream processing framework is used to combine data processing operators into a complete data processing flow; complex data governance logic is constructed through the series, parallel, and branching of operators.

[0138] The processed data is output to the target system through the data sink connector of the stream processing framework (such as Kafka, HDFS, HBase, etc.) for use by downstream applications.

[0139] The checkpoint mechanism and state management mechanism of the stream processing framework are used to achieve fault tolerance and failure recovery in the data processing process; checkpoints are generated regularly to save the state information of the operator, so that data processing can be resumed from the most recent checkpoint when a failure occurs.

[0140] Continuously optimize and tune the execution of data governance strategies based on monitoring indicators and execution logs; improve the efficiency and quality of data governance by adjusting operator parameters, optimizing data processing logic, and improving resource allocation; and continuously iterate and update data governance strategies to adapt to changes in the data environment and the evolution of business needs.

[0141] This step designs a data governance policy execution engine based on the stream processing framework. By building general data processing operators, the data governance policy is converted into an executable data processing process, and the streaming computing capabilities of the stream processing framework are used to achieve efficient and reliable real-time data governance. It also introduces a real-time monitoring and tuning mechanism to continuously optimize the effectiveness and efficiency of data governance and adapt to changes in the data environment and the evolution of business needs.

[0142] Step 5: Introduce federated distillation technology to compress the deep reinforcement learning model trained by federated learning into a lightweight version through distillation and deploy it together with the data processing operator. Figure 6 , which is the deployment flow chart of the federated distillation and lightweight strategy model in this step.

[0143] Establish a federated distillation mechanism to compress the complex deep reinforcement learning model trained by federated learning into a lightweight version for easy deployment and use in the data governance policy execution engine.

[0144] The deep reinforcement learning model trained by federated learning is used as the teacher model; the teacher model usually has high performance and complexity, but may face challenges such as large model size and high inference latency in actual deployment.

[0145] A lightweight student model is used to learn knowledge from the teacher model. The architecture of the student model should be simpler and more compact to reduce model complexity and inference latency, while maintaining performance as close as possible to that of the teacher model.

[0146] Define the distillation loss function to measure the learning effect of the student model during the distillation process; the distillation loss function includes imitation loss and task loss.

[0147] Imitation loss measures the degree to which the student model imitates the teacher model in the feature space, prompting the student model to learn the feature representation of the teacher model; commonly used imitation losses include mean square error loss and KL divergence loss.

[0148] For example, the mean square error loss is used as the imitation loss of the federated distillation process, and the feature vector of the teacher model is set to , the eigenvector of the student model is , then the mean square error loss is expressed as: ;in, is the number of data samples used to train federated distillation, and Respectively represent The feature vectors of the data samples used for federated distillation in the teacher model and the student model.

[0149] Task loss is used to measure the performance of the student model on the original task, ensuring that the student model does not significantly degrade task performance during the distillation process. Task loss is designed based on the evaluation indicators of specific tasks, such as cross entropy loss for classification tasks and mean squared error loss for regression tasks.

[0150] For example, cross entropy loss is used as the task loss of the federated distillation process, and the predicted probability vector of the teacher model is set to , the predicted probability vector of the student model is , the true label is , then the cross entropy loss can be expressed as: ;

[0151] in, The first data sample in the training federated distillation The true labels of samples, For students to model The predicted probability vector for each sample.

[0152] The final distillation loss function can be expressed as a weighted sum of imitation loss and task loss: ;in, and Is the balance factor, which is used to adjust the weight of imitation loss and task loss; for the balance factor and , can be set initially , let the student model learn more about the structural knowledge of the teacher model; gradually increase , enhancing the student model’s ability to fit the true labels.

[0153] In a federated learning environment, the student model is distilled and trained. Each participating node uses local data and the teacher model for distillation to update the parameters of the student model. At the same time, through the federated averaging algorithm, the parameters of the student model of each node are regularly aggregated to obtain a global student model.

[0154] With Nodes participating in federated distillation, The student model parameters of each node are , , the global student model parameters are , then the federated averaging algorithm can be expressed as: .

[0155] The lightweight student model is integrated with the data processing operator as a data governance policy execution mechanism; the data processing operator is responsible for cleaning, converting, aggregating and other operations on the data, and the lightweight student model is responsible for generating the optimal data governance operation based on the data status.

[0156] The distilled lightweight student model is deployed along with the data processing operators into the data governance policy execution engine. An update mechanism is also established to regularly distill the new deep reinforcement learning model trained through federated learning and update the lightweight model in the data governance policy execution engine. During the update process, both the old and new versions of the lightweight model are deployed simultaneously and served simultaneously for a certain period of time. When the model is updated, it automatically switches to the latest model.

[0157] This step establishes a federated distillation mechanism to compress the deep reinforcement learning model trained through federated learning into a lightweight version and integrate it with data processing operators to form an efficient data governance policy execution mechanism. This step also establishes a policy update mechanism to regularly distill and update the model, enabling continuous optimization of the data governance policy. Through measures such as model optimization, resource scheduling, and hot policy updates, this step ensures the efficient execution and stable operation of the data governance policy, improving the intelligence and automation of data governance.

[0158] Example 2

[0159] Reference Figure 7 , which is the second embodiment of the present invention, provides an automated data governance strategy generation system based on reinforcement learning.

[0160] The system includes: a data acquisition module, a strategy generation module, a strategy model optimization module, a data governance module and a federated distillation deployment module.

[0161] The data acquisition module collects multiple types of data from multiple data sources and constructs a data quality assessment indicator system for different data types.

[0162] The strategy generation module is used to build a data governance strategy optimization model based on a deep deterministic policy gradient algorithm, and iteratively updates and optimizes the data governance strategy through data state feedback and reward function driving.

[0163] The policy model optimization module introduces a federated learning mechanism to perform multi-node collaborative training on the data governance policy optimization model built based on the DDPG algorithm.

[0164] The data governance module is used to design a data governance policy execution engine based on a stream processing framework, build data processing operators, and execute data governance policies through data processing operators.

[0165] The federated distillation deployment module introduces federated distillation technology to compress the deep reinforcement learning model trained by federated learning into a lightweight version through distillation, and deploys it together with the data processing operator.

[0166] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiments are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are protected by the present invention.

Claims

1. An automated data governance strategy generation method based on reinforcement learning, characterized by: include: Collect multiple types of data from multiple data sources and build a data quality assessment indicator system for different data types; Build a data governance policy optimization model based on the Deep Deterministic Policy Gradient (DDPG) algorithm. Define the state space of the DDPG algorithm, using the set of data quality assessment indicators as the state space; use data governance operations as the action space, where each action represents a governance operation performed on the dataset; and design a reward function for the DDPG algorithm to evaluate the benefits of the state after executing the action. Constructing a policy network based on a deep deterministic policy gradient algorithm, including designing an actor network and a critic network; the actor network generates data governance operations based on the state; the critic network is used to evaluate the value of state-action pairs; Train a deep deterministic policy gradient algorithm model, continuously update the parameters of the actor network and critic network through data state feedback and reward function drive, and through the constraints of the rule engine and expert system, to optimize the data governance strategy; through data state feedback and reward function drive, iteratively update and optimize the data governance strategy; Introducing a federated learning mechanism to conduct multi-node collaborative training on the data governance strategy optimization model built based on the DDPG algorithm; Design a data governance policy execution engine based on a stream processing framework, build data processing operators, and execute data governance policies through data processing operators; Federated distillation technology is introduced to compress the deep reinforcement learning model trained by federated learning into a lightweight version through distillation and deploy it together with data processing operators. The deep reinforcement learning model trained by federated learning is used as the teacher model. A lightweight student model is used to learn knowledge from the teacher model. In a federated learning environment, the student model is distilled and trained. Each participating node uses local data and the teacher model for distillation to update the parameters of the student model. At the same time, through the federated averaging algorithm, the parameters of the student model of each node are regularly aggregated to obtain a global student model.

2. The method for generating an automated data governance strategy based on reinforcement learning according to claim 1, characterized in that: The multi-type data includes structured data and unstructured data; Construct a data quality assessment indicator system, set quantitative indicators from the dimensions of completeness, accuracy, consistency, timeliness and interpretability of the collected structured data; for the collected unstructured data, additional indicators of structural integrity, semantic accuracy and relevance are added.

3. The method for generating an automated data governance strategy based on reinforcement learning according to claim 2, characterized in that: Introducing a federated learning mechanism to perform multi-node collaborative training on the deep deterministic policy gradient algorithm; During the training process of federated learning, each node independently performs local model training and regularly aggregates and updates parameters with the central server.

4. The method for generating an automated data governance strategy based on reinforcement learning according to claim 3, characterized in that: Training the local model of federated learning: Each node, based on the local dataset, performs data governance operations and observes rewards and state transitions, using a deep deterministic policy gradient algorithm to update the local Actor network parameters and Critic network parameters, thereby optimizing the local policy model. At the end of each federated learning round, each node uploads the local Actor network parameters and Critic network parameters to the central server; the central server aggregates the collected parameters to generate global Actor network parameters and global Critic network parameters; The central server distributes the aggregated global parameters to each node, and each node uses the global parameters to update the local model.

5. The method for generating an automated data governance strategy based on reinforcement learning according to claim 4, characterized in that: Build a stream processing framework for data governance and build data processing operators within the framework; The constructed data processing operators include: data cleaning operators, data conversion operators, data aggregation operators and data connection operators. The data cleaning operators include: outlier processing operators, missing value processing operators and data standardization operators; the data conversion operators include: data type conversion operators, data format conversion operators and data encoding conversion operators; the data aggregation operators include group aggregation operators and window aggregation operators; the data connection operators include: inner connection operators and outer connection operators.

6. The method for generating an automated data governance strategy based on reinforcement learning according to claim 5, characterized in that: Integrate the lightweight student model with data processing operators as a data governance policy execution mechanism; The lightweight student model generates the optimal data governance operation based on the data status, and the data processing operator performs data governance operations on the collected data.

7. An automated data governance strategy generation system based on reinforcement learning, which is used to implement the automated data governance strategy generation method based on reinforcement learning according to any one of claims 1 to 6, characterized in that: include: Data acquisition module, policy generation module, policy model optimization module, data governance module, and federated distillation deployment module; The data acquisition module collects multiple types of data from multiple data sources and builds a data quality assessment indicator system for different data types; The strategy generation module is used to build a data governance strategy optimization model based on a deep deterministic policy gradient algorithm, and iteratively updates and optimizes the data governance strategy through data state feedback and reward function driving. The policy model optimization module introduces a federated learning mechanism to perform multi-node collaborative training on the data governance policy optimization model built based on the DDPG algorithm; The data governance module is used to design a data governance policy execution engine based on a stream processing framework, build data processing operators, and execute data governance policies through data processing operators; The federated distillation deployment module introduces federated distillation technology to compress the deep reinforcement learning model trained by federated learning into a lightweight version through distillation, and deploys it together with the data processing operator.

Images