Data governance method and apparatus, computer device, and storage medium

Abstract

This application belongs to the field of data governance technology, and relates to a data governance method, apparatus, computer equipment, and storage medium. It includes: generating scene labels based on state data from multiple governance nodes; intelligently generating governance actions through a reinforcement learning model; collecting feedback after execution at a selected target node; and synchronizing this feedback to all nodes in the network, thereby achieving dynamic optimization of the governance strategy and system collaboration. This application achieves real-time intelligent decision-making for governance strategies through scene labels generated from multi-source data and a reinforcement learning model. Simultaneously, by synchronizing the execution feedback of a single node to the entire network, the system gains continuous optimization capabilities, solving the problem of insufficient adaptability of traditional static rule-based governance methods in complex and ever-changing distributed environments, and significantly improving the accuracy of governance and the overall efficiency of the system.

Description

Technical Field

[0001] This application relates to the field of data governance technology, and in particular to a data governance method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Existing data governance widely adopts distributed data systems, which deploy multiple governance nodes for collaborative data management and have become the basic architecture in this field. Existing technologies typically employ static governance strategies, that is, pre-defined rules are distributed to each node, and the nodes match and execute fixed governance actions based on their local states.

[0003] However, this solution has significant drawbacks. Its governance strategy is rigid and cannot adaptively adjust based on the real-time state of the system. When a node's action is ineffective, the system cannot provide real-time feedback and optimize the behavior of other nodes, making it unable to cope with dynamic and complex real-world scenarios and resulting in low governance efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a data governance method, apparatus, computer equipment, and storage medium to solve the problem of low governance efficiency caused by the inability to adaptively adjust according to the real-time operating status of the system.

[0005] To address the aforementioned technical problems, this application provides a data governance method, which employs the following technical solution: A data governance method includes the following steps: Status data is obtained from multiple governance nodes, and scene labels are generated based on the status data; Acquire the data to be processed, input the scene label and the data to be processed into a preset reinforcement learning model, and generate target governance actions; Select a target governance node from the plurality of governance nodes, perform the target governance action on the target governance node, and collect feedback information from the target governance node; The feedback information from the target governance node is synchronized to the remaining governance nodes, and the remaining governance nodes are controlled to execute the target governance action based on the feedback information.

[0006] Furthermore, the aforementioned status data includes data status information, business status information, and node status information. The generation of scene tags based on the aforementioned status data includes: The data status information, the business status information, and the node status information are weighted to obtain a target weight; the scene label is generated based on the target weight, the data status information, the business status information, and the node status information.

[0007] Furthermore, the above-mentioned input of the scene label and the data to be processed into a preset reinforcement learning model to generate target governance actions includes: The scene label is fused with the data to be processed to construct a state space; based on the state space, multiple candidate governance actions and the action probability of each candidate governance action are output through the policy network of the reinforcement learning model; according to the action probability of each candidate governance action, a governance action is selected from the multiple candidate governance actions as the target governance action.

[0008] Furthermore, the aforementioned execution of the target governance action at the target governance node and collection of feedback information from the target governance node include: Obtain a preset data governance framework, integrate the target governance action and the data governance framework to generate an integrated framework; according to the integrated framework, schedule the data processing operators in the target governance node to execute the target governance action.

[0009] Furthermore, synchronizing the feedback information from the target governance node to the remaining governance nodes includes: The feedback information from the target governance node is encrypted to obtain encrypted feedback information; the encrypted feedback information is then synchronized to the remaining governance nodes.

[0010] Furthermore, the aforementioned target governance actions are multiple, and the step of selecting a target governance node from the multiple governance nodes and performing the target governance action on the target governance node further includes: Select multiple target governance nodes from the multiple governance nodes, assign multiple target governance actions to the multiple target governance nodes, and control the multiple target governance nodes to execute the multiple target governance actions.

[0011] Furthermore, the above-mentioned selection of multiple target governance nodes from the multiple governance nodes and assignment of multiple target governance actions to the multiple target governance nodes includes: The system acquires performance parameters of multiple governance nodes, selects multiple target governance nodes from the multiple governance nodes based on the performance parameters, acquires the mapping relationship between the multiple target governance actions and the multiple target governance nodes, and assigns the multiple target governance actions to the multiple target governance nodes based on the mapping relationship and the performance parameters.

[0012] To address the aforementioned technical problems, this application also provides a data governance device, which employs the following technical solution: A data governance device, comprising: The scene tag generation module is used to obtain status data from multiple governance nodes and generate scene tags based on the status data. The governance action generation module is used to acquire the data to be processed, input the scene label and the data to be processed into a preset reinforcement learning model, and generate the target governance action. A single-node execution module is used to select a target governance node from multiple governance nodes, execute the target governance action on the target governance node, and collect feedback information from the target governance node; A multi-node execution module is used to synchronize the feedback information of the target governance node to the remaining governance nodes, and control the remaining governance nodes to execute the target governance action based on the feedback information.

[0013] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution: A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the data governance method described above.

[0014] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below: A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the data governance method described above.

[0015] Compared with the prior art, this application has the following main advantages: The data governance method disclosed in this application obtains state data from multiple governance nodes and generates scene labels based on the state data, providing accurate and reliable basis for governance actions; it obtains data to be processed, inputs the scene labels and data to be processed into a preset reinforcement learning model to generate target governance actions, enabling the governance actions to dynamically adapt to complex scenarios and overcome the rigidity of traditional static rules; it selects a target governance node from multiple governance nodes, executes the target governance action on the target governance node, collects feedback information from the target governance node, and transforms the execution effect into quantifiable evaluation data, creating conditions for system optimization; it synchronizes the feedback information of the target governance node to the remaining governance nodes, and controls the remaining governance nodes to execute the target governance action according to the feedback information, forming a value loop of the entire method, thereby improving the governance efficiency of data governance. Attached Figure Description

[0016] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the data governance method according to this application; Figure 3 This is a schematic diagram of a structure of an embodiment of the data governance apparatus according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0021] like Figure 1As shown, the system architecture 100 may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0022] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104, and can receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0023] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 (Moving Picture Experts Group Audio Layer Ⅲ) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptops, and desktop computers, etc.

[0024] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0025] It should be noted that the data governance method provided in this application embodiment is generally executed by a server, and correspondingly, the data governance device is generally located in the server.

[0026] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0027] Continue to refer to Figure 2 A flowchart of an embodiment of the data governance method according to this application is shown. The data governance method includes the following steps: Step S201: Obtain status data from multiple governance nodes and generate scene labels based on the status data.

[0028] In this embodiment, the data governance method runs on electronic devices (e.g., Figure 1 The server shown can send or receive data via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, Wi-Fi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future-developed wireless connection methods.

[0029] It should be noted that traditional methods do not fully utilize the real-time status data of each governance node, leading to a disconnect between governance strategies and actual scenarios. This makes them unable to handle complex situations such as dynamic data changes and fluctuations in node operating status in a distributed environment. Therefore, it is necessary to acquire status data from multiple governance nodes and generate scenario labels to provide comprehensive and accurate scenario-based information for the intelligent generation of subsequent governance actions.

[0030] Governance nodes refer to various computing nodes participating in the data governance process, including but not limited to servers, edge devices, and terminal computing units. Node deployment can encompass centralized, distributed, and edge architectures, and their function is to handle data storage, data processing, and the execution of governance actions. Status data refers to the collection of various data generated by governance nodes during operation, reflecting the node's own operational status and related data governance information. It serves as the basis for generating scenario tags and is not limited to specific data types or dimensions. Scenario tags are identifying information extracted from the status data of multiple governance nodes through preset rules or algorithms, characterizing the features of the current data governance scenario.

[0031] In this embodiment, firstly, the set of governance nodes for which status data needs to be collected is defined. The number of nodes can be flexibly adjusted according to the scale of governance and is not limited to a fixed number. Then, by adapting to the communication methods of the governance nodes (including but not limited to existing or future-developed communication technologies such as wired and wireless communication), status data is collected from each governance node in real time. The collection frequency can be dynamically adjusted according to the real-time requirements of the governance scenario to ensure data timeliness. Finally, based on the collected multi-node status data, through a preset tag generation logic (the logic form can be flexibly adapted to governance needs and is not limited to a specific algorithm), the characteristics of the current governance scenario are extracted to form a scenario tag that can uniquely represent the scenario. The tag format and content can be flexibly defined according to subsequent governance decision-making needs.

[0032] Step S202: Obtain the data to be processed, input the scene label and the data to be processed into a preset reinforcement learning model, and generate the target governance action.

[0033] It should be noted that the drawback of static governance methods is that the governance actions are fixed and cannot be dynamically adjusted according to changes in the scenario and the characteristics of the data to be processed. Therefore, it is necessary to acquire the data to be processed, input it together with the scenario labels into a preset reinforcement learning model, and utilize the dynamic decision-making capability of the reinforcement learning model to generate target governance actions that are adapted to the current scenario and data characteristics.

[0034] Data to be processed refers to various types of data that need to achieve goals such as quality optimization, compliance verification, and risk prevention through the data governance process. Data types can include structured, unstructured, and semi-structured data, and data sources include, but are not limited to, data generated by business systems, externally collected data, and data transmitted across nodes. A pre-trained reinforcement learning model refers to a reinforcement learning model that has been pre-trained and deployed for data governance decision-making. The model type can be selected according to governance needs and is not limited to a specific reinforcement learning algorithm. Its function is to autonomously generate governance actions adapted to the current scenario based on the input scenario labels and data to be processed. Target governance actions refer to the specific operational instructions output by the reinforcement learning model based on scenario labels and data to be processed, used to achieve data governance goals. These include, but are not limited to, various governance-related operations such as data cleaning, data transformation, compliance review, risk identification, and data synchronization.

[0035] In this embodiment, the data to be processed can be obtained from the data source or data storage node through a data access interface (the interface type can be adapted to different data sources and data types), ensuring complete data access without loss of key information. The scene labels generated in step S201 and the obtained data to be processed are organized according to a preset format and input into a pre-deployed reinforcement learning model. The input method can be adapted to the model deployment architecture (centralized, distributed, etc.). Based on the input scene labels and data to be processed, the reinforcement learning model autonomously outputs target governance actions adapted to the current scene and data characteristics through its own decision-making logic (including but not limited to policy network calculation, action probability evaluation, etc.). The number of actions can be single or multiple depending on the governance needs.

[0036] Step S203: Select a target governance node from the plurality of governance nodes, perform the target governance action on the target governance node, and collect feedback information from the target governance node.

[0037] It's important to note that directly executing the target governance action on all nodes could lead to resource waste and low efficiency. Furthermore, the lack of feedback on the effectiveness of these actions means the system won't know their actual impact, hindering subsequent optimization. Therefore, it's necessary to first select target governance nodes to execute the action, verifying its feasibility and effectiveness through pilot testing. Then, based on the collected feedback, a decision should be made on whether to roll it out across the entire network. This approach ensures governance efficiency, conserves resources, and provides data support for continuous system optimization.

[0038] A target governance node refers to a specific node selected from multiple governance nodes to execute target governance actions. The selection criteria can be flexibly set according to governance needs, and the requirement is that it can efficiently undertake the execution tasks of the target governance actions. Feedback information refers to various data generated by the target governance node during and after the execution of the target governance actions, which can reflect the effectiveness of the governance actions and changes in the node's operating status, including but not limited to the success rate of governance action execution, the improvement of data governance quality, and changes in node resource consumption.

[0039] In this embodiment, based on preset node selection criteria (which can be selected according to factors such as governance action characteristics and node operating status), target governance nodes that meet the requirements are selected from multiple governance nodes. The number of nodes can be single or multiple depending on the governance needs. The target governance action is issued to the selected target governance node, which executes the target governance action using its own computing resources and data processing capabilities. The execution priority can be dynamically adjusted according to the node's operating status. Throughout the entire process of the target governance action being executed by the target governance node, data acquisition tools capture node operating data, governance action execution progress data, and governance result data in real time, integrating them to form feedback information to ensure the completeness and timeliness of the feedback information.

[0040] Step S204: Synchronize the feedback information of the target governance node to the remaining governance nodes, and control the remaining governance nodes to execute the target governance action based on the feedback information.

[0041] It's important to note that directly instructing the remaining governance nodes to execute the target governance actions without considering feedback information may lead to the repetition of execution problems encountered by the target node, or poor governance results due to node differences. Therefore, it's necessary to synchronize feedback information to the remaining governance nodes, allowing them to adjust their execution logic based on the feedback. This ensures the effectiveness and consistency of governance actions, while simultaneously enabling collaborative governance across the entire network, forming a closed loop from execution and feedback to promotion, thereby improving overall governance efficiency.

[0042] The remaining governance nodes refer to all governance nodes that did not participate in the first round of governance action execution, except for the target governance node that has already executed the target governance action. They are the carriers for the subsequent promotion and execution of the target governance action.

[0043] In this embodiment, the feedback information collected from the target governance nodes is standardized to ensure a uniform information format and clear content, facilitating parsing and use by the remaining governance nodes. By using information transmission methods adapted to the distributed environment (including but not limited to encrypted transmission, distributed message queues, and direct inter-node communication), the processed feedback information is synchronized to all remaining governance nodes, ensuring the security, integrity, and timeliness of information transmission. Upon receiving the feedback information, the remaining governance nodes determine the suitability of the target governance action based on the feedback information and execute the target governance action according to preset control logic (including but not limited to direct execution, execution after parameter adjustment, and execution after partial action optimization), thus enabling the feedback information to guide the execution process.

[0044] This application achieves real-time intelligent decision-making for governance strategies through scene labels generated from multi-source data and reinforcement learning models. Simultaneously, by synchronizing the execution feedback from individual nodes to the entire network, the system gains continuous optimization capabilities, addressing the insufficient adaptability of traditional static rule-based governance methods in complex and ever-changing distributed environments, and significantly improving the accuracy of governance and the overall efficiency of the system.

[0045] In some optional implementations of this embodiment, the aforementioned status data includes data status information, business status information, and node status information. The step of generating scene tags based on the status data includes: The data status information, the business status information, and the node status information are weighted to obtain a target weight; the scene label is generated based on the target weight, the data status information, the business status information, and the node status information.

[0046] Data status information refers to a set of information characterizing the attributes and characteristics of the data to be processed, including but not limited to data integrity, accuracy, consistency, timeliness, data type (structured / unstructured / semi-structured), data scale, data distribution characteristics, and data sensitivity level, directly reflecting the quality and characteristics of the data. Business status information refers to a set of information related to business scenarios and objectives associated with data governance, including but not limited to business type (e.g., financial transactions, medical diagnosis, government approvals), business process stage, compliance requirements (e.g., privacy protection clauses, industry standards), governance objective priority (e.g., prioritizing data compliance, prioritizing improving data processing efficiency), and business urgency (e.g., ordinary business, urgent business), reflecting the business background and needs of data governance. Node status information refers to a set of status information of each governance node participating in data governance during operation, including but not limited to node computing load, memory utilization, remaining storage resources, network transmission rate, task execution progress, node health (e.g., presence of failure risk), and node type (e.g., edge node, cloud node), reflecting the node's operational capabilities and resource status.

[0047] In this embodiment, based on the needs of the data governance scenario (such as business urgency, data characteristics, node resource status, etc.), the key factors affecting weight adjustment are identified. These factors can come from historical governance experience, real-time scenario characteristics, or preset rules. According to the determined adjustment criteria, the initial weights of data status information, business status information, and node status information are dynamically adjusted. For example, when the "urgency" in the business status information is detected as "extremely high," the weight of the business status information is increased. When the "data sensitivity level" in the data status information is "extremely high," the weight of the data status information is increased. When the "computing load" in the node status information is "excessively high," the weight of the node status information is increased, ultimately obtaining the target weight.

[0048] According to the target weights, data status information, business status information, and node status information are weighted and calculated. The feature values ​​of the three types of information are then fused according to their weight proportions to form a comprehensive feature vector. This vector highlights the features of information with high weights. Feature extraction and scene matching are then performed on the weighted and fused comprehensive feature vector to extract scene labels that can represent the characteristics of the current scene. The label content is consistent with the weighted features.

[0049] This application uses dynamic weight adjustment to enable scenario tags to focus on the characteristics of the current governance scenario and avoid interference from non-critical information. At the same time, it adapts to different types of governance scenarios (such as routine business, emergency business, and high-load node business), enhancing the flexibility and adaptability of scenario tags.

[0050] In some optional implementations of this embodiment, the above-mentioned inputting the scene label and the data to be processed into a preset reinforcement learning model to generate target governance actions includes: The scene label is fused with the data to be processed to construct a state space; based on the state space, multiple candidate governance actions and the action probability of each candidate governance action are output through the policy network of the reinforcement learning model; according to the action probability of each candidate governance action, a governance action is selected from the multiple candidate governance actions as the target governance action.

[0051] The state space is a high-dimensional vector space constructed by fusing scenario labels and data to be processed. It contains features of the current governance scenario and the data to be processed, serving as the input for the reinforcement learning model to understand the current governance environment. Each dimension in the space corresponds to a specific feature (such as scenario urgency, data sensitivity level, data scale, etc.). The policy network of the reinforcement learning model is the component responsible for generating governance actions. It takes the state space as input and outputs possible governance actions and their corresponding probabilities through neural network computation (such as fully connected layers, convolutional layers, attention mechanisms, etc.). Candidate governance actions refer to multiple potential governance actions output by the policy network that may be applicable to the current governance scenario. Each action corresponds to a specific governance operation (such as data cleaning, sensitive information desensitization, outlier detection, data compression, compliance verification, etc.), and the number can be dynamically adjusted according to the complexity of the scenario.

[0052] In this embodiment, feature vectors of scene labels are extracted. For example, "high concurrency - sensitive data - medium load" is transformed into a vector containing three dimensions, each corresponding to an attribute value of the label. Feature vectors of the data to be processed (such as data type, scale, sensitivity level, completeness, etc.) are extracted and transformed into multi-dimensional vectors. Through feature concatenation (connecting the two types of vectors end to end) or feature fusion algorithms (such as attention mechanisms to highlight highly correlated features), the scene label feature vectors and the data to be processed feature vectors are integrated into a high-dimensional comprehensive vector, i.e., the state space, which fully contains the correlation features between "scene and data".

[0053] The constructed state space is input into the policy network of the reinforcement learning model. The policy network can be computed through a multi-layer neural network. For example, the input layer receives the state space vector, the hidden layer processes features through activation functions, and the output layer normalizes the probability using the softmax function. The output layer corresponds to multiple candidate governance actions, and each action node outputs its corresponding action probability, forming a mapping relationship of "candidate action - action probability". The probability value reflects the degree of fit of the action in the current state space.

[0054] The probabilities of all candidate governance actions are ranked, and the candidate action with the highest probability is selected. If there are special constraints (such as a high-probability action requiring a specific resource that the current node does not have), the candidate action with the highest probability that satisfies the constraint can be selected as the target governance action.

[0055] This application constructs a state space by fusing scene labels with the data to be processed, enabling reinforcement learning models to understand the relationship between scenes and data, and reducing the mismatch between actions and scene or data characteristics.

[0056] In some optional implementations of this embodiment, the above-mentioned execution of the target governance action on the target governance node and collection of feedback information from the target governance node include: Obtain a preset data governance framework, integrate the target governance action and the data governance framework to generate an integrated framework; according to the integrated framework, schedule the data processing operators in the target governance node to execute the target governance action.

[0057] A pre-designed data governance framework refers to a structured system that is designed and deployed in advance to standardize the entire data governance process. It includes elements such as governance process specifications, functional module divisions, interface standards, and resource scheduling rules. It serves as the fundamental carrier for executing data governance actions and can be adapted to different industries (such as finance, healthcare, and government) and governance scenarios (such as data cleaning, compliance review, and risk control). Framework forms include, but are not limited to, modular frameworks, process-oriented frameworks, and distributed collaborative frameworks. Data processing operators refer to functional units within the target governance node used to perform specific data processing operations. They are the technical carriers for realizing the target governance actions. Each operator corresponds to a specific data processing logic (e.g., a de-identification operator is responsible for replacing sensitive information, a cleaning operator is responsible for correcting erroneous data, and a detection operator is responsible for identifying abnormal patterns). Operators can be called independently or used in combination.

[0058] An integrated framework refers to a structured system with complete execution capabilities formed by integrating target governance actions with a pre-set data governance framework. It retains the process specifications and resource scheduling capabilities of the original framework while including target governance actions for the current scenario, serving as a key bridge connecting governance decisions and actual execution.

[0059] In this embodiment, a basic framework matching the current governance scenario type is called from the system's preset framework library. For example, a framework containing a compliance verification module is called for a financial scenario, and a framework containing a privacy protection module is called for a medical scenario.

[0060] Analyze the execution requirements of the target governance actions, such as the required interface types, dependent pre-operations, and resource consumption budgets. Based on the pre-defined framework's process specifications and module structure, embed the target governance actions into the corresponding execution nodes of the framework. For example, embed the "sensitive data de-identification" action after the framework's "data preprocessing" module and before the "data storage" module. Adapt the interfaces between the actions and the framework to ensure that the actions can call the framework's basic functions such as resource scheduling and logging. Verify the integrity of the integrated framework and generate a directly executable integrated framework.

[0061] The system analyzes the operational logic corresponding to the target governance actions within the integrated framework, matches data processing operators with the appropriate functions within the target governance nodes (e.g., the "outlier detection" action matches a "statistical detection operator" or a "machine learning detection operator"). Based on the resource scheduling rules of the integrated framework, it allocates node resources (such as CPU, memory space, and runtime slices) to the matched operators. Operators are then started according to the execution order specified by the integrated framework, such as sequential execution (cleaning before detection) or parallel execution (simultaneous de-identification and format conversion).

[0062] This application integrates target governance actions with a pre-defined framework to ensure that action execution conforms to standardized processes and avoids chaotic operation sequences. Simultaneously, the framework-based resource scheduling mechanism avoids resource contention and reduces the average execution latency of target governance actions.

[0063] In some optional implementations of this embodiment, synchronizing the feedback information of the target governance node to the remaining governance nodes includes: The feedback information from the target governance node is encrypted to obtain encrypted feedback information; the encrypted feedback information is then synchronized to the remaining governance nodes.

[0064] In this embodiment, an appropriate encryption method is selected based on the sensitivity level of the feedback information (e.g., "extremely high", "high", "medium", "low"). Highly sensitive information (such as medical privacy data) preferentially uses asymmetric encryption (encrypted with a public key, decryptable only by the private key), while ordinary sensitive information can use symmetric encryption (encrypted and decrypted with a shared key). Simultaneously, a hash value (e.g., SHA-256) is added to the information for subsequent integrity verification.

[0065] In the target governance node, the encryption module invokes the selected encryption algorithm to encrypt the original data of the feedback information, generating ciphertext. Simultaneously, the hash value of the original information is calculated and stored in association with the ciphertext (the hash value can be transmitted in plaintext to verify whether the information has been tampered with). The ciphertext feedback information is then combined with the corresponding hash value to form the encrypted feedback information, ensuring that only the node holding the decryption key can restore and verify the information.

[0066] A communication channel between the target governance node and the remaining governance nodes is established using encrypted transmission protocols (such as Transport Layer Security, TLS / Secure Sockets Layer, SSL) to prevent the transmission link from being eavesdropped on. Through distributed synchronization mechanisms (such as Raft-based consistency synchronization or message queue-based point-to-point transmission), the encrypted feedback information (ciphertext + hash value) is sent to all remaining governance nodes, ensuring that each node receives the complete information. Upon receiving the encrypted feedback information, the remaining governance nodes first decrypt the ciphertext using the corresponding key to obtain the original feedback information, and then verify the integrity of the information by comparing the hash value (if the hash value matches, the information has not been tampered with; if it does not match, the information is rejected).

[0067] This application employs encryption to prevent unauthorized parties from deciphering the feedback information during transmission, reducing the risk of sensitive information leakage. Simultaneously, hash value verification ensures the feedback information has not been tampered with, enhancing the credibility of information received by the remaining governance nodes.

[0068] In some optional implementations of this embodiment, the above-mentioned target governance action is multiple, and the step of selecting a target governance node from the multiple governance nodes and performing the target governance action on the target governance node further includes: Select multiple target governance nodes from the multiple governance nodes, assign multiple target governance actions to the multiple target governance nodes, and control the multiple target governance nodes to execute the multiple target governance actions.

[0069] Multiple target governance actions refer to a set of governance actions generated for complex data governance scenarios that need to be completed collaboratively. Each action corresponds to a specific governance task (such as data cleaning, sensitive information desensitization, cross-node data synchronization, compliance verification, etc.). There may be dependencies between actions (such as cleaning before desensitization) or parallel relationships (such as cleaning and format conversion being performed simultaneously).

[0070] In this embodiment, the total requirements of multiple target governance actions are analyzed, including the computing power requirements, memory requirements, data processing type (structured / unstructured), and execution timeliness requirements (e.g., real-time / near real-time) for each target governance action. Based on node status information (e.g., computing power load, remaining memory, historical processing type, and response latency), candidate nodes that meet the action requirements are selected. According to the number and complexity of actions and the capabilities of candidate nodes, a sufficient number of nodes are selected (e.g., 3-5 governance nodes are selected when there are 5 actions) to ensure that the total capability of the node combination covers all action requirements and that the load is balanced among nodes; for example, avoiding situations where one node undertakes 3 high-load actions while other nodes only undertake 1 low-load action.

[0071] Clearly define whether each target governance action is executed in parallel (without dependencies) or sequentially (with sequential dependencies, such as action 1 to action 2 to action 3). Prioritize assigning high-load actions to nodes with high computing power, and assign specific types of data processing actions to nodes proficient in those types (e.g., assign unstructured data actions to nodes with GPU acceleration). For sequential actions, assign them to the same node or a group of nodes with low-latency communication; for parallel actions, assign them to different nodes to achieve parallel processing. Map each target governance action to its corresponding target governance node, generating a "node-action" allocation table to define the execution task for each node.

[0072] Action execution instructions are sent to each target governance node based on the allocation table. Serial actions are started sequentially according to their dependency order, while parallel actions are started simultaneously. Real-time data collection is performed on the action execution progress (e.g., completion percentage), resource usage changes, and intermediate results for each node, which are then aggregated and analyzed by the collaborative control module. If a node times out or fails, the collaborative control module reassigns its unfinished actions to other idle nodes. If a node is overloaded, some actions are dynamically migrated to lower-loaded nodes to ensure the overall task is completed as expected. After all actions have been executed, the execution results from each node are collected and integrated into a complete governance result.

[0073] In some embodiments, a single target governance action can be executed jointly by multiple target governance nodes. That is, an action is broken down into multiple sub-tasks, each completed by a different node, and the results of the sub-tasks are finally aggregated to form the complete action execution result. Multiple target governance actions can also form a cross-correspondence with multiple target governance nodes, where each node can execute parts of multiple actions, and each action can also be collaboratively participated in by multiple nodes, representing the most flexible collaboration mode in complex scenarios. Multiple target governance actions can also be executed by the same target governance node, where one node undertakes multiple related or independent governance actions, utilizing its own resources to complete the execution of all actions.

[0074] This application overcomes the performance bottleneck of a single node by enabling multiple nodes to collaboratively execute multiple actions, thereby improving the overall governance efficiency in complex scenarios. Simultaneously, it can flexibly handle dependencies between governance actions, adapting to diverse and complex governance scenarios.

[0075] In some optional implementations of this embodiment, the above-mentioned selection of multiple target governance nodes from multiple governance nodes and assignment of multiple target governance actions to the multiple target governance nodes includes: The system acquires performance parameters of multiple governance nodes, selects multiple target governance nodes from the multiple governance nodes based on the performance parameters, acquires the mapping relationship between the multiple target governance actions and the multiple target governance nodes, and assigns the multiple target governance actions to the multiple target governance nodes based on the mapping relationship and the performance parameters.

[0076] Performance parameters refer to a set of indicators used to quantitatively characterize the processing capabilities and operational status of governance nodes. They serve as the basis for evaluating whether a node can adapt to the requirements of governance actions and include, but are not limited to, computing power, data processing throughput, task response latency, resource utilization, historical task success rate, and supported data types. Mapping relationships refer to the pre-established correspondence between multiple target governance actions and multiple target governance nodes based on the performance parameters of multiple target governance nodes. This serves as the direct basis for action allocation.

[0077] In this embodiment, based on the requirements of governance actions, the performance parameters to be collected (such as computing power, throughput, latency, etc.) are clearly defined. The parameter types can be dynamically expanded according to the characteristics of the actions (such as adding a "GPU computing power" parameter when processing unstructured data). Using the monitoring tools built into the nodes (such as operating system performance counters, dedicated monitoring), the performance parameters of each governance node are collected at a preset frequency to ensure that the data reflects the real-time status of the nodes (such as current computing power load rather than peak computing power). The collected raw parameters are converted into a unified quantization format, for example, latency is divided into "low (latency <100ms), medium (100-500ms), high (latency >500ms)" levels to facilitate subsequent filtering and matching.

[0078] Analyze the total performance requirements of multiple target governance actions, and set minimum thresholds for each parameter, such as "all target governance actions require a total computing power ≥ 2000 GFLOPS" and "the latency requirement for a single action is ≤ 300ms". Compare the performance parameters of each governance node with the required thresholds, and eliminate nodes that do not meet the basic thresholds (e.g., nodes with computing power < 300 GFLOPS cannot handle any high-load actions), retaining candidate nodes that meet the conditions. Select multiple nodes from the candidate nodes, ensuring that the total performance of the combination covers the requirements of all actions (e.g., total throughput ≥ total data volume of actions / execution time), while also taking load balancing into account, and finally determine multiple target governance nodes.

[0079] Obtain a mapping table, which can be preset based on historical governance experience (e.g., de-identification actions are adapted to nodes with encryption chips by default), or generated dynamically through analysis, such as by calculating the matching degree between action requirements and node characteristics using algorithms to generate temporary mapping relationships. The mapping relationships must be specific to the correspondence between governance actions and governance nodes. For example, high-priority real-time actions correspond to nodes with response latency <100ms, unstructured data processing actions correspond to nodes with image / text processing libraries installed, and action groups with data dependencies correspond to nodes within the same subnet, etc. Rules must cover attributes such as action type, priority, and dependencies.

[0080] Check the mapping table for conflicts (such as assigning multiple actions to the same node that exceed its performance). If conflicts exist, adjust the node with the second highest adaptability and remap it. Distribute each target governance action to the corresponding target governance node according to the mapping table, clarifying the node's execution task and time window.

[0081] This application selects nodes and maps them to performance parameters, thus matching action requirements with node capabilities and reducing action execution timeout rates. Simultaneously, quantitative parameter evaluation avoids subjective selection bias and improves the accuracy of target governance node adaptation.

[0082] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0083] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0084] Further reference Figure 3 As a response to the above Figure 2To implement the method shown, this application provides an embodiment of a data governance device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0085] like Figure 3 As shown, the data governance device 300 described in this embodiment includes: a scene tag generation module 301, a governance action generation module 302, a single-node execution module 303, and a multi-node execution module 304. Wherein: The scene tag generation module 301 is used to obtain status data from multiple governance nodes and generate scene tags based on the status data. The governance action generation module 302 is used to acquire the data to be processed, input the scene label and the data to be processed into a preset reinforcement learning model, and generate the target governance action. The single-node execution module 303 is used to select a target governance node from multiple governance nodes, execute the target governance action on the target governance node, and collect feedback information from the target governance node; The multi-node execution module 304 is used to synchronize the feedback information of the target governance node to the remaining governance nodes, and control the remaining governance nodes to execute the target governance action according to the feedback information.

[0086] The data governance device provided in this application achieves real-time intelligent decision-making for governance strategies through scene labels generated from multi-source data and reinforcement learning models. Simultaneously, by synchronizing the execution feedback of individual nodes to the entire network, the system gains continuous optimization capabilities, solving the problem of insufficient adaptability of traditional static rule-based governance methods in the face of complex and ever-changing distributed environments, and significantly improving the accuracy of governance and the overall efficiency of the system.

[0087] In some optional implementations of this embodiment, the scene label generation module 301 is further configured to: The data status information, the business status information, and the node status information are weighted to obtain a target weight; the scene label is generated based on the target weight, the data status information, the business status information, and the node status information.

[0088] In some optional implementations of this embodiment, the governance action generation module 302 is further configured to: The scene label is fused with the data to be processed to construct a state space; based on the state space, multiple candidate governance actions and the action probability of each candidate governance action are output through the policy network of the reinforcement learning model; according to the action probability of each candidate governance action, a governance action is selected from the multiple candidate governance actions as the target governance action.

[0089] In some optional implementations of this embodiment, the single-node execution module 303 is further configured to: Obtain a preset data governance framework, integrate the target governance action and the data governance framework to generate an integrated framework; according to the integrated framework, schedule the data processing operators in the target governance node to execute the target governance action.

[0090] In some optional implementations of this embodiment, the multi-node execution module 304 is further configured to: The feedback information from the target governance node is encrypted to obtain encrypted feedback information; the encrypted feedback information is then synchronized to the remaining governance nodes.

[0091] In some optional implementations of this embodiment, the single-node execution module 303 is further configured to: Select multiple target governance nodes from the multiple governance nodes, assign multiple target governance actions to the multiple target governance nodes, and control the multiple target governance nodes to execute the multiple target governance actions.

[0092] In some optional implementations of this embodiment, the single-node execution module 303 is further configured to: The system acquires performance parameters of multiple governance nodes, selects multiple target governance nodes from the multiple governance nodes based on the performance parameters, acquires the mapping relationship between the multiple target governance actions and the multiple target governance nodes, and assigns the multiple target governance actions to the multiple target governance nodes based on the mapping relationship and the performance parameters.

[0093] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0094] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41, 42, and 43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0095] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0096] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for data governance methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0097] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the data governance method.

[0098] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0099] The computer equipment provided in this application enables real-time intelligent decision-making for governance strategies through scene labels generated from multi-source data and reinforcement learning models. Simultaneously, by synchronizing the execution feedback of individual nodes to the entire network, the system gains continuous optimization capabilities, solving the problem of insufficient adaptability of traditional static rule-based governance methods in the face of complex and ever-changing distributed environments, and significantly improving the accuracy of governance and the overall efficiency of the system.

[0100] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the data governance method described above.

[0101] The computer-readable storage medium provided in this application enables real-time intelligent decision-making for governance strategies through scene labels generated from multi-source data and reinforcement learning models. Simultaneously, by synchronizing the execution feedback of individual nodes to the entire network, the system gains continuous optimization capabilities, solving the problem of insufficient adaptability of traditional static rule-based governance methods in complex and ever-changing distributed environments, and significantly improving the accuracy of governance and the overall efficiency of the system.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0103] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A data governance method, characterized in that, Includes the following steps: Status data is obtained from multiple governance nodes, and scene tags are generated based on the status data. The governance nodes refer to various computing nodes participating in the data governance process, and the status data includes data status information, business status information, and node status information. Generating scene tags based on the status data includes: adjusting the weights of the data status information, business status information, and node status information to obtain a target weight; and generating the scene tags based on the target weight and the data status information, business status information, and node status information. The process involves acquiring data to be processed, inputting the scene label and the data to be processed into a preset reinforcement learning model, and generating a target governance action. The target governance action refers to a specific operational instruction output by the reinforcement learning model based on the scene label and the data to be processed, used to achieve a data governance objective. The step of inputting the scene label and the data to be processed into the preset reinforcement learning model to generate the target governance action includes: fusing the scene label and the data to be processed to construct a state space; based on the state space, outputting multiple candidate governance actions and the action probability of each candidate governance action through the policy network of the reinforcement learning model; and selecting one governance action from the multiple candidate governance actions as the target governance action based on the action probability of each candidate governance action. A target governance node is selected from the plurality of governance nodes, the target governance action is executed on the target governance node, and feedback information of the target governance node is collected; wherein, the feedback information refers to various types of data generated by the target governance node during and after the execution of the target governance action, which can reflect the execution effect of the governance action, changes in the node's operating status, etc. The feedback information from the target governance node is synchronized to the remaining governance nodes, and the remaining governance nodes are controlled to execute the target governance action based on the feedback information; wherein, the remaining governance nodes refer to all the other governance nodes that did not participate in the first round of governance action execution, except for the target governance node that has already executed the target governance action.

2. The data governance method according to claim 1, characterized in that, The step of executing the target governance action on the target governance node and collecting feedback information from the target governance node includes: Obtain a preset data governance framework, integrate the target governance action and the data governance framework, and generate an integrated framework; According to the integrated framework, the data processing operators in the target governance node are scheduled to execute the target governance action.

3. The data governance method according to claim 1, characterized in that, The step of synchronizing the feedback information from the target governance node to the remaining governance nodes includes: The feedback information from the target governance node is encrypted to obtain encrypted feedback information; The encrypted feedback information is synchronized to the remaining governance nodes.

4. The data governance method according to claim 1, characterized in that, The target governance action is multiple, and the step of selecting a target governance node from the multiple governance nodes and executing the target governance action on the target governance node further includes: Select multiple target governance nodes from the multiple governance nodes, and assign multiple target governance actions to the multiple target governance nodes; Control the multiple target governance nodes to execute the multiple target governance actions.

5. The data governance method according to claim 4, characterized in that, The step of selecting multiple target governance nodes from the multiple governance nodes and assigning multiple target governance actions to the multiple target governance nodes includes: Obtain the performance parameters of multiple governance nodes, and select multiple target governance nodes from the multiple governance nodes based on the performance parameters of the multiple governance nodes; Obtain the mapping relationship between the multiple target governance actions and the multiple target governance nodes; Based on the mapping relationship and the performance parameters, the multiple target governance actions are assigned to the multiple target governance nodes.

6. A data governance device, characterized in that, include: A scene tag generation module is used to obtain status data from multiple governance nodes and generate scene tags based on the status data. The governance nodes refer to various computing nodes participating in the data governance process, and the status data includes data status information, business status information, and node status information. Generating scene tags based on the status data includes: adjusting the weights of the data status information, business status information, and node status information to obtain a target weight; and generating the scene tag based on the target weight and the data status information, business status information, and node status information. A governance action generation module is used to acquire data to be processed, input the scene label and the data to be processed into a preset reinforcement learning model, and generate a target governance action. The target governance action refers to a specific operation instruction output by the reinforcement learning model based on the scene label and the data to be processed, used to achieve a data governance objective. The step of inputting the scene label and the data to be processed into the preset reinforcement learning model to generate the target governance action includes: fusing the scene label and the data to be processed to construct a state space; based on the state space, outputting multiple candidate governance actions and the action probability of each candidate governance action through the policy network of the reinforcement learning model; and selecting one governance action from the multiple candidate governance actions as the target governance action according to the action probability of each candidate governance action. A single-node execution module is used to select a target governance node from multiple governance nodes, execute the target governance action on the target governance node, and collect feedback information from the target governance node; wherein, the feedback information refers to various types of data generated by the target governance node during and after the execution of the target governance action, which can reflect the execution effect of the governance action, changes in the node's operating status, etc. A multi-node execution module is used to synchronize the feedback information of the target governance node to the remaining governance nodes, and control the remaining governance nodes to execute the target governance action according to the feedback information; wherein, the remaining governance nodes refer to all the other governance nodes that did not participate in the first round of governance action execution, except for the target governance node that has already executed the target governance action.

7. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data governance method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data governance method as described in any one of claims 1 to 5.

Images