Data management method and device, equipment and storage medium

Abstract

The invention discloses a data management method and device, equipment and a storage medium. When the method provided by the embodiment of the invention is executed, firstly, to-be-processed data and a data governance request can be obtained, and the data governance request is vectorized to obtain a semantic vector; and then querying reference governance information which is in semantic fit with the data governance request in a data governance knowledge base based on the semantic vector, taking a cue word used for constraining an output format of the large language model, the data governance request and the reference governance information as input, and performing governance planning generation by utilizing the large language model to obtain a data governance plan. And then, analyzing the data governance plan to determine a target governance execution scheme, and further performing data governance on the to-be-processed data in combination with the data governance plan and the target governance execution scheme. According to the method, the dynamic change requirement in a complex business scene can be met, and the data management efficiency and quality are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data governance method, apparatus, device, and storage medium. Background Technology

[0002] With the acceleration of enterprise digital transformation, massive amounts of multi-source, heterogeneous data are generated from internal business systems, IoT devices, and external channels. This data exhibits significant semantic differences and is often stored in various formats such as voice, text, forms, and logs, leading to challenges in data governance, including difficulties in standardization, long processing chains, and low levels of automation. Traditional data governance solutions largely rely on manual rule configuration or fixed process engines, lacking adaptability and intelligence, and failing to meet the demands of real-time data processing and efficient decision-making in dynamic and complex environments.

[0003] In recent years, large language models have been widely used in data processing and business process automation in information systems due to their powerful natural language understanding, knowledge reasoning, and context learning capabilities. However, when faced with real-time processing and intelligent decision-making of multi-source, heterogeneous data streams, large language models suffer from governance gaps caused by data format differences and semantic conflicts, making it difficult to quickly respond to dynamic data changes and output accurate decision-making basis. Therefore, the application of large language models in data governance cannot meet the dynamic needs of complex business scenarios, thus limiting the efficiency and quality of data governance. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a data governance method, apparatus, device, and storage medium that can achieve efficient governance of complex, multi-source, and heterogeneous data.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] A data governance method, the method comprising:

[0007] Obtain the data to be processed and the data governance request, and vectorize the data governance request to obtain a semantic vector;

[0008] Based on the semantic vector, query the data governance knowledge base for reference governance information that is semantically compatible with the data governance request; the data governance knowledge base includes process experience and tool capabilities required for various data governance scenarios.

[0009] The prompt words, the data governance request, and the reference governance information are used as inputs to generate a governance plan using a large language model, resulting in a data governance plan; the prompt words are used to constrain the output format of the large language model.

[0010] The data governance plan is analyzed to determine the target governance execution plan. The target governance execution plan includes multiple target governance tools, target invocation order, and target parameter configuration. The target invocation order describes the invocation order of the multiple target governance tools. The target parameter configuration describes the parameter configuration of each target governance tool.

[0011] The data to be processed is governed by combining the data governance plan and the target governance execution plan.

[0012] In one possible implementation, the data governance knowledge base includes a pipeline library and a data service library;

[0013] In one possible implementation, querying the data governance knowledge base based on the semantic vector for reference governance information semantically compatible with the data governance request includes:

[0014] Based on the semantic vectors, governance process matching and service matching are performed in the pipeline library and the data service library respectively to obtain the target governance process and the target data service. The target governance process includes the target task execution script and the execution record of the target task execution script. The target task execution script is used to describe the specific execution steps of data governance. The target data service includes the target function document and the calling method of the target function document. The target function document is used to describe the core capabilities and usage rules of the data governance tool or data governance service.

[0015] The target governance process and the target data service are combined to obtain the reference governance information.

[0016] In one possible implementation, the pipeline library includes multiple historical governance processes, each labeled with a corresponding task description; the data service library includes multiple historical data services, each labeled with a corresponding service description.

[0017] The step of performing governance process matching and service matching based on the semantic vector in the pipeline library and the data service library respectively to obtain the target governance process and the target data service includes:

[0018] Using an embedding model, the task description with the highest semantic similarity to the semantic vector is queried in the pipeline library, and the historical governance process corresponding to the task description is used as the target governance process.

[0019] Using an embedding model, the service description with the highest semantic similarity to the semantic vector is queried in the data service library, and the historical data service corresponding to the service description is taken as the target data service.

[0020] In one possible implementation, the analysis of the data governance plan to determine the target governance implementation plan includes:

[0021] The data governance plan is analyzed by an intelligent agent to determine the initial governance execution plan and the confidence level of the governance tools. The initial governance execution plan includes multiple initial governance tools, an initial invocation order, and initial parameter configurations. The confidence level of the governance tools is used to quantitatively evaluate the adaptability and reliability of the multiple initial governance tools.

[0022] If the confidence level of the governance tool is greater than or equal to the confidence threshold, then the initial governance execution plan is determined as the target governance execution plan;

[0023] If the confidence level of the governance tool is less than the confidence threshold, the agent is repeatedly triggered to perform data governance planning analysis and confidence level judgment until the confidence level of the governance tool is greater than the confidence threshold, at which point the process stops and the initial governance execution plan at this point is determined as the target governance execution plan.

[0024] In one possible implementation, the data governance of the data to be processed, combining the data governance plan and the target governance execution scheme, includes:

[0025] Transform the data governance plan into a governance script;

[0026] Using the target governance execution scheme as the core constraint, the governance script is used as the execution tool to perform data governance on the data to be processed.

[0027] In one possible implementation, the method further includes:

[0028] The monitoring process uses the target governance execution plan as the core constraint, the governance script as the execution tool, and performs data governance on the data to be processed, with the execution log including task status, execution time, and exception information.

[0029] If the exception information is empty, the governance task is confirmed to have been successfully completed.

[0030] If the exception information is not empty, the governance script is adjusted based on the exception information, and the adjusted governance script is executed again to perform data governance and subsequent steps on the data to be processed, with the target governance execution scheme as the core constraint and the governance script as the execution tool.

[0031] In one possible implementation, the method further includes:

[0032] If the governance task is confirmed to be successfully completed, the data governance request, execution script, execution log and governance result data of this governance task will be integrated into a new governance process;

[0033] The new governance process is archived into the pipeline library of the data governance knowledge base.

[0034] A data governance apparatus, the apparatus comprising:

[0035] The acquisition unit is used to acquire data to be processed and data governance requests.

[0036] The vectorization unit is used to vectorize the data governance request to obtain a semantic vector;

[0037] The governance information query unit is used to query reference governance information in the data governance knowledge base that is semantically compatible with the data governance request based on the semantic vector; the data governance knowledge base includes process experience and tool capabilities required for various data governance scenarios.

[0038] The governance plan generation unit is used to take the prompt words, the data governance request, and the reference governance information as inputs, and use a large language model to generate a governance plan to obtain a data governance plan; the prompt words are used to constrain the output format of the large language model.

[0039] The governance planning and analysis unit is used to analyze the data governance plan and determine the target governance execution plan. The target governance execution plan includes multiple target governance tools, target invocation order, and target parameter configuration. The target invocation order describes the invocation order of the multiple target governance tools. The target parameter configuration describes the parameter configuration of each target governance tool.

[0040] The data governance unit is used to perform data governance on the data to be processed by combining the data governance plan and the target governance execution scheme.

[0041] A data governance device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the data governance method described above.

[0042] A computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the data governance method described above.

[0043] Compared with the prior art, this application has the following beneficial effects:

[0044] This application provides a data governance method, apparatus, device, and storage medium. Specifically, when executing the data governance method provided in this application embodiment, the process first involves acquiring the data to be processed and the data governance request, and then vectorizing the governance request to obtain a semantic vector. This vectorization process enables the semantic information of the governance request to be captured and represented more accurately, ensuring a precise understanding of complex and heterogeneous data sources. Next, based on the semantic vector, reference governance information matching the request semantics is queried from the data governance knowledge base. The data governance knowledge base contains data governance process experience and tool capabilities support in multiple different scenarios, providing strong knowledge and technical support for data governance and ensuring the ability to flexibly respond to different business needs. Subsequently, by using prompt words, data governance requests, and reference governance information as input, a governance plan is generated using a large language model. The use of prompt words can effectively constrain the output format of the large language model, ensuring that the generated governance plan meets actual needs and standardization requirements. After further analysis of the governance plan, the optimal target governance execution scheme can be determined. The target governance execution scheme includes the target governance tool, the tool calling order, and the corresponding parameter configuration. By precisely controlling the calling order and parameter settings of the governance tools, the efficient execution of the governance process is ensured. Finally, by combining the data governance plan and the target governance execution plan, comprehensive data governance is carried out on the data to be processed. Through automation and intelligent methods, the data governance process becomes more efficient and accurate, better meeting the needs of dynamic and complex business scenarios.

[0045] This application can automatically generate suitable governance solutions based on specific data governance requests by querying reference governance information in a data governance knowledge base. This knowledge base includes process experience and tool capabilities supporting various data governance scenarios, ensuring the flexibility and adaptability of data governance and enabling it to cope with dynamic changes in different business scenarios. Simultaneously, by vectorizing data governance requests, it can effectively capture and represent semantic information within the requests, thereby improving the understanding of complex and heterogeneous data sources and avoiding governance gaps caused by data format differences or semantic conflicts. Therefore, it can perform more accurate real-time processing and decision-making when facing multi-source, heterogeneous data streams. Furthermore, through the analysis of governance plans, the target governance execution plan can be precisely determined, including tool selection, invocation order, and parameter configuration. This approach can automatically arrange the order and parameter configuration of governance tools, avoiding manual intervention and improving the efficiency and quality of data governance. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of a data governance method provided in this application embodiment;

[0048] Figure 2 A flowchart of a reference governance information query method provided in this application embodiment;

[0049] Figure 3 A flowchart illustrating another data governance method provided in this application embodiment;

[0050] Figure 4 This is a schematic diagram of the structure of a data governance device provided in an embodiment of this application. Detailed Implementation

[0051] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0052] With the acceleration of enterprise digital transformation, enterprises face massive, multi-source, and heterogeneous data. These data exhibit significant differences in format and semantics, leading to challenges in data governance such as standardization, long processing chains, and low automation. Traditional data governance solutions often rely on manual rules or fixed processes, lacking adaptability and intelligence, and struggling to meet the real-time processing and decision-making needs in dynamic and complex environments. While large language models have some applications in information system data processing and business process automation, they are clearly insufficient when enterprises frequently face multi-source heterogeneous data (such as simultaneously processing text logs, spreadsheet orders, and audio customer service records) and real-time governance needs (such as real-time cleaning of transaction data and real-time anomaly detection and decision-making). Specifically, when processing multi-source heterogeneous data, large language models are prone to governance gaps due to differences in data format or semantics (e.g., text tools cannot process spreadsheet data, requiring manual tool switching), and cannot quickly respond to data changes or adjust governance strategies in a timely manner (e.g., the inability to automatically optimize detection rules when a certain type of anomaly suddenly increases).

[0053] To address this issue, this application provides a data governance method, apparatus, device, and storage medium. First, it acquires the data to be processed and the governance request, vectorizing the request into semantic vectors to accurately capture the core requirements. Then, relying on a governance knowledge base containing multi-scenario process experience and tool capabilities, it matches appropriate reference governance information through semantic vectors, providing experiential support for solution generation. Subsequently, it uses prompts (constraining output format), the governance request, and reference information as input, and utilizes a large language model to generate a structured data governance plan. Next, it analyzes the plan to determine an execution scheme including target tools, calling order, and parameter configuration, clarifying the specific implementation path. Finally, it combines the plan and the execution scheme to complete the governance of the data to be processed. This application overcomes the limitations of traditional data governance solutions in dynamic and complex environments, meeting the data governance needs of modern enterprises in their digital transformation.

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] See Figure 1 The figure is a flowchart of a data governance method provided in an embodiment of this application, as shown below. Figure 1 As shown, the data governance method may include steps S101-S105:

[0056] S101: Obtain the data to be processed and the data governance request, and vectorize the data governance request to obtain a semantic vector.

[0057] In the initial stage of the data governance process, two core inputs need to be acquired simultaneously: first, the data to be processed (i.e., the raw data that needs to be cleaned, standardized, or optimized for quality, such as order data from an Enterprise Resource Planning (ERP) system or chat logs from a customer service system); second, the data governance request. This request must clearly contain key semantic information related to governance, typically covering the time range (limiting the time interval of the data to be governed), data type (specifying the specific form of the data to be governed), and anomaly category (defining the type of data problem to be addressed). In some scenarios, governance objectives can also be added (e.g., "improve data integrity to 98%"). To enable the model to accurately understand this natural language or semi-structured request, it needs to be transformed into a low-dimensional, dense semantic vector. This transforms the semantic information of the request into a computable and matchable standardized format, providing a unified basis for subsequent retrieval of suitable information in the knowledge base.

[0058] For example, a data governance request may specifically include:

[0059] "Process structured data from e-commerce platform order systems from March 1, 2024 to March 31, 2024, and fix issues such as 'missing delivery address' and 'abnormal payment amount (less than 0 or more than 100,000 yuan)'." Here, "March 1, 2024 to March 31, 2024" is the time range, "structured data from e-commerce platform order systems" is the data type, and "missing delivery address" and "abnormal payment amount (less than 0 or more than 100,000 yuan)" are the exception categories.

[0060] S102: Based on the semantic vector, query the data governance knowledge base for reference governance information that is semantically compatible with the data governance request.

[0061] After vectorizing the data governance request, the next core step is to leverage semantic vectors to achieve precise matching between requirements and experience. Using the vector representing the core semantics of the request as the retrieval basis, similarity calculations and information filtering are performed in the data governance knowledge base to ultimately locate reference governance information that is highly compatible with the semantics of the current governance requirement. This reference information is not generalized content, but rather practical experience that can directly support solution design. For example, for a request to "repair missing values ​​in order data," it might match the process steps from similar historical scenarios: "first fill in missing numerical fields using the mean, then complete missing textual fields using rule matching," or a "parameter configuration case of data cleaning tool A" suitable for this task.

[0062] The data governance knowledge base, serving as the foundation for retrieval, covers multiple core data governance scenarios for enterprises (such as data cleaning, anomaly detection, format standardization, and sensitive information de-identification). For each scenario, it stores two key supporting information categories: first, "process experience," which includes verified and effective operational steps, their sequence, and fault-tolerance strategies for handling similar issues in the past; and second, "tool capability support," including a list of tools adapted to the scenario (such as tools for handling text anomalies and numerical anomalies), functional descriptions of each tool, applicable data types, and historical call parameter examples. This "multi-scenario + process + tool" knowledge base architecture ensures that regardless of the data type (such as ERP order data or customer service text data) or governance objective (such as fixing anomalies or improving integrity), targeted reference information can be retrieved, laying an experiential foundation for generating feasible governance solutions.

[0063] S103: Using the prompt word, the data governance request, and the reference governance information as input, a governance plan is generated using a large language model to obtain the data governance plan.

[0064] To ensure that the governance plan generated by the large language model is both relevant to actual needs and provides clear guidance for implementation, a generation logic of "multi-dimensional input + format constraints" needs to be constructed. First, three types of core input information are integrated: the original data governance request (clearly defining "what problem needs to be solved", such as "fixing the address missing problem in March order data"), reference governance information retrieved from the knowledge base (providing experience support on "how to do it in the past", such as the selection of tools and steps for fixing similar missing values), and prompt words for standardizing the output. Then, these three types of information are input into the large language model together, allowing the model to generate a governance plan based on understanding the needs, drawing on experience, and combining its own semantic reasoning capabilities.

[0065] The key role of prompts is to constrain the output format of the large language model, avoiding the generation of vague textual descriptions and ensuring that the plan has structured and decomposable characteristics. For example, prompts can be set as "Please output in the form of a list of 'task steps - corresponding tools - core constraints'. Each task step must clearly specify the source of input data and the processing goal. Tools must be labeled with the data types they are compatible with. Constraints must specify the parameter range or execution priority." Through such constraints, the data governance plan finally output by the large language model will present a clear logical framework (such as "Step 1: Filter March order data → Tool: Data filtering tool → Constraint: Only keep records with 'empty address field'; Step 2: Complete missing addresses → Tool: Address matching tool → Constraint: Call the enterprise address database interface, and the matching success rate must be ≥90%)", providing a direct basis for subsequent transformation into specific execution plans.

[0066] S104: Analyze the data governance plan and determine the target governance implementation plan.

[0067] After obtaining a structured data governance plan, it needs to be further broken down into actionable solutions. Through detailed analysis of the governance plan, the core execution elements required to complete the governance tasks are identified, and the target governance execution plan is finally determined. This analysis process is not simply extracting information, but rather a comprehensive integration of task logic and tool characteristics: for example, the step in the plan of "first filtering abnormal order data, then completing missing fields" will be transformed into clear tool selection (filtering tool, field completion tool), the order in which the tools are called (run the filtering tool first, and its output serves as the input for the completion tool), and the specific parameters of each tool (the filtering tool needs to set an "abnormal order judgment threshold," and the completion tool needs to be configured with "field matching rules").

[0068] The finalized data governance implementation plan includes three key elements: First, "multiple target governance tools," which is a list of tools corresponding to each governance step and validated to suit the current data type and task requirements (such as Natural Language Processing (NLP) cleaning tools for handling text anomalies and statistical completion tools for handling missing values); second, "target invocation order," clarifying the logical sequence between tools (such as sequential execution to avoid data dependency conflicts or parallel execution to improve efficiency), ensuring smooth process flow; and third, "target parameter configuration," setting specific operating parameters for each tool (such as the "sensitive field list" for de-identification tools and the "anomaly scoring threshold" for detection tools), preventing tool failures or result deviations due to ambiguous parameters. By clarifying these three elements, the abstract governance plan can be transformed into a clear and directly executable operational guide, providing a precise basis for subsequent data governance implementation.

[0069] S105: Perform data governance on the data to be processed by combining the data governance plan and the target governance execution plan.

[0070] When advancing the governance of pending data, it is necessary to use data governance planning as the "direction guide" and the target governance execution plan as the "operational details." The two work together to form a complete governance logic: the data governance planning clarifies the core objectives and overall framework of governance (such as "fixing the missing addresses in March order data + improving data integrity to 98%)", defining the core direction of the governance process and avoiding deviation from the requirements; while the target governance execution plan provides the specific path to achieve the objective, including the tools to be called (such as address matching tools and data filtering tools), the order in which the tools are called (first filter out order data with missing addresses, and then use the matching tool to complete the addresses), and the parameter configuration of each tool (such as the address matching tool calling the enterprise customer database interface and setting the matching timeout to 30 seconds).

[0071] In the actual governance process, the system first automatically connects the data to be processed to the corresponding tools for step-by-step processing based on the tool call logic and parameters in the target governance execution plan (e.g., first extracting a subset of orders with empty addresses using a filtering tool, and then passing this subset to an address matching tool to complete the completion). Simultaneously, the entire process uses the goals in the data governance plan as the verification standard. For example, after each tool processing step is completed, the system checks whether the current governance result meets the "data integrity improvement" requirement in the plan (e.g., whether the non-empty rate of the address field gradually approaches 98% after completion). If tool call anomalies or results fail to meet the standards, the system will correct the situation by combining the planning goals and the fault tolerance rules in the execution plan (e.g., retrying tool calls and adjusting parameters) to ensure that the governance of the data to be processed not only meets the initial requirements but also achieves efficient implementation.

[0072] Based on the content of S101-S105, the process first involves synchronously acquiring the data to be processed and the data governance request, and then converting the request into a semantic vector through vectorization, laying the foundation for subsequent accurate matching. Secondly, relying on a governance knowledge base encompassing multi-scenario process experience and tool capabilities, semantic vectors are used to query reference information that matches the semantics of the request, ensuring that the generated solution is supported by experience. Next, prompt words (used to standardize the output format), the original governance request, and the reference information are used as input to generate a guiding governance plan using a large language model. Then, the governance plan is analyzed in depth to clarify the execution plan, including the target tools, the calling order, and parameter configuration, giving the governance actions a clear path. Finally, the governance plan and execution plan are combined to complete the governance of the data to be processed. This application, relying on a governance knowledge base containing multi-scenario process experience and tool capabilities, combined with the accurate capture of semantic information through request vectorization, can efficiently understand complex heterogeneous data sources, avoid governance gaps caused by semantic conflicts, and automatically generate governance solutions adapted to specific requests, flexibly responding to different business dynamics. Meanwhile, by analyzing the governance plan, the selection of tools, the order of invocation, and the parameter configuration are clarified, reducing manual intervention throughout the process. This not only improves the real-time processing and decision-making accuracy of multi-source heterogeneous data, but also further enhances the efficiency and quality of data governance.

[0073] In one possible implementation, the data governance knowledge base uses a classification architecture of "pipeline library" and "data service library" to achieve structured storage of governance experience and tool resources. The two have a clear division of labor and work together to support the precise matching of data governance needs.

[0074] Specifically, the pipeline library serves as a storage carrier for governance process experience, including multiple historical governance processes that have been validated in practice (such as the "e-commerce platform order data missing value repair process" and the "financial system transaction log anomaly detection process"). To facilitate accurate matching with subsequent data governance requests, each historical governance process is clearly labeled with a corresponding task description. This description must clearly summarize the core governance objectives, applicable data scenarios, and key processing directions of the process. For example, the task description of a certain historical governance process could be set as "For structured order data from the retail system from January to March 2024, repair the missing values ​​of the 'delivery address' and 'contact phone number' fields, improving data integrity to over 95%." Through this concrete labeling, the application scenarios and governance objectives of historical processes are immediately clear, providing a clear comparison basis for semantic vector matching.

[0075] The data service library, serving as an integrated carrier for governance tools and service resources, includes several directly accessible historical data services (such as "address field completion service," "text semantic segmentation service," and "numerical outlier detection service"). To support accurate matching, each historical data service is labeled with a corresponding service description. This description must detail the service's core functions; for example, the service description for a historical data service could be set as "address field completion service." By clearly defining service capabilities and applicable conditions, it ensures that subsequent tool resources matching the governance request requirements can be quickly located, avoiding matching errors caused by ambiguous service information.

[0076] This design, which combines "process + service" databases in parallel and includes descriptive annotations, not only enables the categorized management of governance experience and tool resources but also provides a clear comparison dimension for subsequent semantic vector-based query matching, ensuring efficient retrieval of reference information that is semantically compatible with the current governance request.

[0077] In one possible implementation, such as Figure 2 As shown, step S102 queries the data governance knowledge base based on the semantic vector for reference governance information that is semantically compatible with the data governance request. This can be specifically implemented through steps S201-S202:

[0078] S201: Based on the semantic vector, perform governance process matching and service matching in the pipeline library and the data service library respectively to obtain the target governance process and the target data service.

[0079] When querying reference governance information, the semantic vector of the data governance request should be used as the core retrieval basis. Targeted matching should be performed on the pipeline library and data service library in the data governance knowledge base to obtain two types of key reference information:

[0080] On one hand, governance processes are matched within a pipeline database storing historical governance processes. By calculating the similarity between semantic vectors and the "task descriptions" of each historical governance process in the database (e.g., if the request semantic is "repair missing addresses in March orders," then historical processes with descriptions containing "order data" and "address missing repair") are matched to select target governance processes that fit the current needs. This target governance process not only includes the target task execution script (clearly defining the specific data governance operations in a step-by-step manner, such as "1. Filter records with empty addresses in March orders; 2. Call the address matching tool to complete the fields; 3. Verify the completeness of the completion results"), but also includes historical execution records of the script (such as past success rates, average processing time, and any anomalies encountered and their solutions), providing empirical support for the feasibility of the process.

[0081] On the other hand, service matching is performed in the data service library storing historical data services. This matching is also based on the similarity between semantic vectors and the "service descriptions" of each historical data service (e.g., if a request requires "address completion," then services with descriptions containing "address completion" or "supports order data" are matched). The target data service is then determined. This target data service includes target function documentation (detailing the core capabilities of the corresponding governance tool or service, such as "address completion based on recipient's phone number and city information, supporting three-level matching across provinces, cities, and districts nationwide") and corresponding calling methods (e.g., parameter formats, interface addresses, and timeout rules required for the call), clearly defining the usage logic and implementation path of the tool / service.

[0082] Through these two types of matching, the process framework (i.e., target governance process) and tool resources (i.e. target data service) supporting the design of governance solutions can be obtained simultaneously, laying the foundation for subsequent integration of reference governance information and generation of feasible governance plans.

[0083] S202: Combine the target governance process and the target data service to obtain the reference governance information.

[0084] After obtaining the target governance process and target data services, it is necessary to deeply integrate and correlate these two types of information, focusing on the core principle of "ensuring that the governance solution has both process logic and feasibility of implementation," to ultimately form complete reference governance information. This integration process will use the "task execution script" in the target governance process as the core framework, precisely embedding the tool capabilities and usage rules in the target data services into the corresponding process steps: for example, the step of "filtering order data with missing addresses" in the task execution script will correspond to the functional documentation of the "data filtering tool" in the associated data service library (explaining that the tool supports filtering by field null values ​​and time range) and calling methods (such as the table name, filtering fields, and time parameter format to be passed in); while the step of "completing missing addresses" will be associated with the core capability description of the "address matching tool" (such as supporting completion of city information associated with mobile phone numbers), while supplementing the historical execution records of this tool in the target governance process (such as past completion success rates, common anomalies, and solutions).

[0085] By integrating process steps with tool details in a one-to-one correspondence, the final reference governance information is no longer a simple superposition of two types of data, but forms a complete logical chain of "what to do first (process steps) → what to do (adaptive tools) → how to do it (calling methods) → how well it is done (historical execution records)". This clarifies the overall governance framework and provides specific tool support and experience reference for each step, ensuring that when the large language model generates governance plans, it can generate solutions that are both logically consistent with the process and directly implementable based on this information, avoiding the disconnect of "reasonable planning but lack of tool support".

[0086] In one possible implementation, the step of performing governance process matching and service matching based on the semantic vector in the pipeline library and the data service library respectively to obtain the target governance process and the target data service includes:

[0087] Using an embedding model (e.g., text-embedding-ada-002), the task description with the highest semantic similarity to the semantic vector is queried in the pipeline library, and the historical governance process corresponding to this task description is used as the target governance process. Similarly, using an embedding model, the service description with the highest semantic similarity to the semantic vector is queried in the data service library, and the historical data service corresponding to this service description is used as the target data service.

[0088] Specifically, the matching process between the pipeline library and the data service library is based on the core logic of "semantic similarity first," and accurate retrieval is achieved with the help of an embedding model. The specific operation is as follows:

[0089] For matching governance processes in the pipeline library: First, clarify the core role of the embedding model (e.g., text-embedding-ada-002), which can transform text information into vectors with computable semantic similarity. Since each historical governance process in the pipeline library is labeled with a task description (e.g., "fix missing e-commerce order address data"), the system will first use the embedding model to calculate the similarity between the "semantic vector of the data governance request" and the "vectors of all historical governance process task descriptions in the pipeline library"; then, select the task description with the highest similarity and directly determine its corresponding historical governance process as the target governance process, ensuring that the process is highly consistent with the current request in terms of governance goals and applicable scenarios.

[0090] For service matching in the data service library: the logic is consistent with that of the pipeline library, and the retrieval is also based on the embedding model. Since each historical data service in the data service library is labeled with a service description (such as "supports Chinese address field completion based on mobile phone number, adapting to structured order data"), the system will use the embedding model to calculate the similarity between the "semantic vector of the data governance request" and the "vectors of all historical data service descriptions", and select the service description with the highest similarity. The corresponding historical data service is the target data service, ensuring that the core capabilities of the service accurately match the tool requirements of the current governance request.

[0091] This matching method, which relies on an embedding model and uses semantic similarity as the filtering standard, can effectively avoid the problem of missing semantic associations in keyword matching. It ensures that the target governance process and target data service retrieved from the two types of sub-databases are most suitable for the current request at the semantic level, providing a reliable foundation for the subsequent integration of reference governance information and the generation of governance plans.

[0092] In one possible implementation, the analysis of the data governance plan to determine the target governance implementation plan includes:

[0093] The data governance plan is analyzed by an intelligent agent to determine the initial governance execution plan and the confidence level of the governance tools. The initial governance execution plan includes multiple initial governance tools, an initial invocation order, and initial parameter configurations. The confidence level of the governance tools is used to quantitatively evaluate the adaptability and reliability of the multiple initial governance tools. If the confidence level of the governance tools is greater than or equal to a confidence threshold, the initial governance execution plan is determined as the target governance execution plan. If the confidence level of the governance tools is less than the confidence threshold, the steps of data governance plan analysis and confidence level judgment by the intelligent agent are repeatedly triggered until the confidence level of the governance tools is greater than the confidence threshold, at which point the initial governance execution plan at this point is determined as the target governance execution plan.

[0094] Specifically, in determining the target governance execution plan, the generated data governance plan is first deeply decomposed and analyzed using an intelligent agent. The agent combines the governance goals, task steps, and tool capability information in the data service repository to initially output the initial governance execution plan and the confidence level of the governance tools. The initial governance execution plan clarifies the core elements required to complete the task, including multiple initial governance tools adapted to each step (such as data filtering tools and address matching tools), the initial calling order between tools (filtering first, then completing, avoiding data redundancy), and the initial parameter configuration of each tool. The confidence level of the governance tools is assessed through quantitative indicators (such as the compatibility rate between the tool and the current data type, and the historical call success rate) to evaluate the adaptability and reliability of these initial tools in the current scenario. For example, if an initial tool has a 95% success rate in processing similar order data in the past, its corresponding confidence level will be high.

[0095] Next, the confidence level verification process begins: a confidence threshold is preset (e.g., 0.5, which can be adjusted according to the business requirements for governance accuracy). If the confidence level of the governance tools output by the agent is greater than or equal to the threshold, it indicates that the compatibility and reliability of the tool set in the initial solution have met the implementation requirements, and it is directly determined as the target governance execution solution. If the confidence level is less than the threshold, it indicates that some initial tools may have compatibility deviations (e.g., the tool does not support the current data format) or insufficient reliability (e.g., the historical success rate is too low). At this time, the agent will re-trigger the execution analysis process. The agent will re-select tools, adjust the calling order, or optimize the parameter configuration based on the reason for the previous low confidence level (e.g., insufficient compatibility of a certain tool), generate a new initial governance execution solution and corresponding confidence level, and perform threshold verification again.

[0096] This cycle continues until the confidence level of the governance tool exceeds a threshold, at which point the initial governance execution plan is determined as the target governance execution plan. Through dynamic optimization of the agent and strict control of the confidence level, it is ensured that the final output execution plan not only meets the requirements of the governance plan but also has high reliability and feasibility for implementation, avoiding interruptions in the governance process or deviations in results due to tool compatibility issues.

[0097] In one possible implementation, such as Figure 3 As shown, step S105 combines the data governance plan and the target governance execution scheme to perform data governance on the data to be processed, which can be specifically implemented through steps S301-S302:

[0098] S301: Transform the data governance plan into a governance script.

[0099] To transform data governance planning from a guiding framework into operational instructions that can be automatically executed by the execution engine, a structured transformation from governance planning to governance scripts is required. This process is not a simple text transcoding, but rather involves breaking down each governance requirement in the plan into specific, implementable, and coded steps, incorporating technical implementation logic.

[0100] For example, if the governance plan specifies "Step 1: Filter records in the e-commerce order table where the 'shipping address' field is empty between March 1st and March 31st, 2024; Step 2: Call the address matching tool and pass in the 'recipient's phone number' and 'city' fields to complete the address; Step 3: Verify the non-empty rate of the completed address field, which must reach over 95%", the conversion process will transform these steps one by one into instructions that the execution engine can recognize: Step 1 corresponds to the data query script (specifying the data source table address, the time conditions for filtering, and the field null value conditions); Step 2 corresponds to the tool call script (including the interface address of the address matching tool, the names of the parameters to be passed in, and the data mapping relationship); Step 3 corresponds to the result verification script (setting the non-empty rate calculation logic and the threshold for meeting the standard, and triggering a retry mechanism if the standard is not met).

[0101] The entire transformation process must ensure that the script is completely consistent with the core objectives and steps of the plan, while also taking into account technical feasibility. For example, the script syntax must be compatible with the execution environment (such as SQL or Python), the tool call instructions must conform to the API specifications of the corresponding tools, and the parameter passing must match the data field format. The final governance script can not only accurately reproduce the planning requirements, but also run automatically in the governance system, providing an executable carrier for subsequent governance operations in conjunction with the implementation plan.

[0102] S302: Using the target governance execution scheme as the core constraint, the governance script is used as the execution tool to perform data governance on the data to be processed.

[0103] When conducting actual governance of data, it is necessary to build a collaborative mechanism of "constraints defining rules and scripts executing" to ensure that the governance process not only meets the implementation requirements but also accurately achieves the planned goals. On the one hand, the target governance execution plan is used as the core constraint, defining key rules that cannot be deviated from for the governance operation. For example, the plan specifies requirements such as "the XX address matching tool needs to be called," "the tool calling order is 'filtering → completion → verification'," and "the completion tool needs to be passed the parameter 'recipient's phone number + city'." These requirements will become the hard guidelines for script execution, avoiding problems such as misuse of tools, reversed processes, or parameter configuration deviations. If the script execution encounters a situation that does not conform to the constraints of the plan (such as mistakenly calling other tools), the system will automatically intercept and prompt for adjustment, ensuring that the governance path always conforms to the feasible plan.

[0104] On the other hand, the transformed governance scripts are used as direct execution tools, relying on their coded instructions to automate data processing: the scripts automatically read the data to be processed (such as March e-commerce order data) according to the steps of "filtering abnormal data → calling tools to complete the data → verifying the governance results," and call the corresponding services according to the tools and parameters constrained by the execution plan to complete operations such as address completion, and record the execution status of each step in real time. For example, in the address completion step, the script will strictly follow the constraints of "using XX tool" and "parameter passing rules" in the execution plan to pass the specified field data to the tool interface to ensure that the completion logic is consistent with the plan requirements; after the governance is completed, it will also check whether the result meets the target of "data integrity must reach 95%" in the execution plan through the built-in verification logic of the script. If it does not meet the target, it will trigger a retry according to the fault tolerance rules in the plan. Finally, under the dual guarantee of "constraints controlling the direction and scripts implementing the work," the precise governance of the data to be processed is completed.

[0105] In one possible implementation, the method further includes:

[0106] The process involves monitoring the data governance of the data to be processed, using the governance script as an execution tool, with the target governance execution scheme as the core constraint, and recording the execution log. The execution log includes task status, execution time, and exception information. If the exception information is empty, the governance task is confirmed to have been successfully completed. If the exception information is not empty, the governance script is adjusted based on the exception information, and the process of using the governance script as an execution tool to perform data governance of the data to be processed and subsequent steps is executed again based on the adjusted governance script.

[0107] Specifically, to ensure the stability of the data governance process and the effectiveness of the final results, this application also adds a closed-loop control mechanism of "process monitoring - anomaly handling - script optimization", the specific operation logic of which is as follows:

[0108] First, throughout the entire process of "using the target governance execution plan as the core constraint and the governance script as the execution tool to perform data governance on the data to be processed," the system will monitor and record execution logs in real time. These logs must fully cover key information about the governance process, including task status (e.g., "script initialization in progress," "tool in progress," "governance completed"), execution time (e.g., "filtering step took 2 minutes," "completement step took 5 minutes"), and core exception information (e.g., "tool call timed out," "parameter format error," "data read failure," etc.; if there are no exceptions, the logs will be empty). The execution logs allow for real-time monitoring of the governance progress and provide a basis for subsequent anomaly investigation or process optimization.

[0109] Next, determine the status of the governance task based on the exception information in the execution log: if the exception information is empty, it means that there was no deviation in the entire governance process (such as normal tool call, data processing as expected, and results meeting the standards), and the governance task can be directly confirmed as successfully completed, and the final governed dataset can be output; if the exception information is not empty (such as the log showing "address matching tool call timed out"), then it is necessary to locate the root cause of the problem based on the exception information. For example, "call timeout" may be due to congestion of the tool interface or redundant parameter passing, and "parameter format error" may be due to the field type in the script not matching the tool requirements.

[0110] After identifying the problem, make targeted adjustments to the governance script to address the root cause: if timeouts are caused by interface congestion, add a "timeout retry mechanism" to the script and set a reasonable retry interval; if the problem is caused by incorrect parameter format, correct the format conversion logic of the fields in the script (e.g., convert text-based phone numbers to numeric types). After the adjustments are completed, the steps of "using the target governance execution plan as the core constraint, using the governance script as the execution tool, and performing data governance on the data to be processed" must be executed again, and the execution logs must be monitored again until the exception information is empty and the governance task is successfully completed.

[0111] This mechanism effectively avoids the problem of "termination upon failure of a single execution" through real-time monitoring and dynamic optimization, ensuring that even if temporary anomalies occur, the governance process can be quickly restored through script adjustments, thus guaranteeing the final implementation effect of data governance tasks.

[0112] In one possible implementation, the method further includes:

[0113] If the governance task is confirmed to be successfully completed, the data governance request, execution script, execution log, and governance result data of this governance task will be integrated into a new governance process. This new governance process will then be archived in the pipeline library of the data governance knowledge base.

[0114] Specifically, to continuously enrich the process experience reserves of the data governance knowledge base and improve the planning efficiency of subsequent governance tasks, this application also designed a "governance process accumulation" mechanism to form a cycle of "practice-summarization-reuse," the specific operation of which is as follows:

[0115] Once the execution log confirms that the exception information is empty and the governance task has been successfully completed, the system will initiate process integration. This process will structure and integrate the key information from the entire governance task chain to form a reusable new governance process. The core information integrated covers four categories: data governance requests, execution scripts, execution logs, and governance result data.

[0116] After integration, this new governance process, encompassing "requirements-scripts-processes-results," will be archived in the pipeline library of the data governance knowledge base according to the pipeline library's storage specifications, and a corresponding task description will be added. Subsequently, when similar data governance requests arise, the system can quickly retrieve the newly archived governance process through semantic vector matching, directly using it as a reference to generate a governance plan. This eliminates the need for redesigning the process, significantly improving the response efficiency and reliability of subsequent governance tasks, and enabling the continuous accumulation and reuse of governance experience.

[0117] See Figure 4 , Figure 4 This is a schematic diagram of a data governance device provided in an embodiment of this application. Figure 4 As shown, the data governance device includes:

[0118] Acquisition unit 401 is used to acquire data to be processed and data governance requests;

[0119] Vectorization unit 402 is used to vectorize the data governance request to obtain a semantic vector;

[0120] The governance information query unit 403 is used to query reference governance information in the data governance knowledge base that is semantically compatible with the data governance request based on the semantic vector; the data governance knowledge base includes process experience and tool capabilities required for various data governance scenarios.

[0121] The governance plan generation unit 404 is used to take the prompt words, the data governance request, and the reference governance information as inputs, and use a large language model to generate a governance plan to obtain a data governance plan; the prompt words are used to constrain the output format of the large language model.

[0122] The governance planning and analysis unit 405 is used to analyze the data governance plan and determine the target governance execution plan. The target governance execution plan includes multiple target governance tools, target invocation order, and target parameter configuration. The target invocation order describes the invocation order of the multiple target governance tools. The target parameter configuration describes the parameter configuration of each target governance tool.

[0123] The data governance unit 406 is used to perform data governance on the data to be processed by combining the data governance plan and the target governance execution scheme.

[0124] In one possible implementation, the data governance knowledge base includes a pipeline library and a data service library.

[0125] In one possible implementation, the governance information query unit 403 is specifically used for:

[0126] Based on the semantic vectors, governance process matching and service matching are performed in the pipeline library and the data service library respectively to obtain the target governance process and the target data service. The target governance process includes the target task execution script and the execution record of the target task execution script. The target task execution script is used to describe the specific execution steps of data governance. The target data service includes the target function document and the calling method of the target function document. The target function document is used to describe the core capabilities and usage rules of the data governance tool or data governance service.

[0127] The target governance process and the target data service are combined to obtain the reference governance information.

[0128] In one possible implementation, the pipeline library includes multiple historical governance processes, each labeled with a corresponding task description; the data service library includes multiple historical data services, each labeled with a corresponding service description.

[0129] In one possible implementation, the step of performing governance process matching and service matching based on the semantic vector in the pipeline library and the data service library respectively to obtain the target governance process and the target data service includes:

[0130] Using an embedding model, the task description with the highest semantic similarity to the semantic vector is queried in the pipeline library, and the historical governance process corresponding to the task description is used as the target governance process.

[0131] Using an embedding model, the service description with the highest semantic similarity to the semantic vector is queried in the data service library, and the historical data service corresponding to the service description is taken as the target data service.

[0132] In one possible implementation, the governance planning analysis unit 405 is specifically used for:

[0133] The data governance plan is analyzed by an intelligent agent to determine the initial governance execution plan and the confidence level of the governance tools. The initial governance execution plan includes multiple initial governance tools, an initial invocation order, and initial parameter configurations. The confidence level of the governance tools is used to quantitatively evaluate the adaptability and reliability of the multiple initial governance tools.

[0134] If the confidence level of the governance tool is greater than or equal to the confidence threshold, then the initial governance execution plan is determined as the target governance execution plan;

[0135] If the confidence level of the governance tool is less than the confidence threshold, the agent is repeatedly triggered to perform data governance planning analysis and confidence level judgment until the confidence level of the governance tool is greater than the confidence threshold, at which point the process stops and the initial governance execution plan at this point is determined as the target governance execution plan.

[0136] In one possible implementation, the data governance unit 406 is specifically used for:

[0137] Transform the data governance plan into a governance script;

[0138] Using the target governance execution scheme as the core constraint, the governance script is used as the execution tool to perform data governance on the data to be processed.

[0139] In one possible implementation, the device further includes:

[0140] The monitoring unit is used to monitor the execution log of the process of performing data governance on the data to be processed, with the target governance execution scheme as the core constraint and the governance script as the execution tool; the execution log includes task status, execution time and exception information;

[0141] The confirmation completion unit, if the exception information is empty, is used to confirm that the governance task has been successfully completed;

[0142] The integration unit, if the exception information is not empty, is used to adjust the governance script based on the exception information, and then execute the governance script again based on the adjusted governance script. The governance script is used as an execution tool with the target governance execution scheme as the core constraint to perform data governance and subsequent steps on the data to be processed.

[0143] In one possible implementation, the device further includes:

[0144] The integration unit, if it is confirmed that the governance task has been successfully completed, is used to integrate the data governance request, execution script, execution log and governance result data of this governance task into a new governance process;

[0145] The archiving unit is used to archive the new governance process into the pipeline library in the data governance knowledge base.

[0146] In addition, this application embodiment also provides a data governance device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the data governance method as described above.

[0147] In addition, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the data governance method described above.

[0148] This application, through automated querying of reference governance information in a data governance knowledge base, can generate the most suitable solution for specific data governance requests. The data governance knowledge base contains rich scenario experience and tool support, covering various data governance processes, thus ensuring the method's high flexibility and adaptability in the face of different business needs. This enables rapid adjustment and response to constantly changing business scenarios. Simultaneously, vectorizing data governance requests accurately captures and expresses semantic information within them. This not only enhances the understanding of complex and heterogeneous data sources but also effectively avoids governance defects caused by data format differences or semantic conflicts. This, in turn, enables more accurate data analysis and decision support in the real-time processing and decision-making of multi-source, heterogeneous data streams. Furthermore, through in-depth analysis of the data governance plan, the target governance execution scheme can be clearly defined, including tool selection, invocation order, and parameter configuration. This process is fully automated, avoiding the possibility of manual intervention and ensuring the accuracy of the invocation order and parameter configuration of governance tools, thereby improving the efficiency and quality of the data governance process and optimizing the overall governance effect.

[0149] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0150] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data governance method, characterized by, The method comprises: acquiring to-be-processed data and a data governance request, and vectorizing the data governance request to obtain a semantic vector; querying reference governance information that is semantically adapted to the data governance request in a data governance knowledge base based on the semantic vector; the data governance knowledge base comprises process experience and tool capability support required by multiple data governance scenarios; using a large language model to generate a governance plan by taking a prompt word, the data governance request, and the reference governance information as input, to obtain a data governance plan; the prompt word is used to constrain the output format of the large language model; analyzing the data governance plan to determine a target governance execution scheme; the target governance execution scheme comprises multiple target governance tools, a target calling sequence, and target parameter configurations; the target calling sequence is used to describe the calling sequence of the multiple target governance tools; the target parameter configurations are used to describe the parameter configurations of the respective target governance tools; performing data governance on the to-be-processed data in combination with the data governance plan and the target governance execution scheme.

2. The method of claim 1, wherein, The data governance knowledge base comprises a pipeline library and a data service library; The querying of the reference governance information that is semantically adapted to the data governance request in the data governance knowledge base based on the semantic vector comprises: performing governance process matching and service matching in the pipeline library and the data service library based on the semantic vector, respectively, to obtain a target governance process and a target data service; the target governance process comprises a target task execution script and an execution record of the target task execution script, and the target task execution script is used to describe specific execution steps of data governance; the target data service comprises a target function document and a calling method of the target function document, and the target function document is used to describe the core capability and usage rules of a data governance tool or a data governance service; performing data combination on the target governance process and the target data service to obtain the reference governance information.

3. The method of claim 2, wherein, The pipeline library comprises multiple historical governance processes, and each historical governance process is labeled with a corresponding task description; the data service library comprises multiple historical data services, and each historical data service is labeled with a corresponding service description; The performing of governance process matching and service matching in the pipeline library and the data service library based on the semantic vector, respectively, to obtain a target governance process and a target data service comprises: querying, by using an embedding model, a task description with the highest semantic similarity to the semantic vector in the pipeline library, and taking a historical governance process corresponding to the task description as the target governance process; querying, by using an embedding model, a service description with the highest semantic similarity to the semantic vector in the data service library, and taking a historical data service corresponding to the service description as the target data service.

4. The method of claim 1, wherein, The analyzing of the data governance plan to determine a target governance execution scheme comprises: The agent analyzes the data governance plan to determine an initial governance execution scheme and a governance tool confidence level; the initial governance execution scheme includes multiple initial governance tools, an initial calling sequence, and initial parameter configurations, and the governance tool confidence level is used to quantitatively evaluate the adaptability and reliability of the multiple initial governance tools; If the governance tool confidence level is greater than or equal to a confidence threshold, the initial governance execution scheme is determined as the target governance execution scheme; If the governance tool confidence level is less than the confidence threshold, the steps of triggering the agent to perform data governance plan analysis and confidence level determination are repeated until the governance tool confidence level is greater than the confidence threshold, and the initial governance execution scheme at this time is determined as the target governance execution scheme.

5. The method of claim 1, wherein, The data governance of the to-be-processed data in combination with the data governance plan and the target governance execution scheme includes: Converting the data governance plan into a governance script; Taking the target governance execution scheme as a core constraint and taking the governance script as an execution tool to perform data governance on the to-be-processed data.

6. The method of claim 5, wherein, The method further includes: Monitoring an execution log of the process of taking the target governance execution scheme as a core constraint and taking the governance script as an execution tool to perform data governance on the to-be-processed data; the execution log includes task status, execution time consumption, and exception information; If the exception information is empty, it is confirmed that the governance task is successfully completed; If the exception information is not empty, the governance script is adjusted based on the exception information, and the process of taking the target governance execution scheme as a core constraint and taking the governance script as an execution tool to perform data governance on the to-be-processed data and subsequent steps is performed again based on the adjusted governance script.

7. The method of claim 6, wherein, The method further includes: If it is confirmed that the governance task is successfully completed, the data governance request, the execution script, the execution log, and the governance result data of the current governance task are integrated into a new governance process; The new governance process is archived to a pipeline library in the data governance knowledge base.

8. A data governance apparatus, characterized by, The apparatus includes: An acquisition unit configured to acquire to-be-processed data and a data governance request; A vectorization unit configured to vectorize the data governance request to obtain a semantic vector; A governance information query unit configured to query reference governance information that is semantically adapted to the data governance request from a data governance knowledge base based on the semantic vector; the data governance knowledge base includes process experience and tool capability support required by multiple data governance scenarios; A governance plan generation unit configured to use a large language model to generate a data governance plan by taking a prompt word, the data governance request, and the reference governance information as input; the prompt word is used to constrain the output format of the large language model. The governance planning analysis unit is configured to analyze the data governance planning and determine a target governance execution scheme, wherein the target governance execution scheme comprises a plurality of target governance tools, a target calling sequence and a target parameter configuration, the target calling sequence is used to describe a calling sequence of the plurality of target governance tools, and the target parameter configuration is used to describe a parameter configuration of each target governance tool. The data governance unit is configured to perform data governance on the to-be-processed data in combination with the data governance planning and the target governance execution scheme.

9. A data governance device, characterized by, The data governance method comprises the following steps: The memory, the processor and the computer program stored in the memory and executable on the processor are provided, and the processor implements the data governance method according to any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, and when the instructions run on the terminal device, the terminal device executes the data governance method according to any one of claims 1-7.

Images