Method, system, electronic device and computer readable storage medium for historical data warehouse task arrangement and optimization

Abstract

This invention discloses a method, system, electronic device, and storage medium for organizing and optimizing historical data warehouse tasks. The method performs a global scan of historical tasks, collecting task scripts, scheduling configurations, execution logs, resource consumption records, and metadata related to data lineage, constructing a directed graph of data lineage and task features. Tasks are classified through unsupervised learning, constructing a health profile including activity, complexity, redundancy, dependency health, and resource efficiency. Based on the profile and lineage graph, an optimization plan is generated, including optimization operations, expected benefits, risk levels, impact scope, and explanations of reasons. Optimization is implemented after review and confirmation, and layer-by-layer data consistency checks are performed on the output before and after optimization, with automatic rollback in case of failure. The system also writes optimization cases into a Skill knowledge base and updates the recommended weights of optimization strategies through unsupervised learning. This invention improves the efficiency of historical task organization, the interpretability of optimization, and the security of implementation.

Description

Technical Field

[0001] This invention relates to the fields of data warehouse management, data governance and artificial intelligence technology, and in particular to a method, system and electronic device for organizing and optimizing historical data warehouse tasks based on artificial intelligence. Background Technology

[0002] Data warehouses are used to extract, clean, transform, load, and statistically analyze massive amounts of business data. With the long-term iteration of enterprise information systems, a large number of historical data warehouse tasks typically accumulate in data warehouse clusters. The task scripts, scheduling configurations, execution logs, resource consumption information, and table-level or field-level lineage information for these tasks are often stored in a scattered manner; some tasks may also have missing documentation, changes in original maintenance personnel, and unclear upstream and downstream dependencies. As a result, operations and maintenance personnel find it difficult to quickly determine whether historical tasks are still being used, whether there is duplicate calculation, which tasks will be affected by adjustments, and whether optimizations can maintain the accuracy of the original calculation results.

[0003] In the prior art, patent document CN120578463A, entitled "Data Warehouse Optimization Method, Device and Vehicle for Internet of Vehicles," discloses a method for acquiring the signals, priorities, and task types of data warehouse computing tasks, merging these tasks based on the signal hit rate and correlation between tasks, and combining the priorities and task types. This solution focuses on merging tasks based on the multiplexing relationship of task signals to improve the efficiency of computing resource utilization.

[0004] Patent document CN119046267A, entitled "A Method, Apparatus, Device, and Medium for Optimizing a Data Warehouse," discloses a method for obtaining the lineage of data in a data warehouse, constructing a directed lineage graph including table vertices and field vertices, determining the field utilization rate and field saturation rate of table vertices, and optimizing the data warehouse accordingly. This scheme primarily implements data warehouse optimization based on statistical indicators of table and field lineage.

[0005] Furthermore, patent document CN114238390A identifies and deletes tables to be optimized based on a data warehouse table lineage diagram; patent document CN114595294A involves data warehouse modeling and extraction based on subject, dimension, fact, and aggregation tables. These technologies provide different implementation paths for data warehouse cleaning, modeling, or partial optimization.

[0006] However, when faced with a large set of historical tasks, relying solely on the identification results of inter-task signal correlations, single lineage indicators, or tables to be optimized is insufficient to establish a unified processing chain that covers the entire task picture, dependency risks, resource utilization, and implementation result verification. In particular, without a mechanism for automated verification and failure rollback of task outputs before and after optimization, directly implementing task merging, scheduling adjustments, or SQL rewriting may lead to inconsistencies in data structures or calculation results. Furthermore, without a structured record of successful experiences and lessons learned from existing optimization cases, subsequent optimizations still require repeated manual judgment, making it difficult to continuously improve the accuracy of strategy selection. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, electronic device, and computer-readable storage medium for organizing and optimizing historical data warehouse tasks, so as to at least solve the problems of low efficiency in organizing historical data warehouse tasks, lack of interpretable basis for optimization schemes, difficulty in automatically verifying optimization implementation risks, and difficulty in continuously accumulating optimization experience.

[0008] To achieve the above objectives, the present invention provides a method for organizing and optimizing historical data warehouse tasks, applied to a data warehouse cluster, the method comprising: S1. Perform a global scan of the historical data warehouse tasks in the data warehouse cluster, and collect the task script, scheduling configuration, execution log, resource consumption record and lineage metadata of each historical data warehouse task; construct an end-to-end data lineage directed graph based on the metadata, and construct the task characteristics of each historical data warehouse task according to execution frequency, resource consumption, task complexity and lineage depth. S2. Cluster the task features using an unsupervised learning algorithm to automatically classify the tasks in the historical data warehouse, and construct a task health profile including activity, complexity, redundancy, dependency health, and resource efficiency based on the classification results, the data lineage directed graph, and the resource consumption records. S3. Identify the problem to be optimized based on the task health profile and the directed graph of data lineage, and match the optimization strategy corresponding to the problem to be optimized from the Skill knowledge base to generate an optimization plan; wherein, the optimization plan includes optimization operation, expected benefit, risk level, impact scope assessment and optimization reason explanation; S4. In response to the review and confirmation of the optimization scheme, automatically execute the optimization operation; S5. Perform a layer-by-layer data consistency check between the optimized task output and the unoptimized task output, and automatically roll back to the unoptimized state if the check fails. S6. Store the pre-optimization state, the optimization operation, the post-optimization result, and the verification conclusion as optimization cases in the Skill knowledge base. Use an unsupervised learning algorithm to extract success patterns and failure lessons from the optimization cases, and update the recommendation weights of each optimization strategy in the Skill knowledge base for subsequent optimization of historical data warehouse tasks.

[0009] Further, in step S1, constructing an end-to-end data lineage directed graph based on the metadata includes: determining table-level dependencies and field-level dependencies based on static code analysis of task scripts and / or runtime tracing of execution logs, and writing the table-level dependencies and field-level dependencies into the data lineage directed graph.

[0010] Furthermore, in step S2, the automatic classification includes active tasks, zombie tasks, redundant tasks, and abnormal tasks; the zombie tasks are tasks that have not been executed within a preset time period, and the abnormal tasks include tasks whose resource consumption exceeds the resource consumption threshold corresponding to their respective cluster.

[0011] Furthermore, in step S3, the problem to be optimized includes at least one of duplicate tasks, resource waste, and complex lineage chains; the optimization operation includes at least one of task merging, scheduling rearrangement, SQL rewriting, redundancy cleanup, and lineage pruning.

[0012] Further, in step S3, the risk level and the scope of impact assessment are determined as follows: based on the data lineage directed graph, related tasks that have upstream and downstream dependencies with the task corresponding to the problem to be optimized are identified, and the risk level and the scope of impact are determined by combining the task priority, execution frequency and lineage depth of the related tasks.

[0013] Furthermore, in step S5, the layer-by-layer data consistency verification includes performing schema consistency verification, data row count verification, aggregation result verification, and sampled data comparison on the task output results before and after optimization in sequence; if any verification fails, the automatic rollback is triggered.

[0014] Furthermore, in step S6, an unsupervised learning algorithm is used to extract success patterns and lessons learned from the optimization cases, and the recommendation weights of each optimization strategy in the Skill knowledge base are updated. This includes: using association rule mining and / or cluster analysis to determine the association between optimization strategies, types of problems to be optimized, risk levels, and verification conclusions; and adjusting the recommendation weights of the optimization strategies based on the association and the historical verification pass rates corresponding to each optimization strategy.

[0015] This invention also provides a system for organizing and optimizing historical data warehouse tasks, comprising: The data acquisition layer is used to perform a global scan of historical data warehouse tasks in the data warehouse cluster, collecting task scripts, scheduling configurations, execution logs, resource consumption records, and lineage metadata. The AI ​​intelligent analysis layer is used to construct an end-to-end directed graph of data lineage. It uses unsupervised learning algorithms to automatically classify tasks in the historical data warehouse and construct task health profiles. Based on the task health profiles and the directed graph of data lineage, it identifies problems to be optimized, matches optimization strategies from the Skill knowledge base, and generates optimization schemes that include optimization operations, expected benefits, risk levels, impact scope assessments, and explanations of optimization reasons. An optimized execution layer is used to execute the optimization operation in response to review and confirmation of the optimization scheme, and to perform layer-by-layer data consistency checks on the optimized task output and the unoptimized task output, automatically rolling back to the unoptimized state if the check fails; and The continuous learning layer stores the pre-optimization state, optimization operation, post-optimization result, and verification conclusion as optimization cases into the Skill knowledge base. It also uses unsupervised learning algorithms to extract successful patterns and lessons learned from failures to update the recommendation weights of each optimization strategy in the Skill knowledge base.

[0016] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the above-described method.

[0017] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method.

[0018] Compared with the prior art, the present invention has the following beneficial effects: Compared to solutions that only address task signal reuse, table field lineage statistics, or table cleanup for optimization, this invention collects task scripts, scheduling configurations, execution logs, resource consumption records, and lineage metadata. By constructing a multi-dimensional task health profile through unsupervised clustering, it can generate a global overview at the task level that includes activity, complexity, redundancy, dependency health, and resource efficiency, thereby improving the coverage of historical task identification and classification.

[0019] By combining task health profiles with directed graphs of data lineage to identify problems to be optimized, and generating optimization solutions that include optimization operations, expected benefits, risk levels, impact assessments, and explanations of the reasons for optimization, each optimization suggestion can be matched with specific data and impact scope, thereby improving the interpretability of the solution and the efficiency of review and decision-making.

[0020] By performing optimization operations after review and confirmation, and using schema consistency, number of data rows, aggregation results, and sampled data comparison to form a layer-by-layer data consistency check, and rolling back to the state before optimization when any check fails, the risk of data result errors caused by task merging, scheduling rearrangement, SQL rewriting, or redundancy cleanup can be reduced.

[0021] By constructing optimization cases from the pre-optimization state, optimization operation, post-optimization result, and verification conclusion, and extracting successful patterns and lessons learned from failures through unsupervised learning, and dynamically adjusting the recommendation weights of optimization strategies, historical optimization experience can be transformed into reusable strategy knowledge, thereby improving the strategy matching efficiency and verification pass rate for subsequent optimization of similar tasks. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method for organizing and optimizing historical data warehouse tasks according to the present invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0024] Example 1 like Figure 1 As shown in the figure, this embodiment provides a method for organizing and optimizing historical data warehouse tasks, applied to a data warehouse cluster. The method includes: Step S1: Perform a global scan of historical data warehouse tasks in the target data warehouse cluster, collecting the task script, scheduling configuration, execution log, resource consumption records, and lineage metadata for each historical data warehouse task. The scan scope should at least include the task script, scheduling configuration, execution log, resource consumption records, and lineage metadata. The scheduling configuration may include trigger period, dependent tasks, priority, retry rules, and resource queues; the execution log may include task start time, end time, execution status, failure reason, and number of runs; and the resource consumption records may include CPU usage, memory usage, storage read / write operations, and execution duration.

[0025] In step S1, an end-to-end directed data lineage graph is constructed based on the collected metadata, and task characteristics for each historical data warehouse task are built according to execution frequency, resource consumption, task complexity, and lineage depth. Specifically, static code analysis can be performed on task scripts to extract source tables, target tables, and field mapping relationships from SQL statements, script variables, or task configurations; runtime tracing can also be performed on execution logs to supplement dynamically generated scripts, temporary tables, or runtime dependencies. Using tasks, tables, and fields as nodes in the graph, and read, write, or dependency relationships as directed edges, a directed data lineage graph with both table-level and field-level characteristics can be formed. Task complexity can be characterized by one or more of the following: script length, number of associated tables, number of SQL nesting levels, number of operators, or number of dependent tasks; lineage depth can be characterized by the longest upstream path length or the longest downstream path length of the task in the directed data lineage graph.

[0026] Step S2: Cluster the task features constructed in Step S1 using an unsupervised learning algorithm to automatically classify tasks in the historical data warehouse, and construct a task health profile based on the classification results, the data lineage directed graph, and resource consumption records. Input each task feature into an unsupervised learning algorithm, such as K-Means, DBSCAN, hierarchical clustering, Isolation Forest, or One-Class SVM, to obtain the task classification results.

[0027] In step S2, based on the clustering results and preset business rules, tasks are divided into active tasks, zombie tasks, redundant tasks, and abnormal tasks. Zombie tasks are defined as tasks that have not been executed within a preset time period and have no effective downstream dependencies; redundant tasks are defined as tasks with the same or highly similar output targets, input sources, and core computational logic; abnormal tasks are defined as tasks whose resource consumption exceeds the resource consumption threshold corresponding to their cluster. The preset time period and resource consumption threshold can be configured by administrators or calculated from the quantiles of historical data.

[0028] In step S2, the task health profile includes at least activity level, complexity, redundancy, dependency health, and resource efficiency. Activity level can be determined by the most recent execution time and execution frequency; complexity can be determined by script complexity and the number of dependencies; redundancy can be determined by task logic similarity, input-output overlap, or the proportion of repeated calculations; dependency health can be determined by the integrity of lineage relationships, the number of upstream and downstream dependencies, and lineage depth; resource efficiency can be determined by the relationship between resource consumption and actual output scale. The profile can be stored and displayed using vectors, tag sets, scoring results, or a combination thereof.

[0029] Step S3: Based on the task health profile constructed in Step S2 and the directed graph of data lineage constructed in Step S1, identify the problems to be optimized, and match the optimization strategies corresponding to the problems from the Skill knowledge base to generate an optimization solution. Problems to be optimized may include at least one of the following: repetitive tasks, long-term inactive tasks, resource-wasting tasks, abnormal tasks, and complex lineage chains. For each problem to be optimized, the system matches an optimization strategy from the Skill knowledge base, such as task merging, schedule reordering, SQL rewriting, redundancy cleanup, or lineage pruning.

[0030] The Skill knowledge base is used to store optimization experience and best practices. A Skill entry may include at least the problem type, applicable conditions, optimization operation template, risk control conditions, historical verification pass rate, and recommendation weight. The system generates optimization plans based on the matched Skill entries. The optimization plan includes at least specific optimization operations, expected benefits, risk level, impact scope assessment, and an explanation of the optimization reasons. The expected benefits may include estimated CPU savings, memory savings, storage savings, reduction in scheduling time, or reduction in the number of tasks; the explanation of the optimization reasons should at least specify the task profile indicators or lineage basis that triggered the plan.

[0031] In step S3, the system starts with the task to be optimized and retrieves its upstream and downstream related tasks in the directed graph of data lineage. Then, it determines the scope of impact and risk level by combining the priority, execution frequency, and lineage depth of the related tasks. If there are many related tasks, the priority of the related tasks is high, or the downstream lineage path is deep, the corresponding optimization scheme is marked as high risk, and the list of related tasks and their impact relationships are displayed in the review report.

[0032] Step S4: In response to the review and confirmation of the optimization plan generated in Step S3, the optimization operation is executed automatically. The review report generated by the system may include a comparison before and after optimization, a detailed description of the plan, risk level, scope of impact, and expected benefit analysis. The data warehouse team selects to confirm implementation, reject implementation, or adjust the implementation scope through the review interface. In response to confirmation of implementation, the optimization execution layer automatically performs the corresponding operations, such as modifying scheduling configurations, merging or deleting tasks, rewriting SQL statements, adjusting resource allocation, or updating lineage configurations. Before performing optimization, the task scripts, scheduling configurations, and baseline output results can be saved as the pre-optimization state for verification and rollback.

[0033] Step S5: Perform layer-by-layer data consistency checks between the optimized task output and the unoptimized task output; automatically roll back to the unoptimized state if the check fails. After optimization, the system will sequentially perform schema consistency checks, data row count checks, aggregation result checks, and sampled data comparisons between the optimized and unoptimized task outputs. Schema consistency checks compare field names, field types, and field order; data row count checks compare the number of output records or their allowable error range; aggregation result checks compare counts, sums, maximum values, minimum values, or average values ​​obtained by grouping according to preset dimensions; and sampled data comparisons compare specific record content according to preset sampling rules.

[0034] In step S5, if any of the above checks fails, the rollback controller restores the task scripts, scheduling configurations, and related task states saved before optimization, thereby rolling back to the state before optimization. When all checks pass, the system confirms that the optimization is complete and records the actual resource consumption, actual execution time, and actual check results. For business scenarios that allow for latency or data window differences, time windows, allowable error ranges, or hash check rules can be configured for the corresponding checks.

[0035] Step S6: Store the pre-optimization state, optimization operation, post-optimization result, and verification conclusion as optimization cases in the Skill knowledge base. Use an unsupervised learning algorithm to extract success patterns and lessons learned from these optimization cases, updating the recommendation weights of each optimization strategy in the Skill knowledge base. The continuous learning layer uses association rule mining and / or cluster analysis to determine the relationships between optimization strategies, problem types to be optimized, risk levels, and verification conclusions. Based on these relationships and the historical verification pass rates of each optimization strategy, the recommendation weights of the strategies are adjusted. For example, strategies that have passed multiple consecutive verifications for the same problem type and similar risk levels have their recommendation weights increased; strategies with high failure rates or frequent rollback triggers have their recommendation weights decreased or more stringent application conditions added. The updated Skill knowledge base automatically takes effect in the next round of task optimization.

[0036] The above steps can be performed by the same electronic device or by multiple services deployed in a data warehouse cluster working together.

[0037] Example 2 This embodiment provides a system for organizing and optimizing historical data warehouse tasks, including a data acquisition layer, an AI intelligent analysis layer, an optimization execution layer, and a continuous learning layer.

[0038] The data acquisition layer includes a task metadata collector, a lineage extractor, and a log and monitoring data collector. The task metadata collector is used to obtain task scripts and scheduling configurations; the lineage extractor is used to construct a directed graph of data lineage based on static code analysis and / or runtime tracing; and the log and monitoring data collector is used to obtain execution logs and resource consumption records.

[0039] The AI-powered intelligent analysis layer comprises a task profiling engine, an anomaly detection model, a lineage analysis engine, an optimization strategy matcher, and an interpretability attribution module. The task profiling engine performs task feature extraction, unsupervised clustering, and health profile construction; the anomaly detection model identifies zombie tasks, redundant tasks, and anomalous tasks; the optimization strategy matcher matches optimization strategies from the Skill knowledge base; and the interpretability attribution module generates explanations of optimization reasons, impact assessments, and risk levels.

[0040] The optimization execution layer includes a review report generator, an automated implementation engine, a data consistency checker, and a rollback controller. The review report generator outputs solutions that can be confirmed by the data warehouse team; the automated implementation engine performs task merging, scheduling reordering, SQL rewriting, redundancy cleanup, or lineage pruning; the data consistency checker performs layer-by-layer checks on schema consistency, data row count, aggregation results, and sampled data; and the rollback controller restores the system to its pre-optimization state if checks fail.

[0041] The continuous learning layer includes a Skill knowledge base, an unsupervised learning engine, a policy weight manager, and a feedback loop collector. The feedback loop collector collects optimization cases; the unsupervised learning engine extracts patterns from the optimization cases; the policy weight manager updates the policy recommendation weights based on results such as the validation pass rate; and the Skill knowledge base provides updated rules, cases, and weights for subsequent optimization policy matching.

[0042] Example 3 This embodiment provides an electronic device and a computer-readable storage medium. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in Embodiment 1. The electronic device may be a server, a cloud computing node, an edge computing node, a data warehouse management device, or a distributed computing system composed of multiple physical devices.

[0043] The memory may include non-volatile storage media, volatile storage media, or combinations thereof. The computer-readable storage medium may be any tangible medium capable of storing computer programs and enabling a processor to execute the methods described above, such as a hard disk, solid-state drive, flash memory, read-only memory, random access memory, or optical disk.

[0044] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for organizing and optimizing historical data warehouse tasks, characterized in that, Applied to a data warehouse cluster, the method includes: S1. Perform a global scan of the historical data warehouse tasks in the data warehouse cluster, and collect the task script, scheduling configuration, execution log, resource consumption record and lineage metadata of each historical data warehouse task; construct an end-to-end data lineage directed graph based on the metadata, and construct the task characteristics of each historical data warehouse task according to execution frequency, resource consumption, task complexity and lineage depth. S2. Cluster the task features using an unsupervised learning algorithm to automatically classify the tasks in the historical data warehouse, and construct a task health profile including activity, complexity, redundancy, dependency health, and resource efficiency based on the classification results, the data lineage directed graph, and the resource consumption records. S3. Identify the problem to be optimized based on the task health profile and the directed graph of data lineage, and match the optimization strategy corresponding to the problem to be optimized from the Skill knowledge base to generate an optimization plan; wherein, the optimization plan includes optimization operation, expected benefit, risk level, impact scope assessment and optimization reason explanation; S4. In response to the review and confirmation of the optimization scheme, automatically execute the optimization operation; S5. Perform a layer-by-layer data consistency check between the optimized task output and the unoptimized task output, and automatically roll back to the unoptimized state if the check fails. S6. Store the pre-optimization state, the optimization operation, the post-optimization result, and the verification conclusion as optimization cases in the Skill knowledge base. Use an unsupervised learning algorithm to extract success patterns and failure lessons from the optimization cases, and update the recommendation weights of each optimization strategy in the Skill knowledge base for subsequent optimization of historical data warehouse tasks.

2. The method for organizing and optimizing historical data warehouse tasks according to claim 1, characterized in that, In step S1, an end-to-end data lineage directed graph is constructed based on the metadata, including: determining table-level dependencies and field-level dependencies based on static code analysis of task scripts and / or runtime tracing of execution logs, and writing the table-level dependencies and field-level dependencies into the data lineage directed graph.

3. The method for organizing and optimizing historical data warehouse tasks according to claim 1, characterized in that, In step S2, the automatic classification includes active tasks, zombie tasks, redundant tasks, and abnormal tasks; the zombie tasks are tasks that have not been executed within a preset time period, and the abnormal tasks include tasks whose resource consumption exceeds the resource consumption threshold corresponding to their respective cluster.

4. The method for organizing and optimizing historical data warehouse tasks according to claim 1, characterized in that, In step S3, the problem to be optimized includes at least one of duplicate tasks, resource waste, and complex lineage chains; the optimization operation includes at least one of task merging, scheduling rearrangement, SQL rewriting, redundancy cleanup, and lineage pruning.

5. The method for organizing and optimizing historical data warehouse tasks according to claim 1, characterized in that, In step S3, the risk level and the scope of impact assessment are determined as follows: based on the data lineage directed graph, related tasks that have upstream and downstream dependencies with the task corresponding to the problem to be optimized are identified, and the risk level and the scope of impact are determined by combining the task priority, execution frequency and lineage depth of the related tasks.

6. The method for organizing and optimizing historical data warehouse tasks according to claim 1, characterized in that, In step S5, the layer-by-layer data consistency verification includes performing schema consistency verification, data row count verification, aggregation result verification, and sampled data comparison on the task output results before and after optimization in sequence; if any verification fails, the automatic rollback is triggered.

7. The method for organizing and optimizing historical data warehouse tasks according to claim 1, characterized in that, In step S6, unsupervised learning algorithms are used to extract success patterns and lessons learned from the optimization cases, and the recommendation weights of each optimization strategy in the Skill knowledge base are updated. This includes: using association rule mining and / or cluster analysis to determine the association between optimization strategies, types of problems to be optimized, risk levels, and verification conclusions; and adjusting the recommendation weights of the optimization strategies based on the association and the historical verification pass rates corresponding to each optimization strategy.

8. A system for organizing and optimizing historical data warehouse tasks, characterized in that, include: The data acquisition layer is used to perform a global scan of historical data warehouse tasks in the data warehouse cluster, collecting task scripts, scheduling configurations, execution logs, resource consumption records, and lineage metadata. The AI ​​intelligent analysis layer is used to construct an end-to-end directed graph of data lineage. It uses unsupervised learning algorithms to automatically classify tasks in the historical data warehouse and construct task health profiles. Based on the task health profiles and the directed graph of data lineage, it identifies problems to be optimized, matches optimization strategies from the Skill knowledge base, and generates optimization schemes that include optimization operations, expected benefits, risk levels, impact scope assessments, and explanations of optimization reasons. The optimization execution layer is used to execute the optimization operation in response to the review and confirmation of the optimization scheme, and to perform layer-by-layer data consistency verification between the optimized task output and the task output before optimization. If the verification fails, it will automatically roll back to the state before optimization. as well as The continuous learning layer stores the pre-optimization state, optimization operation, post-optimization result, and verification conclusion as optimization cases into the Skill knowledge base. It also uses unsupervised learning algorithms to extract successful patterns and lessons learned from failures to update the recommendation weights of each optimization strategy in the Skill knowledge base.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images