DAG Workflow for Automated Model Self-Learning Iteration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning model update and iteration processes are cumbersome and costly, requiring complex code development and management, which hinders efficient adaptation to user behavior changes.
Innovation Solution
Implementing a directed acyclic graph (DAG) to manage the whole flow of model self-taught learning services, breaking down the process into subgraphs for data acquisition, training, and model release, thereby simplifying the management and reducing development complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex code development and management is used for model update and iteration, then model adaptation capability is improved, but management difficulty and cost increase
Solution Approach 1:
The system enables automated model self-learning and self-updating through the DAG framework. The model automatically acquires service data, performs training iterations, and updates itself without requiring complex manual code development and management, thereby improving adaptability while reducing management complexity
Solution Approach 2:
The model update process is segmented into distinct subgraphs (data acquisition subgraph, training subgraph, release subgraph) within the DAG framework. Each subgraph handles specific tasks independently, making the overall complex process manageable through modular organization and reducing management difficulty
2Productivity
If complex code development and management is used for model update and iteration, then model iteration process is completed, but management cost increases
Solution Approach 1:
Multiple model iteration tasks are merged into a unified DAG management framework. The framework consolidates data acquisition, training, and release operations into a single coordinated system, improving overall iteration efficiency while reducing redundant management costs through shared resources and centralized control
Solution Approach 2:
The automated self-learning mechanism reduces the need for manual code development and management interventions. The system performs self-service model updates, thereby improving iteration efficiency while reducing the labor costs associated with complex code management
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Provided are a data processing method and apparatus, and a storage medium. The data processing method includes: acquiring a target directed acyclic graph, DAG, corresponding to the service processing logic of a model self-taught learning service, where the service processing logic includes execution logic for acquiring service data generated by an online released service model, execution logic for training a to-be-trained service model based on the service data, and execution logic for releasing the trained service model online; and performing self-taught learning on the to-be-trained service model according to the target DAG.