Distributed Deep Learning Model Deployment Automation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing technologies face challenges in creating a deep learning-based learning engine due to difficulties in deep network design, parameter tuning, and the repetitive process of utilizing common elements, making it hard for non-experts to build and deploy deep learning models efficiently, especially with prolonged training and inference times when using a single server or device.
Innovation Solution
A system and method for continuous integration and deployment of a deep learning framework that includes edge servers, a distributed deep learning training cloud, a software configuration management repository, and a controller to automate the process of training, deploying, and updating deep learning models, allowing for distributed training and inference, and enabling users without deep learning expertise to utilize pre-learned models with reduced training and inference times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single server or device is used for deep learning training, then the system complexity is low, but the training and inference times are greatly prolonged
Solution Approach 1:
The patent divides the deep learning training system into multiple distributed servers that work in parallel. Each server handles a portion of the training workload, thereby reducing the overall training time while distributing the system complexity across multiple components rather than concentrating it in a single server.
Solution Approach 2:
The patent transitions from a single-server architecture to a multi-server distributed architecture, adding the dimension of spatial distribution. This dimensional change allows parallel processing of training data across multiple nodes, significantly reducing training and inference times while managing complexity through distributed design.
2Adaptability or versatility
If deep learning models are created from scratch, then the models can be customized, but the process requires repetition of common elements and prolonged training time
Solution Approach 1:
The patent implements a service model table that stores pre-configured common elements and templates for deep learning models. Before creating a customized model, users can access and utilize these pre-prepared components, eliminating the need to recreate common elements and significantly reducing the overall training time while maintaining customization capabilities.
Solution Approach 2:
The service model table serves multiple functions: storing model templates, managing common elements, facilitating model customization, and enabling rapid deployment. This universal component handles various aspects of model creation, reducing redundancy and training time while preserving adaptability.
3Extent of automation
If the service model table is manually deployed, then deployment control is precise, but the deployment process is not automated and requires manual intervention
Solution Approach 1:
The patent implements an automated notification mechanism where the service model table monitors for changes and automatically notifies the controller when updates occur. This feedback loop enables automated deployment while maintaining precise control, as the system responds to actual changes rather than requiring manual initiation, thus balancing automation with operational control.
4Device complexity
If deep learning training is performed on a single device, then the resource requirements are simple, but the training time is greatly prolonged depending on the amount of data
Solution Approach 1:
The patent segments the training workload across multiple distributed servers, with each server processing a portion of the data in parallel. This segmentation increases training throughput and productivity while distributing resource requirements across multiple devices rather than concentrating them in a single complex system.
Data Source
AI summary
A system for continuous integration and deployment of a service model using a deep learning framework, includes: a plurality of edge servers configured to provide a deep learning inference service; a distributed deep learning training cloud comprising a plurality of distributed servers, each comprising a deep learning framework application query-based deep learning database server, and a main server configured to manage the plurality of distributed server and to perform distributed training for a learning model; a software configuration management (SCM) repository configured to automatically handle revision, version management, backup, and rollback processes of a service model table, which is an outcome of a service model that is the learning model subjected to distributed training; and a controller configured to, according to a predetermined deployment policy, deploy the service model table to be executed on the edge servers when changes to the service model table occur in the SCM repository.