Model training deployment system and method

Through the automation and intelligent means of model training and deployment system, the problems of cumbersome data management, low labeling efficiency and complex model deployment in traditional model training are solved, efficient and stable model training and rapid deployment are achieved, cost reduction and promotion Sustainable development of AI projects.

CN119005356APending Publication Date: 2024-11-22DMAI (GUANGZHOU) CO LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411045473.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In traditional model training, data management is cumbersome, manual labeling efficiency is low, model training and selection is complex, and the model deployment process is complex, resulting in low efficiency and high cost.

Method used

Provide a model training and deployment system, including data management module, labeling audit module, model training module and model deployment module, and implement data preprocessing, labeling, model training and deployment through automation and intelligent means to achieve unified and automated data management , reduce manual intervention and improve efficiency.

Benefits of technology

It simplifies the implementation path of AI projects, improves the quality and speed of data labeling, reduces manual intervention, realizes efficient and stable model training and rapid deployment, reduces hardware investment costs, and promotes the sustainable development of AI projects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005356A_ABST
    Figure CN119005356A_ABST
Patent Text Reader

Abstract

The invention discloses a model training deployment system, and the system comprises a data management module which is used for importing original data, and carrying out the preprocessing of the original data, and obtaining final data; the annotation auditing module is used for carrying out annotation based on the final data to obtain initial annotation data, carrying out annotation quality monitoring on the initial annotation data, and adjusting abnormal annotation data to obtain final annotation data; the model training module is used for optimizing the model structure based on the optimal optimizer and the optimal model parameters to obtain a new model structure, training the new model structure based on the final annotation data to obtain a trained model, and compressing the trained model to obtain a lightweight model; and the model deployment module is used for selecting an optimal model based on the lightweight model, performing service packaging, performing declarative definition on a model service, adjusting resources required by the model service, performing lossless release on the packaged model, and performing automatic recovery when a fault occurs. And data annotation, model training and model deployment are simply realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of model training technology, and more specifically to a model training deployment system and method. Background Art

[0002] In traditional model training, data collection and organization usually require a lot of manual operations, which are inefficient and prone to errors, especially when transmitting large files, which is prone to file errors and file duplications due to network interruptions. At the same time, traditional data labeling often requires a lot of manpower investment, and the labeling quality varies. It is impossible to coordinate multiple labelers to label at the same time, and manual review is cumbersome. In the model training process, users usually need to manually configure training parameters and select appropriate models, which has a high technical threshold for ordinary users.

[0003] Therefore, how to simply implement data labeling, model training and selection, and model deployment is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] In view of this, the present invention provides a model training and deployment system and method, which effectively solves the problems of cumbersome data management, low efficiency of manual labeling, complex model training and selection, and complex model deployment in traditional model development, and improves the convenience and efficiency of applications for enterprises and individual developers.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A model training and deployment system, comprising: a data management module, a labeling and review module, a model training module and a model deployment module;

[0007] The data management module is used to import raw data and pre-process the raw data to obtain final data;

[0008] The annotation review module is used to perform annotation based on the final data to obtain initial annotation data, and to monitor the annotation quality of the initial annotation data, adjust abnormal annotation data, and obtain final annotation data;

[0009] The model training module is used to process the final labeled data in parallel, select an optimal optimizer, a model structure and optimal model parameters, optimize the model structure based on the optimal optimizer and the optimal model parameters to obtain a new model structure, train the new model structure based on the final labeled data to obtain a trained model, and compress the model to obtain a lightweight model;

[0010] The model deployment module is used to select the optimal model based on lightweight model evaluation and perform service packaging. It is also used to declaratively define the model service, adjust the resources required for the model service, losslessly publish the packaged model, and automatically recover in case of failure.

[0011] Preferably, the data management module comprises: a data import unit, a data processing unit and a data association unit;

[0012] The data import unit is used to batch import data in different forms from different data sources, and perform data deduplication processing when importing the data, and the imported data after processing is used as the original data;

[0013] The data processing unit is used to perform data enhancement based on the original data to obtain enhanced data;

[0014] The data association unit is used to build a unified data label system, and associate metadata information with the enhanced data based on the data label system to obtain the final data.

[0015] Preferably, the data management module further includes: a data storage unit;

[0016] The data storage unit is used to store the original data and the final data through a distributed file system.

[0017] Preferably, the annotation review module includes: an automatic annotation unit, a collaborative annotation unit and an annotation monitoring unit;

[0018] The automatic annotation unit is used to annotate the final data based on a semi-automatic annotation technology driven by deep learning to obtain coarsely annotated data;

[0019] The collaborative labeling unit is used to distribute the final data to be labeled to multiple labelers for collaborative labeling, and the labelers perform manual labeling or labeling based on the rough labeling data obtained by the automatic labeling unit, and integrate the labeled data to obtain initial labeling data;

[0020] The annotation monitoring unit is used to establish an annotation quality evaluation system, automatically review the initial annotation data according to the annotation quality evaluation system, and manually review and adjust the controversial annotation data to obtain the final annotation data.

[0021] Preferably, the model training module includes: a computing resource scheduling unit, an optimizer determination unit, a model structure determination unit, a model training unit and a model compression unit;

[0022] The computing resource scheduling unit is used to process the final labeled data in parallel based on a heterogeneous computing cluster architecture, and to achieve computing resource balance through a load balancing algorithm and a dynamic resource allocation strategy;

[0023] The optimizer determination unit is used to select the optimal optimizer according to the model architecture and training data characteristics, and introduce a collaborative training framework;

[0024] The model structure determination unit is used to automatically determine the model structure based on reinforcement learning, evolutionary algorithm or Bayesian optimization, and determine the optimal hyperparameter based on a hyperparameter tuning algorithm;

[0025] The model training unit is used to optimize the model structure based on the optimal optimizer and the optimal model parameters to obtain a new model structure, and train the new model structure based on the final labeled data to obtain a trained model;

[0026] The model compression unit is used to reduce the accuracy of training data and compress the trained model to obtain the lightweight model.

[0027] Preferably, the model training module further includes: a supervised learning unit;

[0028] The supervised learning unit is used to perform self-learning and in-depth mining of feature representation based on a small amount of labeled data, unlabeled data or inaccurately labeled data input into the model.

[0029] Preferably, the model training module further includes: a training monitoring unit;

[0030] The training monitoring unit is used to monitor and track the changes of key parameters in the model training process in real time, and adjust the training strategy in real time based on the changes.

[0031] Preferably, the model deployment module includes: a model evaluation encapsulation unit, a declaration definition unit, a resource optimization unit, and a publishing and fault recovery unit;

[0032] The model evaluation and packaging unit is used to determine the optimal model based on the lightweight model evaluation, and to package the optimal model into a service;

[0033] The declarative definition unit is used to connect to the Kubernetes cluster, declaratively define the model service through the Helm Chart in YAML format or the Kubernetes native Manifest file, and describe the deployment target state in code form;

[0034] The resource optimization unit is used to realize automatic expansion and contraction of the model service instance and allocate and adjust various resources required for the model service;

[0035] The publishing and fault recovery unit is used to losslessly publish the encapsulated model based on the distribution strategy, realize smooth switching between the new and old model versions based on the inspection mechanism, and automatically recover to the previous stable model version in case of failure.

[0036] Preferably, the model deployment module further includes: a model management unit and a service registration adjustment unit;

[0037] The model management unit is used to customize the Kubernetes Operator, which is responsible for the full life cycle management of the model service in the Kubernetes cluster and is used for model version control and rolling updates;

[0038] The service registration adjustment unit is used to integrate the model service into the service grid system after the model service is released, automatically register and discover the service, and ensure the grayscale release and smooth transition of the new model service through intelligent routing and service weight adjustment.

[0039] A model training deployment method, comprising:

[0040] Importing raw data and preprocessing the raw data to obtain final data;

[0041] Annotating the final data to obtain initial annotated data, monitoring the annotation quality of the initial annotated data, adjusting abnormal annotated data, and obtaining final annotated data;

[0042] The final labeled data are processed in parallel, an optimal optimizer, a model structure and optimal model parameters are selected, the model structure is optimized based on the optimal optimizer and the optimal model parameters to obtain a new model structure, the new model structure is trained based on the final labeled data to obtain a trained model, and the model is compressed to obtain a lightweight model;

[0043] Based on lightweight model evaluation, the optimal model is selected and service-oriented packaging is performed, the model service is declaratively defined, the resources required for the model service are adjusted, the packaged model is losslessly published, and it is automatically restored in case of failure.

[0044] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a model training deployment system and method, which has the following beneficial effects:

[0045] 1. Integration and convenience: The one-stop service of the system of the present invention reduces the complexity brought by cross-system operations. Users do not need to switch between different tools and environments, saving time and energy. From raw data import, labeling, model training to the final model launch, the entire process is completed on one system, greatly simplifying the implementation path of AI projects.

[0046] 2. Efficiency improvement: The data labeling of the present invention has the characteristics of automation and intelligence. It uses the model for rough labeling in advance and provides efficient labeling tools and preview functions, which helps to improve the quality and speed of labeling. The automated training process adopts the optimal training parameter configuration and resource scheduling, reduces manual intervention, and makes model training more efficient and stable.

[0047] 3. Collaboration and management optimization: The present invention facilitates the division of labor among team members through multi-user collaborative annotation and project management functions, while supporting annotation quality review and feedback to ensure the quality consistency of the data set; the training process is transparent and the training effect is monitored in real time, which facilitates developers to quickly adjust the optimization strategy and accelerate the iteration and upgrade of the model.

[0048] 4. Model performance and service quality: The integration of best practices and preset model templates, combined with optimized training strategies, is conducive to obtaining high-performance AI models; the rapid deployment and update mechanism enables the model to be seamlessly transitioned to the actual application environment once training is completed, thereby improving the product's speed to market and service continuity.

[0049] 5. Cost saving and resource utilization: Through unified resource management, computing resources are reasonably allocated, hardware investment costs are reduced, and resource waste is avoided; data and models are centrally managed and easy to reuse, which reduces the cost of repeated labeling and training and promotes the sustainable development of AI projects.

[0050] 6. The system of the present invention realizes the standardization, automation and efficiency of the AI ​​project development process, which not only improves work efficiency and model performance, but also brings faster product iteration cycle and higher input-output ratio to enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 A schematic diagram of the structure of a model training deployment system provided by the present invention.

[0053] Figure 2This is a schematic diagram of the data management module structure provided by the present invention.

[0054] Figure 3 This is a schematic diagram of the structure of the model training module provided by the present invention.

[0055] Figure 4 This is a schematic diagram of the model deployment module structure provided by the present invention.

[0056] Figure 5 A flow chart of a model training deployment method provided by the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Example 1

[0059] like Figure 1 As shown, an embodiment of the present invention discloses a model training deployment system, including: a data management module, a labeling review module, a model training module and a model deployment module;

[0060] The data management module is used to import the original data and pre-process the original data to obtain the final data;

[0061] The annotation review module is used to annotate based on the final data to obtain the initial annotation data, monitor the annotation quality of the initial annotation data, adjust the abnormal annotation data, and obtain the final annotation data;

[0062] The model training module is used to process the final labeled data in parallel, select the optimal optimizer, model structure and optimal model parameters, optimize the model structure based on the optimal optimizer and optimal model parameters to obtain a new model structure, train the new model structure based on the final labeled data to obtain a trained model, and compress the model to obtain a lightweight model;

[0063] The model deployment module is used to select the optimal model based on lightweight model evaluation and encapsulate it as a service. It is also used to declaratively define the model service, adjust the resources required for the model service, losslessly publish the encapsulated model, and automatically recover in case of failure.

[0064] Example 2

[0065] The embodiment of the present invention discloses another preferred model training and deployment system, including: a data management module, a labeling and review module, a model training module and a model deployment module;

[0066] The data management module is used to import the original data and pre-process the original data to obtain the final data.

[0067] Preferably, Figure 2 As shown, the data management module includes: a data import unit, a data processing unit and a data association unit;

[0068] The data import unit is used to batch import data in different forms from different data sources and perform data deduplication processing when importing data. The imported data after processing is used as the original data.

[0069] Preferably, the imported data includes image data in JPEG and PNG formats, including but not limited to:

[0070] 1) Pictures of tools used for indoor and outdoor operations in power grid systems; 2) Pictures of medical equipment; 3) Pictures of calligraphy fonts in elementary school calligraphy classes; 4) Pictures of mathematical formulas in elementary and middle schools; 5) Pictures of faces in security.

[0071] Preferably, the data import unit is provided with an HTTP protocol interface to import data from different data sources (such as local files, network links, databases, etc.), and supports batch uploading, streaming transmission and real-time capture of image data and video data in different formats.

[0072] Preferably, the data import unit uses an internal private protocol when importing data, supports intelligent deduplication, breakpoint resumption and idempotent retransmission, and implements data deduplication based on hash value or deep learning feature comparison technology, thereby ensuring the uniqueness and integrity of the imported data.

[0073] Preferably, the duplicate data confirmation method is as follows: the hash value of the imported data is calculated, and the same value is considered a duplicate; a deep learning model is used to extract high-level features of the image, and when the similarity value reaches a preset threshold, the image can be considered duplicated.

[0074] Preferably, the data import unit can interface with a third-party data provider to directly obtain pre-processed data.

[0075] The data processing unit is used to perform data enhancement based on the original data to obtain enhanced data.

[0076] Preferably, the data processing unit performs data enhancement on the expanded data based on image processing algorithms (such as random rotation, cropping, flipping, brightness adjustment, etc.) to expand the diversity of training samples; or generates realistic synthetic data based on the original data and deep learning technology for data expansion to compensate for the problem of uneven distribution of real-world data.

[0077] Preferably, the original data is input into a generative adversarial network (GAN), which generates new data.

[0078] Preferably, the data processing unit can generate simulation data based on a physical simulation engine; and perform data enhancement based on MixUp, CutMix and AutoAugment data enhancement technologies.

[0079] The data association unit is used to build a unified data labeling system, based on which the data association metadata information is enhanced to obtain the final data.

[0080] Preferably, the data tag system is based on enhanced data-associated metadata information to facilitate later retrieval, filtering and grouping. The metadata information includes but is not limited to: data uploader, data upload time, data classification: such as human body, animals, plants, and other items.

[0081] Preferably, the data management module further includes: a data storage unit;

[0082] The data storage unit is used to store the original data and the final data through the distributed file system to ensure high concurrent access and fast loading.

[0083] Preferably, the data storage unit supports both data version control and authority management.

[0084] The annotation review module is used to annotate based on the final data to obtain initial annotation data, monitor the annotation quality of the initial annotation data, adjust abnormal annotation data, and obtain final annotation data.

[0085] Preferably, the annotation review module includes: an automatic annotation unit, a collaborative annotation unit and an annotation monitoring unit;

[0086] The automatic labeling unit is used to label the final data based on the semi-automatic labeling technology driven by deep learning to obtain coarsely labeled data.

[0087] Preferably, semi-automatic labeling technology includes image segmentation, target detection and OCR, which reduces the burden of manual labeling; semi-automatic labeling technology divides labeling into two processes: automatic labeling by the system: telling the system what standard data is needed (such as a certain object, animal or plant, etc.), and then labeling by a pre-trained model without human intervention. The data labeled by the machine is manually reviewed and adjusted as needed.

[0088] Preferably, the automatic labeling unit is also used to assist labeling by rule-based methods, clustering algorithms or unsupervised learning.

[0089] The collaborative labeling unit is used to distribute the final data to be labeled to multiple labelers for collaborative labeling. The labelers perform manual labeling or label based on the rough labeling data obtained by the automatic labeling unit, and integrate the labeled data to obtain initial labeling data.

[0090] Preferably, the collaborative annotation steps are:

[0091] The collaborative annotation unit distributes the data to be annotated (a picture or a voice) to multiple annotators;

[0092] Each annotator is responsible for annotating a specific target;

[0093] The collaborative labeling unit integrates the labeled data to obtain initial labeling data containing all target labeling information.

[0094] Preferably, the collaborative annotation unit uses an internal private consensus algorithm to resolve conflicts during concurrent annotation, avoiding problems such as dirty annotation, annotation overlap, and ghost annotation.

[0095] Preferably, a private consensus algorithm is used to resolve the concurrent labeling process: if multiple labelers label the same target of the same data, the first labeler who submits will enter the labeling information first; when other labelers submit, if there is no business conflict with the already submitted labeling information, or the conflict can be automatically merged, it will be submitted smoothly; otherwise, the submission will be notified that the submission failed, and the labeler will be notified to pull the existing data for manual integration before submitting.

[0096] Preferably, the business conflict refers to that the results of different annotators for labeling the same object overlap in space.

[0097] Preferably, manual integration means that when there is a conflict in annotation information, it cannot be directly overwritten, and it is necessary to pull the annotation information submitted last time, integrate it with your own information, and then submit it; for example, the previous annotator submitted annotation information a, and the next annotator submitted annotation information b later. Since the version numbers submitted by the two annotators are the same, the latter will be prompted with a conflict. At this time, the latter needs to pull information a, integrate it with the information b he is going to submit, and then submit it.

[0098] The annotation monitoring unit is used to establish an annotation quality evaluation system, automatically review the initial annotation data according to the annotation quality evaluation system, and manually review and adjust the controversial annotation data to obtain the final annotation data.

[0099] Preferably, a labeling quality evaluation system is established so that each labeling result is automatically reviewed by the detection program. If the detection program disagrees with the labeler's labeling, the labeling will be automatically sent to the manual review queue for subsequent confirmation and adjustment by the auditor. The labeling data after manual adjustment will be checked by the pre-trained model to achieve automatic review.

[0100] Preferably, the labeling monitoring unit is also used to replace the traditional cross-validation with a blockchain-based consensus mechanism to ensure the authenticity and non-tamperability of the labeling data.

[0101] The model training module is used to process the final labeled data in parallel, select the optimal optimizer, model structure and optimal model parameters, optimize the model structure based on the optimal optimizer and optimal model parameters to obtain a new model structure, train the new model structure based on the final labeled data to obtain a trained model, and compress the model to obtain a lightweight model.

[0102] Preferably, Figure 3 As shown, the model training module includes: a computing resource scheduling unit, an optimizer determination unit, a model structure determination unit, a model training unit and a model compression unit;

[0103] The computing resource scheduling unit is used to process the final labeled data in parallel based on the heterogeneous computing cluster architecture, and to achieve computing resource balance through load balancing algorithms and dynamic resource allocation strategies;

[0104] Preferably, the computing resource scheduling unit is used to accelerate the NPU of the neural network and the DPU that processes the network data flow of the data center, so as to realize efficient parallel processing of massive training data; through the load balancing algorithm and dynamic resource allocation strategy, high-performance computing resources are fully utilized in the process of large-scale deep learning training to ensure the high-speed and stable advancement of the model training process.

[0105] Preferably, the method for achieving computing resource balance is:

[0106] Install a collector on all physical nodes to collect machine load (CPU / memory / graphics card usage);

[0107] The collector sends the collected load information to the computing resource scheduling unit;

[0108] The computing resource scheduling unit calculates the node whose current load is lower than the threshold in real time based on the load information, and sends the computing task to the node, thereby achieving computing resource balance.

[0109] The optimizer determination unit is used to select the optimal optimizer based on the model architecture and training data characteristics, and introduce a collaborative training framework.

[0110] Preferably, the optimizer determines that the unit integrates a series of optimization algorithms, such as AdamW, RMSprop, Adagrad, and NovoGrad, etc., and introduces a collaborative training framework to further improve the model generalization ability through multi-task learning, transfer learning, meta-learning and other methods.

[0111] The model structure determination unit is used to automatically determine the model structure based on reinforcement learning, evolutionary algorithm or Bayesian optimization, and determine the optimal hyperparameters based on the hyperparameter tuning algorithm.

[0112] Preferably, the model structure determination unit integrates neural architecture search technology to automatically determine the optimal model structure based on reinforcement learning, evolutionary algorithm or Bayesian optimization, and accurately locates the combination that can maximize the model performance in a large-scale hyperparameter space as the optimal hyperparameter based on hyperparameter tuning algorithms such as Hyperband, BayesianOptimization or Grid Search.

[0113] Preferably, the model structure determination unit is also used to automatically determine the model structure based on neural architecture search (NAS).

[0114] The model training unit is used to optimize the model structure based on the optimal optimizer and the optimal model parameters to obtain a new model structure, and train the new model structure based on the final labeled data to obtain a trained model.

[0115] Preferably, during the model training process, the choice of loss function, the type of optimizer, the learning rate adjustment strategy, etc. are all key steps that can be replaced.

[0116] The model compression unit is used to reduce the accuracy of training data and compress the trained model to obtain a lightweight model.

[0117] Preferably, the model compression unit is used to reduce the precision of the training data so that the system supports mixed precision training. By adopting low-precision numerical formats such as FP16 and BFLOAT16, the memory usage and computation time are significantly reduced while maintaining the model accuracy.

[0118] Preferably, the training data is preprocessed, and the data is extracted using the relevant functions of PyTorch or TensorFlow. Different parameters are passed in when calling the function to achieve different data accuracies.

[0119] Preferably, the model compression unit compresses the trained model based on model pruning, quantization or distillation compression technology to obtain a lightweight model, so that the trained model meets the requirements of lightweight deployment while ensuring performance.

[0120] Preferably, model pruning: reducing the size and computational complexity of the model by deleting unnecessary connections or nodes in the model; quantization: converting the floating-point parameters in the model into low-precision integers or fixed-point numbers, thereby reducing the storage space and computational complexity of the model; distillation: migrating the knowledge of a larger model (teacher model) to a smaller model (student model), thereby achieving the purpose of model compression.

[0121] Preferably, the model training module further includes: a supervised learning unit;

[0122] The supervised learning unit is used to input a small amount of labeled data, unlabeled data, or inaccurately labeled data into the model for self-learning and in-depth mining of feature representation.

[0123] Preferably, the supervised learning unit is based on self-supervised learning or unsupervised learning. On the basis of limited labeled data, it uses a small amount of labeled data or unlabeled data for self-learning and in-depth mining of feature representation. It is also based on weakly supervised learning strategies, such as label smoothing, entropy regularization, and clustering-guided learning. When the label quality of the training data is not completely reliable, such as inaccurate labels or high noise, the model performs self-learning, thereby overcoming the reliance of traditional supervised learning on a large amount of manually labeled data.

[0124] Preferably, the model training module further includes: a training monitoring unit;

[0125] The training monitoring unit is used to monitor and track the changes in key parameters during the model training process in real time, and adjust the training strategy in real time based on the changes.

[0126] Preferably, key parameters include: loss function curve, accuracy index and gradient distribution.

[0127] Preferably, the training monitoring unit can present the model convergence status and potential problems in real time, adjust the training strategy based on the changes and presentation of key parameters, and ensure that the model training moves steadily towards the expected goal.

[0128] The model deployment module is used to select the optimal model based on lightweight model evaluation and encapsulate it as a service. It is also used to declaratively define the model service, adjust the resources required for the model service, losslessly publish the encapsulated model, and automatically recover in case of failure.

[0129] Preferably, Figure 4 As shown, the model deployment module includes: a model evaluation encapsulation unit, a statement definition unit, a resource optimization unit, and a release and fault recovery unit;

[0130] The model evaluation and packaging unit is used to determine the optimal model based on lightweight model evaluation and to package the optimal model as a service. The model and its dependencies are packaged into a Docker container image that complies with the OCI specification to ensure the consistency and reliability of the model's operation in any compatible environment.

[0131] Preferably, evaluating and determining the optimal model specifically includes: generating different candidate models through different optimization methods (such as NAS and hyperparameter optimization), and after these candidate models are trained, a validation data set is obtained based on the final labeled data, and the validation data set is tested, and the model with the highest score is selected as the optimal model.

[0132] The declaration definition unit is used to connect to the Kubernetes cluster, declaratively define the model service through the Helm Chart in YAML format or the Kubernetes native Manifest file, and describe the deployment target state in code.

[0133] Preferably, declarative definition means that you only need to tell the model "what to do" without telling it "how to do it". For example, SQL is a declarative language. The biggest advantage of declarative definition is that it is developer-friendly and can be used without in-depth understanding of the relevant technical implementation details.

[0134] Preferably, the deployment target state is described in code form to ensure the transparency and traceability of the deployment process; the Manifest file saved in the YAML file can trace the specific content of each release, thus achieving traceability; Transparency: You only need to declare what to do, without worrying about the details, so the details are transparent to users.

[0135] The resource optimization unit is used to realize automatic expansion and reduction of model service instances and allocate and adjust various resources required for model services.

[0136] Preferably, the resource optimization unit combines Kubernetes' Horizontal Pod Autoscaler (HPA) and Vertical Pod Autoscaler (VPA) to achieve automatic expansion and contraction of model service instances to cope with changes in resource demand caused by business traffic fluctuations; by working closely with the Kubernetes scheduler, the CPU, GPU and other resources required for the model service are reasonably allocated and adjusted to ensure maximum resource utilization efficiency.

[0137] The publishing and fault recovery unit is used to losslessly publish the encapsulated model based on the distribution strategy, achieve smooth switching between the new and old model versions based on the inspection mechanism, and automatically restore to the previous stable model version in case of failure.

[0138] Preferably, the release strategy adopts canary release, blue-green deployment or other release strategies to ensure that the user experience is not interrupted during the service update.

[0139] Preferably, by setting preStop hooks, health check probes, and Pod readiness check mechanisms, smooth switching between the old and new model versions is ensured, and automatic recovery to the previous stable model version is achieved in the event of a failure, ensuring business continuity and stability.

[0140] Preferably, the model deployment module further includes: a model management unit and a service registration adjustment unit;

[0141] The model management unit is used to customize the Kubernetes Operator, which is responsible for the full life cycle management of the model service in the Kubernetes cluster and is used for model version control and rolling updates.

[0142] Preferably, full lifecycle management includes creation, updating, extension and cancellation of model service instances.

[0143] Preferably, the model management unit senses the state changes of the model service based on the Operator listening to CRD (Custom Resource Definition) to achieve automatic model version control and rolling update. For some personalized declaration cycle tasks, they are delegated to Kubernetes Operator.

[0144] Preferably, the monitoring process is as follows: the deployer modifies the Manifest file in the YAML format mentioned above and submits it to the CRD; the Kubernetes operator monitors the CRD, learns about the changes in the Manifest file, and adjusts the deployment according to the changes.

[0145] The service registration adjustment unit is used to integrate the model service into the service grid system after it is released, automatically register and discover the service, and ensure the grayscale release and smooth transition of the new model service through intelligent routing and service weight adjustment.

[0146] Preferably, the service registration adjustment unit is based on Istio or Linkerd service mesh technology. The model service is integrated into the service mesh system after release, and automatically performs service registration and discovery, as well as fine-grained flow control, circuit breaking, and retry strategies.

[0147] Preferably, through intelligent routing and service weight adjustment, the grayscale release and smooth transition of new model services are ensured without affecting the continuity of online business.

[0148] Preferably, grayscale release: a release method that smoothly transitions between new and old versions of the software.

[0149] Preferably, smooth transition: refers to a strategy that ensures that user access is not affected or the impact is minimized during service upgrades or changes; it requires maintaining service continuity and stability during the upgrade process to avoid users from perceiving obvious service interruptions or performance degradation.

[0150] Example 3

[0151] like Figure 5 As shown, a model training deployment method includes:

[0152] Import the original data and preprocess the original data to obtain the final data;

[0153] Annotate the final data to obtain initial annotated data, monitor the annotation quality of the initial annotated data, adjust abnormal annotated data, and obtain final annotated data;

[0154] The final labeled data is processed in parallel, the optimal optimizer, model structure and optimal model parameters are selected, the model structure is optimized based on the optimal optimizer and the optimal model parameters to obtain a new model structure, the new model structure is trained based on the final labeled data to obtain a trained model, and the model is compressed to obtain a lightweight model;

[0155] Based on lightweight model evaluation, the optimal model is selected and service-oriented encapsulation is performed. The model service is declaratively defined, the resources required for the model service are adjusted, the encapsulated model is losslessly published, and it is automatically restored in case of failure.

[0156] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a model training deployment system and method, which has the following beneficial effects:

[0157] 1. Integration and convenience: The one-stop service of the system of the present invention reduces the complexity brought by cross-system operations. Users do not need to switch between different tools and environments, saving time and energy. From raw data import, labeling, model training to the final model launch, the entire process is completed on one system, greatly simplifying the implementation path of AI projects.

[0158] 2. Efficiency improvement: The data labeling of the present invention has the characteristics of automation and intelligence. It uses the model for rough labeling in advance and provides efficient labeling tools and preview functions, which helps to improve the quality and speed of labeling. The automated training process adopts the optimal training parameter configuration and resource scheduling, reduces manual intervention, and makes model training more efficient and stable.

[0159] 3. Collaboration and management optimization: The present invention facilitates the division of labor among team members through multi-user collaborative annotation and project management functions, while supporting annotation quality review and feedback to ensure the quality consistency of the data set; the training process is transparent and the training effect is monitored in real time, which facilitates developers to quickly adjust the optimization strategy and accelerate the iteration and upgrade of the model.

[0160] 4. Model performance and service quality: The integration of best practices and preset model templates, combined with optimized training strategies, is conducive to obtaining high-performance AI models; the rapid deployment and update mechanism enables the model to be seamlessly transitioned to the actual application environment once training is completed, thereby improving the product's speed to market and service continuity.

[0161] 5. Cost saving and resource utilization: Through unified resource management, computing resources are reasonably allocated, hardware investment costs are reduced, and resource waste is avoided; data and models are centrally managed and easy to reuse, which reduces the cost of repeated labeling and training and promotes the sustainable development of AI projects.

[0162] 6. The system of the present invention realizes the standardization, automation and efficiency of the AI ​​project development process, which not only improves work efficiency and model performance, but also brings faster product iteration cycle and higher input-output ratio to enterprises.

[0163] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0164] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model training deployment system, characterized in that: include: Data management module, annotation review module, model training module and model deployment module; The data management module is used to import raw data and pre-process the raw data to obtain final data; The annotation review module is used to perform annotation based on the final data to obtain initial annotation data, and to monitor the annotation quality of the initial annotation data, adjust abnormal annotation data, and obtain final annotation data; The model training module is used to process the final labeled data in parallel, select an optimal optimizer, a model structure and optimal model parameters, optimize the model structure based on the optimal optimizer and the optimal model parameters to obtain a new model structure, train the new model structure based on the final labeled data to obtain a trained model, and compress the model to obtain a lightweight model; The model deployment module is used to select the optimal model based on lightweight model evaluation and perform service packaging. It is also used to declaratively define the model service, adjust the resources required for the model service, losslessly publish the packaged model, and automatically recover in case of failure.

2. A model training deployment system according to claim 1, characterized in that: The data management module includes: a data import unit, a data processing unit and a data association unit; The data import unit is used to batch import data in different forms from different data sources, and perform data deduplication processing when importing the data, and the imported data after processing is used as the original data; The data processing unit is used to perform data enhancement based on the original data to obtain enhanced data; The data association unit is used to build a unified data label system, and associate metadata information with the enhanced data based on the data label system to obtain the final data.

3. A model training deployment system according to claim 2, characterized in that: The data management module further includes: a data storage unit; The data storage unit is used to store the original data and the final data through a distributed file system.

4. A model training deployment system according to claim 1, characterized in that: The annotation review module includes: an automatic annotation unit, a collaborative annotation unit and an annotation monitoring unit; The automatic annotation unit is used to annotate the final data based on a semi-automatic annotation technology driven by deep learning to obtain coarsely annotated data; The collaborative labeling unit is used to distribute the final data to be labeled to multiple labelers for collaborative labeling, and the labelers perform manual labeling or labeling based on the rough labeling data obtained by the automatic labeling unit, and integrate the labeled data to obtain initial labeling data; The annotation monitoring unit is used to establish an annotation quality evaluation system, automatically review the initial annotation data according to the annotation quality evaluation system, and manually review and adjust the controversial annotation data to obtain the final annotation data.

5. A model training deployment system according to claim 1, characterized in that: The model training module includes: a computing resource scheduling unit, an optimizer determination unit, a model structure determination unit, a model training unit and a model compression unit; The computing resource scheduling unit is used to process the final labeled data in parallel based on a heterogeneous computing cluster architecture, and to achieve computing resource balance through a load balancing algorithm and a dynamic resource allocation strategy; The optimizer determination unit is used to select the optimal optimizer according to the model architecture and training data characteristics, and introduce a collaborative training framework; The model structure determination unit is used to automatically determine the model structure based on reinforcement learning, evolutionary algorithm or Bayesian optimization, and determine the optimal hyperparameter based on a hyperparameter tuning algorithm; The model training unit is used to optimize the model structure based on the optimal optimizer and the optimal model parameters to obtain a new model structure, and train the new model structure based on the final labeled data to obtain a trained model; The model compression unit is used to reduce the accuracy of training data and compress the trained model to obtain the lightweight model.

6. A model training deployment system according to claim 5, characterized in that: The model training module also includes: a supervised learning unit; The supervised learning unit is used to perform self-learning and in-depth mining of feature representation based on a small amount of labeled data, unlabeled data or inaccurately labeled data input into the model.

7. A model training deployment system according to claim 5, characterized in that: The model training module further includes: a training monitoring unit; The training monitoring unit is used to monitor and track the changes of key parameters in the model training process in real time, and adjust the training strategy in real time based on the changes.

8. A model training deployment system according to claim 1, characterized in that: The model deployment module includes: a model evaluation encapsulation unit, a declaration definition unit, a resource optimization unit, and a publishing and fault recovery unit; The model evaluation and packaging unit is used to determine the optimal model based on the lightweight model evaluation, and to package the optimal model into a service; The declarative definition unit is used to connect to the Kubernetes cluster, declaratively define the model service through the Helm Chart in YAML format or the Kubernetes native Manifest file, and describe the deployment target state in code form; The resource optimization unit is used to realize automatic expansion and contraction of the model service instance and allocate and adjust various resources required for the model service; The publishing and fault recovery unit is used to losslessly publish the encapsulated model based on the distribution strategy, realize smooth switching between the new and old model versions based on the inspection mechanism, and automatically recover to the previous stable model version in case of failure.

9. A model training deployment system according to claim 8, characterized in that: The model deployment module also includes: a model management unit and a service registration adjustment unit; The model management unit is used to customize the Kubernetes Operator, which is responsible for the full life cycle management of the model service in the Kubernetes cluster and is used for model version control and rolling updates; The service registration adjustment unit is used to integrate the model service into the service grid system after the model service is released, automatically register and discover the service, and ensure the grayscale release and smooth transition of the new model service through intelligent routing and service weight adjustment.

10. A model training deployment method, characterized in that: include: Importing raw data and preprocessing the raw data to obtain final data; Annotating the final data to obtain initial annotated data, monitoring the annotation quality of the initial annotated data, adjusting abnormal annotated data, and obtaining final annotated data; The final labeled data are processed in parallel, an optimal optimizer, a model structure and optimal model parameters are selected, the model structure is optimized based on the optimal optimizer and the optimal model parameters to obtain a new model structure, the new model structure is trained based on the final labeled data to obtain a trained model, and the model is compressed to obtain a lightweight model; Based on lightweight model evaluation, the optimal model is selected and service-oriented packaging is performed, the model service is declaratively defined, the resources required for the model service are adjusted, the packaged model is losslessly published, and it is automatically restored in case of failure.

Citation Information

Cited By

  • Automatic learning engine device based on packaging large model training platform

    CN120066523A