Intelligent application development and deployment platform

Abstract

The invention relates to an intelligent application development and deployment platform, and belongs to the technical field of artificial intelligence. The platform comprises training resources, a model development module, a model training module, a model deployment module and a large model agent. According to the method, flexible scheduling of computing resources is realized on the server, universal goods shelves such as a database, a model library, an algorithm library, a lightweight library and an operator library compatible with multiple development frameworks are created, tools such as multi-modal data labeling, model conversion, mirror image packaging and online development are integrated, services such as intelligent labeling and cloud edge collaboration are preset, rapid updating and iteration of models are supported, and the development efficiency is improved. And carrying out one-key deployment on an end-side algorithm. The platform can form a replicable and generalizable product through the combination of a basic type and a specific component, and the platform can be applied to the same row in the industry and has a wide application prospect.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an intelligent application development and deployment platform. Background Technology

[0002] In China's civilian intelligent computing field, a competitive landscape has emerged with numerous vendors such as Alibaba and Huawei. Commercial intelligence service platforms are all built on their respective cloud computing services, meaning users need to store their data on commercial clouds, potentially leading to issues such as data privacy leaks and security breaches. Furthermore, existing research and commercial intelligent middleware and platforms primarily focus on the integration and optimization of R&D resources, with application scenarios mainly concentrated on development collaboration and rapid response to business needs, while support for edge computing and end-device scenarios is relatively limited.

[0003] The current chaotic state of China's intelligent computing ecosystem presents numerous challenges, including inconsistent intelligent application development and deployment processes, significant differences in model training and inference environments, difficulties in migration and deployment, a lack of intelligent model algorithm libraries, cumbersome intelligent framework configurations, and difficulties in implementing intelligent applications due to hardware and software incompatibility, as well as substantial investment costs. There is an urgent need to conduct research on intelligent application support technologies for multiple types of devices, including research on intelligent application development and deployment support, rapid iteration of intelligent applications, and adaptation and migration technologies for multi-chip platforms on intelligent devices. This research should also develop intelligent computing tool libraries, create intelligent application training and inference platforms, build prototype platforms for typical algorithms and application scenarios, consider the development trends of next-generation computing devices, and conduct application verification. The goal is to transform intelligent computing devices from a "closed architecture, hardware and software bundled" to an "open architecture, hardware and software decoupling," supporting comprehensive domestic production and promoting the systematic and intelligent leapfrog development of intelligent application development platforms. Summary of the Invention (a) Technical problems to be solved The technical problem this invention aims to solve is how to provide an intelligent application development and deployment platform to address practical issues such as inconsistent intelligent application development and deployment processes, significant differences in model training and inference environments, difficulties in migration and deployment, lack of intelligent model algorithm libraries, cumbersome intelligent framework configuration, and difficulties in implementing intelligent applications due to software and hardware incompatibility, as well as high investment costs.

[0004] (II) Technical Solution To address the aforementioned technical problems, this invention proposes an intelligent application development and deployment platform, which includes: training resources, a model development module, a model training module, a model deployment module, and a large-scale model agent; Training resources include: file management module, mirror resources, and algorithm repository; the data processing module provides data annotation processing for the file management module; the model development, training, and deployment modules all rely on mirror resources in the mirror repository and various algorithms in the algorithm repository; the model development module can provide new models to the algorithm repository and mirror repository, and also obtain models from the algorithm repository and mirror repository. The data processing module includes five sub-functions: team creation, manual annotation, intelligent annotation, dataset management, and data access. Team collaborative annotation enables multi-person collaborative annotation, assigning annotation tasks to team members as annotators or reviewers, and reviewers verifying the annotations after they are completed. The model development module includes online development and visualization. It provides users with an online algorithm development environment and debugging support within the platform. The online algorithm development environment is an integrated online algorithm development environment based on JupyterLab, offering the powerful file management, code development, debugging, and resource management functions of native JupyterLab. It provides functions for writing notebooks, operating the terminal, editing Markdown text, opening interactive mode, and viewing CSV files and images. It can export code and documents written in .ipynb format to PDF and HTML formats, and associates JupyterLab file paths with the user's file management module through file mapping. The module also introduces the TensorBoard visualization tool, providing scalar visualization and image visualization to display model-related information from different perspectives. Scalar visualization is used to display various scalar data during training. By observing the curves of these scalar changes, users can intuitively understand the progress and effect of model training, determine whether the model is converging normally, and whether training parameters need to be adjusted. The model training module includes single-machine training and distributed training. The model training module receives algorithm, image, and sample data input from training resources, selects different computing power based on deep learning framework and computing type, supports single-machine training mode and distributed training mode, and provides platform-level support for distributed training. The model deployment module has multiple built-in inference engines, integrates KServe, and supports multiple inference frameworks. It allows users to use inference code from various frameworks to complete model inference and verification, and can publish models online to a central server cluster. It also supports users to package their own images. Users can choose the deployment framework according to their business needs and manage the published model services, including adding, querying, starting, terminating, and viewing details. The publishing of model services consists of the model and the runtime environment. Different deep learning frameworks have different runtime environments, and the module supports adding, editing, and querying runtime environments. When using it, users need to enter basic parameters and parameter configurations, select the framework and image, and enter the execution command and service port number. The large-scale model agent deeply integrates large models with IDEs, enabling seamless integration into the development process through IDE plugins. It parses code structure in real time, generates high-quality code, optimizes complex logic, and provides contextual understanding and terminology standardization services. It achieves code completion, automatic code file generation, code optimization and refactoring, code understanding and explanation, and log analysis and location. The large-scale model agent comes pre-loaded with at least two types of code to assist in the development of large models, used for natural language code generation, code continuation, code commenting, code explanation, code correction, code optimization, and generation of unit tests.

[0005] (III) Beneficial Effects This invention proposes an intelligent application development and deployment platform. This platform enables flexible scheduling of computing resources on the server and provides a general-purpose platform compatible with multiple development frameworks, including databases, model libraries, algorithm libraries, lightweight libraries, and operator libraries. It integrates tools for multimodal data annotation, model conversion, image encapsulation, and online development, as well as pre-built intelligent annotation and cloud-edge collaboration services. It supports rapid model updates and iterations, and one-click deployment of edge-side algorithms. This platform can be combined with basic components to form replicable and scalable products, placing it at the forefront of the industry and demonstrating broad application prospects. Attached Figure Description

[0006] Figure 1 This is the overall flowchart of the present invention; Figure 2 This is a flowchart of the resource management module's unit workflow. Figure 3 This is a schematic diagram of Embodiment 1 of the present invention. Detailed Implementation

[0007] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0008] This invention is an intelligent application development and deployment platform, which includes: training resources, model development module, model training module, model deployment module, and large model intelligent agent, etc.

[0009] Training resources include: a file management module, image resources, and an algorithm repository. The composition of training resources and their relationship to other model processes are as follows: Figure 2 As shown. The data processing module provides data annotation and other processing for the file management module. The model development, training and deployment modules all rely on mirror resources in the mirror repository and various algorithms in the algorithm repository. The model development module can provide new models to the algorithm repository and mirror repository, and can also obtain models from the algorithm repository and mirror repository.

[0010] The algorithm repository maintenance platform includes both algorithm libraries and user-defined algorithm libraries. It provides algorithm code management functions for different deep learning frameworks (such as mainstream frameworks like Paddle, PyTorch, TensorFlow, and MindSpore) and application scenarios (such as image classification, segmentation, detection, and keypoint detection), serving as the entry point for subsequent algorithm training. For algorithms newly developed locally, users simply upload their local algorithm folder to the platform's file management, create a user algorithm from the algorithm resources section, select the newly uploaded algorithm folder, and import the algorithm. During training, a new training task is created, listing all hyperparameters involved in the training script in the corresponding locations, and filling in the parameter values ​​and execution commands to complete the configuration. Inference and validation are similar. The algorithm repository is divided into private, shared, and public repositories. Private repositories are private to logged-in users, shared repositories are shared within the logged-in user's organization, and public repositories are shared by all users. The algorithm repository provides a rich, out-of-the-box library of model algorithms, covering core tasks such as classification, detection, segmentation, and tracking based on infrared, visible light, and SAR data; it includes pre-built language, vision, and multimodal large models; it provides at least 5 carefully selected models for each category; and it supports users in quickly creating and managing a consistent development environment.

[0011] The image repository maintains environment images, categorized and managed across multiple dimensions such as deep learning framework type, framework version, underlying platform type, and support for computing resources. This enables the platform to support training and debugging of mainstream algorithms and framework platforms, freeing algorithm developers from the tedious task of environment construction. The Harbor image repository is used to maintain environment images, providing at least four deep learning images, including PyTorch, TensorFlow, PaddlePaddle, and MindSpore. The repository offers pre-built images based on mainstream deep learning frameworks and supports user-defined environments. Environment version control and one-click switching are supported.

[0012] The file management module provides storage and management capabilities for datasets of various types, including images, videos, text, and audio, storing both unlabeled and labeled datasets. This module can categorize and manage data files uploaded to the platform by user through the file management portal according to data type, application scenario, etc., for subsequent labeling; labeled data can be used for model training in the algorithm resource repository.

[0013] The data processing module comprises five sub-functions: team creation, manual annotation, intelligent annotation, dataset management, and data access. Team collaborative annotation enables multi-person collaborative annotation, assigning annotation tasks to team members as annotators or reviewers, with reviewers verifying the annotations completed. The data processing module handles preprocessing and annotation of at least five data types: video, image, text, audio, and time series; it supports multi-person collaborative annotation, annotation review, and includes a pre-built intelligent annotation service, supporting intelligent annotation for object detection.

[0014] B. Model Development Module The model development module includes online development and visualization. It provides users with an online algorithm development environment and debugging support within the platform, enabling them to complete algorithm development work independently of their personal physical machines, relying solely on the platform. The online algorithm development environment is an integrated online algorithm development environment based on JupyterLab, offering the powerful file management, code development, debugging, and resource management functions of native JupyterLab. As a web-based integrated development environment, JupyterLab provides functions such as notebook writing, terminal operation, Markdown text editing, interactive mode access, and viewing CSV files and images. It can export code and documents written in .ipynb format to PDF, HTML, and other formats, and associates JupyterLab file paths with the user's file management module through file mapping.

[0015] The TensorBoard visualization tool is introduced, offering various visualization types such as scalar visualization and image visualization, capable of displaying model-related information from different perspectives. Scalar visualization (SCALARS) is used to display various scalar data during training, such as the model's loss, accuracy, and learning rate, and how these change with training steps or epochs. By observing the curves of these scalar changes, one can intuitively understand the progress and effectiveness of model training, determine whether the model is converging normally, and whether training parameters need to be adjusted. The model development module also includes a pre-training and fine-tuning module, which provides flexible and efficient model training and debugging capabilities. It allows users to manage the lifecycle of various types of training tasks (such as single-machine, distributed, and cloud-edge collaborative training) with a simple interface and view the task status in real time. It also integrates interactive development tools for online algorithm development and visualization of model structure and training parameters.

[0016] C. Model Training Module The model training module includes single-machine training and distributed training. It receives algorithm, image, and sample data inputs from training resources, selects different computing power based on the deep learning framework and computing type, supports both single-machine and distributed training modes, and provides platform-level support for distributed training, reducing development complexity.

[0017] The model training module also includes a lightweight module, which provides various model optimization, lightweighting, and evaluation tools. It includes at least three built-in fine-tuning strategies, such as efficient parameter fine-tuning and at least three model lightweighting strategies (e.g., pruning and distillation).

[0018] D. Model Deployment Module The model deployment module integrates multiple inference engines, including KServe, and supports various inference frameworks such as PaddleInference, PyTorchServing, and TF-Serving. It allows users to complete model inference verification using inference code from different frameworks and provides online deployment services for models on a central server cluster. Users can also create their own images. The module allows users to select a deployment framework based on their business needs and manage the deployed model services, including adding, querying, starting, terminating, and viewing details. Model service deployment consists of the model and its runtime environment, which varies depending on the deep learning framework. The module supports adding, editing, and querying runtime environments. When using the module, users need to input basic parameters and configuration settings, select the framework and image, and enter the execution command and service port number.

[0019] The model deployment module provides components and environment for intelligent algorithm development, and has typical algorithm development capabilities such as online IDE development, visual workflow orchestration, and batch processing jobs. Among them, the workflow component has componentized workflow orchestration capabilities, provides workflow orchestration tools, supports flexible orchestration of annotation, training, evaluation, transformation, packaging, and deployment, and is used for one-click deployment of edge-side algorithms, providing shelf-style management and services for workflows.

[0020] E. Large-scale intelligent agents The Large Model Agent deeply integrates large models with IDEs (such as VSCode and IntelliJ IDEA). Through IDE plugins, the model can be seamlessly integrated into the development process, parsing code structure in real time, generating high-quality code, optimizing complex logic, and providing services such as contextual understanding and terminology standardization. It achieves code completion, automatic code file generation, code optimization and refactoring, code understanding and explanation, and log analysis and location, thereby significantly improving developers' coding efficiency and code quality. The Large Model Agent comes pre-loaded with at least two code assistance mechanisms for developing large models, used for natural language code generation, code continuation, code commenting, code explanation, code correction, code optimization, and generating unit tests.

[0021] Upload data and model files through the file manager, and create a new workflow (test-0910) in the workflow interface. This completes the workflow of sample annotation, model training, model evaluation, model conversion, model packaging, and model deployment. Clicking the one-click deployment button enables rapid model updates and deployment to edge devices.

[0022] This invention creates an intelligent application development and deployment platform that enables flexible scheduling of computing resources on the server. It provides a general-purpose platform compatible with multiple development frameworks, including databases, model libraries, algorithm libraries, lightweight libraries, and operator libraries. It integrates tools for multimodal data annotation, model conversion, image packaging, and online development, as well as pre-built intelligent annotation and cloud-edge collaboration services. It supports rapid model updates and iterations, and one-click deployment of edge-side algorithms. This platform can be combined with basic components to form replicable and scalable products, placing it at the forefront of the industry and promising broad application prospects.

[0023] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A smart application development and deployment platform, characterized in that, The platform includes: training resources, model development module, model training module, model deployment module, and large model agent; Training resources include: file management module, mirror resources, and algorithm repository; the data processing module provides data annotation processing for the file management module; the model development, training, and deployment modules all rely on mirror resources in the mirror repository and various algorithms in the algorithm repository; the model development module can provide new models to the algorithm repository and mirror repository, and also obtain models from the algorithm repository and mirror repository. The data processing module includes five sub-functions: team creation, manual annotation, intelligent annotation, dataset management, and data access. Team collaborative annotation enables multi-person collaborative annotation, assigning annotation tasks to team members as annotators or reviewers, and reviewers verifying the annotations after they are completed. The model development module includes online development and visualization. It provides users with an online algorithm development environment and debugging support within the platform. The online algorithm development environment is an integrated online algorithm development environment based on JupyterLab, offering the powerful file management, code development, debugging, and resource management functions of native JupyterLab. It provides functions for writing notebooks, operating the terminal, editing Markdown text, opening interactive mode, and viewing CSV files and images. It can export code and documents written in .ipynb format to PDF and HTML formats, and associates JupyterLab file paths with the user's file management module through file mapping. The module also introduces the TensorBoard visualization tool, providing scalar visualization and image visualization to display model-related information from different perspectives. Scalar visualization is used to display various scalar data during training. By observing the curves of these scalar changes, users can intuitively understand the progress and effect of model training, determine whether the model is converging normally, and whether training parameters need to be adjusted. The model training module includes single-machine training and distributed training. The model training module receives algorithm, image, and sample data input from training resources, selects different computing power based on deep learning framework and computing type, supports single-machine training mode and distributed training mode, and provides platform-level support for distributed training. The model deployment module has multiple built-in inference engines, integrates KServe, and supports multiple inference frameworks. It allows users to use inference code from various frameworks to complete model inference and verification, and can publish models online to a central server cluster. It also supports users to package their own images. Users can choose the deployment framework according to their business needs and manage the published model services, including adding, querying, starting, terminating, and viewing details. The publishing of model services consists of the model and the runtime environment. Different deep learning frameworks have different runtime environments, and the module supports adding, editing, and querying runtime environments. When using it, users need to enter basic parameters and parameter configurations, select the framework and image, and enter the execution command and service port number. The large-scale model agent deeply integrates large models with IDEs, enabling seamless integration into the development process through IDE plugins. It parses code structure in real time, generates high-quality code, optimizes complex logic, and provides contextual understanding and terminology standardization services. It achieves code completion, automatic code file generation, code optimization and refactoring, code understanding and explanation, and log analysis and location. The large-scale model agent comes pre-loaded with at least two types of code to assist in the development of large models, used for natural language code generation, code continuation, code commenting, code explanation, code correction, code optimization, and generation of unit tests.

2. The intelligent application development and deployment platform as described in claim 1, characterized in that, The algorithm repository maintenance platform includes an algorithm library and a user-defined algorithm library, providing algorithm code management functions for different deep learning frameworks and application scenarios. It serves as the entry point for subsequent algorithm training. For algorithms newly developed locally, users simply need to upload the local algorithm folder to the platform's file management, then create a user algorithm from the algorithm resources section, select the newly uploaded algorithm folder, and complete the algorithm import. During training, users create a new training task, list all hyperparameters involved in the training script in the corresponding locations, and fill in the parameter values ​​and execution commands to complete the configuration.

3. The intelligent application development and deployment platform as described in claim 2, characterized in that, Algorithm repositories are divided into private, shared, and public. Private repositories are private to logged-in users, shared repositories are shared within the organization of the logged-in user, and public repositories are shared by all users.

4. The intelligent application development and deployment platform as described in claim 2, characterized in that, The algorithm repository provides an out-of-the-box library of model algorithms, covering core tasks such as classification, detection, segmentation, and tracking based on infrared, visible light, and SAR data; it includes pre-built large models for language, vision, and multimodal applications; it provides at least 5 carefully selected models for each category; and it supports users in quickly creating and managing a consistent development environment.

5. The intelligent application development and deployment platform as described in claim 1, characterized in that, The image repository maintains environment images, which are divided and managed in multiple dimensions according to deep learning framework type, framework version, basic platform type, and support type of computing resources. The Harbor image repository is used to maintain environment images, providing deep learning images for PyTorch, TensorFlow, PaddlePaddle, and MindSpore.

6. The intelligent application development and deployment platform as described in claim 1, characterized in that, The file management module provides storage and management capabilities for image, video, text, and audio datasets, storing both unlabeled and labeled datasets. This module categorizes and manages data files uploaded to the platform by users through the file management portal according to data type and application scenario for subsequent labeling. The labeled data can be used for model training in the algorithm resource repository.

7. The intelligent application development and deployment platform as described in claim 1, characterized in that, The data processing module is used for preprocessing and annotation of video, image, text, voice, and time series data types; it supports multi-person collaborative annotation, annotation review, pre-built intelligent annotation services, and intelligent annotation for object detection.

8. The intelligent application development and deployment platform as described in claim 1, characterized in that, The model development module also includes a pre-training and fine-tuning module, which provides flexible and efficient model training and debugging capabilities, allowing users to manage the lifecycle of various types of training tasks with one click through the interface and view the task status in real time. It integrates interactive development tools for online algorithm development, enabling visualization of model structure and training parameters.

9. The intelligent application development and deployment platform as described in claim 1, characterized in that, The model training module also includes a lightweight module, which provides a variety of model optimization, lightweighting and evaluation tools, and has built-in a variety of fine-tuning strategies and model lightweighting strategies.

10. The intelligent application development and deployment platform as described in claim 1, characterized in that, The model deployment module provides components and environment for intelligent algorithm development, with online IDE development, visual workflow orchestration, and batch processing capabilities. Among them, the workflow component has componentized workflow orchestration capabilities, provides workflow orchestration tools, supports flexible orchestration of annotation, training, evaluation, transformation, packaging, and deployment, and is used for one-click deployment of edge-side algorithms, providing shelf-style management and services for workflows.

Images