Fine tuning system and method based on deep learning model
Through fine-tuning systems and methods based on deep learning models, the existing technology problems of high resource dependence, difficulty in data fitting and model customization are solved, and the goal of efficiently training large models in the customized field is achieved.
Patent Information
- Application Number
- CN202411559621.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2025-01-21
AI Technical Summary
Training of existing deep learning models requires a large amount of computing resources and data, resulting in high resource dependence, data fitting problems and model customization difficulties.
Propose a fine-tuning system and method based on deep learning models, reduce computing and storage costs through modularization and parameter efficient fine-tuning technology, use fine-tuning a small number of parameters to reduce data fitting risks, build domain training data, and train large teams with domain expert capabilities. Model.
It achieves efficient training of large models in customized fields while reducing computing time and resource usage, reducing fitting risks and improving model performance in specific tasks.
Smart Images
Figure CN119337927A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of large model technology, and in particular to a fine-tuning system and method based on a deep learning model. Background Art
[0002] In the field of artificial intelligence, deep learning models usually require a lot of computing resources and data to train. The models have hundreds of millions to hundreds of billions of parameters. Although full training on large-scale data can obtain strong generalization capabilities, this process usually requires a long time of computing and storage resources. Therefore, many developers choose to use pre-trained models and then optimize them for specific tasks through fine-tuning technology, but there are the following problems: 1. High resource dependence: Full training requires long-term computing and storage resources, which is too expensive and difficult to run for small teams or individual users who do not have access to a large number of GPU resources; 2. Data fitting problem: Traditional fine-tuning is usually performed on small-scale domain datasets, which may cause the model to overfit the training data and lose its generalization ability; 3. Model customization problem: Pre-trained models are usually trained based on general data and therefore may not perform optimally in specific tasks. Summary of the invention
[0003] The purpose of the present invention is to propose a fine-tuning system and method based on a deep learning model to address the above-mentioned technical problems. Based on cutting-edge deep learning methods, it makes full use of open source technology to make improvements, reduces high computing and storage costs through modularization and parameter-efficient fine-tuning technology, uses a small number of parameter fine-tuning methods to reduce data fitting risks, constructs domain training data, and trains large models with domain expert capabilities through advanced basic general large models.
[0004] A fine-tuning system based on a deep learning model, comprising a data set management module, a model square module, a my model module, and a model fine-tuning module; The data set management module defines Prompt+Response, Prompt+Multi-Response, Plain Text, Promp Set, Prompt-chosen-rejected data types, performs versioned storage on data, incrementally imports data, exports data, and publishes data sets; The model square module manages the basic large model; The My Model module manages the domain model generated by fine-tuning; The model fine-tuning module demonstrates the visual model fine-tuning function, fine-tunes different basic large models, selects different data set versions, sets the resource usage for fine-tuning, and controls the output of the fine-tuned model.
[0005] Furthermore, a fine-tuning system based on a deep learning model, wherein the model fine-tuning module includes a data set selection module, a basic model selection module, a parameter setting module, a resource setting module, a fine-tuning mode module, and a model publishing module; The data set selection module selects the data set defined by the data set management module; The basic model selection module selects the basic large model defined by the model square module; The parameter setting module adjusts the parameters of the model to cope with a variety of different fine-tuning; The resource setting module sets the storage space used by the model, provides a view of the current GPU usage, and ensures that resources are controllable; The storage space includes CPU, memory, number of GPUs, and GPU memory; The fine-tuning module provides different model fine-tuning methods such as POST_PRETRAING, SFT and DPO; The model publishing module processes the fine-tuned model and selects whether to include it in the management of the My Model module.
[0006] Further, a fine-tuning system based on a deep learning model, wherein the data management module includes a HAMI database and a MongoDB database; The HAMI database manages the middleware of heterogeneous devices in Kubernetes, manages different types of heterogeneous devices, shares heterogeneous devices between Pods, makes scheduling decisions based on the topology information and scheduling policies of the devices, eliminates the gaps between different heterogeneous devices, and provides users with a unified management interface. The MongoDB database uses the Binary JSON format to store data and is a non-relational database used to process large amounts of unstructured or semi-structured data.
[0007] Further, in a fine-tuning system based on a deep learning model, the data management module includes a device reuse capability submodule and a device resource isolation capability submodule; The device reuse capability submodule applies for computing power equipment and hard isolation of computing power resources by specifying video memory, and applies for computing power equipment by specifying computing power usage ratio, so as to achieve zero modification to existing programs; The device resource isolation capability submodule supports hard isolation of device resources.
[0008] Further, a fine-tuning system based on a deep learning model, the MongoDB database includes a document storage submodule, a dynamic mode submodule, an extensibility submodule, a query language submodule, a built-in replication submodule, a storage engine submodule, an index support submodule, a transaction support submodule, and a community support submodule; The document storage submodule implements MongoDB database to store data in the form of documents. Each document is a collection of key-value pairs, which is used to store hierarchical data without pre-defining the entire structure to add new fields. The dynamic mode submodule implements the table structure of the MongoDB database to retain documents, and different documents have completely different structures; The scalability submodule supports horizontal expansion and distributes data across multiple servers through sharding technology; The query language submodule provides complex aggregation operations and full-text search query functions; The built-in replication submodule realizes data redundancy through replica sets to ensure data security and system availability; The storage engine submodule enables users to select different storage engines according to their needs; The storage engines include WiredTiger and MMAPv1; The index supports submodules to create indexes for any field in a document and composite indexes for arrays and nested documents; The transaction support submodule allows atomic operations to be performed across multiple documents, collections, and databases; The community support submodule supports community resources and tools, and supports commercial versions by providing professional technologies and services.
[0009] A fine-tuning method based on a deep learning model, comprising the following sub-steps: S1: Management basic model; S11: Model upload: distinguish whether it is a basic large model; S12, weight upload: upload the fine-tuned model weights and select the corresponding base model; S2: Fine-tune the basic large model; S3: Release the new model.
[0010] Furthermore, a fine-tuning method based on a deep learning model, said S2 comprises the following sub-steps: S21: Get the dataset version; S22: Obtaining a basic large model; S23: Select fine adjustment mode; S24: Setting fine-tuning parameters, model type, and fine-tuning method; S25: Set fine-tuning resources, including CPU, memory, number of GPUs, GPU memory, shared memory, and storage space used by the model; S26: Record the fine-tuning process log; S27: The generated model is stored in a shared file system for sharing in a distributed system.
[0011] Furthermore, a fine-tuning method based on a deep learning model, said S3 comprises the following sub-steps: S31: The fine-tuned model is generated as a new model; S32: The fine-tuned model is generated as a new version of the original model; S33: All models are stored in a version management manner.
[0012] The beneficial effects of the present invention are as follows: through a fine-tuning system and method based on a deep learning model, the resource configuration for fine-tuning of a large model is defined, and only some modules are adjusted instead of the entire model, thereby reducing computing time and resource usage; through modularization and PEFT methods, only some parameters of the model are fine-tuned to reduce fitting risks; large models in customized fields are efficiently trained, and efficient training goals are achieved by applying low-rank adaptation, 3D parallelism, FlashAttention, mixed precision, and zero-redundancy optimization technologies. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 It is a structural diagram of a fine-tuning system based on a deep learning model.
[0014] Figure 2 It is a flowchart of a fine-tuning method based on deep learning models. DETAILED DESCRIPTION
[0015] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described with reference to the accompanying drawings.
[0016] As attached Figure 1 As shown, a fine-tuning system based on a deep learning model includes a data set management module, a model square module, a my model module, and a model fine-tuning module; The data set management module defines Prompt+Response, Prompt+Multi-Response, Plain Text, Promp Set, Prompt-chosen-rejected data types, performs versioned storage on data, incrementally imports data, exports data, and publishes data sets; The model square module manages the basic large model; The My Model module manages the domain model generated by fine-tuning; The model fine-tuning module demonstrates the visual model fine-tuning function, fine-tunes different basic large models, selects different data set versions, sets the resource usage for fine-tuning, and controls the output of the fine-tuned model.
[0017] Furthermore, a fine-tuning system based on a deep learning model, wherein the model fine-tuning module includes a data set selection module, a basic model selection module, a parameter setting module, a resource setting module, a fine-tuning mode module, and a model publishing module; The data set selection module selects the data set defined by the data set management module; The basic model selection module selects the basic large model defined by the model square module; The parameter setting module adjusts the parameters of the model to cope with a variety of different fine-tuning; The resource setting module sets the storage space used by the model, provides a view of the current GPU usage, and ensures that resources are controllable; The storage space includes CPU, memory, number of GPUs, and GPU memory; The fine-tuning module provides different model fine-tuning methods such as POST_PRETRAING, SFT and DPO; The model publishing module processes the fine-tuned model and selects whether to include it in the management of the My Model module.
[0018] Further, a fine-tuning system based on a deep learning model, wherein the data management module includes a HAMI database and a MongoDB database; The HAMI database manages the middleware of heterogeneous devices in Kubernetes, manages different types of heterogeneous devices, shares heterogeneous devices between Pods, makes scheduling decisions based on the topology information and scheduling policies of the devices, eliminates the gaps between different heterogeneous devices, and provides users with a unified management interface. The MongoDB database uses the Binary JSON format to store data and is a non-relational database used to process large amounts of unstructured or semi-structured data.
[0019] Further, in a fine-tuning system based on a deep learning model, the data management module includes a device reuse capability submodule and a device resource isolation capability submodule; The device reuse capability submodule applies for computing power equipment and hard isolation of computing power resources by specifying video memory, and applies for computing power equipment by specifying computing power usage ratio, so as to achieve zero modification to existing programs; The device resource isolation capability submodule supports hard isolation of device resources.
[0020] Further, a fine-tuning system based on a deep learning model, the MongoDB database includes a document storage submodule, a dynamic mode submodule, an extensibility submodule, a query language submodule, a built-in replication submodule, a storage engine submodule, an index support submodule, a transaction support submodule, and a community support submodule; The document storage submodule implements MongoDB database to store data in the form of documents. Each document is a collection of key-value pairs, which is used to store hierarchical data without pre-defining the entire structure to add new fields. The dynamic mode submodule implements the table structure of the MongoDB database to retain documents, and different documents have completely different structures; The scalability submodule supports horizontal expansion and distributes data across multiple servers through sharding technology; The query language submodule provides complex aggregation operations and full-text search query functions; The built-in replication submodule realizes data redundancy through replica sets to ensure data security and system availability; The storage engine submodule enables users to select different storage engines according to their needs; The storage engines include WiredTiger and MMAPv1; The index supports submodules to create indexes for any field in a document and composite indexes for arrays and nested documents; The transaction support submodule allows atomic operations to be performed across multiple documents, collections, and databases; The community support submodule supports community resources and tools, and supports commercial versions by providing professional technologies and services.
[0021] As attached Figure 2 As shown, a fine-tuning method based on a deep learning model includes the following sub-steps: S1: Management basic model; S11: Model upload: distinguish whether it is a basic large model; S12, weight upload: upload the fine-tuned model weights and select the corresponding base model; S2: Fine-tune the basic large model; S3: Release the new model.
[0022] Furthermore, a fine-tuning method based on a deep learning model, said S2 comprises the following sub-steps: S21: Get the dataset version; S22: Obtaining a basic large model; S23: Select fine adjustment mode; S24: Setting fine-tuning parameters, model type, and fine-tuning method; S25: Set fine-tuning resources, including CPU, memory, number of GPUs, GPU memory, shared memory, and storage space used by the model; S26: Record the fine-tuning process log; S27: The generated model is stored in a shared file system for sharing in a distributed system.
[0023] Furthermore, a fine-tuning method based on a deep learning model, said S3 comprises the following sub-steps: S31: The fine-tuned model is generated as a new model; S32: The fine-tuned model is generated as a new version of the original model; S33: All models are stored in a version management manner.
[0024] Specific embodiment 1 Model fine-tuning module based on LlamaFactory model The LlamaFactory model is a large language model (LLM) fine-tuning framework that aims to achieve higher efficiency by optimizing the performance of large models, especially during inference and fine-tuning.
[0025] The core technology of the LlamaFactory model is divided into the following parts: Multimodal model integration LlamaFactory usually supports processing multimodal data, such as text, images, audio, etc., so that the fine-tuned model can perform well in multiple domain tasks. Through a modular architecture, users can easily expand the input source of the model and adapt to different data types; Efficient fine-tuning LlamaFactory uses lightweight adaptive fine-tuning, such as Low-Rank Adaptation of LLMs technology, to train large language models. LoRA allows the fine-tuning process to train and update only a small number of parameters without adjusting all parameters of the model, significantly reducing the computational cost and resource requirements of fine-tuning; Distributed training LlamaFactory model optimizes the training efficiency of the model through distributed architecture, and can perform large-scale parallel training on multiple GPUs or even multiple nodes. By optimizing communication strategies and memory management, the training process is accelerated and hardware bottlenecks are reduced. LlamaFactory integrates Prompt Tuning technology, which quickly guides the output of the model by inputting specific prompts, avoiding the adjustment of a large number of parameters in traditional fine-tuning, making the fine-tuning process of LlamaFactory more flexible and scalable.
[0026] Specific embodiment 2 The MinIO object storage system that the data set management module relies on: MinIO is a high-performance object storage system that is compatible with the Amazon S3 API, which means it can be seamlessly integrated with widely used cloud storage services. MinIO is designed for private cloud and edge computing environments, can handle large-scale data sets, and supports a variety of data protection mechanisms, such as Erasure Coding to ensure high availability and persistence of data.
[0027] The main features are: High performance: MinIO can achieve very high throughput and low latency on standard hardware, which is particularly suitable for large-scale data applications that require fast read and write speeds, such as machine learning and big data analysis; Compatible: Fully compatible with the Amazon S3 API, which means that all tools, applications, and services that use S3 can be directly connected to MinIO. This compatibility makes it easy to migrate existing S3-based workloads. Distributed architecture: MinIO can be deployed as a single node or a distributed multi-node cluster. In distributed mode, data redundancy is provided through erasure coding, ensuring that data is not lost even if multiple nodes fail. Security: Supports TLS / SSL encrypted transmission to ensure data security during transmission, and provides authentication and authorization mechanisms, including policy-based access control. Supports server-side encryption, which can encrypt data stored in MinIO; Easy to deploy and manage: MinIO can be easily deployed in containerized environments such as Kubernetes. It provides command line tools and graphical interfaces for management and monitoring; Cross-platform support: It supports multiple operating systems such as Linux, Windows, and macOS. It can also run on the ARM architecture, which is suitable for edge computing scenarios; Open Source: MinIO is open source software, released under the GNU Affero General Public License v3.0 (AGPLv3).
[0028] The open source community is active, with a large number of contributors and support resources; Rich plug-ins and tools: The community provides many tools and plug-ins for operations such as backup, synchronization, and migration. It supports monitoring tools such as Prometheus and Grafana to facilitate performance monitoring and alarm settings.
[0029] Specific embodiment 3 The model fine-tuning module relies on Qwen1.5-14B-Chat: Qwen-1.5-14B-Chat is a version in the Qwen series of large-scale language models launched by Alibaba Cloud. It is optimized for dialogue and chat scenarios. This model is based on the Transformer architecture and has 14 billion parameters. It aims to provide high-quality natural language understanding and generation capabilities and is suitable for a variety of application scenarios.
[0030] It has the following features: Large-scale parameters: With 14 billion parameters, Qwen-1.5-14B-Chat can capture complex language patterns and subtle semantic differences. A large number of parameters helps improve the model's ability to understand and generate long texts. Conversation optimization: The model is specially trained to handle multi-turn conversations and excels at maintaining contextual coherence, being able to understand and respond to user questions or statements while maintaining the fluency and relevance of the conversation; Breadth and depth of knowledge: The training data covers a wide range of subject areas, from general knowledge to professional knowledge. Through extensive pre-training and fine-tuning, the model can demonstrate a deep knowledge background on different topics. Multi-language support: Qwen series models usually support multiple languages. Although they are mainly optimized for Chinese, they can also handle input and output of English and some other languages, which makes them more flexible and practical in international applications; Security and Compliance: We adhere to strict data privacy protection principles, take security and compliance requirements into consideration during design, take measures to avoid generating harmful, false or biased content, and strive to ensure the security and reliability of output.
[0031] This solution defines the resource configuration for fine-tuning large models through a fine-tuning system and method based on deep learning models, and only adjusts some modules instead of the whole model, reducing computing time and resource usage; only fine-tunes some parameters of the model through modularization and PEFT methods to reduce fitting risks; and efficiently trains large models in customized fields, achieving efficient training goals by applying low-rank adaptation, 3D parallelism, FlashAttention, mixed precision, and zero-redundancy optimization technologies.
[0032] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A fine-tuning system based on a deep learning model, characterized in that: Including data set management module, model square module, my model module, model fine-tuning module; The data set management module defines Prompt+Response, Prompt+Multi-Response, Plain Text, Promp Set, Prompt-chosen-rejected data types, performs versioned storage on data, incrementally imports data, exports data, and publishes data sets; The model square module manages the basic large model; The My Model module manages the domain model generated by fine-tuning; The model fine-tuning module demonstrates the visual model fine-tuning function, fine-tunes different basic large models, selects different data set versions, sets the resource usage for fine-tuning, and controls the output of the fine-tuned model.
2. A fine-tuning system based on a deep learning model according to claim 1, characterized in that: The model fine-tuning module includes a data set selection module, a basic model selection module, a parameter setting module, a resource setting module, a fine-tuning method module, and a model publishing module; The data set selection module selects the data set defined by the data set management module; The basic model selection module selects the basic large model defined by the model square module; The parameter setting module adjusts the parameters of the model to cope with a variety of different fine-tuning; The resource setting module sets the storage space used by the model, provides a view of the current GPU usage, and ensures that resources are controllable; The storage space includes CPU, memory, number of GPUs, and GPU memory; The fine-tuning module provides different model fine-tuning methods such as POST_PRETRAING, SFT and DPO; The model publishing module processes the fine-tuned model and selects whether to include it in the management of the My Model module.
3. A fine-tuning system based on a deep learning model according to claim 2, characterized in that: The data management module includes a HAMI database and a MongoDB database; The HAMI database manages the middleware of heterogeneous devices in Kubernetes, manages different types of heterogeneous devices, shares heterogeneous devices between Pods, makes scheduling decisions based on the topology information and scheduling policies of the devices, eliminates the gaps between different heterogeneous devices, and provides users with a unified management interface. The MongoDB database uses the Binary JSON format to store data and is a non-relational database used to process large amounts of unstructured or semi-structured data.
4. The fine-tuning system based on a deep learning model according to claim 3, characterized in that: The data management module includes a device multiplexing capability submodule and a device resource isolation capability submodule; The device reuse capability submodule applies for computing power equipment and hard isolation of computing power resources by specifying video memory, and applies for computing power equipment by specifying computing power usage ratio, so as to achieve zero modification to existing programs; The device resource isolation capability submodule supports hard isolation of device resources.
5. The fine-tuning system based on a deep learning model according to claim 3, characterized in that: The MongoDB database includes a document storage submodule, a dynamic mode submodule, an extensibility submodule, a query language submodule, a built-in replication submodule, a storage engine submodule, an index support submodule, a transaction support submodule, and a community support submodule; The document storage submodule implements MongoDB database to store data in the form of documents. Each document is a collection of key-value pairs, which is used to store hierarchical data without pre-defining the entire structure to add new fields. The dynamic mode submodule implements the table structure of the MongoDB database to retain documents, and different documents have completely different structures; The scalability submodule supports horizontal expansion and distributes data across multiple servers through sharding technology; The query language submodule provides complex aggregation operations and full-text search query functions; The built-in replication submodule realizes data redundancy through replica sets to ensure data security and system availability; The storage engine submodule enables users to select different storage engines according to their needs; The storage engines include WiredTiger and MMAPv1; The index supports submodules to create indexes for any field in a document and composite indexes for arrays and nested documents; The transaction support submodule allows atomic operations to be performed across multiple documents, collections, and databases; The community support submodule supports community resources and tools, and supports commercial versions by providing professional technologies and services.
6. A fine-tuning method based on a deep learning model, implemented by a fine-tuning system based on a deep learning model according to any one of claims 1 to 5, characterized in that: It includes the following sub-steps: S1: Management basic model; S11: Model upload: distinguish whether it is a basic large model; S12, weight upload: upload the fine-tuned model weights and select the corresponding base model; S2: Fine-tune the basic large model; S3: Release the new model.
7. The fine-tuning method based on a deep learning model according to claim 6, characterized in that: The S2 comprises the following sub-steps: S21: Get the dataset version; S22: Obtaining a basic large model; S23: Select fine adjustment mode; S24: Setting fine-tuning parameters, model type, and fine-tuning method; S25: Set fine-tuning resources, including CPU, memory, number of GPUs, GPU memory, shared memory, and storage space used by the model; S26: Record the fine-tuning process log; S27: The generated model is stored in a shared file system for sharing in a distributed system.
8. The fine-tuning method based on a deep learning model according to claim 6, characterized in that: The S3 comprises the following sub-steps: S31: The fine-tuned model is generated as a new model; S32: The fine-tuned model is generated as a new version of the original model; S33: All models are stored in a version management manner.