Development, operation and maintenance platform of large language model and model development and deployment method

Abstract

The invention discloses a development, operation and maintenance platform of a large language model and a model development and deployment method, and the development, operation and maintenance platform adopts a front-end and rear-end separation architecture design, and comprises a front end part and a rear end part, wherein the front end comprises a data set uploading interface, a model training interface, a model evaluation interface, a model deployment interface, a model-adapter merging interface and a model tracing interface; and the rear end comprises a data set management function module, a data set uploading function module, a model training function module, a model evaluation function module, a model deployment function module, a model-adapter merging function module and a model management function module. The data set uploading interface is used for uploading the data set for training and evaluation, the model training interface is used for model training to improve the model performance, the model evaluation interface is used for evaluating the capabilities of the model in different aspects, the model deployment interface is used for providing services for model deployment, and the model development and deployment work is completed.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to an integrated large language model development and operation platform and a model development and deployment method. Specifically, it relates to a model development and operation platform that provides large language model training, evaluation, deployment, dataset management and model management functions, and a method for using the platform to perform model training, evaluation and deployment. Background Art

[0002] In recent years, large language models such as LLaMA, Qwen, and DeepSeek have demonstrated powerful capabilities in multiple fields, including natural language understanding, code generation, and conversational interaction. However, many challenges remain when applying large language models to specific production areas:

[0003] 1. Fragmented toolchain: Existing technologies separate model development from operations and maintenance, leaving training, evaluation, and deployment independent. This results in the need for manual coordination of workflows from model development to deployment. Furthermore, this fragmented toolchain also leads to a lack of unified model version control.

[0004] 2. Dataset Management: Existing dataset management methods lack adaptability to training methods, leading to a disconnect between training data and model training methods. Different training methods often require datasets in different formats, and using mismatched datasets for training can lead to training failures.

[0005] 3. Model Deployment Limitations: Current model deployment methods generally have several pain points. Lightweight frameworks are suitable for deploying small models, but cannot support high-concurrency production environments. Enterprise-level engine configuration methods are complex and require high user requirements.

[0006] 4. Disconnection between data and models: Existing methods lack the association management between models and datasets and model version management, which makes it difficult to trace the model and reproduce the model training process.

[0007] Therefore, there is an urgent need for a model development and operation platform that can integrate the entire process from model training to deployment, provide convenient data set management, flexibly deploy models, and achieve full-link traceability of the model training process, as well as a simple and fast model development and deployment method to reduce the difficulty of implementing large language models in specific production fields. Summary of the Invention

[0008] To reduce the difficulty of implementing large language models in specific production areas, this paper provides a large language model development and operations platform and model development and deployment method that integrates the entire model training and deployment process, provides convenient dataset management, enables flexible model deployment, and enables full traceability of the model training process. This paper supports the training, evaluation, deployment, dataset management, and model management functions of large language models.

[0009] The purpose of the present invention is achieved through the following technical solutions:

[0010] A development and operation platform for large language models, designed with a front-end and back-end separation architecture, consisting of two parts:

[0011] The front end includes a dataset upload interface, a model training interface, a model evaluation interface, a model deployment interface, a model-adapter merging interface, and a model traceability interface;

[0012] The backend includes a dataset management function module, a dataset upload function module, a model training function module, a model evaluation function module, a model deployment function module, a model-adapter merging function module and a model management function module;

[0013] The backend has multiple large language models built in for training, and uses a MySQL database to implement persistent storage of data set information, model information, and its training link information. Each functional module interacts with the frontend through a RESTful API interface.

[0014] The dataset upload interface supports users to perform visual drag-and-drop dataset upload and dataset information configuration operations, can interact with the dataset management function module to check whether a dataset exists, and can interact with the dataset upload function module to upload the dataset;

[0015] The model training interface supports users to visually select a model training method, select a model from a model list, select a data set from a data set list, and configure model training parameters. It can interact with the model management function module to obtain a model list, can interact with the data set management function module to obtain a data set list that meets the training method format requirements, and can interact with the model training function module to implement model training and query the training task status.

[0016] The model evaluation interface supports users to visually select models from the model list, select adapters from the adapter list, select data sets from the data set list and configure model evaluation parameters. It can interact with the model management function module to obtain a model list and a list of adapters for a specified model, can interact with the data set management function module to obtain a list of data sets that meet the requirements, and can interact with the model evaluation function module to implement model evaluation and query the evaluation task status.

[0017] The model deployment interface supports users to visually select models to be deployed using Ollama or vLLM, select models from the model list, select adapters from the adapter list, and configure model deployment parameters. It can interact with the model management function module to obtain a model list and a list of adapters for a specified model, and can interact with the model deployment function module to implement model deployment.

[0018] The model-adapter merging interface supports users to visually select models from the model list and adapters from the adapter list, can interact with the model management function module to obtain the model list and the adapter list of the specified model, and can interact with the model-adapter merging function module to realize the merging of models and adapters;

[0019] The model traceability interface supports users to visually select models from the model list, and can interact with the model management function module to obtain the model list and the process link of model training;

[0020] The dataset management function module supports obtaining dataset lists and dataset information from the database, and can process requests for obtaining dataset lists from the front-end page;

[0021] The dataset upload function module supports receiving and storing datasets, verifying data formats, and adding dataset information to the database, and can process dataset upload requests from the front-end page;

[0022] The model training function module supports creating subprocesses to perform model training tasks, monitoring training task status, and adding model training information to the database, and can handle model training requests and training task status query requests on the front-end page;

[0023] The model evaluation function module supports creating subprocesses to perform model evaluation tasks, monitoring evaluation task status, and adding model evaluation information to the database, and can handle model evaluation requests and evaluation task status query requests on the front-end page;

[0024] The model deployment function module supports creating a subprocess to call Ollama or vLLM to perform model deployment tasks and add the deployed model information to the database, and can handle model deployment requests on the front-end page;

[0025] The model-adapter merging function module can create a sub-process to merge the model and the adapter and add the model merging information to the database, and can process the model and adapter merging request of the front-end page;

[0026] The model management function module can retrieve the database to obtain the model list, specify the model adapter list and model source information, and can process the front-end page's request to obtain the model list, request to obtain a certain model adapter and model tracing request.

[0027] A method for model training and deployment using the above-mentioned model development and operation platform includes the following steps:

[0028] Step S1: The user completes dataset name verification, type selection, and file upload through the dataset upload interface. The backend verifies the data format according to the type and stores the data and records the dataset information in the MySQL dataset table.

[0029] Step S2: The user selects the training method, basic model, and matching dataset through the model training interface. The backend creates a training subprocess and generates a task identifier task_id. After the training is completed, the training results are stored in the MySQL model table or adapter table. The frontend periodically polls the / model_train / task_status interface to obtain the training status.

[0030] Step S3: The user selects the model, adapter, and evaluation dataset through the model evaluation interface. The backend starts the evaluation subprocess and generates a task identifier, task_id. After the evaluation is completed, the evaluation results are stored in the MySQL model evaluation result table. The frontend periodically polls the / model_eval / task_status interface to obtain the evaluation status.

[0031] Step S4: The user selects the deployment framework Ollama or vLLM through the model deployment interface. The backend calls the corresponding engine to deploy the model and records it in the MySQL deployed model table.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] 1. Various tools are integrated into a single platform: Through a front-end and back-end separation architecture, training, evaluation, and deployment are integrated into a single platform. The back-end provides a modular interface, and the front-end connects workflows through a visual interface. This approach simplifies the model development process and facilitates model and dataset maintenance.

[0034] 2. Convenient dataset management: This invention supports datasets in various formats, verifies data formats, and automatically associates them with corresponding training methods to avoid training failures due to data-method mismatch.

[0035] 3. Alleviating the limitations of model deployment: The present invention meets different application scenarios through the collaboration of Ollama and vLLM.

[0036] 4. Introducing data-model association: This invention designs full-link traceability of data and models, which can reproduce the model training process.

[0037] The present invention can complete the model development and deployment work by uploading the data set for training and evaluation through the data set upload interface, using the model training interface to perform model training to improve model performance, evaluating the different aspects of the model capabilities through the model evaluation interface, and providing model deployment services through the model deployment interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the components of the large language model development and operation platform;

[0039] Figure 2 This is a diagram of the interaction between the modules of the large language model development and operation platform;

[0040] Figure 3 This is the interaction sequence diagram of the model dataset upload function;

[0041] Figure 4 It is a timing diagram of the interaction between model training functions;

[0042] Figure 5 It is the interaction sequence diagram of the model evaluation function;

[0043] Figure 6 It is a model deployment function interaction sequence diagram. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is further described below with reference to the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention that does not depart from the spirit and scope of the technical solution of the present invention should be included in the scope of protection of the present invention.

[0045] The present invention provides a development and operation platform for a large language model. The development and operation platform adopts a front-end and back-end separation architecture design, and its component modules are as follows: Figure 1 The interaction between different modules is shown in Figure 2 As shown, it includes two parts: the front end and the back end, where:

[0046] The front-end supports user operations and interacts with the back-end to complete function execution, including the following modules:

[0047] Dataset Upload Interface: This interface allows users to upload datasets required for model training and evaluation. The dataset upload interface supports uploading pre-trained datasets, supervised fine-tuning datasets, preference datasets, and KTO datasets. Before uploading a dataset, users must select the dataset type and assign a unique name. The interface detects changes in dataset names in real time and sends a GET request with the dataset name field dataset_name to the backend dataset management module's service interface / dataset_manage / dataset_exist to check if the dataset name exists. A return value of "Dataset exists" indicates that the dataset name exists and cannot be used and needs to be modified. A return value of "Dataset not found" indicates that the dataset name does not exist and can be used as a unique dataset name. After determining the dataset type and unique name, the frontend can upload the file and send a POST request to the backend dataset upload module's service interface / dataset_upload / {kind} to upload the dataset and submit the dataset name. If the uploaded dataset is a pre-trained dataset, the value "kind" is pretrain. If the uploaded dataset is a supervised fine-tuning dataset, the value "kind" is sft. If the uploaded dataset is a preference dataset, the value "kind" is preference. If the uploaded dataset is a KTO dataset, the kind value is kto. If the interface / dataset_upload / {kind} returns "success", it means the dataset has been uploaded successfully.

[0048] Model Training Interface: This interface provides users with a front-end interface for training models to improve specific aspects of the model's capabilities. The model training interface supports incremental pre-training, supervised fine-tuning, direct preference optimization, simple preference optimization, and KTO (Known To Optimization) methods. When loading, the front-end page sends a GET request to the back-end model management module service interface ( / model_manage / model_list) to obtain a list of models. After the user selects a training method, the front-end page sends a GET request to the back-end dataset management module service interface ( / dataset_manage / dataset_list) with the training method field "kind" to obtain a list of available datasets. When using incremental pre-training, "kind" takes the value "pretrain" to obtain pre-trained datasets. When using supervised fine-tuning, "kind" takes the value "sft" to obtain a list of supervised fine-tuning datasets. When using direct preference optimization or simple preference optimization, "kind" takes the value "preference" to obtain a list of KTO datasets. When using KTO, "kind" takes the value "kto" to obtain a list of KTO datasets. After selecting the training model, dataset, and model training parameters, the user sends a POST request to the backend model training module service interface / model_train / {method} with the model name, dataset name, and model training parameters to train the model. If the / model_train / {method} service executes normally, the task_id field will be returned as the training task identifier, which can be used to access the task execution status. After starting a training task, the frontend page will set a timer to periodically send a POST request to the backend model training module service interface / model_train / task_status with the task_id field to obtain the execution status of the corresponding training task. When the backend model training task completes, the / model_train / task_status interface will return the model training results.

[0049] Model Evaluation Interface: This front-end interface provides users with model evaluation capabilities to assess specific aspects of a model's capabilities. The model evaluation interface supports evaluating the model's natural language generation performance using metrics such as BLEU and ROUGE. Upon loading, the front-end page sends a GET request to the back-end model management module service interface / model_manage / model_list to obtain a list of models. It also sends a GET request with the kind field set to sft to the back-end dataset management module service interface / dataset_manage / dataset_list to obtain a list of datasets available for model evaluation. After the user selects a model for evaluation, the front-end page sends a GET request to the back-end dataset management module service interface / model_manage / model_adapter_list with the model name field model_name to obtain a list of available adapters for the specified model. The user selects a model, model adapter, and evaluation dataset, and configures model evaluation parameters for model evaluation. The front-end page then sends a POST request to the back-end model evaluation module service interface / model_eval / NLG with the model name, adapter name, evaluation dataset name, and evaluation parameters for model evaluation. When the / model_eval / NLG service executes normally, it returns the task_id field, which serves as the evaluation task identifier and is used to access the task's execution status. After starting an evaluation task, the front-end page sets a timer to periodically send a POST request to the / model_eval / task_status service interface of the back-end model evaluation module, carrying the task_id field, to obtain the execution status of the corresponding evaluation task. When the back-end model evaluation task completes, the / model_eval / task_status interface returns the model evaluation results.

[0050] Model Deployment Interface: This interface provides users with model deployment capabilities, allowing them to deploy models and provide external services. The interface supports model deployment in both Ollama and vLLM. Upon loading, the front-end page sends a GET request to the back-end model management module service interface / model_manage / model_list to obtain a list of models. After the user selects a model to deploy, the front-end page sends a GET request to the back-end dataset management module service interface / model_manage / model_adapter_list with the model name field model_name to retrieve a list of available adapters for the specified model. After selecting a model, adapter, and deployment tool, the user configures model deployment parameters and deploys the model. The front-end page sends a POST request to the back-end model deployment module service interface / model_deploy / ollama or / model_deploy / vllm with the model name, adapter name, and corresponding model deployment parameters to deploy the model. If the model deploys successfully, the front-end page will receive a message indicating successful deployment.

[0051] Model-Adapter Merge Interface: The Model-Adapter Merge Interface is a front-end interface that provides the Model-Adapter Merge function, merging a model and its adapter into a single model. When the front-end page loads, it sends a GET request to the back-end Model Management Function Module service interface / model_manage / model_list to obtain a list of models. After the user selects a base model, the front-end page sends a GET request to the back-end Dataset Management Function Module service interface / model_manage / model_adapter_list with the model name field model_name to obtain a list of available adapters for the specified model. After the user selects a model and adapter, the model and adapter are merged. The front-end page sends a POST request to the back-end Model-Adapter Merge Function Module service interface / merge with the model name and adapter name to merge the model and adapter. If the model and adapter merge is complete, the front-end page will receive a message indicating that the model-adapter merge was successful.

[0052] Model Tracing Interface: This front-end interface provides users with model traceability capabilities, allowing them to trace the model training chain. This interface supports tracing the origin of a model, including the initial model, intermediate models, datasets used in each training step, and training parameters. Upon loading, the front-end interface sends a GET request to the back-end model management module service interface ( / model_manage / model_list) to obtain a list of models. After the user selects a base model, the front-end interface sends a GET request to the back-end dataset management module service interface ( / model_manage / trace) with the model name for model tracing. If the function executes normally, the back-end service interface returns information about the model training chain.

[0053] The backend supports receiving frontend requests and performing corresponding operations, including the following modules:

[0054] Dataset Management Module: This module is a backend functional module that provides front-end interfaces with functions for querying dataset lists and determining the existence of dataset names. The dataset management module provides the service interface / dataset_manage / dataset_list to handle GET requests for dataset lists. Upon receiving a request, the module extracts the kind field in the request. If the kind field is empty, it retrieves all dataset names from the MySQL dataset table and returns a list of all dataset names. If the kind field is not empty, it retrieves all tuples in the MySQL dataset table whose kind field value matches the requested kind and extracts the dataset name field. The dataset table fields include id, dataset_name, kind, and num. If the kind value is not empty, it can be pretrain, sft, preference, or kto. The dataset management module provides the service interface / dataset_manage / dataset_exist to handle GET requests for checking the existence of a dataset name. Upon receiving a request, the module extracts the dataset_name field and checks whether a tuple matching the dataset_name field exists in the MySQL dataset table. If so, the response is "Dataset exists"; otherwise, the response is "Dataset not found."

[0055] Dataset Upload Module: This module is a backend function module that provides dataset upload services for front-end pages. The module provides the service interface / dataset_upload / {kind} for processing POST requests for dataset uploads. Upon receiving the request, it extracts the dataset file, dataset_name, and kind. The file's content is validated against the kind. If it does not meet the requirements, it is deleted and a dataset format error message is returned. If it meets the requirements, the dataset entry num is recorded and a record containing dataset_name, kind, and num is added to the MySQL dataset table, along with a unique id. Finally, the dataset is stored in the specified location.

[0056] Model Training Module: This module is a backend module that provides model training functionality for the front-end. It provides a service interface, / model_train / {method}, to receive POST requests for model training. Upon receiving the request, it extracts the model_name, dataset_name, and learning_rate fields, creates a child process to call the llama-factory for model training, and creates a task_id to identify the task process, an output field to collect process output, an error field to record process error information, and a status field to indicate the task's running status. The module then returns the task_id to the front-end. The task_id, output, error, and status fields are added to the train_tasks dictionary. Task_id is used as the key, and output, error, and status are used as the values. A status value of "completed" indicates task completion, "failed" indicates task failure, and "running" indicates task execution. Upon successful task execution, the module records the trained base model, training method, dataset used, configured training parameters, and generated artifacts in MySQL. If the artifact is an adapter, a record is added to the model adapter table; if the artifact is a model, a record is added to the model table. The model training module provides a service interface, / model_train / task_status, for receiving POST requests to query the training task status. After receiving the request, the module extracts the task_id field and retrieves its corresponding status from train_tasks. If the status is complete, the model training results are returned; if it is failed, the training failure information is returned; if it is running, the task is running.

[0057] Model Evaluation Module: This module is a backend module that provides model evaluation services to the frontend. The module provides the service interface / model_eval / NLG to receive POST requests for model evaluation. Upon receiving the request, it extracts the model_name, dataset_name, and top_p fields, creates a child process to call llama-factory for model evaluation, and creates a task_id to identify the task process, output to collect process output, error to record process error information, and status to identify the task's running status. The module then returns the task_id to the frontend. The task_id, output, error, and status fields are added to the eval_tasks dictionary. Task_id serves as the keyword, and output, error, and status serve as the values. A status value of "completed" indicates that the task is complete; a value of "failed" indicates that the task has failed; and a value of "running" indicates that the task is in progress. Upon successful task execution, the module records the model, dataset, evaluation parameters, and results in the model evaluation results table in MySQL. The model evaluation module provides a service interface, / model_eval / task_status, for receiving POST requests to query the evaluation task status. After receiving the request, the module extracts the task_id field and retrieves its corresponding status from train_tasks. If the status is complete, the model evaluation result is returned; if it is failed, the evaluation failure information is returned; if it is running, the task is still executing.

[0058] Model Deployment Module: This module is a backend module that provides model deployment services for the frontend. The module provides the service interfaces / model_deploy / ollama and / model_deploy / vllm to process POST requests for model deployment to Ollama or vLLM. Upon receiving the request, the module extracts the model_name and adapter_name fields, creates a child process to call Ollama or vLLM to deploy the model, and adds the model name, adapter name, and deployment tool information to the deployed models table in MySQL. Upon completion, a successful model deployment message is returned to the frontend.

[0059] Model-Adapter Merge Functional Module: This module is a backend functional module that provides model and adapter merging services for the front-end interface. The module provides a service interface, / merge, for processing POST requests for merging models and adapters. Upon receiving the request, the module extracts the model_name and adapter_name fields, creates a child process, and calls llama-factory to merge the model and adapter. The module retrieves adapter information from the MySQL model adapter table and stores the trained base model, training method, dataset used, configured training parameters, and model_name in the MySQL model table for model traceability. After the operation is completed, the module returns the merge completion information to the front-end.

[0060] Model Management Module: This module is a backend functional module that provides services to the front-end page for obtaining a model list, a list of adapters for a specific model, and a training link for a specific model. The module provides the service interface / model_manage / model_list to handle GET requests for obtaining a model list. Upon receiving this request, the module retrieves all model names from the MySQL model table and returns it to the front-end. The module also provides the service interface / model_manage / trace to handle GET requests for tracing a model training link. Upon receiving this request, the module extracts the model_name field. The model then performs the following operations: 1. Creates a return result array and adds a data item containing only model_name to the result array. 2. Query the MySQL model table to retrieve model information for the model named model_name, including the base model, training method, dataset, and configured training parameters. 3. If the model originates from another model, a data item containing the base model, training method, dataset, and configured training parameters is added to the result array, and the base model name is assigned to model_name, and step 2 is executed. If the model originates from the platform itself, step 4 is executed. 4. The result array is returned to the front-end, and execution ends. The model management function module provides a service interface / model_manage / model_adapter_list for processing GET requests for obtaining a certain model adapter. After receiving the request, the module extracts the model_name field and retrieves all adapters of the model_name model from MySQL and returns them to the front end.

[0061] The present invention also provides a method for training, evaluating, and deploying a large language model using the large language model development and operation platform, the method comprising the following steps:

[0062] Step S1: The user completes the dataset name verification, type selection and file upload through the dataset upload interface. The backend verifies the data format according to the type and stores it and records the dataset information to the MySQL dataset table. The user configures the dataset information and uploads the dataset to the backend by dragging and dropping. Figure 3 As shown in the figure, the specific steps are as follows: the user enters the dataset name on the dataset upload interface, and the front-end sends a real-time GET request to the / dataset_manage / dataset_exist interface with the dataset_name parameter. The dataset management module queries the MySQL dataset table to verify the uniqueness of the dataset name and returns the verification result: Dataset exists (name conflict) or Dataset not found (name available). If the name is available, the user selects the dataset type (pretrain, sft, preference, or kto), and the front-end uploads the file to the / dataset_upload / {kind} interface via a POST request. The back-end performs data format verification based on the type. If verification passes, the record is inserted into the MySQL dataset table (including the fields dataset_name, kind, and num) and returns a success status code. The front-end displays a successful upload prompt.

[0063] Step S2: The user selects the training method, basic model and matching data set through the model training interface. The backend creates a training subprocess and generates a task identifier task_id. After the training is completed, the training results are stored in the MySQL model table or adapter table. The frontend periodically polls the / model_train / task_status interface to obtain the training status. Figure 4 As shown, the specific steps are as follows: The user opens the model training interface. The front-end calls

[0064] / model_manage / model_list retrieves a list of all models and calls / dataset_manage / dataset_list to retrieve the corresponding dataset based on the training method. The user selects: training method (method), base model, dataset, and learning rate parameter. The frontend sends a POST request to the / model_train / {method} interface. The backend creates a training task (generates task_id), starts a child process that calls llama-factory for training, records the task status in the train_tasks dictionary, and returns task_id. The frontend periodically polls the / model_train / task_status interface, and the backend returns the task status (running, completed, or failed). After training is complete, the training information is recorded in the MySQL model table / adapter table, and the frontend displays the training results.

[0065] In this step, the method for obtaining the corresponding data set according to the training method is as follows:

[0066] When the training method is incremental pre-training, automatically filter the dataset with kind=pretrain;

[0067] When the training method is instruction-supervised fine-tuning, automatically filter the dataset with kind=sft;

[0068] When the training method is preference optimization, the dataset with kind=preference is automatically selected;

[0069] When the training method is KTO, the dataset with kind=kto is automatically filtered.

[0070] Step S3: The user selects the model, adapter, and evaluation dataset through the model evaluation interface. The backend starts the evaluation subprocess and generates a task identifier task_id. After the evaluation is completed, the evaluation results are stored in the MySQL model evaluation result table. The frontend periodically polls the / model_eval / task_status interface to obtain the evaluation status. Figure 5 As shown in the figure, the specific steps are as follows: The user opens the model evaluation interface. The front-end loads the model list through the / model_manage / model_list interface and the SFT dataset list through / dataset_manage / dataset_list?kind=sft. After the user selects the model to be evaluated, the front-end calls / model_manage / model_adapter_list to obtain the adapter list for that model. The user selects the adapter and dataset and configures the evaluation parameters. The front-end sends a POST request to the / model_eval / NLG interface. The back-end starts a child process that calls llama-factory to perform the evaluation task, generates a task_id, records the task in eval_tasks, and returns the task_id. The front-end periodically polls / model_eval / task_status to query the task status. After the task completes, the evaluation results are written to the MySQL evaluation result table, the task status is changed to completed, and the evaluation results are added to the data item corresponding to task_id. After the task completes, the front-end obtains the task completion status and evaluation result information through / model_eval / task_status.

[0071] Step S4: The user selects the deployment framework (Ollama or vLLM) through the model deployment interface, and the backend calls the corresponding engine to deploy the model and records it in the MySQL deployed model table, such as Figure 6As shown in the figure, the specific steps are as follows: When the user opens the model deployment interface, the front-end accesses / model_manage / model_list to load the model list. After the user selects a model, the front-end accesses / model_manage / model_adapter_list to load the adapter list. The user selects the deployment platform (Ollama or vLLM) and configures key parameters before executing the deployment. The front-end sends a POST to / model_deploy / ollama or / model_deploy / vllm. The back-end calls the Ollama create command through a child process to create a model service or starts the vLLM API service process and adds the deployed model information to MySQL for subsequent management. After the deployment is complete, the module returns a deployment success message to the front-end. After that, users can perform unified model service configuration through OneAPI.

Claims

1. A large language model development and operation platform, characterized by The development and operation platform adopts a front-end and back-end separation architecture design, including two parts: the front-end and the back-end. The front end includes a dataset upload interface, a model training interface, a model evaluation interface, a model deployment interface, a model-adapter merging interface, and a model traceability interface; The backend includes a dataset management function module, a dataset upload function module, a model training function module, a model evaluation function module, a model deployment function module, a model-adapter merging function module and a model management function module; The backend has multiple large language models built in for training, and uses a MySQL database to implement persistent storage of data set information, model information, and its training link information. Each functional module interacts with the frontend through a RESTful API interface. The dataset upload interface supports users to perform visual drag-and-drop dataset upload and dataset information configuration operations, can interact with the dataset management function module to check whether a dataset exists, and can interact with the dataset upload function module to upload the dataset; The model training interface supports users to visually select a model training method, select a model from a model list, select a data set from a data set list, and configure model training parameters. It can interact with the model management function module to obtain a model list, can interact with the data set management function module to obtain a data set list that meets the training method format requirements, and can interact with the model training function module to implement model training and query the training task status. The model evaluation interface supports users to visually select models from the model list, select adapters from the adapter list, select data sets from the data set list and configure model evaluation parameters. It can interact with the model management function module to obtain a model list and a list of adapters for a specified model, can interact with the data set management function module to obtain a list of data sets that meet the requirements, and can interact with the model evaluation function module to implement model evaluation and query the evaluation task status. The model deployment interface supports users to visually select models to be deployed using Ollama or vLLM, select models from the model list, select adapters from the adapter list, and configure model deployment parameters. It can interact with the model management function module to obtain a model list and a list of adapters for a specified model, and can interact with the model deployment function module to implement model deployment. The model-adapter merging interface supports users to visually select models from the model list and adapters from the adapter list, can interact with the model management function module to obtain the model list and the adapter list of the specified model, and can interact with the model-adapter merging function module to realize the merging of models and adapters; The model traceability interface supports users to visually select models from the model list, and can interact with the model management function module to obtain the model list and the process link of model training; The dataset management function module supports obtaining dataset lists and dataset information from the database, and can process requests for obtaining dataset lists from the front-end page; The dataset upload function module supports receiving and storing datasets, verifying data formats, and adding dataset information to the database, and can process dataset upload requests from the front-end page; The model training function module supports creating subprocesses to perform model training tasks, monitoring training task status, and adding model training information to the database, and can handle model training requests and training task status query requests on the front-end page; The model evaluation function module supports creating subprocesses to perform model evaluation tasks, monitoring evaluation task status, and adding model evaluation information to the database, and can handle model evaluation requests and evaluation task status query requests on the front-end page; The model deployment function module supports creating a subprocess to call Ollama or vLLM to perform model deployment tasks and add the deployed model information to the database, and can handle model deployment requests on the front-end page; The model-adapter merging function module can create a sub-process to merge the model and the adapter and add the model merging information to the database, and can process the model and adapter merging request of the front-end page; The model management function module can retrieve the database to obtain the model list, specify the model adapter list and model source information, and can process the front-end page's request to obtain the model list, request to obtain a certain model adapter and model tracing request.

2. The large language model development and operation platform according to claim 1 is characterized in that The dataset upload interface is a front-end page for providing users with the function of uploading datasets required for model training and evaluation. The dataset upload interface supports uploading pre-trained datasets, instruction-supervised fine-tuning datasets, preference datasets, and KTO datasets. Before uploading a dataset, users need to select the type of dataset and give it a unique name; The front-end interface will detect changes in the dataset name in real time and send a GET request to the service interface / dataset_manage / dataset_exist of the back-end dataset management function module, carrying the dataset name field dataset_name to check whether the dataset name exists. The return information "Dataset exist" indicates that the dataset name exists and cannot be used and needs to be modified. The return result "Dataset notfound" indicates that the dataset name does not exist and can be used as a unique dataset name. After the dataset type and unique dataset name are determined, the front-end uploads the file and sends a POST request to the service interface / dataset_upload / {kind} of the back-end dataset upload function module to upload the dataset and submit the dataset name information. When the uploaded dataset is a pre-training dataset, the value of kind is pretrain. When the uploaded dataset is an instruction-supervised fine-tuning dataset, the value of kind is sft. When the uploaded dataset is a preference dataset, the value of kind is preference. When the uploaded dataset is a KTO dataset, the value of kind is kto. The interface / dataset_upload / {kind} returns the information "success", indicating that the dataset has been uploaded successfully. The model training interface is a front-end interface that provides users with training models to improve certain aspects of the model's capabilities. The model training interface supports incremental pre-training methods, instruction-supervised fine-tuning methods, direct preference optimization methods, simple preference optimization methods, and KTO methods. When loading, the front-end page sends a GET request to the back-end model management function module service interface / model_manage / model_list to obtain a model list. After the user selects a training method, the front-end page sends a GET request to the back-end dataset management function module service interface / dataset_manage / dataset_list and carries the training method field kind to obtain a list of available datasets. When the incremental pre-training method is used, kind takes the value pretrain to obtain a pre-trained dataset. When the instruction-supervised fine-tuning method is used, kind takes the value sft to obtain a list of instruction-supervised fine-tuning datasets. When the direct preference optimization method or the simple preference optimization method is used, kind takes the value preference. Get, when using the KTO method, kind takes the value kto to obtain the KTO dataset list; after the user selects the training model, the dataset to be used and configures the model training parameters, a POST request is sent to the back-end model training function module service interface / model_train / {method} with the model name, dataset name and model training parameters to train the model; if the service of the interface / model_train / {method} is executed normally, the task_id field will be returned as the identifier of the training task for accessing the task execution status; after starting the training task, the front-end page will set a timer to send a POST request to the service interface / model_train / task_status of the back-end model training function module at regular intervals with the task_id field to obtain the execution status of the corresponding training task; when the back-end model training task is completed, the / model_train / task_status interface will return the model training result.

3. The large language model development and operation platform according to claim 1 is characterized in that The model evaluation interface is a front-end interface that provides users with a model evaluation function to evaluate a certain aspect of the model's capabilities. The model evaluation interface supports the evaluation of the model's natural language generation effect, and the evaluation indicators include BLEU and ROUGE. When the front-end page is loaded, it sends a GET request to the back-end model management function module service interface / model_manage / model_list to obtain a model list and sends a GET request to the back-end dataset management function module service interface / dataset_manage / dataset_list with the field kind set to sft to obtain a list of datasets that can be used for model evaluation. After the user selects an evaluation model, the front-end page sends a GET request to the back-end dataset management function module service interface / model_manage / model_adapter_list with the model name field model_name to obtain the available adapters for the specified model. Adapter list; the user selects the model, model adapter and evaluation data set and configures the model evaluation parameters for model evaluation. The front-end page sends a POST request to the back-end model evaluation function module service interface / model_eval / NLG and carries the model name, adapter name, evaluation data set name and evaluation parameters for model evaluation; the interface / model_eval / NLG service is executed normally and the task_id field is returned as the identifier of the evaluation task for accessing the task execution status; after starting the evaluation task, the front-end page will set a timer to send a POST request to the back-end model evaluation function module service interface / model_eval / task_status at regular intervals and carry the task_id field to obtain the execution status of the corresponding evaluation task; when the back-end model evaluation task is completed, the interface / model_eval / task_status will return the model evaluation result; The model deployment interface is a front-end interface that provides users with model deployment functions to deploy models and provide services externally. The model deployment interface supports deploying models in Ollama and in vLLM. When loading, the front-end page sends a GET request to the back-end model management function module service interface / model_manage / model_list to obtain a model list. After the user selects the model to be deployed, the front-end page sends a GET request to the back-end dataset management function module service interface / model_manage / model_adapter_list and carries the model name field model_name to obtain a list of available adapters for the specified model. After the user selects the model, adapter and deployment tool, configure the model deployment parameters for model deployment. The front-end page sends a POST request to the back-end model deployment function module service interface / model_deploy / ollama or / model_deploy / vllm and carries the model name, adapter name and corresponding model deployment parameters for model deployment. If the model is deployed normally, the information model deployment success will be returned to the front-end page.

4. The large language model development and operation platform according to claim 1 is characterized in that The model-adapter merging interface is a front-end interface that provides a model-adapter merging function to merge a model and its adapter into one model. When the front-end page is loaded, it sends a GET request to the back-end model management function module service interface / model_manage / model_list to obtain a list of models. After the user selects a basic model, the front-end page sends a GET request to the back-end dataset management function module service interface / model_manage / model_adapter_list and carries the model name field model_name to obtain a list of available adapters for the specified model. After the user selects a model and an adapter, the model and the adapter are merged. The front-end page sends a POST request to the back-end model-adapter merging function module service interface / merge and carries the model name and the adapter name to merge the model and the adapter. If the model and the adapter are merged, the information "Model-adapter merge successful" will be returned to the front-end page. The model tracing interface is a front-end interface that provides users with a model traceability function to trace the model training link. The model tracing interface supports tracing the source of the model, including the initial model, intermediate models, and the data sets and training parameters used in each training step. When the front-end page is loaded, it sends a GET request to the back-end model management function module service interface / model_manage / model_list to obtain a model list. After the user selects a basic model, the front-end page sends a GET request to the back-end data set management function module service interface / model_manage / trace with the model name for model tracing. If the function is executed normally, the back-end service interface returns the model training link information.

5. The large language model development and operation platform according to claim 1 is characterized in that The dataset management function module is a backend function module that provides a front-end interface with a query dataset list and determines whether a dataset name exists. The dataset management function module provides a service interface / dataset_manage / dataset_list for processing a GET request to obtain a dataset list. After receiving the request, the kind field in the request is extracted. When the kind field value is empty, the dataset table in MySQL is retrieved to obtain all dataset names and a list of all dataset names is returned. When kind is not empty, all tuples whose kind field values in the MySQL dataset table match the requested kind are retrieved and the dataset name field is extracted and returned. The fields of the dataset table include id, dataset_name, kind, and num. When the kind value is not empty, it is pretrain, sft, preference, or kto. The dataset management function module provides a service interface / dataset_manage / dataset_exist for processing a GET request to check whether a dataset name exists. After receiving the request, the dataset_name field is extracted and the MySQL dataset table is retrieved to determine whether a tuple matching the dataset_name field exists. If so, Dataset exists is returned; otherwise, Dataset not found is returned. The dataset upload module is a backend module that provides dataset upload services for the front-end page. It provides a service interface, / dataset_upload / {kind}, for processing POST requests for dataset uploads. Upon receiving the request, it extracts the dataset file, dataset_name, and kind. It then verifies the file's content based on the kind. If it doesn't meet the requirements, it deletes it and returns a dataset format error message. If it meets the requirements, it records the dataset entry num and adds a record containing dataset_name, kind, and num to the MySQL dataset table, along with a unique id. Finally, it stores the dataset in the specified location.

6. The large language model development and operation platform according to claim 1 is characterized in that The model training function module is a back-end function module that provides model training functions for the front-end page. The model training function module provides a service interface / model_train / {method} to receive a POST request for model training. After receiving the request, the model_name, dataset_name, and learning_rate fields are extracted, a child process is created to call llama-factory for model training, and task_id is created to identify the task process, output is used to collect process output, error is used to record process error information, and status is used to identify the task running status, and task_id is returned to the front-end. task_id, output, error, and status will be added to the train_tasks dictionary, where task_id is the keyword and output, error, and status are the values. When the status value is completed, it means the task is completed; when it is failed, it means the task failed; and when it is running, it means the task is being executed. After the task is successfully executed, the module records the basic model, training method, used dataset, configured training parameters, and generated products in MySQL. If the product is an adapter, a record is added to the model adapter table; if the product is a model, a record is added to the model table. The model training function module provides a service interface / model_train / task_status for receiving POST requests for querying the status of training tasks. After receiving the request, the task_id field is extracted and its corresponding status is retrieved from train_tasks. If the status is complete, the model training result is returned; if it is failed, the training failure information is returned; if it is running, the task is running. The model evaluation function module is a back-end function module that provides model evaluation services for the front-end. The model evaluation function module provides a service interface / model_eval / NLG to receive a POST request for model evaluation. After receiving the request, it extracts the model_name, dataset_name, and top_p fields, creates a sub-process to call llama-factory for model evaluation, and creates task_id for identifying the task process, output for collecting process output, error for recording process error information, and status for identifying the task running status, and returns task_id to the front-end; task_id, output, error, and status will be added to The eval_tasks dictionary uses task_id as the key and output, error, and status as the values. A status value of "completed" indicates task completion, "failed" indicates task failure, and "running" indicates task execution. Upon successful task execution, the module records the model, dataset, evaluation parameters, and evaluation results in the model evaluation result table in MySQL. The model evaluation module provides a service interface, / model_eval / task_status, for receiving POST requests to query the evaluation task status. Upon receiving the request, it extracts the task_id field and retrieves its corresponding status from train_tasks. A status of "complete" returns the model evaluation result, a "failed" returns the evaluation failure information, and a "running" returns the task execution status.

7. The large language model development and operation platform according to claim 1, characterized in that The model deployment function module is a back-end function module that provides model deployment services for the front-end page. The model deployment function module provides service interfaces / model_deploy / ollama and / model_deploy / vllm for processing POST requests for model deployment to Ollama or vLLM. After receiving the request, the module extracts the model_name and adapter_name fields, creates a child process to call Ollama or vLLM to deploy the model and adds the model name, adapter name and deployment tool information to the deployed model table in MySQL. After the deployment is completed, the model deployment success information is returned to the front-end; The model-adapter merging function module is a back-end function module that provides model and adapter merging services for the front-end interface. The model-adapter merging function module provides a service interface / merge for processing POST requests for merging models and adapters. After receiving the request, the module extracts the model_name and adapter_name fields, creates a child process to call llama-factory to merge the model and adapter; the module obtains the adapter information from the MySQL model adapter table and stores the basic model, training method, used data set, configured training parameters and model_name in the MySQL model table for model traceability; after the operation is completed, the module returns the merge completion information to the front-end.

8. A method for model development and deployment using the development and operation platform according to any one of claims 1 to 7, characterized in that The method comprises the following steps: Step S1: The user completes dataset name verification, type selection, and file upload through the dataset upload interface. The backend verifies the data format according to the type and stores the data and records the dataset information in the MySQL dataset table. Step S2: The user selects the training method, basic model, and matching dataset through the model training interface. The backend creates a training subprocess and generates a task identifier task_id. After the training is completed, the training results are stored in the MySQL model table or adapter table. The frontend periodically polls the / model_train / task_status interface to obtain the training status. Step S3: The user selects the model, adapter, and evaluation dataset through the model evaluation interface. The backend starts the evaluation subprocess and generates a task identifier, task_id. After the evaluation is completed, the evaluation results are stored in the MySQL model evaluation result table. The frontend periodically polls the / model_eval / task_status interface to obtain the evaluation status. Step S4: The user selects the deployment framework Ollama or vLLM through the model deployment interface. The backend calls the corresponding engine to deploy the model and records it in the MySQL deployed model table.

9. The model development and deployment method according to claim 8, characterized in that In step S2, the method for obtaining the corresponding data set according to the training method is as follows: When the training method is incremental pre-training, automatically filter the dataset with kind=pretrain; When the training method is instruction-supervised fine-tuning, automatically filter the dataset with kind=sft; When the training method is preference optimization, the dataset with kind=preference is automatically selected; When the training method is KTO, the dataset with kind=kto is automatically filtered.

Images