Processing system and method for deploying llm model resources, computer device, and resource invocation method for llm models

Abstract

A processing system and method for deploying LLM model resources, a computer device, and a method for retrieving LLM model resources are disclosed. The processing system includes a configuration server and a computer under test (DUT). The configuration server stores a model list. The DUT has device resources and a quantization adjustment program. The DUT executes the quantization adjustment program on the model under test. The quantized model is the adjusted model. The quantization adjustment program generates a configuration file based on the adjusted model and device resources. The quantization adjustment program transmits the configuration file to the configuration server, and the configuration server adds the model's device information to the model list. The computer device obtains the model list and the corresponding configuration file from the configuration server based on the device resources. The computer device quantizes the LLM model according to the configuration file. The computer device executes a target program based on the quantized LLM model.

Claims

1. A processing system for deploying LLM model resources, comprising: Configure a server and store a list of models; The system includes a computer under test (DUT), a device resource, and a quantization adjustment program. The DUT executes the quantization adjustment program on a model under test (DUT), which includes multiple model cores. The quantization adjustment program selects a quantization precision option and adjusts each model core. The adjusted model core is a quantized core, and the quantized DUT is a quantized model. The quantization adjustment program generates a configuration file based on the quantized model and the device resource. The quantization adjustment program transmits model information of the DUT, the device resource, and the configuration file to a configuration server. The configuration server establishes a model-to-device pair information based on the received model information, device resource, and configuration file, and adds the model-to-device pair information to the model list.

2. The processing system for deploying LLM model resources as described in claim 1, wherein the device resources include a graphics processing memory and a system memory, and the capacity of the graphics processing memory is less than or equal to that of the system memory.

3. The processing system for deploying LLM model resources as described in claim 1, wherein the quantization tuning program selects any one of the tuned cores, and the quantization tuning program selects either the graphics processing memory or the system memory based on the capacity of the selected tuned core, until the tuned cores select the corresponding memory; wherein, Select the tuned cores of the graphics processing memory as a first core group, and select the tuned cores of the system memory as a second core group.

4. The processing system for deploying LLM model resources as described in claim 3, wherein the total capacity of the first core group is less than or equal to the capacity of the graphics processing memory, and the total capacity of the second core group is less than or equal to the system memory.

5. The processing system for deploying LLM model resources as described in claim 3, wherein the quantization tuning program generates the configuration file for each tuned core and selected memory and the quantization precision option, the quantization tuning program sends the configuration file to the configuration server, and the configuration server records the received configuration file in the model list.

6. The processing system for deploying LLM model resources as described in claim 1, further comprising a source server connected to the computer under test, the source server having multiple original models, the computer under test obtaining one of the original models from the source server, the obtained model being the model under test.

7. A computer device, comprising: A device resource, including a graphics processing memory and a system memory; a communication unit, network-connected to a configuration server and a source server; a storage unit, storing a target program and a tuned model; and a processing unit, connected to the graphics processing memory, the system memory, the communication unit, and the storage unit. The processing unit initializes the target program, obtains a model list from the configuration server based on the device resource, selects a primary model from the model list, obtains the primary model from the source server, obtains a corresponding configuration file from the configuration server based on the primary model, quantizes multiple model cores of the primary model according to a quantization precision option in the configuration file, and generates multiple tuned cores. The quantized primary model is the tuned model. The processing unit configures these tuned cores and the tuned model into the graphics processing memory or the system memory according to the configuration file.

8. The computer apparatus of claim 7, wherein the target program sequentially calls the corresponding tuned core from the graphics processing memory and the system memory based on association information.

9. A resource retrieval method for an LLM model, comprising: When a computer device initializes a target program, the computer device connects to a configuration server via a device resource network; the computer device obtains a model list from the configuration server and selects a primary model from the model list; the computer device connects to a source server via the primary model and the model list network and obtains the primary model from the source server; the computer device obtains a corresponding configuration file from the configuration server based on the primary model; the computer device adjusts multiple model cores of the primary model according to a quantization precision option in the configuration file, and the adjusted model cores are called adjusted cores; and the computer device places the multiple adjusted cores of the primary model and the adjusted model into a graphics processing memory and a system memory.

10. A method for deploying LLM model resources, comprising: A computer under test executes a quantization adjustment program; The quantization adjustment program quantizes a model under test based on a device resource of the computer under test. The model under test includes multiple model cores. The quantization adjustment program selects a quantization precision option and adjusts each model core. The adjusted model core is an adjusted core, and the quantized model under test is an adjusted model. The quantization adjustment program generates a configuration file based on the adjusted model and the device resource. The quantization adjustment program transmits model information of the model under test, the device resource, and the configuration file to a configuration server. The configuration server establishes a model-to-device information based on the received model information, device resource, and configuration file, and adds the model-to-device information to a model list.

11. The method for deploying LLM model resources as described in claim 10, wherein the step of "the quantization adjustment program generating the configuration file based on the adjusted model and the device resources" includes: The quantitative adjustment program selects any of the adjusted cores; The quantization tuning program selects either a graphics processing memory or a system memory based on the capacity of the selected tuned core; and the quantization tuning program repeatedly selects the remaining tuned cores until all the tuned cores have been selected for their corresponding memory, the tuned cores of the graphics processing memory are selected as a first core group, and the tuned cores of the system memory are selected as a second core group.

12. The method for deploying LLM model resources as described in claim 11, further comprising: The quantization adjustment program generates the configuration file for each of the adjusted cores and selected memory modules, along with the quantization precision option. The quantization adjustment program sends the configuration file to the configuration server; and the configuration server records the received configuration file in the model list.

Images