Generative model reasoning service deployment method and device based on AI questions and answers

Abstract

The invention discloses a generative model reasoning deployment method and device based on AI questions and answers, and the method comprises the steps: constructing the basic configuration of an AI questions and answers system, and carrying out the model deployment; designing a database and storing model metadata, a model example code, a GPU (Graphics Processing Unit) configuration guide and a Dockerfile script template; receiving a user reasoning service request, generating a deployment task ID and recording task information; performing model existence check, and if the model is missing, automatically downloading model metadata according to a stored Git warehouse address; extracting corresponding configuration data and example code details from the database, and dynamically filling a script template with the configuration data and example code details to generate a Dockerfile script; executing a mirror image starting script constructed in the container, running a mirror image and returning a service address; according to the invention, the efficiency and convenience of model service deployment are improved, the threshold of AI technology application is reduced, the resource configuration is optimized, and the user experience is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically to a generative model reasoning deployment method and device based on AI question and answer. Background Art

[0002] At present, artificial intelligence (AI) technology has become a key force in promoting innovation in all walks of life. As the core link in the application of AI technology, the efficiency and convenience of the deployment of AI model reasoning services directly affect the actual application effect of AI technology.

[0003] However, there are many challenges in the traditional model inference service deployment process, mainly including the following aspects: (1) Time-consuming deployment: Traditional model inference service deployment often requires complex configuration and environment construction. From model training to the launch of inference service, the entire process takes a long time, which is not conducive to rapid iteration and application; (2) Cumbersome deployment process: A large number of technical details need to be handled during the deployment process, including hardware selection, software configuration, environment construction, etc., which not only increases the complexity of deployment, but also raises the technical threshold; (3) Resource waste: Under the traditional deployment model, resource utilization efficiency is not high, and resources are often wasted due to improper configuration or unreasonable resource allocation; (4) High technical threshold: Deploying and maintaining inference services usually requires professional technicians, which limits the possibility of non-technical background personnel participating in and using AI technology.

[0004] Therefore, how to improve the efficiency of model reasoning service deployment, lower the technical threshold, optimize resource allocation, and enhance user experience is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a generative model reasoning deployment method and device based on AI question and answer to solve some of the technical problems mentioned in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A generative model reasoning deployment method based on AI question answering includes the following steps:

[0008] S1. Build the basic configuration of the AI ​​question-answering system and deploy the ChatGLM3-6b model;

[0009] S2. Design a database to store model metadata, model sample code, GPU configuration guide, and Dockerfile script template for model service containerization;

[0010] S3. Initialize the AI ​​question-answering system and load the ChatGLM3-6b model, receive the inference service request submitted by the user to build a specific model, generate a deployment task ID and record the task information;

[0011] S4. Check the existence of the model. If the model is missing, automatically download the model metadata according to the stored Git repository address;

[0012] S5. Based on the specific model information requested by the user, extract the corresponding configuration data and sample code details from the database, dynamically fill them into the preset Dockerfile script template, and generate a Dockerfile script;

[0013] S6. Start the container based on the newly generated image, execute the image startup script built inside the container, complete the model loading and API service initialization, and feedback the service access address to the user. The user can directly access the deployed AI question-and-answer service through the address.

[0014] Preferably, the basic configuration of the AI ​​question answering system includes environment preparation, software stack, and various libraries and dependencies required by the model.

[0015] Preferably, the specific contents of step S2 include:

[0016] S21. Capture the latest model list of the model library and obtain the model metadata, and store it in the database together with the Git repository address of the model;

[0017] S22. Download the model sample code from the official GitHub repository or Hugging Face page of the model, review and organize it, and classify and archive it into the database;

[0018] S23. According to the model characteristics and parameter configuration in the sample code, the amount of GPU resources required for different scenarios is calculated, the configuration suggestions are recorded and stored according to the GPU model, and a configuration guide is formed and stored in the database.

[0019] Preferably, the construction method of the Dockerfile script template in step S2 is specifically as follows:

[0020] Select a suitable base image;

[0021] Write Dockerfile instructions to define the image building process step by step;

[0022] Use the ARG instruction in Dockerfile to define variable parameters, so that the same template can quickly generate customized images according to different requirements;

[0023] Compile Dockerfile through the Dockerbuild command to generate the final container image.

[0024] Preferably, the defined image building process specifically includes pulling the base image, installing additional dependencies, copying model files and sample codes, setting environment variables, and configuring container startup commands.

[0025] Preferably, step S5 further comprises saving the generated Dockerfile script as a template to a database.

[0026] Preferably, the image startup script includes environment variable setting, model loading and service initialization, and dynamically adjusts the behavior according to the environment variables defined in the Dockerfile to adapt to different deployment environments and configuration requirements.

[0027] A generative model reasoning deployment device based on AI question and answer, based on the generative model reasoning deployment method based on AI question and answer, comprising: an AI question and answer system, a database, a user interface and a reasoning service generation system;

[0028] The inference service generation system includes a task creation module, a model acquisition module, a Dockerfile script generation module, an image startup script construction module, and an image running module;

[0029] AI question-answering system, used to build the basic configuration of the AI ​​question-answering system and deploy the ChatGLM3-6b model;

[0030] A database for storing model metadata, model sample code, GPU configuration guides, and Dockerfile script templates for containerizing model services;

[0031] User interface, used by users to submit inference service requests for building specific models, receive inference service access addresses, and access deployed AI question-and-answer services;

[0032] The task creation module is used to receive the inference service request submitted by the user to build a specific model, generate a deployment task ID and record the task information;

[0033] Model acquisition module, used to check the existence of the model. If the model is missing, the model metadata is automatically downloaded according to the stored Git repository address;

[0034] The Dockerfile script generation module is used to extract the corresponding configuration data and sample code details from the database based on the specific model information requested by the user, dynamically fill them into the preset Dockerfile script template, and generate the Dockerfile script;

[0035] The image running module is used to start the container based on the newly generated image, execute the image startup script built by the image startup script construction module, complete the model loading and API service initialization, and feedback the service access address to the user.

[0036] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the generative model reasoning deployment method based on AI question and answer.

[0037] A processing terminal includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, a generative model reasoning deployment method based on AI question and answer is implemented.

[0038] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a generative model reasoning deployment method and device based on AI question and answer, which improves the efficiency and convenience of model service deployment, lowers the threshold for the application of artificial intelligence (AI) technology, optimizes resource allocation, enhances user experience, and helps to provide strong support for the widespread application and rapid iteration of AI technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 A schematic diagram of a generative model reasoning deployment method based on AI question and answer provided by the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The embodiment of the present invention discloses a generative model reasoning deployment method based on AI question answering, such as Figure 1 , including the following steps:

[0043] S1. Build the basic configuration of the AI ​​question-answering system and deploy the ChatGLM3-6b model;

[0044] S2. Design a database to store model metadata, model sample code, GPU configuration guide, and Dockerfile script template for model service containerization;

[0045] S3. Initialize the AI ​​question-answering system and load the ChatGLM3-6b model, receive the inference service request submitted by the user to build a specific model, generate a deployment task ID and record task information, such as task status, request parameters, etc.;

[0046] S4. Check the model existence. If the model is missing, automatically download the model metadata according to the stored Git repository address.

[0047] Specifically, after receiving the task, the system queries the local or cloud storage to determine whether the requested model has been cached. If the model is missing, the system will automatically initiate a download request based on the previously stored Git repository address. During the download process, the system monitors the download progress in real time and provides breakpoint resume support when necessary to ensure that the model data arrives at the local storage intact.

[0048] S5. Based on the specific model information requested by the user, extract the corresponding configuration data and sample code details from the database, dynamically fill them into the preset Dockerfile script template, and generate a Dockerfile script;

[0049] S6. Start the container based on the newly generated image, execute the image startup script built inside the container, complete the model loading and API service initialization, and feedback the service access address to the user. The user can directly access the deployed AI question-and-answer service through the address.

[0050] Preferably, the basic configuration of the AI ​​question-answering system includes environmental preparation, including ensuring that the system has sufficient computing resources, especially GPU resources, installing the necessary software stack, such as the Python environment, PyTorch or similar deep learning frameworks, and various libraries and dependencies required for the model; after completing the basic configuration, the model can be easily loaded and called through toolkits such as Hugging Face Transformers.

[0051] Preferably, the specific contents of step S2 include:

[0052] S21. Capture the latest model list of the model library and obtain the model metadata, and store it in the database together with the Git repository address of the model;

[0053] S22. Download the model sample code from the official GitHub repository or Hugging Face page of the model, review and organize it, and classify and archive it into a database, such as by model category, purpose or function;

[0054] S23. Based on the model characteristics and parameter configurations in the sample code, calculate the amount of GPU resources required in different scenarios. For example, considering the computational intensity and memory usage of the model, it may be necessary to configure a high-end GPU with large video memory, such as NVIDIA RTX 3090, and sufficient VRAM to avoid memory overflow. Record configuration suggestions and store them by category according to GPU model to form a configuration guide and store it in the database.

[0055] In this embodiment, for step S21, by writing an intelligent crawler program, the API interface or web page of Hugging Face is regularly accessed to collect metadata of the model, including but not limited to the model name, version, publisher, download link, introduction and performance indicators, and the metadata is stored in the database after being sorted for fast retrieval and comparison; the database design takes into account flexibility and scalability, and adopts relational databases such as MySQL or non-relational databases MongoDB to adapt to the growing demand for model information storage; for each model, its Git repository address is recorded, providing a direct path for automatic download and update of the model.

[0056] As for step S22, in this embodiment, it also includes establishing a code version control system, such as Git, to ensure that historical changes of the code can be traced, which is convenient for maintenance and backtracking.

[0057] For step S23, in this embodiment, the configuration suggestions also include considering the possibility of multi-card parallel computing to further improve throughput; the configuration suggestions are recorded in detail and stored according to the GPU model classification to form a detailed configuration guide, which is convenient for operation and maintenance personnel to quickly match and adjust the configuration according to actual hardware conditions.

[0058] Preferably, the construction method of the Dockerfile script template in step S2 is specifically as follows:

[0059] Select a suitable base image, such as an Ubuntu-based Python environment image, and ensure that all necessary system libraries and Python packages are pre-installed;

[0060] Write Dockerfile instructions to define the image building process step by step;

[0061] Use the ARG instruction in Dockerfile to define variable parameters, such as GPU enable flags, image tags, etc., so that the same template can quickly generate customized images according to different requirements;

[0062] Compile the Dockerfile through the Dockerbuild command to generate the final container image.

[0063] Preferably, the defined image building process specifically includes pulling the base image, installing additional dependencies, copying model files and sample codes, setting environment variables, and configuring container startup commands.

[0064] Preferably, step S5 further comprises saving the generated Dockerfile script as a template to a database.

[0065] In this embodiment, step S5 includes replacing the parameters in the template, such as specifying the version number of the model, adjusting the resource limit parameters to adapt to the GPU type specified by the user or automatically detected, etc. The generated Dockerfile not only meets the current deployment requirements, but is also saved as a template to facilitate rapid response to subsequent similar requests.

[0066] Preferably, the image startup script includes environment variable setting, model loading and service initialization, and dynamically adjusts the behavior according to the environment variables defined in the Dockerfile to adapt to different deployment environments and configuration requirements.

[0067] A generative model reasoning deployment device based on AI question and answer, based on the generative model reasoning deployment method based on AI question and answer, comprising: an AI question and answer system, a database, a user interface and a reasoning service generation system;

[0068] The inference service generation system includes a task creation module, a model acquisition module, a Dockerfile script generation module, an image startup script construction module, and an image running module;

[0069] AI question-answering system, used to build the basic configuration of the AI ​​question-answering system and deploy the ChatGLM3-6b model;

[0070] A database for storing model metadata, model sample code, GPU configuration guides, and Dockerfile script templates for containerizing model services;

[0071] User interface, used by users to submit inference service requests for building specific models, receive inference service access addresses, and access deployed AI question-and-answer services;

[0072] The task creation module is used to receive the inference service request submitted by the user to build a specific model, generate the deployment task ID and record the task information. It is implemented through the RESTful API to ensure the separation of the front-end and back-end, and improve the maintainability and scalability of the system;

[0073] Model acquisition module, used to check the existence of the model. If the model is missing, the model metadata is automatically downloaded according to the stored Git repository address;

[0074] The Dockerfile script generation module is used to extract the corresponding configuration data and sample code details from the database based on the specific model information requested by the user, dynamically fill them into the preset Dockerfile script template, and generate the Dockerfile script;

[0075] The image running module is used to start the container based on the newly generated image, execute the image startup script built by the image startup script construction module, complete the model loading and API service initialization, and feedback the service access address to the user.

[0076] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the generative model reasoning deployment method based on AI question and answer.

[0077] A processing terminal includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, a generative model reasoning deployment method based on AI question and answer is implemented.

[0078] The present invention uses an AI question-answering generative model to build a highly automated reasoning service deployment platform. Users only need to clarify service requirements through simple question-answering interactions, and the system can automatically complete model selection, downloading, Docker image building, and service operation, compressing the originally cumbersome deployment process to minutes, greatly improving deployment efficiency and enabling model services to quickly respond to changes in business requirements.

[0079] Through the code-free development concept, even business personnel without programming knowledge can easily manage the entire life cycle of the inference service, including service creation, operation status monitoring, and dynamic allocation and shutdown of resources. This design breaks down technical barriers, makes internal collaboration within the enterprise smoother, and accelerates the integration and application of AI technology in various business fields.

[0080] The real-time update function of the integrated model library can automatically obtain and store the latest AI model data and sample codes on the market, providing users with a rich model selection space. Users can easily choose the most suitable model version according to the application scenario, and the system automatically handles the download and version update of the model, ensuring the cutting-edge and accuracy of the service;

[0081] It can intelligently recommend or adaptively adjust GPU configuration to maximize the operating efficiency of the inference service. During the dynamic generation of the Dockerfile template, the system automatically adjusts resource configuration according to model requirements, maximizing resource utilization and reducing operating costs.

[0082] By utilizing Docker container technology, the present invention ensures the consistency and isolation of the service environment, effectively avoiding problems caused by environmental differences; at the same time, the lightweight characteristics of the container make service deployment more flexible, and fast and secure migration and expansion can be achieved regardless of whether it is local deployment or cloud deployment.

[0083] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0084] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A generative model reasoning deployment method based on AI question answering, characterized in that: The following steps are involved: S1. Build the basic configuration of the AI ​​question-answering system and deploy the ChatGLM3-6b model; S2. Design a database to store model metadata, model sample code, GPU configuration guide, and Dockerfile script template for model service containerization; S3. Initialize the AI ​​question-answering system and load the ChatGLM3-6b model, receive the inference service request submitted by the user to build a specific model, generate a deployment task ID and record the task information; S4. Check the existence of the model. If the model is missing, automatically download the model metadata according to the stored Git repository address; S5. Based on the specific model information requested by the user, extract the corresponding configuration data and sample code details from the database, dynamically fill them into the preset Dockerfile script template, and generate a Dockerfile script; S6. Start the container based on the newly generated image, execute the image startup script built inside the container, complete the model loading and API service initialization, and feedback the service access address to the user. The user can directly access the deployed AI question-and-answer service through the address.

2. According to the method for deploying generative model reasoning based on AI question answering according to claim 1, it is characterized in that: The basic configuration of the AI ​​question-answering system includes environment preparation, software stack, and various libraries and dependencies required by the model.

3. According to the method for deploying generative model reasoning based on AI question answering according to claim 1, it is characterized in that: The specific contents of step S2 include: S21. Capture the latest model list of the model library and obtain the model metadata, and store it in the database together with the Git repository address of the model; S22. Download the model sample code from the official GitHub repository or Hugging Face page of the model, review and organize it, and classify and archive it into the database; S23. According to the model characteristics and parameter configuration in the sample code, the amount of GPU resources required for different scenarios is calculated, the configuration suggestions are recorded and stored according to the GPU model, and a configuration guide is formed and stored in the database.

4. According to the method for deploying generative model reasoning based on AI question-answering according to claim 1, it is characterized in that: The specific construction method of the Dockerfile script template in step S2 is: Select a suitable base image; Write Dockerfile instructions to define the image building process step by step; Use the ARG instruction in Dockerfile to define variable parameters, so that the same template can quickly generate customized images according to different requirements; Compile the Dockerfile through the Dockerbuild command to generate the final container image.

5. The method for deploying generative model reasoning based on AI question-answering according to claim 1, characterized in that: The defined image building process includes pulling the base image, installing additional dependencies, copying model files and sample codes, setting environment variables, and configuring container startup commands.

6. The method for deploying generative model reasoning based on AI question answering according to claim 1, characterized in that: Step S5 also includes saving the generated Dockerfile script as a template to a database.

7. The method for deploying generative model reasoning based on AI question-answering according to claim 1, characterized in that: The image startup script includes environment variable settings, model loading, and service initialization. It dynamically adjusts behavior based on the environment variables defined in the Dockerfile to adapt to different deployment environments and configuration requirements.

8. A generative model reasoning deployment device based on AI question answering, characterized in that: A generative model reasoning deployment method based on AI question-answering according to any one of claims 1 to 7, comprising: an AI question-answering system, a database, a user interface, and a reasoning service generation system; The inference service generation system includes a task creation module, a model acquisition module, a Dockerfile script generation module, an image startup script construction module, and an image running module; AI question-answering system, used to build the basic configuration of the AI ​​question-answering system and deploy the ChatGLM3-6b model; A database for storing model metadata, model sample code, GPU configuration guides, and Dockerfile script templates for containerizing model services; User interface, used by users to submit inference service requests for building specific models, receive inference service access addresses, and access deployed AI question-and-answer services; The task creation module is used to receive the inference service request submitted by the user to build a specific model, generate a deployment task ID and record the task information; Model acquisition module, used to check the existence of the model. If the model is missing, the model metadata is automatically downloaded according to the stored Git repository address; The Dockerfile script generation module is used to extract the corresponding configuration data and sample code details from the database based on the specific model information requested by the user, dynamically fill them into the preset Dockerfile script template, and generate the Dockerfile script; The image running module is used to start the container based on the newly generated image, execute the image startup script built by the image startup script construction module, complete the model loading and API service initialization, and feedback the service access address to the user.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements a generative model reasoning deployment method based on AI question and answer as described in any one of claims 1 to 7.

10. A processing terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, it implements a generative model reasoning deployment method based on AI question and answer as described in any one of claims 1 to 7.

Images