Large model service processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a large model service processing method and device, electronic equipment and a storage medium, and can be widely applied to the technical field of model processing. After the operation request instruction of the large model service is preset, the service container of the preset large model service and the service mirror image corresponding to the service container are constructed according to the operation request instruction, and the service mirror image is stored in the preset mirror image warehouse; meanwhile, after a first model file and a first extension plug-in file of a preset large model service are decoupled and plugged in according to the operation request instruction to obtain a second model file and a second extension plug-in file, the preset large model service is deployed according to a service mirror image in a preset mirror image warehouse, the second model file and the second extension plug-in file; therefore, the file quantity of the service mirror image is effectively reduced, the transmission and deployment cost can be effectively reduced, the deployment verification period is shortened, and the network bandwidth and flow occupied by transmission are reduced.

Description

Technical Field

[0001] The present application relates to the field of model processing technology, and in particular to a processing method and device for a large model service, an electronic device and a storage medium. Background Art

[0002] In related technologies, large model application containers are lightweight, portable containers packaged from large model applications and their dependencies. Since large model services rely on various large models, the volume of large model service containers can basically reach more than ten GB or even hundreds of GB, which leads to extremely high transmission and deployment costs for large model services each time. The deployment verification cycle of large model services after fine-tuning and training is also longer, and the network bandwidth and traffic occupied by transmission are also larger.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a processing method and device, electronic device and storage medium for large model services, which can effectively reduce transmission and deployment costs, shorten the deployment verification cycle, and reduce the network bandwidth and traffic occupied by transmission.

[0005] To achieve the above purpose, one aspect of an embodiment of the present application provides a processing method for a large model service, the method comprising the following steps:

[0006] Obtain the operation request instruction of the preset large model service;

[0007] Constructing a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction;

[0008] Storing the service image in a preset image warehouse;

[0009] Decouple and plug-in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction to obtain a second model file and a second extension plug-in file;

[0010] The preset large model service is deployed according to the service image in the preset image repository, the second model file and the second extension plug-in file.

[0011] In some embodiments, the step of constructing a service container for the preset large model service according to the operation request instruction includes:

[0012] Loading the dependency package of the preset large model service according to the operation request instruction, and packaging the project code of the preset large model service;

[0013] A service container for the preset large model service is constructed according to the dependency package and the packaged project code.

[0014] In some embodiments, storing the service image in a preset image repository includes:

[0015] Acquire a first storage operation instruction of the service image, where the first storage operation instruction is triggered by a first preset control of a preset graphical interface;

[0016] The service image is stored in a preset image repository according to the first storage operation instruction.

[0017] In some embodiments, the decoupling and plugging the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction includes:

[0018] Acquire the mount configuration information of the preset file system according to the operation request instruction;

[0019] Acquire a target file list according to the mount configuration information, where the target file list has a mapping relationship with the preset file system;

[0020] Creating a first folder and a second folder in the target file list;

[0021] storing the first model file of the preset large model service into the first folder, and storing the first extension plug-in file of the preset large model service into the second folder;

[0022] The first folder storing the first model file is synchronized to the preset file system according to the mapping relationship, and the second folder storing the first extension plug-in file is synchronized to the preset file system according to the mapping relationship.

[0023] In some embodiments, storing the first model file of the preset large model service into the first folder, and storing the first extension plug-in file of the preset large model service into the second folder, comprises:

[0024] Acquire a second storage operation instruction and a third storage operation instruction, wherein the second storage operation instruction is triggered by a second preset control of the preset graphical interface, and the third storage operation instruction is triggered by a third preset control of the preset graphical interface;

[0025] According to the second storage operation instruction, the first model file of the preset large model service is stored in the first folder; and according to the third storage operation instruction, the first extension plug-in file of the preset large model service is stored in the second folder.

[0026] In some embodiments, synchronizing the first folder storing the first model file to the preset file system according to the mapping relationship, and synchronizing the second folder storing the first extension plug-in file to the preset file system according to the mapping relationship, includes:

[0027] According to the mapping relationship, synchronizing the first folder storing the first model file to the preset file system through a preset protocol;

[0028] And according to the mapping relationship, the second folder storing the first extension plug-in file is synchronized to the preset file system through a preset protocol.

[0029] In some embodiments, the deploying of the preset large model service according to the service image in the preset image repository, the second model file, and the second extension plug-in file includes:

[0030] Mapping the second model file and the second extension plug-in file to the service container persistent volume to obtain a third model file and a third extension plug-in file;

[0031] When the running instruction of the service container is obtained, the third model file and the third extension plug-in file are read from the persistent volume of the service container, and the service image is read from the preset image repository;

[0032] The preset large model service is deployed based on the read third model file, the third extension plug-in file and the service image.

[0033] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a processing device for a large model service, the device comprising:

[0034] The first module is used to obtain the operation request instruction of the preset large model service;

[0035] The second module is used to construct a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction;

[0036] The third module is used to store the service image in a preset image warehouse;

[0037] A fourth module is used to decouple and plug-in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction to obtain a second model file and a second extension plug-in file;

[0038] The fifth module is used to deploy the preset large model service according to the service image in the preset image repository, the second model file and the second extension plug-in file.

[0039] To achieve the above object, another aspect of an embodiment of the present application provides an electronic device, including:

[0040] at least one processor;

[0041] at least one memory for storing at least one program;

[0042] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0043] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0044] The embodiments of the present application include at least the following beneficial effects: The present application provides a processing method and device for a large model service, an electronic device and a storage medium. After the operation request instruction of the preset large model service is set, the scheme builds a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction, and stores the service image in a preset image warehouse. At the same time, according to the operation request instruction, the first model file and the first extension plug-in file of the preset large model service are decoupled and plug-in to obtain the second model file and the second extension plug-in file, and then the preset large model service is deployed according to the service image, the second model file and the second extension plug-in file in the preset image warehouse, thereby effectively reducing the file volume of the service image, and then effectively reducing the transmission and deployment costs, shortening the deployment verification cycle and reducing the network bandwidth and traffic occupied by the transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of a processing method of a large model service provided in an embodiment of the present application;

[0046] Figure 2 It is a flow chart of constructing a service container of a preset large model service according to an operation request instruction provided in an embodiment of the present application;

[0047] Figure 3 It is a flowchart of decoupling and plugging in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction provided by the embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of an application for decoupling and plugging in files of a preset large model service provided by an embodiment of the present application;

[0049] Figure 5 is a flowchart of step S150 provided in an embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of the preset large model service deployment provided in the embodiment of the present application;

[0051] Figure 7 This is a schematic diagram of a scenario for deploying a Stable Diffusion image generation model service provided in an embodiment of the present application;

[0052] Figure 8 It is an application flow chart of service container construction and service deployment provided by an embodiment of the present application;

[0053] Fig. 9 It is a structural diagram of a processing device for a large model service provided in an embodiment of the present application;

[0054] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application.

[0056] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0057] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0060] Kubemetes (K8s) is a popular open source container orchestration platform that can be used to automate the deployment, expansion, and management of application container clusters. In K8s, Docker is the most commonly used container runtime environment.

[0061] Docker is an open platform for developing, deploying, and running containerized environments for applications. Docker allows developers to package applications and their dependencies into a lightweight, portable container, ensuring that the application runs consistently in any environment.

[0062] Pop is the smallest deployable unit of Kubemetes, consisting of one or more containers. Pop is a group of densely deployed containers that share the same network and storage resources and are scheduled to run on the same node.

[0063] Docker Persistent Volumes (PVs) is a storage mechanism used to save container data. The data still exists even after the container is deleted, which can solve the problem of impermanent container state, because by default, the container data will be lost when the container stops running. Persistent volumes can be bound to the host's file system path, or external storage solutions (such as network storage) can be used. Persistent volumes are very useful for applications that need to store data for a long time, such as database services.

[0064] Docker Hub Repository is a public Docker image repository that allows developers to store and distribute Docker images. Users can upload their own Docker images to Docker Hub, or download images shared by others. Docker Hub also provides a series of tools and services, such as automated builds, continuous integration / continuous deployment (CI / CD), etc., to help developers manage and use Docker images more efficiently.

[0065] AList (AList File System) is a lightweight file management system that supports the management of multiple cloud storage services and local file systems. AList provides a unified interface to access different types of storage resources, such as Alibaba Cloud OSS, Tencent Cloud COS, Google Drive, SFTP, WebDAV, etc. Users can upload, download, and browse files through the Web interface, and even perform automated management through the API.

[0066] NFS (Network File System): is a distributed file system protocol that allows computers on a network to share each other's file systems. Through NFS, users can read and write files on remote computers as if the files were located on a local disk. NFS is widely used in Unix and Unix-like systems. NFS can improve file access efficiency, simplify file management in a network environment, and support interoperability between multiple operating systems.

[0067] PyTorch is an open source Python machine learning library. PyTorch's predecessor is Torch. Its underlying framework is the same as Torch, but many contents are rewritten in Python. It is not only more flexible and supports dynamic graphs, but also provides Python interfaces. PyTorch can be seen as numpy with GPU support, but also as a powerful deep neural network with automatic differentiation function.

[0068] HTTP (Hyper Text Transfer Protocol) is an application layer protocol for distributed, collaborative and hypermedia information systems. It is the basis of data communication on the World Wide Web and the most widely used network transmission protocol on the Internet. HTTP is a transmission protocol used to transmit hypertext from a WWW server to a local browser, which can make the browser more efficient and reduce network transmission. HTTP not only ensures that computers transmit hypertext documents correctly and quickly, but also determines which part of the document is transmitted and which part of the content is displayed first (such as text before graphics).

[0069] GUI (Graphical User Interface) refers to a computer operation user interface that uses graphics to display. Graphical user interface is an interface display format for communication between people and computers, allowing users to use input devices such as a mouse to manipulate icons or menu options on the screen to select commands, call files, start programs, or perform other daily tasks. Compared with character interfaces that use keyboard input of text or character commands to complete routine tasks, graphical user interfaces have many advantages.

[0070] Mirroring is a file format that can make many files into a mirror file. When it is placed on a disk with programs such as GHOST and opened with GHOST, it can be restored into many files. Mirroring is a type of redundancy. The data on one disk has an identical copy on another disk.

[0071] In related technologies, with the widespread application of artificial intelligence in various industries, the demand for deployment of various large model services is becoming more and more common. In the application process, artificial intelligence applications rely on various basic services such as semantic large models and multimodal large models. Compared with traditional Web services, large model services are heavily dependent on the combined application of various large models, which makes the volume of the container (Docker) file of the large model service basically reach more than ten GB or even hundreds of GB. Due to the large file size of the large model service, the transmission and deployment cost of each large model service will become extremely high, resulting in a longer deployment verification cycle for the large model service after fine-tuning and training.

[0072] At present, in the application process of large model services, by default, the model, extension plug-in and other files in the service container and the business code of the model service are packaged uniformly into the Docker service container of the model service. When the Docker service image is built, it is transmitted to the target DockerHub image repository through the HTTP network. Due to the large size of the generated Docker image (generally ranging from tens of GB to hundreds of GB), the network bandwidth will be occupied for a long time during the transmission process and a huge amount of traffic will be generated. At the same time, after the Docker image is pushed, the Docker image needs to be pulled from the DockerHub repository to the local computer when deploying the Docker service on the host server. The pulling process is similar to the transmission process, which will also occupy the bandwidth for a long time and generate a huge amount of traffic. When the Docker image is downloaded, due to the large size of the Docker image, the Docker Pod service of the deployment service needs to allocate high memory and CPU resources. In the existing large model service deployment solution, each construction, push, download and deployment of the large model service container image takes at least 1 hour, which will seriously affect the efficiency of business development and verification.

[0073] In view of this, an embodiment of the present application provides a processing method and device for a large model service, an electronic device and a storage medium. After the operation request instruction of the preset large model service is preset, the present application constructs a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction, and stores the service image to the preset image warehouse. At the same time, according to the operation request instruction, the first model file and the first extension plug-in file of the preset large model service are decoupled and plug-in to obtain the second model file and the second extension plug-in file, and then the preset large model service is deployed according to the service image, the second model file and the second extension plug-in file in the preset image warehouse, thereby effectively reducing the file volume of the service image, and then effectively reducing the transmission and deployment costs, shortening the deployment verification cycle and reducing the network bandwidth and traffic occupied by the transmission.

[0074] It is understandable that the processing method of the large model service provided in the embodiment of the present application relates to the field of model processing technology. The processing method of the large model service provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in the blockchain network; the software can be an application of the processing method for implementing the large model service, etc., but is not limited to the above form.

[0075] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0076] Figure 1 is an optional flowchart of the processing method of the large model service provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S150:

[0077] Step S110, obtaining an operation request instruction of a preset large model service;

[0078] Step S120: construct a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction;

[0079] Step S130, storing the service image in a preset image repository;

[0080] Step S140: Decouple and plug in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction to obtain a second model file and a second extension plug-in file;

[0081] Step S150: Deploy the preset large model service according to the service image, the second model file and the second extension plug-in file in the preset image repository.

[0082] It is understandable that the preset big model may include but is not limited to a semantic big model, a multimodal big model or an image recognition big model. The preset big model service refers to the process of deploying the preset big model into the production environment. The service container may be a container obtained by packaging the application of the preset big model and its dependencies, and the container can be transplanted to apply the preset big model service. Exemplarily, when the preset big model is a semantic big model, after obtaining the application scenario of the semantic big model, the application and its dependencies required for the semantic big model service are determined based on the application scenario, and then the corresponding application and dependencies are packaged into a lightweight and portable service container, so that when the service is deployed, the deployment operation can be completed by transplanting the service container. Specifically, the environment in which the service container is located includes but is not limited to Docker.

[0083] It is understandable that the operation request instruction of the preset large model service can be an instruction triggered by the relevant personnel clicking the corresponding control in the graphical user interface. When the relevant personnel clicks the control in the graphical user interface with the mouse, the backend can receive the corresponding operation request instruction and perform the corresponding operation based on the operation request instruction.

[0084] In an embodiment of the present application, after obtaining the service container corresponding to the large model service, the present embodiment can construct a service image corresponding to the service container, so that when the large model service is deployed, the service image can be called to achieve the function of large model service deployment, thereby effectively improving the stability and security of the service container. Specifically, the present embodiment can better manage and use images by storing service images in a preset image repository. Among them, when the environment in which the service container is located is Docker, the preset image repository can be the Docker Hub repository corresponding to Docker, thereby improving the stability of the image storage path.

[0085] In the embodiment of the present application, in order to reduce the file size of the service image, the service image in the embodiment does not contain the model file and the extension plug-in file of the preset large model service. Specifically, after the operation request instruction is triggered in the graphical user interface, the first model file and the first extension plug-in file of the preset large model service are decoupled and plug-ined, so that when the service is deployed, the corresponding model file and extension plug-in file can be obtained from the plug-in interface, thereby effectively reducing the file size of the service image, effectively reducing the transmission and deployment costs, shortening the deployment verification cycle and reducing the network bandwidth and traffic occupied by the transmission.

[0086] In the embodiments of the present application, Figure 2 As shown, the process of constructing a service container of a preset large model service according to an operation request instruction includes but is not limited to steps S210 to S220:

[0087] Step S210: Load the dependency package of the preset large model service according to the operation request instruction, and package the project code of the preset large model service;

[0088] Step S220: construct a service container for the preset large model service according to the dependent package and the packaged project code.

[0089] It is understandable that the project code can be the code for implementing the scenario function to be applied by the preset large model service, and the dependency package can be the program code corresponding to the environment on which the application scenario depends. Specifically, the project code and the dependency package have a mutually dependent relationship. When the dependency package is missing, the project code will not be able to run normally to obtain the corresponding scenario effect. This embodiment synchronously packages the project code of the preset large model when loading the dependency package of the preset large model service, and then builds a service container for the preset large model service based on the loaded dependency package and the packaged project code, so that the service container can contain both the dependency package and the project code, thereby effectively improving the functional stability of the service container after transplantation and deployment.

[0090] In an embodiment of the present application, after obtaining the service image corresponding to the service container, when the relevant personnel click the first preset control in the preset graphical interface, the first storage operation instruction of the service image can be triggered, and the backend can store the service image to the preset image warehouse according to the first storage operation instruction, so that the service image can be managed and used more efficiently. Exemplarily, when the service container is a container product in Docker, the service image corresponding to the service container is a Docker image, so the Docker image can be saved to the Docker Hub warehouse. It is understandable that in this embodiment, the Docker image can be pushed to the corresponding location of the DockerHub warehouse for storage by pushing.

[0091] In the embodiments of the present application, Figure 3 As shown, the process of decoupling and plugging the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction includes but is not limited to steps S310 to S350:

[0092] Step S310: Obtaining the mount configuration information of the preset file system according to the operation request instruction;

[0093] Step S320: obtaining a target file list according to the mount configuration information, wherein the target file list has a mapping relationship with a preset file system;

[0094] Step S330, creating a first folder and a second folder in the target file list;

[0095] Step S340: storing the first model file of the preset large model service into the first folder, and storing the first extension plug-in file of the preset large model service into the second folder;

[0096] Step S350: synchronize the first folder storing the first model file to the preset file system according to the mapping relationship, and synchronize the second folder storing the first extension plug-in file to the preset file system according to the mapping relationship.

[0097] It is understandable that the operation request instruction can be an instruction triggered by the operation of the relevant personnel in the graphical user interface to characterize the need to perform the preset large model service processing. When it is determined according to the operation request instruction that the preset large model service needs to be decoupled and plug-in-externalized, this embodiment obtains the mount configuration information of the file system after determining the corresponding file system, so as to determine the location of the corresponding file of the preset large model service in the preset file system through the mount configuration information. Then, according to the mount configuration information, the target file list is obtained, that is, after obtaining the target file list that has a mapping relationship with the corresponding position of the mount configuration information, the model file and the extension plug-in file of the preset large model service are stored in the corresponding folder in the target file list, thereby effectively improving the file management efficiency and facilitating subsequent calls.

[0098] In an embodiment of the present application, relevant personnel can click the second preset control and the third preset control in the preset graphical interface to trigger the second storage operation instruction and the third storage operation instruction, so that the backend can store the first model file of the preset large model service into the first folder according to the second storage operation instruction; and store the first extension plug-in file of the preset large model service into the second folder according to the third storage operation instruction, so that the corresponding files can be accurately stored in the corresponding folders, thereby improving the accuracy of file management.

[0099] It is understandable that the interaction process between the preset file system and other systems can be interacted through a preset protocol. The preset protocol may include but is not limited to http, ftp, https, etc., and the specific preset protocol content can be determined according to the actual application. Specifically, after completing the operation process of storing the corresponding file in the corresponding folder, the present embodiment can synchronize the first folder containing the first model file to the preset file system through the preset protocol based on the mapping relationship in the preset file system; and synchronize the second folder containing the first extension plug-in file to the preset file system through the preset protocol based on the mapping relationship, so that the accuracy file content can be obtained from the preset file system during the application process.

[0100] For example, in the ALIst file management system, the decoupling plug-in of the preset large model service file (GUI large model file) is used as an example, and NFS is used as the preset file system. Figure 4 As shown, the processing process includes but is not limited to the following steps:

[0101] Step S410: After entering the ALIst file management system through the operation request instruction, obtain NFS mount configuration information in the ALIst file management system;

[0102] Step S420: after determining the storage location in NFS according to the NFS mount configuration information, read the file list mapped to NFS in the ALIst file management system as the target file list;

[0103] Step S430: Create a models folder (first folder) and an extensions folder (second folder) in the target file list in the ALIst file management system;

[0104] Step S440, entering the models folder and the extensions folder in the ALIst file management system;

[0105] Step S450: Upload the model file to the models folder and the extension plug-in file to the extensions folder in the ALIst file management system through the http protocol, and synchronize the models folder storing the model file and the extensions folder storing the extension plug-in file to the NFS for storage based on the mapping relationship between the ALIst file management system and NFS;

[0106] Step S460, receiving the response information returned by NFS, and after determining that the file synchronization is successful based on the response information, it can be determined that the model file corresponding to the preset large model service completes the processing process of decoupling the plug-in.

[0107] In the embodiments of the present application, Figure 5 As shown, the process of deploying the preset large model service according to the service image in the preset image repository, the second model file and the second extension plug-in file includes but is not limited to steps S510 to S530:

[0108] Step S510: Map the second model file and the second extension plug-in file to the service container persistent volume to obtain a third model file and a third extension plug-in file;

[0109] Step S520: When the running instruction of the service container is obtained, the third model file and the third extension plug-in file are read from the persistent volume of the service container, and the service image is read from the preset image repository;

[0110] Step S530: Deploy the preset large model service based on the read third model file, the third extension plug-in file and the service image.

[0111] It is understandable that the service container persistent volume can be set in the container to store the corresponding data. Figure 6As shown, when Docker is used as the container of the embodiment of the present application, the service container persistent volume can be a Docker container persistent volume, so that the service container persistent volume can be set in the Docker service. In this embodiment, after the second model file and the second extension plug-in file are mapped to the service container persistent volume, even if Docker stops running, the second model file and the second extension plug-in file in the service container persistent volume will not disappear.

[0112] Specifically, Figure 6 Taking the scenario shown as an example, when the files in NFS are mapped to the Docker container persistent volume, the Docker container persistent volume will contain model files, extension plug-in files and other contents. When the Docker container service is running in the remote Docker Hub warehouse, the model files and extension plug-in files corresponding to the service can be read from the Docker container persistent volume, and the service image can be read in the Docker Hub warehouse at the same time, and then the preset large model service deployment is performed based on the read information. Since the service image read by the embodiment of the present application does not contain large model files and extension plug-in files, the file size of the service image is greatly reduced, which can effectively reduce the transmission and deployment costs, shorten the deployment verification cycle and reduce the network bandwidth and traffic occupied by the transmission.

[0113] It is understandable that in Figure 6 In the application process, the present application embodiment includes but is not limited to the following four stages:

[0114] GUI-based decoupling and transmission of large model files: Relevant personnel use the AList file system on the local development workstation through the GUI interactive interface to upload large model files and extension plug-in files using visual drag-and-drop controls;

[0115] NFS storage stage: By configuring the AList file system in advance, large model files and extension plug-in files are stored and mounted on NFS;

[0116] Docker container service building and pushing stage: Relevant personnel write Dockerfile scripts to configure and build service images. The building process will preload the building dependency packages, package the project code, build the corresponding service container and the service image corresponding to the service container, and push the service image to the remote Docker Hub repository to wait for deployment and operation;

[0117] Docker container service deployment and mounting phase: After the service image is pushed, the Docker (service container) deployment will be started. At this time, by mapping the NFS storage to the container persistent volume, after running the Docker container service, the large model file and the extension plug-in file can be successfully loaded into the host machine through the plug-in method to perform the preset large model service deployment, thereby achieving the effect of accelerating deployment.

[0118] In some embodiments, when the method of the present application is applied to a scenario where a Stable Diffusion image generation model service is deployed, such as Figure 7 As shown in the figure, the deployment process involves the local development workstation, NFS storage, Docker container server, and remote Docker Hub repository. During the deployment process, the interaction process of these four ends is as follows:

[0119] First, perform a large model file decoupling plug-in operation between the local development workstation and the NFS storage end. Specifically, after the relevant personnel log in to the Alist system through the user interface, upload the model files and extensions files required for the Stable Diffusion raw image model service, and then transfer the model files and extensions files to the network file system of the NFS storage end through the http protocol for storage;

[0120] Secondly, Stable0Diffusion container construction and Stable Diffusion container image push are performed between the Docker container server and the remote Docker Hub warehouse. Specifically, this embodiment pre-downloads the Stable Diffusion model service dependency package in the Docker container server, packages the project code, builds the Docker container product corresponding to Stable Diffusion, and the container image corresponding to the Docker container, and pushes the container image to the remote Docker Hub warehouse for storage;

[0121] Finally, the deployment operation of the StableDiffusion image generation model service is performed between the NFS storage end, the Docker container service end, and the remote Docker Hub warehouse. Specifically, when the Docker container service corresponding to the StableDiffusion service is run in the remote Docker Hub warehouse, the files stored on the NFS storage end are mapped to the StableDiffusion service container persistent volume, so that the Stable Diffusion service container persistent volume contains models files and extensions files, and then the corresponding models files and extensions files are obtained from the Stable Diffusion service container persistent volume. After obtaining the container image at the same time, the deployment operation of the Stable Diffusion image generation model service can be completed.

[0122] It is understandable that if Figure 8 As shown, in the process of service container construction and service deployment, including but not limited to steps S810 to S860:

[0123] Step S810, after obtaining the FROM base image through the command "cloud-aigc-sd:0.0.1", copy (COPY) the host machine warehouse large file (repositories.zip) and the host machine's warm storage folder (. / csche / );

[0124] Step S820, copy (COPY) the module folder (modules), extension folder (extensions) or model folder (models) of the host machine, and determine the corresponding file path mount point in nfs;

[0125] Step S830, run the loaded version (RUN pip install), learning library (PyTorch) and dependent packages, and expose the port (EXPOSE);

[0126] Step S840: When the command input (CMD) is executed to start the container, the Docker container product can be built;

[0127] Step S850: After executing the drag command, the image corresponding to the constructed Docker container product is saved to the Docker remote warehouse;

[0128] Step S860: When running the image service (Docker run Stable Diffusion), the extension folder (extensions) and the model folder (models) can be obtained by accessing the sharing method, and the service deployment operation can be completed by combining the module folder (modules).

[0129] From the above content, it can be seen that the embodiment of the present application mounts NFS storage through the AList file system, so that non-development and operation and maintenance personnel can also upload the large model files of the local workstation to the NFS storage space in a visual manner based on the GUI, thereby realizing the storage resource sharing and reuse of the resources of the large model file among multiple Docker container services; at the same time, by mapping the shared resources of the NFS storage based on the Docker container service persistent volume to the file directory path of the corresponding container service when the service is started, the large model service of the Docker container can be easily loaded into the large model file in the NFS storage, so that the container service of the large model service and the external large model file can be decoupled through the plug-in, and the container image and the large model file can be uploaded in steps, so as to achieve the function of uploading the large model resource once and reusing it multiple times, effectively reducing network overhead and time cost.

[0130] Reference Fig. 9 The embodiment of the present application provides a processing device for a large model service, the device comprising:

[0131] The first module 910 is used to obtain an operation request instruction for a preset large model service;

[0132] The second module 920 is used to construct a service container of a preset large model service and a service image corresponding to the service container according to the operation request instruction;

[0133] The third module 930 is used to store the service image in a preset image warehouse;

[0134] The fourth module 940 is used to decouple and plug-in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction to obtain the second model file and the second extension plug-in file;

[0135] The fifth module 950 is used to deploy the preset large model service according to the service image, the second model file and the second extension plug-in file in the preset image repository.

[0136] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0137] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the processing method of the large model service when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0138] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0139] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0140] The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0141] The memory 1020 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1020 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1020, and the processor 1010 calls and executes the processing method of the large model service in the embodiment of this application;

[0142] Input / output interface 1030, used to implement information input and output;

[0143] The communication interface 1040 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0144] Bus 1050 , which transmits information between various components of the device (e.g., processor 1010 , memory 1020 , input / output interface 1030 , and communication interface 1040 );

[0145] The processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 are connected to each other in communication within the device via a bus 1050 .

[0146] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processing method of the large model service is implemented.

[0147] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0148] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0150] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0151] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0153] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0154] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0156] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0159] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for processing a large model service, characterized in that: The method comprises the following steps: Obtain the operation request instruction of the preset large model service; Constructing a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction; Storing the service image in a preset image warehouse; Decouple and plug-in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction to obtain a second model file and a second extension plug-in file; The preset large model service is deployed according to the service image in the preset image repository, the second model file and the second extension plug-in file.

2. The method according to claim 1, characterized in that The step of constructing the service container of the preset large model service according to the operation request instruction includes: Loading the dependency package of the preset large model service according to the operation request instruction, and packaging the project code of the preset large model service; A service container for the preset large model service is constructed according to the dependency package and the packaged project code.

3. The method according to claim 1, characterized in that The storing the service image in a preset image repository includes: Acquire a first storage operation instruction of the service image, where the first storage operation instruction is triggered by a first preset control of a preset graphical interface; The service image is stored in a preset image repository according to the first storage operation instruction.

4. The method according to claim 1, characterized in that: The decoupling and plugging in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction includes: Acquire the mount configuration information of the preset file system according to the operation request instruction; Acquire a target file list according to the mount configuration information, where the target file list has a mapping relationship with the preset file system; Creating a first folder and a second folder in the target file list; storing the first model file of the preset large model service into the first folder, and storing the first extension plug-in file of the preset large model service into the second folder; The first folder storing the first model file is synchronized to the preset file system according to the mapping relationship, and the second folder storing the first extension plug-in file is synchronized to the preset file system according to the mapping relationship.

5. The method according to claim 4, characterized in that The step of storing the first model file of the preset large model service into the first folder, and storing the first extension plug-in file of the preset large model service into the second folder, comprises: Acquire a second storage operation instruction and a third storage operation instruction, wherein the second storage operation instruction is triggered by a second preset control of the preset graphical interface, and the third storage operation instruction is triggered by a third preset control of the preset graphical interface; According to the second storage operation instruction, the first model file of the preset large model service is stored in the first folder; and according to the third storage operation instruction, the first extension plug-in file of the preset large model service is stored in the second folder.

6. The method according to claim 4, characterized in that The step of synchronizing the first folder storing the first model file to the preset file system according to the mapping relationship, and synchronizing the second folder storing the first extension plug-in file to the preset file system according to the mapping relationship, includes: According to the mapping relationship, synchronizing the first folder storing the first model file to the preset file system through a preset protocol; And according to the mapping relationship, the second folder storing the first extension plug-in file is synchronized to the preset file system through a preset protocol.

7. The method according to claim 1, characterized in that The deploying of the preset large model service according to the service image in the preset image repository, the second model file and the second extension plug-in file includes: Mapping the second model file and the second extension plug-in file to the service container persistent volume to obtain a third model file and a third extension plug-in file; When the running instruction of the service container is obtained, the third model file and the third extension plug-in file are read from the persistent volume of the service container, and the service image is read from the preset image repository; The preset large model service is deployed based on the read third model file, the third extension plug-in file and the service image.

8. A processing device for a large model service, characterized in that: The device comprises: The first module is used to obtain the operation request instruction of the preset large model service; The second module is used to construct a service container of the preset large model service and a service image corresponding to the service container according to the operation request instruction; The third module is used to store the service image in a preset image warehouse; A fourth module is used to decouple and plug-in the first model file and the first extension plug-in file of the preset large model service according to the operation request instruction to obtain a second model file and a second extension plug-in file; The fifth module is used to deploy the preset large model service according to the service image in the preset image repository, the second model file and the second extension plug-in file.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images