Method and device for deploying large language model based on computing power of intelligent computing center

Abstract

The invention provides a method and device for deploying a large language model based on computing power of an intelligent computing center, and the method is applied to a VCluster cluster of the intelligent computing center, and comprises the steps: S1, obtaining a preset configuration file which comprises the name of a target large language model and the network address of a private cloud platform; s2, according to the name of the target large language model, obtaining an encapsulation file of the target large language model encapsulated in NVIDIA NIM service, and according to the network address of the private cloud platform, transmitting and storing the encapsulation file to the private cloud platform; and S3, deploying the target large model to the private cloud platform according to the packaging file. The target large model deployment method has high security and high deployment efficiency.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of computing power infrastructure, and in particular, to a method and device for deploying a large language model based on the computing power of an intelligent computing center. Background Art

[0002] With the development of artificial intelligence technology and computing power technology, the concept of intelligent computing center has emerged. "Intelligent computing center" refers to the use of large-scale heterogeneous computing power resources, including general computing power and intelligent computing power, mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithms. Intelligent computing center covers facilities, hardware, software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0003] The Intelligent Computing Center supports VCluster clusters. VCluster (virtual cluster) is a technology used in cloud computing and virtualization environments that allows users to create multiple virtual clusters on a physical server. These virtual clusters can run independently and simulate the behavior of real clusters. They are usually used in scenarios such as development, testing, and resource isolation.

[0004] Existing large language models are often deployed directly in the production environment. The operation of large language models will affect users in the production environment, especially when the large language model fails, which will interfere with users in the production environment and have low security. In addition, the production environment is mostly a non-containerized environment. In order to deploy large language models, the preparation and management of resources in the production environment require special planning, which increases the difficulty of deploying large language models, and the deployment convenience and efficiency are low. In addition, the encapsulated files of large language models have a huge amount of data. When deploying large language models, the encapsulated files need to be solved. The existing deployment methods have slow solution rates and low deployment efficiency. Summary of the invention

[0005] The embodiment of the present invention provides a method and device for deploying a large language model based on the computing power of an intelligent computing center, so as to solve the problems that the existing large language model is often deployed directly in a production environment, the operation of the large language model will affect users in the production environment, especially when the large language model fails, it will interfere with users in the production environment, and the security is low; and the production environment is mostly a non-containerized environment. In order to deploy the large language model, the preparation and management of resources in the production environment need to be specially planned, which increases the difficulty of deploying the large language model, and the deployment convenience and efficiency are low. In addition, the encapsulated file of the large language model has a huge amount of data. When deploying the large language model, the encapsulated file needs to be solved. The existing deployment method has a slow solution rate and low deployment efficiency.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a method for deploying a large language model based on the computing power of an intelligent computing center, which is applied to a VCluster cluster of an intelligent computing center, including:

[0008] Step S1: Obtain a preset configuration file, wherein the configuration file includes: the name of the target large language model and the network address of the private cloud platform;

[0009] Step S2: according to the name of the target large language model, obtaining a package file of the target large language model packaged in the NVIDIA NIM service, and delivering and storing the package file to the private cloud platform according to the network address of the private cloud platform;

[0010] Step S3: deploying the target large model to the private cloud platform according to the encapsulation file.

[0011] Optionally, the configuration file further includes: a cluster name, a service name, and a port number corresponding to the service name of the VCluster cluster;

[0012] The step S2 comprises:

[0013] Step S21: according to the cluster name, the service name and the port number, enable the ingress function of the VCluster cluster to access the NVIDIA NIM service from outside;

[0014] Step S22: Obtaining an acquisition permission, wherein the acquisition permission is used to allow the VCluster cluster to obtain the package file from the NVIDIANIM service;

[0015] Step S23: Based on the ingress function and the acquisition permission, obtain the package file from the NVIDIA NIM service.

[0016] Optionally, the ingress function of the VCluster cluster accessing the NVIDIA NIM service from outside includes:

[0017] Access the NVIDIA NIM service through the curl command-line tool, and / or the API interface.

[0018] Optionally, a simple storage service is used as a delivery relay for delivering the packaged file from the NVIDIA NIM service to the private cloud platform.

[0019] Optionally, step S2 includes:

[0020] Step S24: Obtain the package file from an image repository that hosts the image of the NVIDIA NIM service.

[0021] Optionally, the private cloud platform is a NAS private cloud platform.

[0022] In a second aspect, an embodiment of the present invention provides a device for deploying a large language model based on the computing power of an intelligent computing center, which is applied to a VCluster cluster of an intelligent computing center, including:

[0023] An acquisition module, used in step S1: acquiring a preset configuration file, wherein the configuration file includes: a name of a target large language model and a network address of a private cloud platform;

[0024] An execution module, for step S2: obtaining a package file of the target large language model packaged in the NVIDIA NIM service according to the name of the target large language model, and delivering and storing the package file to the private cloud platform according to the network address of the private cloud platform;

[0025] The deployment module is used in step S3: deploying the target large model to the private cloud platform according to the encapsulation file.

[0026] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the method for deploying a large language model based on the computing power of an intelligent computing center as described in any one of the first aspects.

[0027] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the method for deploying a large language model based on the computing power of an intelligent computing center as described in any one of the first aspects are implemented.

[0028] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method for deploying a large language model based on the computing power of an intelligent computing center as described in any one of the first aspects.

[0029] In the embodiment of the present invention, through step S1: obtaining a preset configuration file, the configuration file includes: the name of the target large language model, the network address of the private cloud platform; step S2: according to the name of the target large language model, obtaining the encapsulation file of the target large language model encapsulated in the NVIDIA NIM service, and, according to the network address of the private cloud platform, delivering and storing the encapsulation file to the private cloud platform; step S3: according to the encapsulation file, deploying the target large model to the private cloud platform, the embodiment of the present invention can realize the deployment of the target large language model on the private cloud platform, the private cloud platform is isolated from the production environment, so that the operation of the target large language model will not affect the users in the production environment, and has high security. In addition, the production environment is isolated, and the embodiment of the present invention does not need to specially plan the preparation and management of resources in the production environment when deploying the target large language model, which reduces the difficulty of deploying the large language model, and the deployment is convenient and efficient. Further, the embodiment of the present invention uses the computing power of the intelligent computing center to realize the deployment of the large language model, and the solution rate of the encapsulated file is high, which improves the deployment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 This is a flow chart of a method for deploying a large language model based on the computing power of an intelligent computing center according to an embodiment of the present invention;

[0032] Figure 2 This is a second flow chart of a method for deploying a large language model based on the computing power of an intelligent computing center according to an embodiment of the present invention;

[0033] Figure 3 It is a principle block diagram of a device for deploying a large language model based on the computing power of an intelligent computing center according to an embodiment of the present invention;

[0034] Figure 4 The figure is a principle block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0037] In the technical solution of the present disclosure, words such as “connect”, “couple” or “connected” are not limited to physical or mechanical connections, but may include electrical connections.

[0038] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0039] First, the technical terms involved in the present invention are briefly explained below.

[0040] The "computing power" mentioned in the present invention is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to execute certain computing requirements. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to the society through computing power infrastructure.

[0041] The "computing power" (Computational Power, CP) described in the present invention is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating point operations performed per second (FLOPS: Floating Point Operations Per Second, 1EFLOPS = 10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0042] The "Network Power" (NP) described in the present invention is a manifestation of the data transmission capability of computing facilities, including comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. The carrying capacity involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities. In the embodiment of the present invention, the carrying capacity uses the memory bandwidth.

[0043] The "Storage Power" (SP) described in the present invention is the comprehensive ability of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB = 2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability.

[0044] The "computing power infrastructure" described in the present invention is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission and application of information, and presents characteristics such as diversity and ubiquity, intelligence and agility, safety and reliability, green and low-carbon. It is of great significance to promote industrial transformation and upgrading, enable my country's scientific and technological innovation, meet people's better life and realize high-efficiency social governance.

[0045] The “computing power” mentioned in the present invention includes general computing power, intelligent computing power and super computing power.

[0046] The “general computing power” mentioned in the present invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0047] The "intelligent computing power" described in the present invention is a computing platform for large-scale deployment of special chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) for various innovative applications of artificial intelligence, such as natural language processing and machine vision.

[0048] The "super computing power" mentioned in the present invention is mainly the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0049] The "intelligent computing center" described in the present invention refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning scenarios) by using large-scale heterogeneous computing power resources, including general computing power (CPU: Central Processing Unit) and intelligent computing power (GPU: Graphics Processing Unit, FPGA: Field Programmable Gate Array, ASIC: Application Specific Integrated Circuit, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0050] The "computing resources" mentioned in the present invention refer to the technologies and facilities with information calculation, transmission, storage and application capabilities required for the development of the digital society, including but not limited to computing resources such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, as well as supporting and guarantee resources such as wind, fire, water and electricity.

[0051] The VCluster (virtual cluster) described in the present invention is a technology used in cloud computing and virtualization environments, allowing users to create multiple virtual clusters on a physical server. These virtual clusters can run independently, simulating the behavior of real clusters, and are usually used in scenarios such as development, testing, and resource isolation.

[0052] The "large language model" described in the present invention refers to a large language model (LLM), which is a language model with a large parameter scale, designed to understand and generate human language, trained with a large amount of text data, and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc. Among them, the large language model can be a large language model such as GPT4, GLM-4 or Llama-3.

[0053] NVIDIA NIM is a set of optimized enterprise-grade generative AI microservices that includes an inference engine and provides services to the outside world through an API interface, helping enterprises and individual developers to more easily develop and deploy AI-driven applications.

[0054] NIM uses industry-standard APIs to support AI use cases across multiple fields, including LLM, visual language model (VLM), and models for speech, image, video, 3D, drug development, medical imaging, etc. At the same time, it is based on NVIDIA Triton TM Inference Server, NVIDIA TensorRT TM , NVIDIA TensorRT-LLM, and PyTorch to provide optimal latency and throughput on accelerated infrastructure.

[0055] To further reduce complexity, NIM decouples the model and the runtime environment and packages each model or model series in the form of a container image.

[0056] The intelligent computing center VCluster deeply integrates the new generation of cloud native technology to provide high-performance Kubernetes container cluster management services with containers as the core. It can provide NIM with a stable, reliable and high-performance operating environment, and realize a powerful combination of model use and operation.

[0057] At the same time, the release and operation of model services are also inseparable from release management, network access, observation and other capabilities. VCluster deeply integrates the intelligent computing center's high-performance computing (ECS / bare metal), network (VPC / EIP / CLB), storage (EBS / TOS / NAS), elastic container instance (VCI) and other services, and is horizontally connected with cloud products such as image warehouse, continuous delivery, managed Prometheus, log service, microservice engine, etc., which can realize the full-link process of NIM service construction, deployment, release, monitoring, etc., helping enterprises to build and expand customized large language models (LLMs) based on their own data more flexibly and agilely, and create a truly enterprise-level intelligent and automated infrastructure.

[0058] The embodiment of the present invention provides a method for deploying a large language model based on the computing power of an intelligent computing center, which is applied to a VCluster cluster of an intelligent computing center. Figure 1 As shown, Figure 1 This is a flow chart of a method for deploying a large language model based on the computing power of an intelligent computing center according to an embodiment of the present invention, including:

[0059] Step S1: Obtain a preset configuration file, the configuration file including: the name of the target large language model and the network address of the private cloud platform;

[0060] Step S2: according to the name of the target large language model, obtain the package file of the target large language model packaged in the NVIDIA NIM service, and transmit and store the package file to the private cloud platform according to the network address of the private cloud platform;

[0061] Step S3: Deploy the target large model to the private cloud platform according to the encapsulation file.

[0062] It should be noted that a configuration file refers to a file used to store software or system settings, usually in text format. The configuration file includes the behavior, parameters and options of the system defined by the user or administrator. Specifically in the embodiment of the present invention, the configuration file includes: the name of the target large language model and the network address of the private cloud platform.

[0063] Based on the name of the target large language model in the configuration file and the network address of the private cloud platform, the embodiment of the present invention further executes step S2, that is, according to the name of the target large language model, obtaining the encapsulated file of the target large language model encapsulated in the NVIDIA NIM service, and, according to the network address of the private cloud platform, delivering and storing the encapsulated file to the private cloud platform.

[0064] In some embodiments of the present invention, the encapsulation file may be a containerized file of the target large language model. Specifically, containerization includes: using Docker to package the target language model and its dependencies into a container for easy deployment and management. Create a Dockerfile (equivalent to the encapsulation file in the present invention) to define how to build a container image.

[0065] It should be noted that, in the embodiment of the present invention, deploying the target large model to the private cloud platform refers to the process of migrating the target large language model to the private cloud platform (ie, the production environment) so that the target large language model can perform actual reasoning or prediction.

[0066] During the execution of the embodiment of the present invention, the VCluster cluster can call the computing power resources of the intelligent computing center. For example, in the deployment process, the decoding of packaged files and other tasks are involved. The computing power of the intelligent computing center can be used to achieve rapid solution and improve deployment efficiency. For example, based on the "carrying capacity" of the computing power resources of the intelligent computing center, the rate at which packaged files are transferred and stored to the private cloud platform can be increased, thereby improving deployment efficiency.

[0067] In the embodiment of the present invention, through step S1: obtaining a preset configuration file, the configuration file includes: the name of the target large language model, the network address of the private cloud platform; step S2: according to the name of the target large language model, obtaining the encapsulation file of the target large language model encapsulated in the NVIDIA NIM service, and, according to the network address of the private cloud platform, delivering and storing the encapsulation file to the private cloud platform; step S3: according to the encapsulation file, deploying the target large model to the private cloud platform, the embodiment of the present invention can realize the deployment of the target large language model on the private cloud platform, the private cloud platform is isolated from the production environment, so that the operation of the target large language model will not affect the users in the production environment, and has high security. In addition, the production environment is isolated, and the embodiment of the present invention does not need to specially plan the preparation and management of resources in the production environment when deploying the target large language model, which reduces the difficulty of deploying the large language model, and the deployment is convenient and efficient. Further, the embodiment of the present invention uses the computing power of the intelligent computing center to realize the deployment of the large language model, and the solution rate of the encapsulated file is high, which improves the deployment efficiency.

[0068] In some embodiments of the present invention, optionally, the configuration file further includes: a cluster name of the VCluster cluster, a service name, and a port number corresponding to the service name;

[0069] See also Figure 2 As shown, Figure 2 This is a second flow chart of a method for deploying a large language model based on the computing power of an intelligent computing center according to an embodiment of the present invention. Step S2 includes:

[0070] Step S21: According to the cluster name, service name and port number, enable the ingress function for external access to the NVIDIANIM service of the VCluster cluster;

[0071] Step S22: Obtaining the acquisition permission, where the acquisition permission is used to allow the VCluster cluster to obtain the package file from the NVIDIA NIM service;

[0072] Step S23: Based on the ingress function and the obtained permissions, obtain the package file from the NVIDIA NIM service.

[0073] It should be noted that Ingress is a resource type in Kubernetes that is used to manage the way external users access services within the cluster. It provides a simple way to expose HTTP and HTTPS routes to services within the cluster. The Ingress function allows access rules for multiple services to be managed through a single entry point (usually a load balancer). Specifically in the embodiment of the present invention, the Ingress function allows the VCluster cluster to manage the access rules of the NVIDIA NIM service through a single entry point, so that the VCluster cluster can access the NVIDIA NIM service externally, which can achieve convenient access and have higher access efficiency.

[0074] In some embodiments of the present invention, optionally, the ingress function of externally accessing the NVIDIA NIM service of the VCluster cluster includes:

[0075] Access the NVIDIA NIM service through the curl command-line tool and / or the API interface.

[0076] In the embodiment of the present invention, by accessing the NVIDIA NIM service through the curl command line tool and / or the API interface, secure and fast access to the NVIDIA NIM service can be achieved.

[0077] In some embodiments of the present invention, a simple storage service is optionally used as a delivery relay for delivering packaged files from the NVIDIA NIM service to the private cloud platform.

[0078] Simple Storage Service (S3) refers to Amazon S3, an object storage service provided by Amazon Web Services (AWS). S3 allows users to store and retrieve any amount of data and is suitable for a variety of purposes, including backup, archiving, data analysis, and content distribution. Here are some key points about the Simple Storage Service.

[0079] S3 provides up to 99.9999999999% (11 nines) of data persistence, and data is automatically redundantly stored in multiple physical locations to ensure high availability. S3 provides multi-layer security mechanisms, including access control lists (ACLs), bucket policies, IAM (identity and access management) policies, etc., to ensure data security. In an embodiment of the present invention, by using a simple storage service as a transmission relay for transmitting packaged files from the NVIDIA NIM service to the private cloud platform, the efficiency of data transmission from the NVIDIA NIM service to the private cloud platform can be improved, and high data security is achieved during the transmission process.

[0080] In some embodiments of the present invention, optionally, step S2 includes:

[0081] Step S24: Obtain a package file from an image repository hosting an image of the NVIDIA NIM service.

[0082] In the embodiment of the present invention, the packaged file can be obtained without accessing the NVIDIA NIM service, but from the image repository of the image hosting the NVIDIA NIM service. The complex operations and authentication procedures required for external access to the NVIDIA NIM service are avoided, and the efficiency of obtaining the packaged file can be improved. In some optional embodiments, the image repository is locally deployed, which can further improve the efficiency of obtaining the packaged file.

[0083] In some embodiments of the present invention, optionally, the private cloud platform is a NAS private cloud platform.

[0084] Network Attached Storage (NAS) is a convenient storage solution for users and businesses that need centralized storage and file sharing. Through its flexibility and scalability, NAS can meet a variety of storage needs for homes and businesses.

[0085] NAS private cloud platform is to use NAS (Network Attached Storage) devices as private cloud platform. NAS private cloud platform allows users to centrally store and manage data in their own network, while providing functions similar to public cloud services. By configuring NAS devices as private clouds, users can enjoy the flexibility and security of data access, backup and sharing. Using NAS devices to build a private cloud platform is a flexible and secure data storage solution suitable for home and enterprise users. Through centralized storage, remote access and rich functions, users can effectively manage and protect their data.

[0086] The embodiment of the present invention provides a device for deploying a large language model based on the computing power of an intelligent computing center, which is applied to a VCluster cluster of an intelligent computing center. Figure 3 As shown, Figure 3 This is a principle block diagram of an apparatus for deploying a large language model based on the computing power of an intelligent computing center according to an embodiment of the present invention. The apparatus 30 for deploying a large language model based on the computing power of an intelligent computing center includes:

[0087] The acquisition module 31 is used in step S1: acquiring a preset configuration file, wherein the configuration file includes: the name of the target large language model and the network address of the private cloud platform;

[0088] The execution module 32 is used for step S2: obtaining a package file of the target large language model packaged in the NVIDIA NIM service according to the name of the target large language model, and delivering and storing the package file to the private cloud platform according to the network address of the private cloud platform;

[0089] The deployment module 33 is used in step S3: deploying the target large model to the private cloud platform according to the encapsulation file.

[0090] In some embodiments of the present invention, optionally, the configuration file further includes: a cluster name of the VCluster cluster, a service name, and a port number corresponding to the service name;

[0091] The execution module 32 is also used in step S21: according to the cluster name, the service name and the port number, enable the ingress function of the VCluster cluster to access the NVIDIA NIM service externally;

[0092] The execution module 32 is further used in step S22: obtaining an acquisition permission, wherein the acquisition permission is used to allow the VCluster cluster to obtain the package file from the NVIDIA NIM service;

[0093] The execution module 32 is also used in step S23: based on the ingress function and the acquisition permission, obtain the package file from the NVIDIA NIM service.

[0094] In some embodiments of the present invention, optionally, the ingress function of the VCluster cluster externally accessing the NVIDIA NIM service includes:

[0095] Access the NVIDIA NIM service through the curl command-line tool, and / or the API interface.

[0096] In some embodiments of the present invention, optionally, a simple storage service is used as a delivery relay for delivering the packaged file from the NVIDIA NIM service to the private cloud platform.

[0097] The execution module 32 is further used in step S24: obtaining the package file from the image repository hosting the image of the NVIDIA NIM service.

[0098] In some embodiments of the present invention, optionally, the private cloud platform is a NAS private cloud platform.

[0099] The device for deploying a large language model based on the computing power of an intelligent computing center provided in the embodiment of the present application can achieve Figure 1 to Figure 2The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0100] An embodiment of the present invention provides an electronic device 40, see Figure 4 As shown, Figure 4 This is a principle block diagram of an electronic device 40 according to an embodiment of the present invention, including a processor 41, a memory 42, and a program or instruction stored in the memory 42 and executable on the processor 41. When the program or instruction is executed by the processor, any step in the method for deploying a large language model based on the computing power of an intelligent computing center according to the present invention is implemented.

[0101] An embodiment of the present invention provides a readable storage medium, on which programs or instructions are stored. When the programs or instructions are executed by a processor, the various processes of an embodiment of a method for deploying a large language model based on the computing power of an intelligent computing center, such as any of the above, are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0102] The readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0103] An embodiment of the present invention also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of any of the above-mentioned method embodiments for deploying a large language model based on the computing power of an intelligent computing center are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0104] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for deploying a large language model based on the computing power of an intelligent computing center, characterized in that: VCluster clusters used in intelligent computing centers include: Step S1: Obtain a preset configuration file, wherein the configuration file includes: the name of the target large language model and the network address of the private cloud platform; Step S2: according to the name of the target large language model, obtaining a package file of the target large language model packaged in the NVIDIA NIM service, and delivering and storing the package file to the private cloud platform according to the network address of the private cloud platform; Step S3: deploying the target large model to the private cloud platform according to the encapsulation file.

2. The method for deploying a large language model based on the computing power of an intelligent computing center according to claim 1, characterized in that: The configuration file also includes: the cluster name of the VCluster cluster, the service name, and the port number corresponding to the service name; The step S2 comprises: Step S21: according to the cluster name, the service name and the port number, enable the ingress function of the VCluster cluster to access the NVIDIANIM service from outside; Step S22: Acquire the acquisition permission, where the acquisition permission is used to allow the VCluster cluster to obtain the package file from the NVIDIA NIM service; Step S23: Based on the ingress function and the acquisition permission, obtain the package file from the NVIDIA NIM service.

3. The method for deploying a large language model based on the computing power of an intelligent computing center according to claim 2, characterized in that: The ingress function of the VCluster cluster to access the NVIDIA NIM service externally includes: Access the NVIDIA NIM service through the curl command-line tool, and / or the API interface.

4. The method for deploying a large language model based on the computing power of an intelligent computing center according to claim 2 or 3, characterized in that: A simple storage service is used as a delivery relay for delivering the packaged file from the NVIDIA NIM service to the private cloud platform.

5. The method for deploying a large language model based on the computing power of an intelligent computing center according to claim 1, characterized in that: The step S2 comprises: Step S24: Obtain the package file from an image repository that hosts the image of the NVIDIA NIM service.

6. The method for deploying a large language model based on the computing power of an intelligent computing center according to claim 1, characterized in that: The private cloud platform is a NAS private cloud platform.

7. A device for deploying a large language model based on the computing power of an intelligent computing center, characterized in that: VCluster clusters used in intelligent computing centers include: An acquisition module, used in step S1: acquiring a preset configuration file, wherein the configuration file includes: a name of a target large language model and a network address of a private cloud platform; An execution module, for step S2: obtaining a package file of the target large language model packaged in the NVIDIA NIM service according to the name of the target large language model, and delivering and storing the package file to the private cloud platform according to the network address of the private cloud platform; The deployment module is used in step S3: deploying the target large model to the private cloud platform according to the encapsulation file.

8. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the method for deploying a large language model based on the computing power of an intelligent computing center as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, which, when executed by a processor, implement the steps in the method for deploying a large language model based on the computing power of an intelligent computing center as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps of the method for deploying a large language model based on the computing power of an intelligent computing center as described in any one of claims 1 to 6.

Images