Control device, control method, and control program

Abstract

A control device (10) comprises a collection unit (111) that collects information about the the usage situation of resources for executing an AI model from a plurality of data centers, an acquisition unit (112) that acquires splitting information for splitting the AI model, and a deployment unit (113) that deploys AI models obtained by splitting the AI model on the basis of the splitting information to respective target data centers in accordance with the usage situation of the resources at the plurality of data centers.

Description

Control device, control method, and control program

[0001] The present disclosure relates to a control device, a control method, and a control program.

[0002] A technique is known in which an AI model is divided and each divided AI model is deployed to a target node.

[0003] Japanese National Patent Publication No. 2024-529206

[0004] A control device according to an aspect of the present disclosure is a control device capable of communicating with a plurality of data centers, and comprises: a collection unit that collects information on the usage status of resources for executing an AI model from the plurality of data centers; an acquisition unit that acquires division information for dividing the AI model; and a deployment unit that deploys each of the AI models divided based on the division information to each target data center according to the usage status of the resources in the plurality of data centers.

[0005] A control method according to an aspect of the present disclosure is a control method used in a control device capable of communicating with a plurality of data centers, and comprises: a collection step of collecting information on the usage status of resources for executing an AI model from the plurality of data centers; an acquisition step of acquiring division information for dividing the AI model; and a deployment step of deploying each of the AI models divided based on the division information to each target data center according to the usage status of the resources in the plurality of data centers.

[0006] A control program according to an aspect of the present disclosure is a control program for causing a computer to function as a control device, and causes the computer to function as: a collection unit that collects information on the usage status of resources for executing an AI model from the plurality of data centers; an acquisition unit that acquires division information for dividing the AI model; and a deployment unit that deploys each of the AI models divided based on the division information to each target data center according to the usage status of the resources in the plurality of data centers.

[0007] This is a diagram illustrating the outline of the wireless communication system related to this disclosure. This is a block diagram showing the functional configuration of the control device related to this disclosure. This is a diagram illustrating an example of a method for dividing the AI ​​model related to this disclosure. This is a flowchart showing an example of a process executed by the control device related to this disclosure.

[0008] Hereinafter, one embodiment of this disclosure will be described in detail with reference to the drawings. For ease of understanding, the background and challenges of this disclosure will be described first, followed by a detailed description of the disclosure.

[0009] In wireless communication systems such as 5G, distributed systems are known to provide highly secure and low-latency services by distributing data centers equipped with computing infrastructure.

[0010] In recent years, the development of Large Language Models (LLMs) has been active. A common method of providing such LLMs is through cloud services with large amounts of GPU resources. However, accessing such cloud services via the internet results in communication latency issues.

[0011] Now, let's consider providing LLM using the distributed system described above. If the GPU resources of the central data center located in the middle of the network are strained, it is conceivable to provide LLM by utilizing the available GPU resources of distributed data centers located closer to users as a complement to the central data center.

[0012] However, due to GPU resource capacity limitations, a single distributed data center may not have enough capacity to provide LLM (Limited Liability Management). Furthermore, even if capacity is sufficient, a single distributed data center may still be unable to provide LLM depending on the availability of GPU resources.

[0013] Therefore, the control device according to this disclosure includes: a collection unit that collects information on the resource usage status for executing an AI model from multiple data centers; an acquisition unit that acquires division information for dividing the AI ​​model; and a deployment unit that deploys each of the AI ​​models divided based on the division information to each target data center according to the resource usage status in the multiple data centers.

[0014] As described above, by dividing the AI ​​model and deploying each divided AI model to a separate data center, it becomes possible to utilize the available resources in each data center, thus enabling efficient use of GPU resources. Furthermore, since the data centers targeted for deployment are located close to the users, running the AI ​​model in these data centers results in lower latency compared to accessing the cloud via the internet. Details are explained below.

[0015] <Example Configuration of Wireless Communication System 1> Figure 1 is a diagram illustrating an example configuration of wireless communication system 1 according to this embodiment.

[0016] As shown in Figure 1, the wireless communication system 1 includes a central data center 100, a distributed data center 200, and a distributed data center 300. Hereafter, when the distributed data center 200 and the distributed data center 300 are not distinguished, they will simply be referred to as "distributed data centers." Similarly, when the central data center 100, the distributed data center 200, and the distributed data center 300 are not distinguished, they will simply be referred to as "data centers." The central data center 100, the distributed data center 200, and the distributed data center 300 are able to communicate with each other.

[0017] The wireless communication system 1 may be configured as a system that integrates applications for a radio access network (RAN) and AI (Artificial Intelligence) applications on the same platform. Such a system is also called AI-RAN. AI-RAN is a technology that utilizes the computing infrastructure of a base station to provide highly immediate services with low latency to users and devices around the base station. In AI-RAN, AI applications and machine learning applications are deployed at the network edge (terminal side) through the RAN. This configuration promotes the creation of new industries and solutions that leverage low latency and confidentiality. The network may be a mobile communication system such as 3G, 4G, 5G, or 6G, LTE (Long Term Evolution), Wi-Fi (registered trademark), or an in-house LAN.

[0018] The central data center 100 may be distributed among several others within the area covered by the wireless communication system 1. However, the number of central data centers 100 is not limited to several; for example, there may be just one.

[0019] The central data center 100 functions as an orchestrator, collecting and managing metrics such as GPU resources, frequency resources, throughput, packet loss rate, connection time, availability, and error rate from distributed data centers. This manages the data flow across the entire network, enabling efficient communication.

[0020] Multiple (two in Figure 1) distributed data centers 200 and 300 are located in the hierarchical layers below the central data center 100. The number of distributed data centers is not limited to two; many distributed data centers may be located. Furthermore, the distributed data centers may be arranged hierarchically. For example, several more distributed data centers may be located in the hierarchical layers below the distributed data center 200. In addition, some of the distributed data centers may be configured as base stations.

[0021] Distributed data centers process data originating in specific regions, enabling real-time responses. This minimizes latency and allows for services tailored to the specific needs of each region.

[0022] The User Equipment (UE) connected to the distributed data center 200 is typically a user-portable smartphone. Alternatively, the UE may be a tablet, mobile phone, or the like. Another example is a wearable device such as a Head-Mounted Display (HMD) or smartwatch. Yet another example is a stationary Personal Computer (PC), laptop PC, or the like. Finally, the UE may be an electronic device mounted in a vehicle.

[0023] The central data center 100, the distributed data center 200, and the distributed data center 300 may be configured to run both RAN applications and AI applications, as described above. That is, the central data center 100, the distributed data center 200, and the distributed data center 300 may function as a RIC (RAN Intelligent Controller). The RAN may be a virtualized vRAN (Virtual RAN) or a physical RAN.

[0024] The central data center 100, the distributed data center 200, and the distributed data center 300 have container-type virtualization application execution environments such as Docker® installed, enabling them to deploy and run containers. In each data center, a cluster consisting of one or more containers generated by such virtualization technology may be constructed. For example, a Kubernetes cluster managed by a container management tool such as Kubernetes® may be constructed. Processors on the constructed cluster may run containerized RAN applications and AI applications. A container is a collection of files, libraries, etc., necessary to run an application. In Kubernetes, applications are executed by placing containers in pods. A pod is the smallest deployable unit in Kubernetes. Container operations are performed by an orchestration tool, which defines information to identify the node (each data center) on which the container runs, and information to access the services running on the container.

[0025] Figure 1 illustrates a configuration in which a containerized RAN application 210 is deployed to a distributed data center 200, and the distributed data center 200 executes the RAN application 210.

[0026] A control device 10 is provided in the central data center 100. This control device 10 functions as the orchestrator described above. A specific configuration example of the control device 10 will be described later. An example of the orchestrator will be explained. The control device 10 constantly collects and stores metrics from the central data center 100, the distributed data center 200, and the distributed data center 300. For example, the control device 10 acquires resource information for the GPU 101 provided in the central data center 100, resource information for the GPU 201 provided in the distributed data center 200, and resource information for the GPU 301 provided in the distributed data center 300. When the control device 10 receives a request from a user to execute an AI application, it determines whether the requested instance is specified as being deployable in separate instances. As a specific example, the control device 10 may determine whether an environment for executing a partitioned AI model is available in the distributed data center 200 and the distributed data center 300 near the user.

[0027] The control device 10 divides the AI ​​model, containerizes each of the divided AI models, and deploys the containerized AI models to each target data center according to the availability of GPU resources in each data center. Figure 1 illustrates a configuration in which the AI ​​model is divided into three parts, and the divided models 102 to 302 are deployed to each data center.

[0028] The control device 10 refers to metrics from the central data center 100, the distributed data center 200, and the distributed data center 300 to determine the availability of GPU resources.

[0029] <Basic Configuration of Control Device 10> Figure 2 shows an example of the basic configuration of the control device 10. As shown in Figure 2, the control device 10 includes a processor 11, memory 12, storage 13, and a communication I / F 14. These are electrically connected to each other via a communication bus.

[0030] The processor 11 executes instructions written in the program. The processor 11 consists of an arithmetic unit, registers, and / or peripheral circuits. The processor 11 is not particularly limited, but can be implemented as, for example, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or an FPGA (Field-Programmable Gate Array). Although Figure 2 shows one processor, it is not limited to this, and multiple processors may be provided.

[0031] Memory 12 is a computer-readable recording medium that temporarily stores programs and data processed by programs, etc. While not particularly limited, memory 12 can be volatile memory such as DRAM (Dynamic Random Access Memory). Such memory 12 may also be called registers, cache, or main memory.

[0032] Storage 13 is a computer-readable recording medium that stores data and programs. Such storage 13 is composed of, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc. Storage 13 may also be a portable recording medium such as a flexible disk, optical disk, compact disk, or Blu-ray® disk. Storage 13 is sometimes also called an auxiliary storage device.

[0033] Communication I / F 14 is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.

[0034] The control device 10 may be configured as a physical machine or as a virtual machine (VM).

[0035] The processor 11 functions as a data collection unit 111, an acquisition unit 112, and a deployment unit 113 by reading a program from the storage 13 and executing the program using the memory 12 as a workspace. The units described as "~units" here may be replaced with "~circuits," "~devices," or "~equipment," or they may be replaced with "~steps," "~procedures," or "~processes." In other words, the units described as "~units" may be implemented by programs stored in the storage 13 as described above, or by hardware such as elements, devices, boards, and wiring alone, or by a combination of software and hardware. These functions will be described below.

[0036] <Collection Unit 111> The collection unit 111 collects metrics such as GPU resources, frequency resources, throughput, packet loss rate, connection time, availability, and error rate from the distributed data center 200 and the distributed data center 300. The collection unit 111 may also collect metrics from the central data center 100. The storage medium in which the collected metrics are stored is not particularly limited, but for example, the collected metrics may be stored in storage 13.

[0037] <Acquisition Unit 112> The acquisition unit 112 acquires the division information necessary to divide the AI ​​model requested by the user. Such division information may include divisible position information indicating the positions where the AI ​​model can be divided. The method of acquiring the division information is not particularly limited, but for example, the division information may be given as metadata to the AI ​​model and then acquired from the AI ​​model as model information. The division positions of the AI ​​model do not have to be set in advance as described above. For example, the division positions of the AI ​​model may be calculated from other model information (architecture information including the number of parameters, number of bits, etc.) and collected metrics. The division information may also include network requirements required to execute the divided AI model. The acquisition unit 112 outputs the acquired division information to the deployment unit 113.

[0038] <Deployment Unit 113> The deployment unit 113 divides the AI ​​model based on the division information acquired from the acquisition unit 112 and deploys each divided AI model to each data center. Here, an example of how to divide the AI ​​model will be explained with reference to Figure 3.

[0039] <An example of how to divide an AI model> Figure 3 illustrates a case where the AI ​​model is a deep learning model consisting of 1 to 30 layers. As mentioned above, in conventional LLM provision processing, all 1 to 30 layers are loaded into the cloud, and batch output is performed for batch input.

[0040] In contrast, in this embodiment, the AI ​​model is divided based on the divisible position information and collected metrics included in the division information described above. Here, we assume that the divisible position information indicates that the model is divisible between layers 10 and 11, and between layers 20 and 21. In this case, the deployment unit 113 can divide the model into three parts: layers 1 to 10 as divided model 102, layers 11 to 20 as divided model 202, and layers 21 to 30 as divided model 302. The algorithm for dividing the AI ​​model and executing the divided AI models is not particularly limited, but for example, a pipeline parallel may be used.

[0041] <Pipeline Parallelization> Pipeline parallelization is a technique that divides a process, task, etc., into multiple stages and executes each stage in parallel. In pipeline parallelization, while one process, task, etc., is being processed in one stage, another process, task, etc., is being processed in another stage. For example, the stages of data input, processing, and output are each processed in parallel by different processors or threads. This allows multiple processes, tasks, etc., to be processed simultaneously, thus improving processing efficiency.

[0042] Regarding which data center to deploy the partitioned model 102, partitioned model 202, and partitioned model 302 partitioned by pipeline parallelism to, the deployment unit 113 can make the determination based on the availability of GPU resources in each data center, the network requirements required to execute the partitioned AI model, and the like. FIG. 3 illustrates an aspect where, based on the availability of GPU resources, network requirements, and the like, the partitioned model 102 is loaded onto the GPU 101 of the central data center 100, the partitioned model 202 is loaded onto the GPU 201 of the distributed data center 200, and the partitioned model 302 is loaded onto the GPU 301 of the distributed data center 300.

[0043] A batch input to the partitioned model 102 is processed by the partitioned model 102, and output as an input batch to the partitioned model 202. A batch input to the partitioned model 202 is processed by the partitioned model 202, and output as an input batch to the partitioned model 302. A batch input to the partitioned model 302 is processed by the partitioned model 302 and then output. In this way, according to pipeline parallelism, data input, processing, and output are processed in parallel, thereby improving processing efficiency.

[0044] <Processing Flow> Next, the flow of processing executed by the control device 10 will be described with reference to FIG. 4. FIG. 4 is a flowchart illustrating an example of processing executed by the control device 10. In this flowchart, the collection unit 111, acquisition unit 112, and deployment unit 113, which are the functions of the control device 10, will be mainly described, and some other functions will also be described.

[0045] <Step S101> In step S101, when the control device 10 receives a request to execute an AI model from a user, it determines whether the requested instance is specified as being capable of distributed deployment. As described above, the control device 10 may determine whether an environment for executing the divided AI model is prepared in a distributed data center near the user. If the determination in step S101 is NO, the process proceeds to step S102, and the control device 10 deploys the AI model to a single data center without dividing the AI model. The "single data center" referred to here may be any data center as long as it has GPU resources capable of executing the AI model. On the other hand, if the determination in step S101 is YES, the process proceeds to step S103.

[0046] <Step S103> In step S103, the collection unit 111 collects metrics such as GPU resources, frequency resources, throughput, packet loss rate, connection time, availability, and error rate from each data center (for example, the central data center 100, the distributed data center 200, and the distributed data center 300). Note that since the collection unit 111 collects metrics at all times, the collection process in step S102 may be performed before the determination process in step S101.

[0047] <Step S104> In step S104, the acquisition unit 112 acquires division information necessary for dividing the AI model requested by the user (for example, dividable position information used in pipeline parallelism), network requirements necessary for executing the divided AI model, and the like.

[0048] <Step S105> In step S105, the deployment unit 113 refers to metrics of distributed data centers that are deployment candidates (for example, the availability of free GPU resources) to determine whether deployment is possible. If the determination in step S105 is NO, the process proceeds to step S102, and the control device 10 deploys the AI model to a single data center without dividing the AI model. On the other hand, if the determination in step S105 is YES, the process proceeds to step S106.

[0049] <Step S106> In step S106, the deployment unit 113 divides the AI ​​model using pipeline parallelism, containerizes each of the divided AI models, and deploys the containerized AI models to each target data center according to the availability of GPU resources in each data center.

[0050] As described above, the control method according to this embodiment includes a collection step (S103) of collecting information on the resource usage status for executing an AI model from a plurality of data centers, an acquisition step (S104) of acquiring partitioning information for partitioning an AI model, and a deployment step (S106) of deploying each of the partitioned AI models to the target data center according to the resource usage status in the plurality of data centers.

[0051] Note that the processing flow shown in the flowchart in Figure 4 is just one example, and steps may be deleted, new steps added, or the processing order rearranged as long as it does not deviate from the main point.

[0052] <Effects and Effects> As described above, the following effects and effects can be obtained according to this embodiment.

[0053] The control device 10 is a device capable of communicating with multiple data centers (for example, a central data center 100, a distributed data center 200, and a distributed data center 300). The control device 10 includes a collection unit 111 that collects information on the resource usage status for executing an AI model (for example, LLM) from the multiple data centers, an acquisition unit 112 that acquires partitioning information for partitioning the AI ​​model, and a deployment unit 113 that deploys each of the partitioned AI models to the target data center according to the resource usage status in the multiple data centers.

[0054] As described above, by dividing the AI ​​model and deploying each divided AI model to a separate data center, it becomes possible to utilize the available GPU resources in each data center, thus enabling efficient use of GPU resources. Furthermore, since the data center to which the AI ​​model is deployed (for example, distributed data center 200) is located close to the user, running the AI ​​model in that data center results in lower latency compared to accessing the cloud via the internet.

[0055] In recent years, "multimodal AI," which integrates and processes data in different formats such as text data, audio data, image data, video data, and sensor data using AI, has emerged, making the growth of Large-Scale Memory (LLMs) inevitable. As LLMs grow larger, the required GPU resources also increase, which presents a problem. However, with the above configuration, the AI ​​model can be divided and provided separately, making it possible to handle even massive LLMs.

[0056] The control device 10 may also deploy each of the AI ​​models, which have been divided using pipeline parallelism, to each target data center. Alternatively, the control device 10 may containerize each of the AI ​​models, which have been divided using pipeline parallelism, and deploy the containerized AI models to each target data center.

[0057] According to the above configuration, processing efficiency is improved because multiple processes and tasks of the AI ​​model can be processed simultaneously.

[0058] <Other Embodiments> In the embodiments described above, an example was given in which the divided AI model is deployed to the central data center 100, the distributed data center 200, and the distributed data center 300, but the embodiments are not limited thereto. For example, the divided AI model may be deployed only to each of the distributed data centers arranged hierarchically below the central data center 100. This further improves latency by processing the AI ​​model closer to the user.

[0059] Furthermore, when an AI model is divided, parts of the AI ​​model may be shared between different systems (a system is defined as the entire AI model from the first layer to the last layer). For example, suppose an AI model is divided into two, with one divided model 102 deployed on the GPU 101 of the central data center 100, and the other divided model 202 deployed on the GPU 201 of the distributed data center 200 (see Figure 1). Subsequently, even if the divided model 202 is deployed on the GPU 301 of the distributed data center 300 due to changes in metrics, the divided model 102 deployed on the GPU 101 of the central data center 100 can still be utilized. Sharing parts of the AI ​​model in this way enables flexible responses to the ever-changing communication environment.

[0060] In the above explanation, LLM was used as an example of an AI model, but AI models are not limited to large-scale language models like LLM. AI models may be large-scale AI models other than large-scale language models. For example, diffusion models used for image generation can be cited as large-scale AI models. Deep learning model partitioning using layer-direction partitioning, similar to pipeline parallelism, is also possible for large-scale AI models other than language models.

[0061] <Example of implementation by software> The functions of the control device 10 (hereinafter simply referred to as "device") can be realized by a control program that causes a computer to function as the device, and by a program that causes a computer to function as each control block of the device (particularly the collection unit 111, the acquisition unit 112, and the deployment unit 113).

[0062] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0063] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0064] Furthermore, some or all of the functions of each of the above control blocks can also be implemented by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of this disclosure. In addition, it is also possible to implement the functions of each of the above control blocks by, for example, a quantum computer.

[0065] Furthermore, each of the processes described in the above embodiments may be executed by AI. In this case, the AI ​​may operate on the control device described above, or it may operate on other devices (for example, an edge computer or a cloud server).

[0066] <Summary> The control device according to Embodiment 1 of the present disclosure is a control device capable of communicating with a plurality of data centers, and comprises: a collection unit that collects information from the plurality of data centers regarding the usage status of resources for executing an AI model; an acquisition unit that acquires division information for dividing the AI ​​model; and a deployment unit that deploys each of the AI ​​models divided based on the division information to each target data center according to the resource usage status in the plurality of data centers.

[0067] In the second aspect of this disclosure, the control device may be configured such that, in the first aspect described above, the deployment unit deploys each of the AI ​​models, which have been divided using pipeline parallelism, to each target data center.

[0068] The control device according to aspect 3 of the present disclosure may be configured such that, in aspect 1 or 2 above, the deployment unit containerizes each AI model divided using the pipeline parallel and deploys the containerized AI models to each target data center.

[0069] The control device according to aspect 4 of the present disclosure may be configured such that, in any one of aspects 1 to 3 above, the AI ​​model is a large-scale AI model.

[0070] A control method according to aspect 5 of the present disclosure is a control method used in a control device capable of communicating with a plurality of data centers, and includes a collection step of collecting information from the plurality of data centers regarding the usage status of resources for executing an AI model; an acquisition step of obtaining division information for dividing the AI ​​model; and a deployment step of deploying each of the AI ​​models divided based on the division information to each target data center according to the resource usage status in the plurality of data centers.

[0071] A control program according to aspect 6 of the present disclosure is a control program for causing a computer to function as a control device, wherein the computer is configured to function as: a collection unit that collects information on the usage status of resources for executing an AI model from the plurality of data centers; an acquisition unit that acquires division information for dividing the AI ​​model; and a deployment unit that deploys each of the AI ​​models divided based on the division information to each target data center according to the resource usage status in the plurality of data centers.

[0072] <Additional Notes> This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure.

[0073] This disclosure demonstrates that by utilizing available GPU resources in each data center to run AI models, low latency can be achieved, creating an innovative technological foundation for the telecommunications business and contributing to the achievement of Sustainable Development Goal 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

[0074] 1 Wireless communication system 10 Control unit 11 Processor 12 Memory 13 Storage 14 Communication interface 100 Central data center 200, 300 Distributed data centers 101, 201, 301 GPU 102, 202, 302 Split model 111 Collection unit 112 Acquisition unit 113 Deployment unit

Claims

1. A control device capable of communicating with multiple data centers, comprising: a collection unit that collects information from the multiple data centers regarding the usage status of resources for executing an AI model; an acquisition unit that acquires division information for dividing the AI ​​model; and a deployment unit that deploys each of the AI ​​models divided based on the division information to each target data center according to the resource usage status in the multiple data centers.

2. The control device according to claim 1, wherein the deployment unit deploys each of the divided AI models using pipeline parallelism to each target data center.

3. The control device according to claim 2, wherein the deployment unit containerizes each of the AI ​​models divided using the pipeline parallel and deploys the containerized AI models to each target data center.

4. The control device according to claim 1, wherein the AI ​​model is a large-scale AI model.

5. A control method used in a control device capable of communicating with multiple data centers, comprising: a collection step of collecting information from the multiple data centers regarding the usage status of resources for executing an AI model; an acquisition step of obtaining division information for dividing the AI ​​model; and a deployment step of deploying each of the AI ​​models divided based on the division information to each target data center according to the resource usage status in the multiple data centers.

6. A control program for causing a computer to function as a control device, the control program causing the computer to function as: an information collection unit that collects information from multiple data centers regarding the usage status of resources for executing an AI model; an information acquisition unit that acquires information for dividing the AI ​​model; and a deployment unit that deploys each of the AI ​​models divided based on the information to the target data center according to the resource usage status in the multiple data centers.

Images