Information processing device, multi-agent LLM system, and control program

Abstract

An information processing device (1) comprises: an acquisition unit (11) that acquires DC metrics, which are metrics of each of a plurality of data centers (20, 30), and LLM metrics, which are metrics of each of a plurality of agent LLMs provided in the plurality of data centers (20, 30); and an eviction candidate determination unit (12) that determines one or more eviction candidates, which are candidates to be evicted, from among the plurality of agent LLMs, said determination being made on the basis of the DC metrics and LLM metrics.

Description

Information Processing Apparatus, Multi-Agent LLM System, and Control Program

[0001] The present disclosure relates to an information processing apparatus, a multi-agent LLM system, and a control program.

[0002] Conventionally, multi-agent LLMs have been known (for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2025-002046

[0004] To solve the above problems, an information processing apparatus according to an aspect of the present disclosure includes an acquisition unit that acquires DC metrics, which are metrics of each of a plurality of data centers, and LLM metrics, which are metrics of each of a plurality of agent LLMs provided in the plurality of data centers, and an eviction candidate determination unit that determines one or more eviction candidates that are candidates to be evicted from the plurality of agent LLMs based on the DC metrics and the LLM metrics.

[0005] To solve the above problems, a multi-agent LLM system according to an aspect of the present disclosure includes a plurality of data centers, a plurality of agent LLMs deployed in the plurality of data centers, an acquisition unit that acquires DC metrics, which are metrics of each of the plurality of data centers, and LLM metrics, which are metrics of each of the plurality of agent LLMs provided in the plurality of data centers, and an eviction candidate determination unit that determines one or more eviction candidates that are candidates to be evicted from the plurality of agent LLMs based on the DC metrics and the LLM metrics.

[0006] It is a schematic diagram for explaining the outline of the multi-agent LLM system according to the present disclosure. It is a block diagram showing the functional configuration of the information processing apparatus according to the present disclosure. It is a block diagram showing an example of the configuration of each data center. It is a flowchart showing an example of the operation of the information processing apparatus. It is a schematic diagram showing a state where the information processing apparatus appropriately arranges the multi-agent LLM based on the DC metrics and the LLM metrics.

[0007] Hereinafter, one embodiment of the present disclosure will be described in detail with reference to the drawings. For ease of understanding, the background and challenges of the present disclosure will be described first, followed by a detailed description of the present disclosure. <(Multi) Agent LLM> An "Agent LLM (Agent-Type Large-Scale Language Model)" is a large-scale AI language model designed to perform a specific task. While a typical LLM generates text on its own, an Agent LLM has the ability to perform more complex tasks in conjunction with external tools and / or systems. Some Agent LLMs are specialized for generating text, images, videos, or audio.

[0008] A multi-agent LLM refers to a system that allows multiple agent LLMs to work together. Each agent LLM has a specific task and / or role, and they work together to solve complex problems or perform tasks efficiently.

[0009] One example of collaboration is when agents LLM exchange information with each other to solve a problem. Another example of role division is when one agent LLM collects information, and another agent LLM analyzes that information.

[0010] Multi-agent LLMs are often deployed across multiple distributed data centers (DCs). Reasons for this include improved scalability, high availability and redundancy, low latency, and regional data management.

[0011] Improved scalability can be achieved by distributing agents across multiple data centers, thereby scaling up processing capacity on a large scale, and by distributing agents across different data centers, which allows for more efficient load balancing.

[0012] In terms of high availability and redundancy, deploying each agent in multiple data centers allows processing to continue in other data centers even if one data center goes down, thus improving system availability.

[0013] One way to achieve low latency is by positioning agents geographically closer to the user, which reduces response time and enables the delivery of services with low latency.

[0014] Regarding regional data management, placing data in specific regions allows for compliance with data protection regulations or local requirements (such as the European General Data Protection Regulation).

[0015] Here, multi-agent LLM requires differences for each agent LLM to fulfill its respective purpose or role. There are two ways to introduce these differences: (1) by introducing differences in the model weights themselves through learning, and (2) by introducing differences through prompts (KV cache). Both methods (1) and (2) require a very large amount of data to introduce sufficient differences, so each agent LLM cannot easily move between data centers.

[0016] On the other hand, in communication between agent LLMs, especially in multimodal AI, sufficient bandwidth may be required, so there is room for improvement in the proper placement of each agent LLM. <Overview of Information Processing Device 1> The information processing device 1 in this disclosure includes an acquisition unit that acquires DC metrics, which are metrics of each of the multiple data centers, and LLM metrics, which are metrics of each of the multiple agent LLMs provided in the multiple data centers, and an eviction candidate determination unit that determines one or more eviction candidates from among the multiple agent LLMs to be evicted based on the DC metrics and the LLM metrics.

[0017] According to the above configuration, the information processing device 1 determines one or more evicting candidates from among multiple agent LLMs based on DC metrics and LLM metrics. Therefore, the information processing device 1 can determine evicting candidates, which are prerequisites for the proper placement of multi-agent LLMs, with high accuracy, and can facilitate the subsequent proper placement of multi-agent LLMs.

[0018] The overview of the information processing device 1 will be described below with reference to Figures 1 and 2. Figure 1 is a schematic diagram illustrating the overview of the multi-agent LLM system 100 according to this disclosure. Figure 2 is a block diagram showing the functional configuration of the information processing device 1 according to this disclosure.

[0019] Referring to Figure 1, the multi-agent LLM system 100 comprises a central data center 20 and one or more distributed data centers 30.

[0020] The central data center 20 is the data center that provides overall management for the multi-agent LLM system 100. The central data center 20 is responsible for large-scale data processing (AI analysis, big data analysis, etc.), master database management, backup and disaster recovery, and / or maintaining a high-security environment.

[0021] The distributed data centers 30 are responsible for data processing in each region (reducing latency by providing data closer to users), load balancing (load distribution, traffic optimization, etc.), ensuring redundancy (failover in case of failure, improved availability, etc.), and / or complying with regional data regulations.

[0022] The central data center 20 and the distributed data center 30 each have different agent LLMs deployed. "Different language models" means they have different architectures or hyperparameters, including differences in layer structure, number of neurons, training methods, and / or optimization algorithms. Furthermore, the datasets used for training and / or tuning methods may also differ, potentially resulting in differences in the quality and / or characteristics of the generated text.

[0023] This allows multi-agent LLMs to have different roles and / or areas of expertise, such as an LLM strong in mathematics, an LLM strong in law, or an LLM strong in medicine. Therefore, multi-agent LLMs can cooperate to efficiently handle complex tasks. In this way, multi-agent LLMs can provide high flexibility and scalability by combining the strengths of each agent LLM.

[0024] The central data center 20 does not necessarily need to have an agent LLM installed, but in the following explanation, we will assume that an agent LLM is installed in the central data center 20.

[0025] Next, with reference to Figure 2, the information processing device 1 according to this disclosure will be described. The central data center 20 is equipped with the information processing device 1. The information processing device 1 may be provided in the distributed data center 30, but in the following description, the information processing device 1 will be provided in the central data center 20.

[0026] The information processing device 1 comprises a control unit 10, a storage unit 17, and a communication unit 18. The control unit 10 includes an acquisition unit 11, an evicto candidate determination unit 12, an evicto target determination unit 13, and a deployment destination determination unit 14.

[0027] The control unit 10 is a control device such as a CPU or GPU that oversees the entire information processing device 1, and also operates as an acquisition unit 11, an evicto candidate determination unit 12, an evicto target determination unit 13, and a deployment destination determination unit 14.

[0028] The acquisition unit 11 acquires DC metrics, which are metrics for the central data center 20 and the distributed data center 30, and LLM metrics, which are metrics for each of the multiple agent LLMs installed in the central data center 20 and the distributed data center 30.

[0029] The Evict Candidate Determination Unit 12 determines one or more Evict candidates from among multiple Agent LLMs based on DC metrics and LLM metrics.

[0030] The Evict target determination unit 13 determines an Evict target from among one or more Evict candidates based on predetermined conditions.

[0031] The deployment destination determination unit 14 determines from among the central data center 20 and distributed data centers 30 which data center will be the redeployment destination for the Evict target.

[0032] The storage unit 17 is a storage device such as a memory that stores various types of information, at least temporarily. For example, the storage unit 17 stores a program that operates the information processing device 1 as an orchestrator, and the specifications of each node. The control unit 10 executes the program, causing the information processing device 1 to operate as an orchestrator.

[0033] The communication unit 18 is an interface that performs communication processing with other devices. <DC metrics, LLM metrics> Next, DC metrics and LLM metrics will be explained with reference to Figure 3. Figure 3 is a block diagram showing an example of the configuration of each data center.

[0034] Each data center physically houses multiple servers, network equipment, and IT infrastructure such as storage.

[0035] A cluster 50 is a logical unit in which one or more nodes 60 collaborate to run applications and / or services. A cluster 50 is typically built on nodes 60 within a data center. The cluster 50 is distributed across multiple servers within the data center, which provides its hardware infrastructure. A cluster 50 may also be configured virtually, in which case it may span multiple data centers.

[0036] Node 60 is a physical or virtual server on which the workload is running.

[0037] Pod 70 is a collection of containers 80. Pod 70 resides within node 60 and has at least one container.

[0038] A container 80 is a unit that packages an application and the middleware, such as libraries, that the application uses.

[0039] <Implementation Example> As mentioned above, the acquisition unit 11 acquires DC metrics and LLM metrics. To do this, the acquisition unit 11 uses Kubernetes® and Prometheus® as an example. Kubernetes is a container orchestration tool that deploys, scales, and manages applications. Kubernetes efficiently manages multiple containers to ensure availability and scalability. Prometheus is a monitoring and alerting tool that monitors the performance of the Kubernetes cluster and applications. In terms of the division of roles between the two, Kubernetes is responsible for managing containers, and Prometheus monitors their operation to understand their performance and health.

[0040] More specifically, Prometheus acquires agent LLM metrics (hereinafter referred to as "LLM metrics") and node (data center) metrics (hereinafter referred to as "DC metrics") as observability tools.

[0041] LLM metrics include, for example, per-pod resource usage (CPU, memory, GPU, etc.), per-pod traffic, and inter-pod latency. LLM metrics may also include other metrics such as inference throughput, error rate, and number of requests.

[0042] DC metrics may include, for example, node-level resource usage (CPU, memory, GPU, etc.), node-level traffic, and node-level traffic capacity. Node metrics may also include other metrics such as the number of running processes, system uptime, and network errors.

[0043] In particular, when redeploying the agent LLM, a large amount of communication traffic is expected. Therefore, it is preferable to include metrics related to the communication traffic in the LLM matrix and / or the DC matrix.

[0044] Note that the acquisition unit 11 may use other tools having functions similar to those of Docker / Kubernetes and Prometheus / ElasticSearch stacks. <Operation example of the information processing apparatus 1> Next, an operation example of the information processing apparatus 1 will be described with reference to FIG. 4. FIG. 4 is a flowchart showing an example of the operation of the information processing apparatus 1.

[0045] First, the acquisition unit 11 acquires the DC metrics of each of the central data center 20 and the distributed data center 30 and the LLM metrics of each of the plurality of agent LLMs provided in the central data center 20 and the distributed data center 30 (S2). The acquisition unit 11 may acquire the DC metrics and the LLM metrics from Prometheus. Alternatively, the acquisition unit 11 may be Prometheus itself.

[0046] Subsequently, the moving average of each metric is calculated (S4). This step may be executed by the acquisition unit 11 or the event candidate determination unit 12, or may be calculated by another arithmetic unit. The method of calculating the moving average is to take the average of data within a specific period, and the time interval thereof may be set arbitrarily. For example, the moving average may be calculated as the moving average over 15 minutes.

[0047] The step of S4 may be omitted. However, by calculating the numerical values of each matrix based on the moving average, the event candidate determination unit 12 can determine event candidates, which are the premise for properly arranging the multi-agent LLM, with high accuracy, and the subsequent proper arrangement of the multi-agent LLM can also be realized with higher accuracy.

[0048] Next, the eviction candidate determination unit 12 determines whether there is a node with a resource utilization rate of the threshold value or more based on the DC metrics (S6). For example, when any one of the conditions of CPU 80% or more, memory 80% or more, and GPU 80% or more (utilization rate in MIG unit) is satisfied, the eviction candidate determination unit 12 determines that there is a node with a resource utilization rate of the threshold value or more (YES in S6). CPU 80% or more, memory 80% or more, and GPU 80% or more are an example, and are not limited to these numerical values. Also, the threshold values of the CPU, memory, and GPU respectively do not need to be the same value, and may be different from each other.

[0049] In the case of YES in S6, the eviction candidate determination unit 12 determines, based on the LLM metrics, one or more eviction candidates that are candidates for being evicted for the agent LLM with a large resource usage amount per pod, and lists them (S8).

[0050] In the case of NO in S6, the process proceeds to S10. In S10, the eviction candidate determination unit 12 determines whether there is a node with a traffic utilization rate of the threshold value or more based on the DC metrics (S10). For example, when there is a node with a transmission and reception amount of 80% or more, the eviction candidate determination unit 12 determines that there is a node with a traffic utilization rate of the threshold value or more (YES in S10). Transmission and reception amount 80% or more is an example, and is not limited to this numerical value.

[0051] In the case of YES in S10, the eviction candidate determination unit 12 determines, based on the LLM metrics, one or more eviction candidates that are candidates for being evicted for the agent LLM with a large traffic amount per pod, and lists them (S8).

[0052] In this way, the eviction candidate determination unit 12 determines one or more eviction candidates that are candidates for being evicted from among a plurality of agent LLMs based on the DC metrics and the LLM metrics.

[0053] Next, in S12, the Evict target determination unit 13 determines whether eviction is possible from among one or more eviction candidates based on predetermined conditions (S12). Specifically, the Evict target determination unit 13 refers to one or more eviction candidates listed in S12. At this time, the Evict target determination unit 13 may refer to the eviction candidate that most significantly exceeds the threshold. Then, the Evict target determination unit 13 refers to the attributes of the referenced eviction candidate and determines whether it is actually possible to evict. The attributes of the eviction candidate may be, for example, information indicating whether the agent LLM can be redeployed. If there is an attribute (attribute information) indicating that the agent LLM can be redeployed, the Evict target determination unit 13 determines that eviction is possible based on that attribute information. In addition, in order to determine whether the agent LLM can be redeployed, the Evict target determination unit 13 may refer to attributes such as the instance priority or node affinity.

[0054] If the answer in S12 is YES, the Evict target determination unit 13 determines an agent LLM (Evict target) that can be evicted (S14).

[0055] Next, the deployment destination determination unit 14 determines from among the central data center 20 and distributed data centers 30 which data center will be the redeployment destination for the Evict target (S16). At this time, the deployment destination determination unit 14 may, as an example, use kube-scheduler, which is one of the components that run within the Kubernetes cluster. For example, the deployment destination determination unit 14 uses kube-scheduler to select a data center that will not place a heavy communication load on the deployment, taking into account resources or communication load, and deploys the Agent LLM (an instance of Agent LLM) that is the Evict target to the selected data center (S16). After that, it sleeps for 15 minutes (S18) and returns to S2. Here, the 15-minute sleep is just an example, and the sleep time can be set arbitrarily.

[0056] If the answer in S12 is NO, proceed to S20. In S20, the Evicto target determination unit 13 determines whether there are other Evicto candidates in the list. If the answer in S20 is YES, return to S12, and the Evicto target determination unit 13 performs the aforementioned processing on the other existing Evicto candidates.

[0057] If the answer in S20 is NO, the system sleeps for 15 minutes (S22) and then returns to S2. Here, the 15-minute sleep is just an example, and the sleep time can be set arbitrarily.

[0058] If the answer in S10 is NO, the process proceeds to S24, where the Evict candidate determination unit 12 determines, based on the LLM metrics, whether there are any pairs where the inter-pod latency is greater than or equal to a threshold. For example, "p95:100ms" is set as the threshold for inter-pod latency. "p95:100ms" is just an example and is not limited to this value.

[0059] If the answer in S24 is YES, the Evict Candidate Determination Unit 12 determines the target pair of agent LLMs as Evict candidates. Then, proceeding to S26, the Evict Target Determination Unit 13 determines whether both agent LLMs, which are Evict candidates, are eligible for Evict. At this time, the Evict Target Determination Unit 13 refers to the attributes of both agent LLMs and determines whether it is actually OK to evict them. The attributes of the Evict candidates may be, for example, information indicating whether the agent LLMs can be redeployed. If there is an attribute (attribute information) indicating that the agent LLMs can be redeployed, the Evict Target Determination Unit 13 determines that Evict is possible based on that attribute information (YES in S26). Then, proceeding to S14.

[0060] If the answer in S24 is NO, proceed to S18. If the answer in S26 is NO, return to S24.

[0061] In this way, the information processing device 1 determines one or more evicting candidates from among multiple agent LLMs based on DC metrics and LLM metrics. Then, the information processing device 1 determines the evicting target based on predetermined conditions and determines the data center to which the evicting target will be redeployed from among multiple data centers.

[0062] With the above configuration, the information processing device 1 can appropriately deploy multi-agent LLMs based on DC metrics and LLM metrics. As a result, the information processing device 1 can improve the availability of the multi-agent LLM system and reduce the bandwidth load across the entire data center.

[0063] Figure 5 is a schematic diagram showing how the information processing device 1 appropriately positions the multi-agent LLM based on DC metrics and LLM metrics. The diagram indicated by reference numeral 500 in Figure 5 shows the state before the information processing device 1 rearranges the multi-agent LLM. The diagram indicated by reference numeral 501 in Figure 5 shows the state after the information processing device 1 rearranges the multi-agent LLM. As shown in Figure 5, the multi-agent LLM system 100 includes a central data center 20 and distributed data centers 30a to 30f.

[0064] Referring to the diagram shown by reference numeral 500 in Figure 5, before the information processing device 1 rearranges the multi-agent LLM, the central data center 20, the distributed data center 30d, and the distributed data center 30f are working together on a single task. At this time, communication from the central data center 20 to the distributed data center 30d is via the distributed data center 30a, and communication from the central data center 20 to the distributed data center 30f is via the distributed data center 30b.

[0065] Next, referring to the diagram shown by reference numeral 501 in Figure 5, after the information processing device 1 has rearranged the multi-agent LLM, the distributed data centers 30b, 30e, and 30f are working together on a single task. At this time, the distributed data centers 30b, 30e, and 30f are communicating without going through other distributed data centers.

[0066] In the example shown in Figure 5, the information processing device 1 rearranges the multi-agent LLM so that the single task is resolved collaboratively by the distributed data centers 30b, 30e, and 30f. This reduces the unnecessary load on the traffic in the multi-agent LLM system 100. In other words, the information processing device 1 acts as a multi-agent orchestrator in the multi-agent LLM system 100 to facilitate the rearrangement of each agent.

[0067] The information processing device 1 continuously acquires metrics from each agent LLM, such as the amount of computational data (Flops) used for inference, memory usage (GB), and bandwidth consumed for communication to other agent LLMs (Mbps), as well as the allowable computational data, memory amount, and bandwidth from each distributed data center 30. When the metrics of an agent LLM in any of the distributed data centers 30 exceed a threshold, the information processing device 1 acts as a trigger. Specifically, the information processing device 1 deploys an instance of the agent LLM to the distributed data center 30 that can host the agent LLM and where the communication load score calculated from the metrics is the lowest when the agent LLM is actually hosted. Alternatively, the information processing device 1 may deploy an instance of the agent LLM to the distributed data center 30 where the communication load score calculated from the metrics is the lowest. The central data center 20 also acts as a load balancer for the entire multi-agent LLM system 100 to ensure that traffic is also routed to the new instance.

[0068] As a result, the information processing device 1 can improve the availability of the multi-agent LLM system and reduce the bandwidth load across the entire data center.

[0069] In Figure 5, a single task is shown collaborating with one multi-agent LLM group in the multi-agent LLM system 100. However, multiple tasks may collaborate simultaneously with separate multi-agent LLM groups. In this case, the information processing device 1 may deploy the multi-agent LLMs such that one agent LLM is shared by each of the separate multi-agent LLM groups.

[0070] Furthermore, the information processing device 1 may not only redeploy all data centers within the information processing device 1 at once, but may also gradually optimize their placement based on the relationships between each agent LLM.

[0071] Furthermore, in the multi-agent LLM system 100, the information processing device 1 may rearrange the agent LLMs according to logic (Figure 4, etc.) based on the amount of each type of inference generated (text, images, videos, or audio, etc.).

[0072] Furthermore, in the multi-agent LLM system 100, the information processing device 1 may rearrange the agent LLMs according to logic (Figure 4, etc.) based not only on current metrics but also on predicted future metrics. In this case, the predicted future metrics may be derived, for example, by AI.

[0073] Alternatively, the information processing device 1, which operates as a multi-agent orchestrator, may leave older instances in the distributed data center 30. Or, the information processing device 1 may set an upper limit on the number of agent LLMs for the entire multi-agent LLM system 100, and delete older instances when that upper limit is reached.

[0074] The information processing device 1 may be configured to utilize a portion of the computing resources of a base station device of a mobile communication network, which is implemented as a vRAN (virtualized RAN). For example, RUs (radio units) may be installed at multiple sites as devices that process the antenna portion and the lowest layer of the PHY layer of the base station, and the modulation and demodulation of signals, or the retransmission of lost signals, may be controlled by a DU (Distributed Unit) located in a data center or the like. Furthermore, a RIC (RAN Intelligent Controller) that controls the operation of the RUs and DUs may be provided in the data center or the like. Either the information processing device 1 in the central data center 20 or a physical server in the distributed data center 30 may be equipped with a RIC (RAN Intelligent Controller), which controls the RAN (Radio Access Network). The information processing device 1 may be configured to utilize a portion of the computing resources of the RIC, for example.

[0075] Furthermore, the information processing device 1 can also be effectively utilized in AI on RAN. Specifically, AI on RAN is a technology that uses the computing infrastructure of a base station to provide highly immediate services with low latency to users and devices around the base station. AI on RAN deploys AI (Artificial Intelligence) and / or machine learning technology applications at the network edge through RAN (Radio Access Network). As a result, AI on RAN promotes the creation of new industries and solutions that leverage low latency and confidentiality. In such an AI on RAN, the information processing device 1 with the above configuration can be effectively utilized.

[0076] <Example of implementation by software> The functions of the information processing device 1 (hereinafter referred to as "device") can be realized by a program that causes a computer to function as the device, and by a program that causes a computer to function as each control block of the device (in particular, the acquisition unit 11, the evicto candidate determination unit 12, the evicto target determination unit 13, and the deployment destination determination unit 14).

[0077] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0078] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0079] Furthermore, some or all of the functions of each of the above control blocks can also be implemented by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of this disclosure. In addition, it is also possible to implement the functions of each of the above control blocks by, for example, a quantum computer.

[0080] Furthermore, each process described in the above embodiments may be performed by AI (Artificial Intelligence). In this case, the AI ​​may operate on the control device described above, or it may operate on other devices (e.g., an edge computer or a cloud server). <Summary> This disclosure describes at least the following aspects.

[0081] An information processing device according to Embodiment 1 of the present disclosure includes an acquisition unit that acquires DC metrics, which are metrics for each of a plurality of data centers, and LLM metrics, which are metrics for each of a plurality of agent LLMs provided in the plurality of data centers, and an eviction candidate determination unit that determines one or more eviction candidates from among the plurality of agent LLMs to be evictioned based on the DC metrics and the LLM metrics.

[0082] According to the above configuration, the information processing device according to Embodiment 1 of the present disclosure can determine one or more evict candidates from among multiple agent LLMs based on DC metrics and LLM metrics. Therefore, the information processing device according to Embodiment 1 of the present disclosure can determine evict candidates, which are prerequisites for the proper placement of multi-agent LLMs, with high accuracy, and can facilitate the proper placement of agent LLMs thereafter.

[0083] The information processing device according to Embodiment 2 of the present disclosure includes, in Embodiment 1, an Evicto target determination unit that determines an Evicto target from among the one or more Evicto candidates based on predetermined conditions.

[0084] According to the above configuration, the information processing device according to aspect 2 of this disclosure can determine the target of the Evict with high accuracy. The predetermined conditions may be, for example, the attributes of the agent LLM (such as whether or not redeployment is set to be disabled).

[0085] The information processing device according to aspect 3 of the present disclosure includes, in aspect 2, a deployment destination determination unit that determines from among the plurality of data centers the data center to be the redeployment destination for the Evict target.

[0086] According to the above configuration, the information processing device according to aspect 3 of this disclosure can determine the redeployment destination for the Evict target, thereby improving the appropriate placement of the agent LLM.

[0087] The information processing device according to Embodiment 4 of the present disclosure, in any of Embodiments 1 to 3, wherein the DC metrics include at least one of the node-level resource capacity, node-level communication volume, and node-level communication capacity of the nodes constituting the cluster in the plurality of data centers, and the LLM metrics include at least one of the pod-level resource usage, pod-level communication volume, and inter-pod latency of the pods included in the node.

[0088] According to the above configuration, the information processing device according to aspect 4 of this disclosure can determine evictor candidates with higher accuracy and contribute to the appropriate placement of agent LLMs thereafter.

[0089] The information processing device according to aspect 5 of this disclosure is installed in a central data center among the plurality of data centers in any of aspects 1 to 4.

[0090] Generally, a central data center manages systems centrally and is responsible for efficient resource allocation. Furthermore, a central data center centralizes system operation and monitoring, offering advantages in terms of cost reduction, enhanced security, and improved fault tolerance.

[0091] Therefore, according to the above configuration, the information processing device according to aspect 5 of this disclosure can enjoy the characteristics of a central data center as described above.

[0092] A multi-agent LLM system according to aspect 6 of this disclosure comprises: a plurality of data centers; a plurality of agent LLMs deployed in the plurality of data centers; an acquisition unit that acquires DC metrics, which are metrics for each of the plurality of data centers, and LLM metrics, which are metrics for each of the plurality of agent LLMs provided in the plurality of data centers; and an evicto candidate determination unit that determines one or more evicto candidates from among the plurality of agent LLMs based on the DC metrics and the LLM metrics.

[0093] According to the above configuration, the multi-agent LLM system according to aspect 6 of the present disclosure can achieve the same effects as the information processing device according to aspect 1 of the present disclosure.

[0094] The control program according to aspect 7 of the present disclosure is a control program for causing a computer to function as an information processing device according to aspect 1, and according to the above configuration which causes the computer to function as the acquisition unit and the evicto candidate determination unit, the control program according to aspect 7 of the present disclosure can achieve the same effects as the information processing device according to aspect 1 of the present disclosure.

[0095] This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure.

[0096] (Additional Notes) This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure.

[0097] 1. Information Processing Device 10 Control Unit 11 Acquisition Unit 12 Evict Candidate Determination Unit 13 Evict Target Determination Unit 14 Deployment Destination Determination Unit 17 Storage Unit 18 Communication Unit 20 Data Center 20 Central Data Center 30, 30a, 30b, 30d, 30e, 30f Distributed Data Center 50 Cluster 60 Node 70 Pod 80 CPU 100 Multi-Agent LLM System

Claims

1. An information processing device comprising: an acquisition unit that acquires DC metrics, which are metrics for each of multiple data centers, and LLM metrics, which are metrics for each of multiple agent LLMs installed in the multiple data centers; and an evicting candidate determination unit that determines one or more evicting candidates from among the multiple agent LLMs to be evicted based on the DC metrics and the LLM metrics.

2. The information processing apparatus according to claim 1, comprising an evicting target determination unit that determines an evicting target from among the one or more evicting candidates based on predetermined conditions.

3. The information processing apparatus according to claim 2, further comprising a deployment destination determination unit that determines from among the plurality of data centers the data center to be used as the redeployment destination for the Evict target.

4. The information processing device according to claim 1 or 2, wherein the DC metrics include at least one of the node-level resource capacity, node-level communication volume, and node-level communication capacity of the nodes constituting the cluster in the plurality of data centers, and the LLM metrics include at least one of the pod-level resource usage, pod-level communication volume, and inter-pod latency of the pods included in the node.

5. An information processing device according to claim 1 or 2, which is installed in a central data center among the plurality of data centers.

6. A multi-agent LLM system comprising: multiple data centers; multiple agent LLMs deployed in the multiple data centers; an acquisition unit that acquires DC metrics, which are metrics for each of the multiple data centers, and LLM metrics, which are metrics for each of the multiple agent LLMs provided in the multiple data centers; and an evicto candidate determination unit that determines one or more evicto candidates from among the multiple agent LLMs based on the DC metrics and the LLM metrics.

7. A control program for causing a computer to function as an information processing device according to claim 1, wherein the computer functions as the acquisition unit and the evicto candidate determination unit.

Images