Information processing apparatus, robot, control system, control program, and recording medium

Abstract

To use an LLM of a suitable size according to the degree of difficulty of a task in controlling a robot.SOLUTION: An information processing device (20, 20a) according to one aspect of the present invention includes a classifier (28) configured to receive information indicating a task to be executed by a robot (30) controlled using any one of a plurality of LLMs (100) having different sizes from each other, and to output information indicating a result of execution of the task by the robot (30) under control using the LLM (100), and a determination unit (22) configured to determine which LLM (100) of the plurality of LLMs (100) is to be used for control of the robot (30) when the robot (30) is caused to execute an arbitrary task, using the classifier (28).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a robot, a control system, a control program, and a recording medium. [Background technology]

[0002] In recent years, research has been conducted into the application of LLMs (Large Language Models) to robot control. By using huge LLMs with a large number of input parameters, it becomes possible to have robots execute highly difficult tasks. Non-Patent Document 1 discloses research into robot control using LLMs and foundation models. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Integration of Large-scale Language Model and World Models in Robot Control", Akira Kinose, Ryo Okumura, Tadahiro Taniguchi, The 37th Annual Conference of the Japanese Society for Artificial Intelligence, 2023. Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a problem in that the constant use of a huge LLM can be a costly factor.

[0005] One aspect of the present invention has been made in consideration of the above problem, and aims to enable an LLM of an appropriate size to be used in controlling a robot according to the difficulty of the task. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to one embodiment of the present invention includes a classifier that receives input information indicating a task to be instructed to be executed by a robot controlled using one of a plurality of LLMs having different sizes, and outputs information indicating the result of the robot executing the task through control using one of the LLMs, and a determination unit that uses the classifier to determine which of the plurality of LLMs to use to control the robot when an arbitrary task is to be executed by the robot.

[0007] Another aspect of the present invention provides a control system comprising a robot, an information processing device, and a plurality of server devices, each of which is equipped with a plurality of LLMs of different sizes, and the robot is controlled using one of the plurality of LLMs, and the information processing device comprises a classifier that receives information indicating a task to be instructed to the robot and outputs information indicating the result of the robot performing the task through control using one of the LLMs, a determination unit that uses the classifier to determine which of the plurality of LLMs to use to control the robot when the robot is to perform an arbitrary task, and a communication unit that transmits information indicating a request to have the robot perform the task to the server device equipped with the LLM determined to be used to control the robot.

[0008] The control program of an information processing device that causes a computer to operate as each part (software element) of the information processing device according to each aspect of the present invention, thereby realizing the information processing device on the computer, and a computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention.

[0009] The control program may use various machine learning techniques in the process of causing a computer to operate as each of the above units or in other processes. In this case, the program using the machine learning technique may run on a server or on another device (for example, an edge computer or a cloud server). [Effects of the Invention]

[0010] According to one aspect of the present disclosure, in controlling a robot, an LLM of an appropriate size according to the difficulty of the task can be used. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is an example of a block diagram showing a functional configuration of a control system according to a first embodiment. [Figure 2] 4 is an example of a flowchart showing the flow of a control process executed by the control system according to the first embodiment. [Figure 3] 4 is an example of a flowchart showing the flow of a learning process executed by the control system according to the first embodiment. [Figure 4] 3 is an example of a conceptual diagram illustrating a learning process executed by the control system according to the first embodiment. [Figure 5] FIG. 10 is an example of a block diagram showing the functional configuration of a control system according to a second embodiment. [Figure 6] 10 is an example of a flowchart showing the flow of a control process executed by a control system according to a second embodiment. [Figure 7] 10 is an example of a flowchart showing the flow of a learning process executed by the control system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] 1. Embodiment 1 Hereinafter, one embodiment of the present invention will be described in detail.

[0013] [1.1. Control system configuration example] 1 is an example of a block diagram showing the functional configuration of a control system 1 according to this embodiment. The control system 1 is a system for causing a robot to execute a task by control using any one of a plurality of LLMs.

[0014] Pick-and-place, which grasps an object and moves it to another location, is an example of a task, but in this embodiment, a case will be described in which the task is a static task in which execution time does not affect success or failure. A case in which the task is a dynamic task in which execution time affects success or failure and real-time performance is required will be described in embodiment 2 below. To further supplement, the "task of hanging clothes from the floor on a hanger" is an example of a static task, and the "task of catching a ball" is an example of a dynamic task.

[0015] The control system 1 includes one or more information processing devices 20, robots 30, learning instruction devices 40, DC (Delivery Controller) servers 50, MEC (Mobile Edge Computing) servers 51, and edge servers 52. The DC servers 50, MEC servers 51, and edge servers 52 are examples of server devices according to the present disclosure.

[0016] The information processing device 20 is a device for selecting one of the LLMs 100 depending on the difficulty of the task to be executed by the robot 30, and offloading the task to the selected LLM 100. As will be described in detail later, the DC server 50, MEC server 51, and edge server 52 each include an LLM 100. In other words, the information processing device 20 is a device for offloading control of causing the robot 30 to execute the task to one of the server devices. In the present disclosure, when there is no need to distinguish between the LLMs 100a to 100c, any or all of them may be referred to as "LLM 100," as described above.

[0017] The information processing device 20 includes a control unit 21, a storage unit 27, and a communication unit 29.

[0018] The control unit 21 is a control device such as a CPU that controls the entire information processing device 20, and includes a determination unit 22 and a learning unit .

[0019] When causing the robot 30 to execute an arbitrary task, the determination unit 22 determines which of the multiple LLMs 100 should be used to control the robot 30, using a classifier 28, which is a type of learning model.

[0020] The classifier 28 receives information indicating a task to be instructed to be executed by the robot 30, and outputs information indicating the result of the robot 30 executing the task under control using any one of a plurality of LLMs 100 of different sizes. The feature amount of the task is an example of the information indicating the task.

[0021] The learning unit 23 inputs training data into the classifier 28 to train the classifier 28. Specifically, the learning unit 23 trains the classifier 28 using training data in which information indicating a task to be instructed to be executed by the robot 30 is used as an explanatory variable, and information indicating a result of the robot 30 executing the task under control using any one of a plurality of LLMs 100 of different sizes is used as a target variable.

[0022] The storage unit 27 is a storage device such as a memory that at least temporarily stores various types of information, and stores, for example, data such as a parameter set that defines the classifier 28. From another perspective, the storage unit 27 can be considered to be equipped with the classifier 28 that operates under the control of the control unit 21. Furthermore, the learning unit 23 performs learning of the classifier 28 by updating the values ​​of the parameter set.

[0023] The communication unit 29 performs communication processing with external devices such as the DC server 50 under the control of the control unit 21 .

[0024] Note that a single information processing device 20 may be associated with one or more robots 30, or the robot 30 may be provided with the information processing device 20. Also, any one of the DC server 50, the MEC server 51, and the edge server 52 may be provided with the information processing device 20.

[0025] The robot 30 is a robot that is controlled using any one of the LLMs 100 included in the DC server 50, the MEC server 51, and the edge server 52 at least when executing a task, and is equipped with one or more types of sensors 31, an input unit 32, and a communication unit 33. Note that the robot 30 may also be configured to include any one of the server devices, especially the edge server 52.

[0026] The sensor 31 senses the situation around the robot 30 and actions on the robot 30. The sensor 31 may include an image sensor, a sound sensor, a force sensor, and the like.

[0027] While the robot 30 is being controlled using the LLM 100, in other words, while the LLM 100 is controlling the robot 30, the LLM 100 periodically transmits control signals to the robot 30, and the robot 30 periodically transmits sensing results to the LLM 100. Furthermore, the results of sensing by the sensor 31 are not only referenced for the continued operation of the robot 30, but can also be input to the learning instruction device 40 for use in the learning process of training the classifier 28.

[0028] Furthermore, the robot 30 can refer to the sensing results of the sensor 31 to determine whether it has succeeded or failed in the task.

[0029] The input unit 32 receives input of instructions from a user to the robot 30. For example, the input unit 32 is realized as a member including a microphone that receives instructions in natural language spoken by the user.

[0030] However, the present disclosure also includes a configuration in which the input unit 32 is realized as a component including a keyboard or the like that accepts character input in natural language, or a component including a camera or the like that accepts input in sign language. Furthermore, the sound sensor, image sensor, etc. that the input unit 32 may be equipped with may be the same as the sensor 31 described above.

[0031] The communication unit 33 performs a process of communicating with an external device such as the DC server 50. For example, the communication unit 33 performs a process of receiving a control signal from any one of the server devices.

[0032] The robot 30 is not limited to a humanoid robot, but may be an animal-type or vehicle-type robot, an aircraft such as a drone, a machine tool, or a combination of these.

[0033] The learning instruction device 40 is a device used in a learning process for training the classifier 28. As will be described in detail later, the learning instruction device 40 determines a task to be performed by the robot 30 according to the sensing results of the sensors 31 provided in the robot 30. The learning instruction device 40 then instructs the robot 30 to perform the determined task using natural language. The learning instruction device 40 also acquires, evaluates, and accumulates information related to the results of task execution from the robot 30.

[0034] Auto RT (Robotics Transformer) by Google Deep Mind (registered trademark) is an example of a device that has a function equivalent to the learning instruction device 40.

[0035] The DC server 50, the MEC server 51, and the edge server 52 are server devices each equipped with an LLM 100 used to control the robot 30, and are candidate server devices for task offloading. However, the size (model size) of the LLM 100 equipped in each server device is different from each other. As a specific example of the number of input parameters, the LLM 100a of the DC server 50 is about 1000 B (billion), the LLM 100b of the MEC server 51 is about 55 B, and the LLM 100c of the edge server 52 is about 7 B.

[0036] Furthermore, the function of a single component included in the control system 1 may be realized by a plurality of other components, and the functions of a plurality of components included in the control system 1 may be realized by a single other component.

[0037] The above describes an example of the configuration of the control system 1. Additionally, each unit included in the control system 1 has a function to execute the processes described below.

[0038] [1.2. Control processing example] Next, a control process executed by the control system 1 according to this embodiment will be described. The control process involves the information processing device 20 determining which of the LLMs 100 included in each server device is to be used to control the robot 30, and the LLM 100 controls the robot 30. The control process according to the present disclosure and a learning process described below include an example of an information processing method executed by the information processing device 20. FIG. 2 is an example flowchart illustrating the process flow. At the start of the control process shown in FIG. 2, the classifier 28 is assumed to have been sufficiently trained by the learning process shown in FIG. 3, which will be described later.

[0039] In S101 (step S101), the input unit 32 of the robot 30 receives an instruction in natural language from a user. The robot 30 transmits the instruction received by the input unit 32 to the information processing device 20 as a task.

[0040] In S102, the control unit 21 of the information processing device 20 extracts feature quantities of the task by breaking down the task received via the communication unit 29 into low-level instruction sets. Here, breaking down the task into low-level instruction sets can be said to break down a single task into one or more simpler operations.

[0041] In S103, the determination unit 22 inputs the extracted feature amount to the classifier 28. As a classification process, the classifier 28 performs a process of outputting a success / failure prediction for each of a plurality of LLMs 100 of different sizes, indicating whether the task will succeed or fail when the robot 30 is controlled using the LLM 100.

[0042] In S104, the determination unit 22 determines the smallest LLM 100 among the LLMs 100 predicted by the classifier 28 to be successful in the task when used to control the robot 30 as the LLM 100 to be used to control the robot 30. This contributes to cost reduction.

[0043] In S105, the control unit 21 transmits the task broken down into a group of low-level commands to a server device equipped with the LLM 100 to be used for controlling the robot 30, thereby offloading the control of the robot 30. From another perspective, the communication unit 29 transmits information indicating a request to have the robot 30 execute the task to the server device equipped with the LLM 100 determined to be used for controlling the robot 30.

[0044] In S106, the LLM 100 provided in the server device to which the control of the robot 30 has been offloaded controls the robot 30 to execute the received task.

[0045] According to the control process of this example described above, in controlling the robot 30, an LLM 100 of an appropriate size according to the difficulty of the task can be used.

[0046] Note that the present disclosure also includes a configuration in which the control unit 21 only performs processing to extract feature quantities of a task in a step corresponding to S102, and then the server device receives a task expressed in natural language from the information processing device 20 and breaks it down into low-level commands in a step corresponding to S105. This is similarly explained in the control processing shown in Fig. 6 and the learning processing shown in Figs. 3 and 7, which will be described later.

[0047] [1.3. Learning process example] Next, a flow of the learning process executed by the control system 1 according to this embodiment, in which the learning unit 23 of the information processing device 20 trains the classifier 28, will be described. Fig. 3 is an example of a flowchart showing the flow of the process. Fig. 4 is an example of a conceptual diagram showing the process.

[0048] In S201, the learning instruction device 40 acquires the sensing results of the sensor 31 from the robot 30. The sensing results include images acquired via an image sensor. That is, the learning instruction device 40 acquires images showing the periphery of the robot 30.

[0049] In S202, a VLM (Vision and Language Model) 401 included in the learning instruction device 40 analyzes the situation of each object captured in the acquired image and supplies text indicating the situation as environmental information to an LLM 402 included in the learning instruction device 40. In this way, the VLM explains to the LLM 402 the situation of each object captured in the image.

[0050] In addition, as an example of a case where the sensing result does not include an image, the learning instruction device 40 may refer to the measurement result by LiDAR (Light Detection and Ranging) and analyze the situation of each object located around the robot 30. Alternatively, the sensing result may include an image and the measurement result by LiDAR.

[0051] In S203, the LLM 402 determines one task to be executed by the robot 30, which is a task according to the situation of each object.

[0052] As a specific example, the LLM 402 may determine the task to be executed by the robot 30 according to the following steps (1) to (4).

[0053] (1) The LLM 402 generates one or more candidate tasks to be executed by the robot 30. For example, the LLM 402 may generate executable actions for each object as tasks.

[0054] (2) For each candidate task, the LLM 402 determines whether the task is one that the robot 30 can perform independently, one that requires user assistance, or one that the robot 30 cannot perform. Here, tasks that the robot 30 cannot perform include prohibited actions, such as actions involving handling sharp objects such as blades. The LLM 402 may also determine the difficulty of each task, or may determine that tasks that require user assistance are more difficult than tasks that the robot 30 can perform independently.

[0055] (3) The LLM 402 filters candidate tasks to be executed by the robot 30, thereby excluding zero or more candidate tasks. The "Filter" in FIG. 4 refers to the excluding process. For example, the LLM 402 not only excludes candidate tasks that the robot 30 cannot execute, but also excludes candidate tasks that require user assistance when the user is far away.

[0056] (4) The LLM 402 determines one task to be executed by the robot 30 from among the remaining candidate tasks. Here, the LLM 402 may determine the one task randomly from among the remaining tasks, may determine the one task based on the difficulty of the task as described above, or may determine the one task based on a priority set in advance for each object.

[0057] In addition, if there are no candidates for a task to be executed by the robot 30 or if no candidates remain, for example, the learning instruction device 40 may instruct the robot 30 to move, and the robot 30 may move, and then the processing from S201 onwards may be repeated.

[0058] Furthermore, in the process of step S203 in which a task to be executed by the robot 30 is determined, the LLM 402 may refer to information accumulated by the learning instruction device 40 and related to the results of task execution by the robot 30. For example, if the robot 30 has failed in executing the most recent task, the LLM 402 may determine a task with a lower level of difficulty as the task to be executed next by the robot 30.

[0059] In S204, the learning instruction device 40 instructs the determined task in natural language to the robot 30. The robot 30 receives the instruction and transmits it to the information processing device 20.

[0060] In S205, the control unit 21 of the information processing device 20 breaks down the received task into low-level commands and extracts the feature of the task. The "feature X" in Fig. 4 means the extracted feature.

[0061] In S206, the determination unit 22 determines one of the LLMs 100 as the LLM 100 to be used for controlling the robot 30. Here, the information processing device 20 may randomly select the LLM 100 to be used for controlling the robot 30.

[0062] In S207, the control unit 21 transmits the task broken down into a group of low-level commands to a server device including the LLM 100 used to control the robot 30, thereby offloading the control of the robot 30.

[0063] In S208, the LLM 100 included in the server device to which the control of the robot 30 has been offloaded controls the robot 30 to execute the received task. As a result, it is determined whether the robot 30 has succeeded or failed in the task.

[0064] In S209, the robot 30 transmits, as a result of the execution of the task, at least information indicating whether the robot 30 has succeeded or failed in executing the task to the information processing device 20 and the learning instruction device 40. The learning instruction device 40 evaluates and accumulates information regarding the result of the execution of the task by the robot 30. "Scoring" in FIG. 4 means the evaluation process.

[0065] In S210, the learning unit 23 of the information processing device 20 inputs training data, in which the feature quantities of the tasks extracted in S205 are used as explanatory variables and the execution results of the tasks controlled using any one of the LLMs 100 are used as objective variables, into the classifier 28. As a result, the classifier 28 is trained on the LLM 100.

[0066] Furthermore, since the features of the task instructed by the learning instruction device 40 are input to the classifier 28, it can be said that, as a result, in the above-mentioned S204, the learning instruction device 40 outputs information indicating the task as information to be used as input to the classifier 28.

[0067] Note that execution results corresponding to LLMs 100 other than the LLM 100 used to control the robot 30 are masked when inputting training data to the classifier 28. In the example shown in Fig. 4, the execution results of the task when an LLM 100b with 55B input parameters is used to control the robot 30 are input to the classifier 28 as training data.

[0068] The learning process shown in the flowchart of FIG. 3 is repeated while the robot 30 moves, either under the direction of the learning instruction device 40 or autonomously, until the classifier 28 has been fully trained for all LLMs 100 that may be used to control the robot 30.

[0069] The learning process in which the learning unit 23 of the information processing device 20 trains the classifier 28 has been described above.

[0070] Note that the present disclosure also includes a configuration in which the learning instruction device 40 performs processing in which the user instructs the robot 30 to execute a task. Furthermore, if the control system 1 performs the above-mentioned control processing in which the LLM 100 controls the robot 30 using the trained classifier 28 and does not perform further learning processing, it is not essential that the control system 1 includes the learning instruction device 40 and that the control unit 21 of the information processing device 20 includes the learning unit 23. These matters also apply to the following embodiment 2.

[0071] 2. Embodiment 2 A second embodiment of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and redundant description will not be repeated.

[0072] In this embodiment, a dynamic task whose success or failure is affected by execution time will be described. When a dynamic task is executed by the robot 30 under control using the LLM 100, if the size of the LLM 100 is too small, the task may fail to be executed because the task is too difficult for the LLM 100. If the size of the LLM 100 is too large, the response speed may be delayed, which may result in failure to execute the task.

[0073] [2.1. Control system configuration example] FIG. 5 is an example of a block diagram showing the functional configuration of a control system 1a according to this embodiment.

[0074] In comparison with the control system 1 shown in FIG. 1, the control system 1a further includes a control unit 21a that includes a task time prediction unit 24 and a communication quality prediction unit 25.

[0075] The task time prediction unit 24 predicts the execution time of a task when the robot 30 is caused to execute the task under control using each LLM 100.

[0076] The communication quality prediction unit 25 predicts the communication quality when a task is being executed, which is the communication quality between each server device and the robot 30. The communication quality may include, for example, communication time, and radio wave conditions and congestion conditions in communication.

[0077] Furthermore, the communication quality predicting unit 25 may predict the communication quality using, for example, the following existing methods. "Data-driven Network Simulation for Performance Analysis of Anticipatory Vehicular Communication Systems", [online], Benjamin Sliwa, Christian Wietfeld, [Retrieved June 10, 2024], Internet<https: / / arxiv.org / abs / 1911.10753> Furthermore, when making their respective predictions, the task time prediction unit 24 and the communication quality prediction unit 25 may refer to information about past execution results of tasks stored in the memory unit 27, or may use various machine learning techniques.

[0078] Furthermore, in the classification process and learning process described above, the classifier 28 according to this embodiment is configured to receive, in addition to the feature quantities of the task, at least information indicating the communication quality predicted by the communication quality prediction unit 25. From another perspective, the information output by the classifier 28 in the above configuration is information indicating the result of the robot 30 executing the task under control using any one of the LLMs 100 under the condition that the communication quality of the communication is that predicted by the communication quality prediction unit 25.

[0079] In one aspect, the classification process according to this embodiment involves the following steps.

[0080] (1) The task time prediction unit 24 predicts the execution time of a task when the robot 30 is caused to execute the task by control using the largest-sized LLM 100. Here, the largest-sized LLM 100 is usually the LLM 100a provided in the DC server 50.

[0081] (2) The communication quality prediction unit 25 predicts the communication quality of the communication between each server device and the robot 30 during the execution time predicted by the task time prediction unit 24. In one aspect, the communication quality prediction unit 25 estimates that the communication quality is the same between any server device and the robot 30.

[0082] (3) The determination unit 22 inputs the feature quantities of the task extracted by the control unit 21a and information indicating the communication quality predicted by the communication quality prediction unit 25 to the classifier 28. As a classification process, the classifier 28 performs a process of outputting information indicating a prediction of whether the task will succeed or fail when the robot 30 is controlled using each LLM 100.

[0083] In the classification process and learning process, the classifier 28 may be configured to receive, in addition to information indicating the feature quantities of the tasks and the communication quality, information indicating the execution time of the tasks predicted by the task time prediction unit 24. Alternatively, the execution time and communication quality of the tasks may be predicted for each LLM 100 that can be used to control the robot 30, and the prediction results for each LLM 100 may be input to the classifier 28.

[0084] In one aspect, in the learning process according to this embodiment, in the step corresponding to S209 in the flowchart of FIG. 3, the robot 30 transmits, to at least the information processing device 20a and the learning instruction device 40, information indicating whether the robot 30 has succeeded or failed in executing the task, as well as the actual time required to execute the task.

[0085] Furthermore, in the learning process, the actual communication quality during the execution time of the task is measured by the robot 30 or a server device equipped with the LLM 100 used to control the robot 30, and information indicating the communication quality is transmitted from the robot 30 or the server device to at least the information processing device 20a and the learning instruction device 40.

[0086] [2.2. Control processing example] Next, the flow of the control process executed by the control system 1a according to this embodiment will be described. Fig. 6 is an example of a flowchart showing the flow of the process. At the start of the control process shown in Fig. 6, it is assumed that the classifier 28 has been sufficiently trained by the training process shown in Fig. 7, which will be described later.

[0087] In S101 and S102, the same processing as in the flowchart of FIG. 2 is performed.

[0088] In S301, the task time prediction unit 24 predicts the execution time of a task when the robot 30 is caused to execute the task by control using the LLM 100a provided in the DC server 50. In the prediction, the task time prediction unit 24 may refer to the feature amount of the task extracted by the control unit 21a.

[0089] In S302, the communication quality prediction unit 25 predicts the communication quality of the communication between each server device and the robot 30 during the execution time predicted by the task time prediction unit 24. For example, the communication quality prediction unit 25 predicts the communication quality of the communication between the DC server 50 and the robot 30, and presumes that the same communication quality will be applied between other servers and the robot 30.

[0090] In S303, the determination unit 22 inputs the feature amount of the task extracted by the control unit 21a and information indicating the communication quality predicted by the communication quality prediction unit 25 to the classifier 28. As a classification process, the classifier 28 performs a process of outputting a success / failure prediction for each of a plurality of LLMs 100 of different sizes, indicating whether the task will succeed or fail when the robot 30 is controlled using the LLM 100.

[0091] In steps S104 to S106, the same processing as in the flowchart of Fig. 2 is performed. As a result, the robot 30 executes the task under control using one of the LLMs 100.

[0092] According to the control process of this example described above, in controlling the robot 30, an LLM 100 of an appropriate size can be used according to the difficulty of the task and the required response speed.

[0093] [2.3. Learning process example] Next, the flow of the learning process executed by the control system 1a according to this embodiment will be described. Fig. 7 is an example of a flowchart showing the flow of the process.

[0094] In steps S201 to S207, the same processes as those in the flowchart of Fig. 3 are performed. For example, in step S205, the control unit 21a of the information processing device 20a performs a process of breaking down the task received from the robot 30 into a group of low-level commands and extracting the feature quantities of the task.

[0095] In S401, the LLM 100 provided in the server device to which the control of the robot 30 has been offloaded controls the robot 30 to execute the received task. As a result, it is determined whether the robot 30 succeeded or failed in the task. Furthermore, it is determined the time actually required for the robot 30 to execute the task and the actual communication quality during the execution of the task.

[0096] In S402, the robot 30 transmits, as the result of the execution of the task, information indicating whether the robot 30 has succeeded or failed in executing the task and the time actually required to execute the task to the learning instruction device 40. The learning instruction device 40 evaluates and accumulates information regarding the result of the execution of the task by the robot 30.

[0097] In S403, the robot 30 transmits information indicating the actual communication quality during the execution time of the task to the information processing device 20a. Note that the processing of step S403 may be executed in parallel with or before the processing of S402. The information indicating the communication quality may also be transmitted to the learning instruction device 40 and used for, for example, evaluating information related to the execution result of the task. As described above, the server device including the LLM 100 used to control the robot 30 may be configured to transmit the information indicating the communication quality to the information processing device 20a, etc.

[0098] In S404, the learning unit 23 of the information processing device 20a inputs training data to the classifier 28, using the task feature amounts extracted in S205 and information indicating the actual communication quality during the task execution time as explanatory variables, and the execution results of the task controlled using any one of the LLMs 100 as objective variables. This allows the classifier 28 to learn about that LLM 100. Note that execution results corresponding to LLMs 100 other than the LLM 100 used to control the robot 30 are masked when inputting the training data to the classifier 28.

[0099] The learning process shown in the flowchart of FIG. 7 is repeated while the robot 30 moves, either under the direction of the learning instruction device 40 or autonomously, until the classifier 28 has been fully trained for all LLMs 100 that can be used to control the robot 30.

[0100] The learning process in which the learning unit 23 of the information processing device 20a trains the classifier 28 has been described above.

[0101] When the time actually required to execute a task is transmitted to the information processing device 20a, the time may also be input to the classifier 28 as training data.

[0102] 3. Supplementary Notes Regarding Embodiments 1 and 2 The success / failure prediction of the task by the classifier 28 does not necessarily need to be performed for all LLMs 100 that can be used to control the robot 30. For example, in the first embodiment where the task is a static task, if the output of the classifier 28 predicts that the task will fail to be executed when controlled using the LLM 100b of the MEC server 51, the determination unit 22 may not use the classifier 28 to predict the success / failure of the task when the LLM 100c of the edge server 52 is used for control. This is because a static task that is too difficult to be executed by the LLM 100b is estimated to be impossible to execute by the LLM 100c without referring to the output of the classifier 28.

[0103] Furthermore, the number of types of LLMs 100 and the server devices equipped therewith is not limited to three, but may be two, four or more.

[0104] 3, 4, and 7, the VLM 401 may be realized by an LLM. Furthermore, with regard to the processing of S203, the LLM 402 may be realized by an LLM that generates one or more candidate tasks to be executed by the robot 30, i.e., an LLM that performs the processing (1) described above in S203, and an LLM that evaluates each task and determines one task to be executed by the robot 30, i.e., an LLM that performs the processing (2) to (4) described above in S203, which are independent LLMs.

[0105] 2 and 6, the storage unit 27 of the information processing device 20 or 20a may store information for implementing the LLM, and the LLM may be used in the process of extracting the feature quantities of the task. The LLM may also be configured to be implemented as a part of the classifier 28.

[0106] As described above, in one embodiment, in the control systems 1 and 1a, in addition to the LLM 100 provided in each server device, three types of LLMs may be used in the learning instruction device 40 and one type of LLM may be used in the information processing devices 20 and 20a.

[0107] Furthermore, the control unit 21a of the information processing device 20a may be configured to determine whether a task instructed to the robot 30 is a static task or a dynamic task, for example, by using the LLM that extracts feature amounts of the task. The information processing device 20a may perform processing in accordance with the first embodiment if the task is a static task, and may perform processing in accordance with the second embodiment if the task is a dynamic task. Furthermore, when the information processing device 20a is associated with multiple robots 30, it may perform processing in accordance with the first embodiment for a task to be executed by one robot 30, and may perform processing in accordance with the second embodiment for a task to be executed by another robot 30.

[0108] [4. Software implementation example] The functions of the information processing devices 20 and 20a and the learning instruction device 40 (hereinafter referred to as the "devices") can be realized by a program for causing a computer to function as the devices, and a program for causing a computer to function as each control block of the devices (particularly each part included in the control units 21 and 21a).

[0109] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0110] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0111] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0112] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0113] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0114] 1, 1a Control System 20, 20a Information processing device 21, 21a Control section 22 Judgment section 23 Learning Department 24 Task Time Prediction Unit 25 Communication Quality Prediction Department 27 Memory section 28 Classifier 29 Communications Department 30 Robot 31 Sensors 32 Input section 40 Learning instruction device 50 DC servers 51 MEC Server 52 Edge Server 100, 100a, 100b, 100c LLM

Claims

1. a classifier that receives information indicating a task to be executed by a robot controlled using one of a plurality of LLMs (Large Language Models) having different sizes, and outputs information indicating a result of the robot executing the task under control using one of the LLMs; a determination unit that determines, using the classifier, which of the plurality of LLMs to use for controlling the robot when causing the robot to execute an arbitrary task; An information processing device comprising:

2. The classifier The information processing apparatus according to claim 1 , further comprising: a prediction of success or failure of the task when the robot is controlled using the LLM;

3. The determination unit The information processing device according to claim 2 , wherein the LLM having the smallest size among the LLMs predicted by the classifier to be successful in the task when used to control the robot is determined to be the LLM to be used to control the robot.

4. a communication quality prediction unit that predicts the communication quality between the robot and a plurality of server devices each including the plurality of LLMs when the task is being executed; The classifier 3. The information processing device according to claim 1, further comprising: a communication quality predictor that receives input of information indicating the communication quality predicted by the communication quality predictor; and an information processor that outputs information indicating a result of the robot executing the task under control using any one of the LLMs under the condition that the communication has the communication quality predicted by the information processor.

5. 3. The information processing device according to claim 1, further comprising a learning unit configured to train the classifier using, as training data, information indicating a task to be instructed to be performed by the robot and information indicating a result of the robot performing the task under control using any one of the LLMs.

6. The robot according to claim 1, comprising the information processing device according to claim 1.

7. A control system including a robot, an information processing device, and a plurality of server devices, Each server device The system includes a plurality of LLMs (Large Language Models) each having a different size, The robot Controlled using any one of the plurality of LLMs, The information processing device includes: a classifier that receives information indicating a task to be instructed to be executed by the robot and outputs information indicating a result of the robot executing the task under control using any one of the LLMs; a determination unit that determines, using the classifier, which of the plurality of LLMs to use for controlling the robot when causing the robot to execute an arbitrary task; A control system comprising: a communication unit that transmits information indicating a request to have the robot execute the task to the server device that has an LLM determined to be used to control the robot.

8. 8. The control system according to claim 7, further comprising a learning instruction device that determines the task according to a sensing result of a sensor provided in the robot, and outputs information indicating the task as information to be used as input to the classifier.

9. 2. A control program for causing a computer to function as the information processing device according to claim 1, the control program causing the computer to function as the determination unit.

10. A computer-readable recording medium on which the control program according to claim 9 is recorded.

Images