Mobile robot control device, control method, robot system, and program

Abstract

To provide a technology for easily controlling the group shape of a mobile robot.SOLUTION: A control device for controlling a plurality of mobile robots, the control device comprising: a target position determination unit configured to give input information including information for specifying a shape to be expressed by the plurality of mobile robots to a large scale language model, acquire output information related to a target position of each of the plurality of mobile robots for expressing the shape from the large scale language model, and determine the target position of each of the plurality of mobile robots based on the output information; and an output unit configured to output the target position to the plurality of mobile robots for movement of the plurality of mobile robots.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control method, a robot system, and a program for a mobile robot. [Background technology]

[0002] Pattern formation by mobile robots is one of the key challenges in robotics, with applications ranging from drone control to autonomous navigation in factories, to exploration and rescue missions. In these applications, it is important for mobile robots to form desired patterns.

[0003] In pattern formation, the designation of target position coordinates has often been done manually, but this has the problem of being a cumbersome and time-consuming task. In response to this, Non-Patent Document 1 discloses a technology that enables patterns (group shapes) to be designated by hand-drawn pictures. However, forming complex patterns requires specialized skills.

[0004] As a related study, Non-Patent Document 2 proposes a method for outputting vector-format images from language. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Sun, G., Zhou, R., Ma, Z. et al. Mean-shift exploration in shape assembly of robot swarms. Nat Commun 14, 3476 (2023).<URL: https: / / doi.org / 10.1038 / s41467-023-39251-5> [Non-patent document 2] Tang, Zecheng, et al. "StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis." arXiv preprint arXiv:2401.17093 (2024).<URL: https: / / doi.org / 10.48550 / arXiv.2401.17093> Summary of the Invention [Problem to be solved by the invention]

[0006] The present disclosure aims to provide a new and useful technology that enables even users without specialized knowledge to easily control the group shape of mobile robots. [Means for solving the problem]

[0007] One aspect of the present disclosure is a control device for controlling multiple mobile robots, comprising: a target position determination means for providing input information including information identifying a shape to be represented by the multiple mobile robots to a large-scale language model; obtaining output information related to a target position of each of the multiple mobile robots for representing the shape from the large-scale language model; and determining the target position of each of the multiple mobile robots based on the output information; and an output means for outputting the target positions to the multiple mobile robots for movement of the multiple mobile robots.

[0008] Large Language Models (LLMs) are a type of machine learning known as generative AI. It is a machine learning model that has the learning ability to generate versatile content. A large-scale language model refers to a language model that has been scaled up in at least one of the amount of calculation, the amount of data, and the number of model parameters. In this disclosure, a language model with 5 billion or more model parameters is called a large-scale language model. Information that specifies a shape is natural language information, The information may be in any form, such as text, voice, gesture, image, or any combination thereof.

[0009] By determining the target position using a language model in this way, even those without specialized knowledge can easily control the group shape of mobile robots.

[0010] In one aspect, the output information includes a target position for each of the plurality of mobile robots. In another aspect, the output information includes a line drawing corresponding to the shape, and the target position determining means may determine the target position for each of the plurality of mobile robots from the line drawing.

[0011] In one aspect, the input information includes task information that describes in natural language a task of determining target positions for each of a plurality of mobile robots from information that specifies the shapes of the mobile robots. The task information can also be said to be information that describes a task to be performed by the language model.

[0012] In one aspect, the input information preferably includes example information including at least one pair of example information specifying a shape and example output information corresponding to the example information. By providing an example pair of input and output in addition to the task information, more accurate results can be obtained.

[0013] In one aspect, the input information includes information specifying a first shape and information specifying a second shape, the target positions include a first target position for expressing the first shape, a second target position for expressing the second shape, and at least one intermediate target position when transitioning from the first shape to the second shape, and the movement instruction means may instruct the plurality of mobile robots to move sequentially to the first target position, the intermediate target position, and the second target position.

[0014] In this way, group shape control can be easily performed to change multiple mobile robots from a first shape to a second shape. The number of intermediate target positions is not limited to one set, but may be multiple sets. The greater the number of intermediate target positions, the smoother the shape transformation can be achieved. Furthermore, this method can be applied not only to changes from a first shape to a second shape, but also to changes of three or more shapes.

[0015] The present disclosure can be understood as a control device including at least some of the above means. The present disclosure can also be understood as a robot system including the above control device and multiple mobile robots. The present disclosure can also be understood as a method including at least some of the above processes, or a program for realizing such a method, or a recording medium on which such a program is non-temporarily recorded. The present disclosure also includes an apparatus or method in which the above means and processes are combined with each other to the greatest extent possible. [Effects of the Invention]

[0016] According to the present disclosure, group shape control of multiple mobile robots can be easily performed. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a schematic diagram of a patterning framework according to the present disclosure; [Figure 2] FIG. 1 is a diagram showing an overview of a robot system according to the present disclosure. [Figure 3] FIG. 2 is a diagram showing a hardware configuration of a control device according to the present disclosure. [Figure 4] FIG. 1 is a diagram showing a functional configuration of a robot system according to the present disclosure. [Figure 5] 10 is a flowchart showing the flow of a robot control process executed by a control device according to the present disclosure. [Figure 6] FIG. 10 shows an example of a prompt sent to an LLM and an example of an LLM's response in the present disclosure. [Figure 7] FIG. 10 shows an example of a prompt sent to an LLM and an example of an LLM's response in the present disclosure. [Figure 8]10A and 10B are diagrams showing examples of LLM outputs when a shape change occurs in the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018] <Application example> First, an example of a robot system to which the present disclosure is applied will be described. The robot system includes multiple mobile robots and a mobile robot control device. The control device controls the mobile robots so that the mobile robots as a whole form a desired shape (pattern).

[0019] FIG. 1 is a diagram that schematically illustrates a framework for pattern formation in a robot system to which the present disclosure is applied. Pattern formation by mobile robots in the present disclosure consists of two phases. The first phase is knowledge-driven pattern generation 10, which uses a large-scale language model to generate a desired pattern, and the second phase is multi-robot cooperation, which controls a group of mobile robots to form a specified shape. The multi-robot cooperation includes multi-agent reinforcement learning (MARL) 20, which uses simulation to learn a joint control policy for the mobile robots, and sim-to-real transfer learning 30 for application from simulation to the real world.

[0020] In the knowledge-driven pattern generation phase, information (e.g., text) specifying the shape to be generated is given as input to a large-scale language model (LLM). The LLM converts the input information into a sequence of n tokens, and outputs a set of target position coordinates G = {g1, g2, ..., gK} for each mobile robot for pattern generation.

[0021] Once the target position coordinates are determined, the coordination phase of the mobile robots begins. In this example, we assume that a group of K homogeneous mobile robots i (i∈B≡{1,2,...,K}) with a uniform radius R moves on a two-dimensional plane to generate a desired pattern. The position of each agent p(i)=(x i ,y i) imposes collision avoidance constraints to prevent robots from colliding with each other while moving. Cooperative control is formulated as follows:

[0022]

number

[0023] where || represents the Euclidean distance defined in the two-dimensional plane, and N(g i ,B)=p(argmin j |g i -p(j)|) is the position g i represents the position of the mobile robot j∈B closest to

[0024] 2 is a diagram illustrating an overview of a robot system 100 according to the present disclosure. The robot system 100 includes a control device 110 and a mobile robot 120. The control device 110 determines target positions for multiple mobile robots to form a predetermined pattern using a large-scale language model (LLM) 130, and controls the multiple mobile robots to move to the determined target positions.

[0025] The control device 110 provides instructions 140 for pattern generation to the LLM 130 to obtain results. The instructions 140 are also called prompts.

[0026] The instruction 140 includes, for example, a task definition, an example answer, and information specifying the shape to be generated. The task definition is information that explains the task to be performed by the LLM 130, and the typical Specifically, it is a statement requesting the LLM 130 to output the target positions of each mobile robot for expressing the shape with multiple robots based on the information specifying the shape to be generated contained in the instruction statement 140. The example answers include examples of information specifying the shape to be generated and example outputs corresponding to these input examples. Including such example answers enables more accurate output to be obtained. The information specifying the shape to be generated specifies the shape to be expressed by multiple mobile robots using natural language. Examples of information specifying the shape to be generated include "circle," "A-shape," and "tree." While an example of specifying a shape using a word is given here, the shape may also be specified using one or more sentences.

[0027] The control device 110 extracts the target positions of the mobile robots 120 from the response 150 of the LLM 130, and transmits control commands to the mobile robots 120 to move to these target positions. Any known method can be used to control the mobile robots 120; the mobile robots 120 may be notified of only the target positions and move autonomously, or the control device 110 may transmit more detailed control commands to the mobile robots 120 to move to the target positions.

[0028] In FIG. 2, the control device 110 transmits the instruction 140 in text format to the LLM 130, but data including audio information, image information, gesture information, etc. may also be transmitted.

[0029] <Embodiment> [composition] 3 schematically shows an example of the hardware configuration of the control device 110 constituting the robot system 100 according to this embodiment. As shown in Fig. 2, the control device 110 according to this embodiment is a computer (information processing device) to which a control unit 201, a storage unit 202, an input device 205, an output device 206, a communication interface 207, and a drive 208 are electrically connected.

[0030] The control unit 201 includes a CPU (Central Processing Unit) which is a hardware processor. The control unit 201 includes a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and is configured to execute information processing based on programs and various data. The control unit 201 (CPU) is an example of a processor resource.

[0031] The storage unit 202 is an example of a memory resource, and is configured, for example, by a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 202 stores various information such as a robot control program 203 and prompt information.

[0032] The robot control program 203 is a program for causing the control device 110 to execute information processing for controlling the robot. The robot control program 203 includes a series of commands for the information processing. The prompt information 204 is information to be included in the instruction statement (prompt) 140 that queries the LLM 130, and corresponds to a system prompt, which will be described later.

[0033] The input device 205 is a device for inputting, such as a mouse, keyboard, keypad, or touch panel device. The output device 206 is a device for outputting, such as a display or speaker. A user can operate the control device 110 by using the input device 205 and the output device 206. The input device 205 and the output device 206 may be integrated into one device, such as a touch panel display.

[0034] The communication interface 207 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is used for wired or wireless communication via a network. The control device 110 can communicate data with the mobile robot 120, the LLM 130, and other computers via the communication interface 207.

[0035] The drive 208 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as programs stored in a storage medium 209. At least one of the robot control program 203 and the prompt information 204 may be stored in the storage medium 209.

[0036] The storage medium 209 is a medium that stores various information such as programs electrically, magnetically, optically, mechanically, or chemically so that the information can be read by a computer or other device, machine, etc. The control device 110 may acquire at least one of the robot control program 203 and the prompt information 204 from the storage medium 209.

[0037] 3 illustrates a disk-type storage medium such as a CD or DVD as an example of the storage medium 209. However, the type of storage medium 209 is not limited to a disk type, and may be other than a disk type. Examples of storage media other than a disk type include semiconductor memories such as flash memories. The type of drive 208 may be selected appropriately depending on the type of storage medium 209.

[0038] It should be noted that, with regard to the specific hardware configuration of the control device 110, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 201 may include multiple hardware processors. The hardware processors may be microprocessors, FPGAs (field-programmable gate arrays), DSPs (digital signal processors), etc. The control device 110 may be configured as a general-purpose server, a general-purpose personal computer (PC), an industrial PC, or the like, in addition to an information processing device designed specifically for the service to be provided.

[0039] FIG. 4 is a block diagram showing the functional configuration of the control device 110 and the mobile robot 120 that constitute the robot system 100.

[0040] By the control unit 201 executing the robot control program 203, the control device 110 functions as a user prompt acquisition unit 301, a system prompt storage unit 302, a prompt generation and transmission unit 303, a target position determination unit 304, and a movement instruction unit 305 shown in Figure 4.

[0041] The user prompt acquisition unit 301 acquires information (user prompt) designated by the user from among the prompts 140 to be given to the LLM 130. The prompt generation and transmission unit 303 generates a prompt by combining the user prompt and the system prompt, and transmits the generated prompt to the LLM 130. The target position determination unit 304 determines the target position of the mobile robot 120 based on the response to the prompt from the LLM 130. The movement instruction unit 305 outputs a control command including the target position to the mobile robot 120.

[0042] The mobile robot 120 may be any mobile device, such as a rotary or fixed-wing drone (UAV), an autonomous mobile robot (AMR), a self-propelled capsule endoscope, an underwater drone (AUV), a self-propelled vacuum cleaner, or a small robot in outer space. Typical examples include flying vehicles and autonomous robots, but the method of movement (wheels, caterpillars, bipeds, multi-legged, propellers, thrust engines) and location of movement (ground, air, water, outer space) are not particularly limited, and any mobile robot may be used. The mobile robot 120 includes a movement instruction acquisition unit 311, a movement control unit 312, and a movement unit 313. The movement instruction acquisition unit 311 acquires a target position from the control device 110. The movement control unit 312 controls the movement unit 313 to move to the target position. The movement unit 313 includes components for moving the mobile robot 120, such as a drive source and propellers or wheels. The mobile robot 120 also includes various sensors, such as an inertial sensor, a GPS device, an ultrasonic sensor, and a camera. Mobile robots 120 can communicate with each other via wireless communication, enabling cooperative control with other mobile robots 120.

[0043] LLM130 is implemented by a computer executing a program, for example, a cloud computing system. LLM130 uses large amounts of text data as training data and is trained using deep learning techniques to be adaptable to various natural language processing tasks. LLM130 performs natural language processing tasks using a deep learning model called a Transformer, for example. Examples of LLM130 include GPT-4o, GPT-4, GPT-3.5, Mixtral, Mistral, LLaMA, Gemini, and Claude3.

[0044] [process] Fig. 5 is a flowchart illustrating the flow of processing performed by the control device 110 according to this embodiment. Figs. 6 to 8 are diagrams illustrating examples of prompts that the control device 110 transmits to the LLM 130 and examples of outputs that the LLM 130 transmits to the control device 110. The flow of processing performed by the control device 110 will be described below with reference to Figs. 5 to 8.

[0045] In step S401, the user prompt acquisition unit 310 acquires information from the user that specifies the shape to be expressed by the mobile robot 120. The information that specifies the shape is input as text in one example, and is input as voice, gesture, or image information in other examples. Examples of the information that specifies the shape include "circle," "A-shape," and "tree."

[0046] In step S402, the prompt generation and transmission unit 303 acquires a system prompt from the system prompt storage unit 320, combines it with the information (user prompt) acquired in step S401, and generates a prompt (input information) to be transmitted to the LLM .

[0047] 6(A) shows an example of a prompt generated by the prompt generating and transmitting unit 303. The prompt 500 includes a system prompt 501 and a user prompt 504, and the system prompt 501 includes task definition information 502 and an input / output example 503.

[0048] The task definition information 502 is information describing in natural language the task to be performed by the LLM 130. As an example, the task definition information 502 describes the content of a task to determine the target positions of multiple mobile robots from shape-identifying information included in the user prompt 504. As shown in FIG. 6A, the task information includes the role of the LLM 130, input information, details of the processing content, output format, etc. Note that the task information shown in FIG. 6A is merely an example; by specifying the task content in more detail, the LLM 130 can be made to perform the intended processing. For example, the task definition information 502 may include instructions such as not adding unnecessary lines, not placing robots (boxes) inside shapes unless instructed to the contrary, making sure the center of the shape roughly coincides with the center of the whole, and maintaining an appropriate distance between robots (boxes) without touching or overlapping.

[0049] Input / output example 503 describes an example of input and an example of output corresponding to this input. This input / output example 503 gives a combination of nine coordinates representing a circle as an example of output for the input "target shape: circle" and "number of boxes: 9". Note that the number of examples may be one (one-shot) or two or more (few-shot). Also, input / output example 503 may not be necessary (zero-shot).

[0050] The task definition information 502 and the input / output example 503 are stored in advance as system prompts in the system prompt storage unit 302. The prompt generating and transmitting unit 303 may use the system prompts (task definition information 502 and input / output example 503) stored in the system prompt storage unit 320 as is, or may use them after partially modifying them.

[0051] The user prompt 504 is generated based on information acquired by the user prompt acquisition unit 310 from the user. The user prompt acquisition unit 310 may acquire and use the user prompt 504 itself shown in FIG. 6(A) from the user, or may generate the user prompt 504 by processing the information received from the user. In this example, the task is to determine the coordinates of boxes to represent a target shape using a given number of boxes (mobile robots), so the user prompt includes a "target shape" and a "number of boxes." The number of boxes may not be explicitly specified by the user, and the LLM 130 may determine an appropriate number.

[0052] FIG. 6 shows an example of a prompt when the target shape is specified by text, but the target shape may be specified by information in a format other than text. For example, the information specifying the target shape may be image information or audio information. FIG. 7 shows an example of a prompt when the target shape can also be specified by image and audio data. The prompt in this example is similar in that it includes task definition information 502, input / output example 503, and user prompt 54. However, input / output example 503 includes, in addition to input / output example 503A when text data is input, input / output example 503B when image data is input, and input / output example 503C when audio data is input. User prompt 504 includes any of text data, image data, and audio data as information specifying the target shape. Note that the prompt shown in FIG. 7 is an example that can accommodate all of text data, image data, and audio data. For example, if only image data is handled as input, input / output example 503 may include only image data input / output example 503B.

[0053] The prompt generation and transmission unit 303 generates a series of information including task definition information 502, input / output example 503, and user prompt 504 as a prompt.

[0054] In step S403, the prompt generation and transmission unit 303 transmits the generated prompt to the LLM 130 and obtains an answer (output information) to the prompt from the LLM 130. Fig. 6(B) is a diagram showing an example of an answer 510 from the LLM 130. Fig. 6(C) shows a diagram in which the coordinates included in the answer 510 are plotted, and it can be seen that the outline of a triangle is expressed by a set of target coordinates.

[0055] In step S404, the target position determination unit 304 determines the target position of the mobile robot 120 based on the response 510. Specifically, the target position determination unit 304 parses the response 510 to obtain a set of target positions.

[0056] In step S405, the movement instruction unit 305 generates and transmits to the mobile robot 120 a control command to move to the target position acquired in step S404. When the movement instruction acquisition unit 311 acquires the movement instruction, the mobile robot 120 transmits to the movement control unit 312 The target position is notified, and the movement control unit 312 controls the movement unit 313 to move to the target position.

[0057] The control method for the mobile robot 120 is not particularly limited, but for example, control using a control policy learned by multi-agent reinforcement learning (MARL) can be adopted. As an example, in order to ensure flexibility in robot control, a cooperation policy that takes cooperation with other mobile robots 120 into consideration is learned without assigning a fixed target position to each mobile robot 120. As an example, the MARL algorithm can be configured as follows.

[0058] The observed values ​​and the action space are defined as follows. In this embodiment, the robot system is an autonomous distributed system, and each agent (robot) acts based on local observations. The observed value z of each agent i is i is the velocity (v i x ,v i y ) and position (x i ,y i ), the positions of the k closest other agents, and the k closest target positions. The agent's behavior is either moving in one of four directions (up, down, left, right) or stopping in place.

[0059] The reward function is designed to achieve pattern formation and collision avoidance. The reward rp for pattern formation is calculated by the following equation: i It is defined as the negative sum of the distances between the agent and its nearest neighbor.

number

[0060] In addition, to prevent collisions, a penalty is imposed when the distance between any two agents is less than the radius of the agent. The penalty rc is defined as follows:

number

[0061] The final reward function is r=r p +αr c Here, the coefficient α is a coefficient for balancing the importance of the two goals of pattern formation and collision avoidance, and is determined appropriately.

[0062] Reinforcement learning is performed by simulation, but sim-to-real transfer may be performed to apply the policy to a real mobile robot 120. Differences between simulation and the real world include, for example, that in simulation, a policy is learned using a velocity offset, whereas in reality, the robot is position-controlled, and that in reality, the robot is differentially actuated and non-holonomic, whereas the simulation environment is holonomic.

[0063] These discrepancies are mitigated by the following approach: first, the velocity offsets in the policy are converted to position offsets by multiplying them by a constant coefficient, and then the target relative position of each robot is converted to the corresponding tentative fetch at each time step. Furthermore, to address the mismatch between holonomic and nonholonomic behavior, the robots are forced to reach the exact target position. Constraints are introduced to prevent the robot from moving continuously to reach the goal position. For example, when the robot receives a new goal position, it moves for a certain time (e.g., 0.1 seconds) and then stops moving. This prevents the robot from reaching the goal position too directly.

[0064] The mobile robot 120 may control its movement using a method other than reinforcement learning, or the control device 110 may determine control parameters for movement and notify the mobile robot 120 of the control parameters.

[0065] Since reinforcement learning acquires behavior that maximizes reward, various route plans are possible depending on the reward design, and route plans that take human preferences into account can also be performed. Specifically, the reinforcement learning module is presented with multiple route candidates, and a human is asked to rank them, and the reward is learned based on the rankings. By performing reinforcement learning based on the reward function determined in this way, it is possible to learn route plans that humans prefer.

[0066] [Beneficial Effects] By using the LLM 130 to specify the position coordinates of each mobile robot when forming a group of mobile robots 120, it is expected that the design time will be significantly reduced. In addition, because it can be controlled by human language instructions, even those without specialized knowledge can carry out the design.

[0067] <Modification> In the above description, one target shape is given and the mobile robot 120 is controlled to form that target shape. In one embodiment, multiple target shapes may be given and the mobile robot 120 may be controlled to change shapes.

[0068] FIG. 7A shows an example of the output 601 of the LLM 130, including a user prompt for changing a shape. The system prompt 501 (task definition information 502 and input / output example 503) can be changed as needed to accommodate changes in the task content. In this example, eight mobile robots are assumed to form a circular pattern and then a triangular pattern. The control device 110 requests the output of one set of target position coordinates for the mobile robots for each of the circular and triangular patterns, as well as target position coordinates (intermediate target positions) for intermediate shapes during transformation. In this example, the output is in the form of a list containing three lists corresponding to Shape 1, Intermediate Shape, and Shape 2, respectively, but the output format is not particularly limited. FIG. 7B shows the output coordinates from the LLM 130. Target coordinate set 611 corresponds to the output of "Shape 1" and is a set of coordinates that causes the mobile robot 120 to form a circle. Target coordinate set 613 corresponds to the output of "Shape 2" and is a set of coordinates that causes the mobile robot 120 to form a triangle. The target coordinate group 612 corresponds to the output of the "intermediate shape" and is a coordinate group that represents the state during the transition from Shape 1 (circle) to Shape 2 (triangle).

[0069] Although the LLM 130 is requested to output only one set of intermediate shapes here, the LLM 130 may be requested to output multiple intermediate shapes. Also, although the transition from Shape 1 to Shape 2, i.e., the transition between two shapes, is handled here, transitions between three or more shapes can also be processed in a similar manner.

[0070] Furthermore, although the above description illustrates an example in which the mobile robot 120 forms a pattern on a two-dimensional plane, the mobile robot 130 may also form a pattern in three-dimensional space. In this case, the target position determination unit 304 determines the target position coordinates of each mobile robot 120 as three-dimensional coordinates. For example, the task definition information 502 includes an instruction to the LLM 130 to output position coordinates for forming a specified pattern in three-dimensional space, and the input / output example 503 includes a list of three-dimensional coordinates. The prompt generation / transmission unit 303 generates a prompt 500 using the task definition information 502 and the input / output example 503.

[0071] In the above embodiment, the LLM 130 transmits a set of target position coordinates to the control device 110. However, the LLM 130 may transmit a line drawing of the target shape to the control device 110, and the control device 110 may determine the target position coordinates of the mobile robot 120 from this line drawing. In this case, the LLM 130 uses a Vision-Language Model so that it can handle line drawings (images). In this modification, The target position determination unit 304 extracts coordinates for the number of mobile robots 120 from the line drawing obtained from the LLM 130 and sets these as the target position coordinates of the mobile robots 120. At this time, it may be possible to prevent imbalance in the distances between the mobile robots 120, or to prioritize corners or areas with high curvature as target positions so as to properly represent the shape.

[0072] <Other embodiments> The above-described embodiment is merely an example, and the present disclosure can be modified and implemented as appropriate within the scope that does not deviate from the gist of the disclosure.

[0073] <Additional Notes> A control device (110) for controlling a plurality of mobile robots (120), a target position determination means (303, 304) for providing input information (500) including information specifying a shape to be expressed by the plurality of mobile robots to a language model (130), obtaining output information (510) related to a target position of each of the plurality of mobile robots for expressing the shape from the language model, and determining the target position of each of the plurality of mobile robots based on the output information; an output means (305) for outputting the target positions for movement of the plurality of mobile robots to the plurality of mobile robots; A control device comprising: [Explanation of symbols]

[0074] 100: Robot system 110: Control device 120: Mobile robot 130: LLM 301: User prompt acquisition unit 302: System prompt storage unit 303: Prompt generation and transmission unit 304: Target position determination unit 305: Movement instruction unit 311: Movement instruction acquisition unit 312: Movement control unit 313: Movement unit

Claims

1. A control device for controlling a plurality of mobile robots, a target position determination means for providing input information including information specifying a shape to be represented by the plurality of mobile robots to a large-scale language model, obtaining output information related to a target position of each of the plurality of mobile robots for representing the shape from the large-scale language model, and determining the target position of each of the plurality of mobile robots based on the output information; an output means for outputting the target positions for movement of the plurality of mobile robots to the plurality of mobile robots; A control device comprising:

2. the output information includes the target position of each of the mobile robots; 2. The control device according to claim 1.

3. the input information includes task definition information describing in a natural language a task of determining target positions of the plurality of mobile robots from the information specifying the shapes; 3. The control device according to claim 2.

4. the input information includes example information including at least one set of an example of information specifying the shape and an example of corresponding output information; 2. The control device according to claim 1.

5. the output information includes a line drawing corresponding to the shape; the target position determination means determines a target position of each of the plurality of robots from the line drawing; 2. The control device according to claim 1.

6. The information specifying the shape is information written in a natural language.

2. The control device according to claim 1.

7. The information specifying the shape is image information.

2. The control device according to claim 1.

8. the input information includes information specifying a first shape and information specifying a second shape; the target positions include a first target position for representing a first shape, a second target position for representing a second shape, and at least one intermediate target position when transitioning from the first shape to the second shape; the output means outputs the first target position, the intermediate target position, and the second target position to the plurality of mobile robots.

2. The control device according to claim 1.

9. A mobile robot, a control means for acquiring a target position for movement from the control device; a control means for controlling the movement to the target position; the target position is determined based on output information obtained by providing input information, including information specifying a shape to be expressed by the mobile robot, to a large-scale language model; A mobile robot characterized by:

10. the control means controls the movement using a control policy that has been learned so that the mobile robot can reach a target position while cooperating with other mobile robots and avoiding collisions; 10. The mobile robot according to claim 9.

11. A control method for controlling a plurality of mobile robots, comprising: providing input information to a large-scale language model, the input information including information specifying shapes to be represented by the plurality of mobile robots; obtaining output information related to a target position of each of the plurality of mobile robots for representing the shape from the large-scale language model, and determining the target position of each of the plurality of mobile robots based on the output information; instructing the plurality of mobile robots to move to the target position; A control method comprising:

12. A program for causing a computer to function as each of the means of the control device according to any one of claims 1 to 8.

13. A program for causing a computer to execute each step of the control method according to claim 11.

Images