Control device for mobile robot, mobile robot, control method, and program

Abstract

A technology according to the present invention that can easily control the shape of a mobile robot group is provided. A control device for controlling a plurality of mobile robots, the control device comprising: a target position determination means that gives input information, including information for identifying a shape which is expressed by the plurality of mobile robots, to a large language model, acquires, from the large language model, output information related to a target position of each of the plurality of mobile robots for expressing the shape, and determines the target positions of the plurality of mobile robots, respectively, on the basis of the output information; and an output means that outputs, to the plurality of mobile robots, the target positions for moving the plurality of mobile robots.

Description

Mobile robot control device, mobile robot, control method, and program

[0001] The present disclosure relates to a control device for a mobile robot, a mobile robot, a control method, and a program.

[0002] Pattern formation by mobile robots is one of the key challenges in robotics, with applications ranging from drone control to autonomous navigation in factories, and even exploration and rescue missions. In these applications, it is important for mobile robots to form desired patterns.

[0003] In pattern formation, the designation of target position coordinates has often been performed manually, which has the problem of being a cumbersome and time-consuming task. Non-Patent Document 1, on the other hand, discloses a technology that enables a pattern (group shape) to be designated by a hand-drawn picture. However, forming complex patterns requires specialized skills.

[0004] As a related study, Non-Patent Document 2 proposes a method for outputting vector-format images from language.

[0005] Sun, G., Zhou, R., Ma, Z. et al. Mean-shift exploration in shape assembly of robot swarms. Nat Commun 14, 3476 (2023).<URL: https: / / doi.org / 10.1038 / s41467-023-39251-5> Tang, Zecheng, et al. "StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis." arXiv preprint arXiv:2401.17093 (2024).<URL: https: / / doi.org / 10.48550 / arXiv.2401.17093>

[0006] The present disclosure aims to provide a new and useful technology that enables even users without specialized knowledge to easily control the group shape of mobile robots.

[0007] One aspect of the present disclosure is a control device for controlling multiple mobile robots, comprising: a target position determination means for providing input information including information identifying a shape to be represented by the multiple mobile robots to a large-scale language model; obtaining output information related to a target position of each of the multiple mobile robots for representing the shape from the large-scale language model; and determining the target position of each of the multiple mobile robots based on the output information; and an output means for outputting the target positions to the multiple mobile robots for movement of the multiple mobile robots.

[0008] A large language model (LLM) is a machine learning model known as generative AI, capable of learning to generate versatile content. A large language model refers to a language model that has increased scale in at least one of the computational complexity, data volume, and number of model parameters. In this disclosure, a large language model is defined as a language model with 5 billion or more model parameters. The information identifying a shape may be in any form, such as natural language information or image information, and more specifically, includes information expressed in text, speech, gestures, images, and any combination thereof.

[0009] By determining the target position using a language model in this way, even those without specialized knowledge can easily control the group shape of mobile robots.

[0010] In one aspect, the output information includes a target position for each of the plurality of mobile robots. In another aspect, the output information includes a line drawing corresponding to the shape, and the target position determining means may determine the target position for each of the plurality of mobile robots from the line drawing.

[0011] In one aspect, the input information includes task information that describes in natural language a task of determining target positions for each of a plurality of mobile robots from information that specifies the shapes of the mobile robots. The task information can also be said to be information that describes a task to be performed by the language model.

[0012] In one aspect, the input information preferably includes example information including at least one pair of an example of information specifying a shape and an example of corresponding output information. By providing an example of a pair of input and output in addition to the task information, more accurate results can be obtained.

[0013] In one aspect, the input information includes information specifying a first shape and information specifying a second shape, the target positions include a first target position for expressing the first shape, a second target position for expressing the second shape, and at least one intermediate target position when transitioning from the first shape to the second shape, and the movement instruction means may instruct the plurality of mobile robots to move sequentially to the first target position, the intermediate target position, and the second target position.

[0014] In this way, group shape control can be easily performed to change multiple mobile robots from a first shape to a second shape. The number of intermediate target positions is not limited to one, but may be multiple sets. The greater the number of intermediate target positions, the smoother the shape transformation can be achieved. Furthermore, this method can be applied not only to changes from a first shape to a second shape, but also to changes of three or more shapes.

[0015] The present disclosure can be understood as a control device including at least some of the above means. The present disclosure can also be understood as a robot system including the above control device and multiple mobile robots. The present disclosure can also be understood as a method including at least some of the above processes, or a program for realizing such a method, or a recording medium on which such a program is non-temporarily recorded. The present disclosure also includes an apparatus or method in which the above means and processes are combined with each other to the greatest extent possible.

[0016] According to the present disclosure, group shape control of multiple mobile robots can be easily performed.

[0017] FIG. 1 is a diagram schematically illustrating a pattern formation framework according to the present disclosure. FIG. 2 is a diagram illustrating an overview of a robot system according to the present disclosure. FIG. 3 is a diagram illustrating a hardware configuration of a control device according to the present disclosure. FIG. 4 is a diagram illustrating a functional configuration of a robot system according to the present disclosure. FIG. 5 is a flowchart illustrating a flow of a robot control process executed by a control device according to the present disclosure. FIG. 6A is a first diagram illustrating an example of a prompt sent to an LLM and an example of a response from the LLM in the present disclosure. FIG. 6B is a second diagram illustrating an example of a prompt sent to an LLM and an example of a response from the LLM in the present disclosure. FIG. 6C is a third diagram illustrating an example of a prompt sent to an LLM and an example of a response from the LLM in the present disclosure. FIG. 7 is a diagram illustrating an example of a prompt sent to an LLM and an example of a response from the LLM in the present disclosure. FIG. 8A is a first diagram illustrating an example of an output from an LLM in the case where a shape change is involved in the present disclosure. FIG. 8B is a second diagram illustrating an example of an output from an LLM in the case where a shape change is involved in the present disclosure.

[0018] <Application Example> First, an example of a robot system to which the present disclosure is applied will be described. The robot system includes multiple mobile robots and a mobile robot control device. The control device controls the mobile robots so that the mobile robots as a whole form a desired shape (pattern).

[0019] FIG. 1 is a diagram that schematically illustrates a framework for pattern formation in a robot system to which the present disclosure is applied. Pattern formation by mobile robots in the present disclosure consists of two phases. The first phase is knowledge-driven pattern generation 10, which uses a large-scale language model to generate a desired pattern. The second phase is multi-robot cooperation, which controls a group of mobile robots to form a specified shape. The multi-robot cooperation includes multi-agent reinforcement learning (MARL) 20, which uses simulation to learn a joint control policy for the mobile robots, and sim-to-real transfer learning 30, which applies the simulation to the real world.

[0020] In the knowledge-driven pattern generation phase, information (e.g., text) specifying the shape to be generated is given as input to a large-scale language model (LLM). The input information is converted into a sequence of n tokens by the LLM, and the LLM outputs a set of target position coordinates G = {g1, g2,..., gK} for each mobile robot for pattern generation.

[0021] Once the target position coordinates are determined, the coordination phase of the mobile robots begins. In this example, we assume that a group of K homogeneous mobile robots i (i∈B≡{1, 2, . . . , K}) with a uniform radius R moves on a two-dimensional plane to generate a desired pattern. The position p(i) of each agent is calculated as follows: ｉ , y ｉ ) imposes collision avoidance constraints to prevent robots from colliding with each other while moving. Cooperative control is formulated as follows:

[0022]

[0023] where |·| represents the Euclidean distance defined on a two-dimensional plane, and N(g ｉ ,B)=p(argmin ｊ |g ｉ -p(j)|) is the position g ｉ represents the position of the mobile robot j∈B closest to

[0024] 2 is a diagram illustrating an overview of a robot system 100 according to the present disclosure. The robot system 100 includes a control device 110 and a mobile robot 120. The control device 110 determines target positions for multiple mobile robots to form a predetermined pattern using a large-scale language model (LLM) 130, and controls the multiple mobile robots to move to the determined target positions.

[0025] The control unit 110 provides instructions 140 for pattern generation to the LLM 130 to obtain results. The instructions 140 are also called prompts.

[0026] The instruction 140 includes, for example, a task definition, an example answer, and information specifying the shape to be generated. The task definition is information explaining the task to be performed by the LLM 130, and is typically a sentence requesting the LLM 130 to output the target positions of each mobile robot for expressing the shape using multiple robots based on the information specifying the shape to be generated included in the instruction 140. The example answer includes an example of information specifying the shape to be generated and an example of output corresponding to this example input. Including such an example answer enables more accurate output. The information specifying the shape to be generated specifies the shape to be expressed by multiple mobile robots using natural language. Examples of information specifying the shape to be generated include "circle," "A-shape," and "tree." While an example of specifying a shape using a word is given here, the shape may also be specified using one or more sentences.

[0027] The control device 110 extracts the target positions of the mobile robots 120 from the response 150 of the LLM 130, and transmits control commands to the mobile robots 120 to move to these target positions. Any known method can be used to control the mobile robots 120; the mobile robots 120 may be notified of only the target positions and move autonomously, or the control device 110 may transmit more detailed control commands to the mobile robots 120 to move to the target positions.

[0028] In FIG. 2, the instruction 140 is transmitted in text format from the control device 110 to the LLM 130, but data including voice information, image information, gesture information, etc. may also be transmitted.

[0029] 3 is a diagram showing an example of the hardware configuration of the control device 110 constituting the robot system 100 according to this embodiment. As shown in Fig. 2, the control device 110 according to this embodiment is a computer (information processing device) to which a control unit 201, a storage unit 202, an input device 205, an output device 206, a communication interface 207, and a drive 208 are electrically connected.

[0030] The control unit 201 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), and is configured to execute information processing based on programs and various data. The control unit 201 (CPU) is an example of a processor resource.

[0031] The storage unit 202 is an example of a memory resource, and is configured, for example, by a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 202 stores various information such as a robot control program 203 and prompt information.

[0032] The robot control program 203 is a program for causing the control device 110 to execute information processing for controlling the robot. The robot control program 203 includes a series of commands for the information processing. The prompt information 204 is information to be included in the instruction statement (prompt) 140 that queries the LLM 130, and corresponds to a system prompt, which will be described later.

[0033] The input device 205 is a device for inputting information, such as a mouse, keyboard, keypad, or touch panel device. The output device 206 is a device for outputting information, such as a display or speaker. A user can operate the control device 110 by using the input device 205 and the output device 206. The input device 205 and the output device 206 may be integrated into one device, such as a touch panel display.

[0034] The communication interface 207 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and is an interface for performing wired or wireless communication via a network. The control device 110 can perform data communication with the mobile robot 120, the LLM 130, and other computers via the communication interface 207.

[0035] The drive 208 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as programs stored in a storage medium 209. At least one of the robot control program 203 and the prompt information 204 may be stored in the storage medium 209.

[0036] The storage medium 209 is a medium that stores various information such as programs electrically, magnetically, optically, mechanically, or chemically so that the information can be read by a computer or other device, machine, etc. The control device 110 may acquire at least one of the robot control program 203 and the prompt information 204 from the storage medium 209.

[0037] 3 illustrates a disk-type storage medium such as a CD or DVD as an example of the storage medium 209. However, the type of the storage medium 209 is not limited to a disk type, and may be a type other than a disk. Examples of storage media other than a disk type include semiconductor memories such as flash memories. The type of the drive 208 may be selected appropriately depending on the type of the storage medium 209.

[0038] Note that, with regard to the specific hardware configuration of the control device 110, components may be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 201 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 202 may be configured with RAM and ROM included in the control unit 201. At least one of the input device 205, the output device 206, the communication interface 207, and the drive 208 may be omitted. The control device 110 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the control device 110 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a general-purpose personal computer (PC), an industrial PC, or the like.

[0039] FIG. 4 is a block diagram showing the functional configuration of the control device 110 and the mobile robot 120 that constitute the robot system 100.

[0040] By the control unit 201 executing the robot control program 203, the control device 110 functions as a user prompt acquisition unit 301, a system prompt storage unit 302, a prompt generation / transmission unit 303, a target position determination unit 304, and a movement instruction unit 305 shown in Figure 4.

[0041] The user prompt acquisition unit 301 acquires information specified by the user (user prompt) from the instruction sentence 140 to be given to the LLM 130. The prompt generation and transmission unit 303 generates a prompt by combining the user prompt and the system prompt, and transmits the generated prompt to the LLM 130. The target position determination unit 304 determines the target position of the mobile robot 120 based on the response to the prompt from the LLM 130. The movement instruction unit 305 outputs a control command including the target position to the mobile robot 120.

[0042] The mobile robot 120 may be any mobile device, such as a rotary-wing or fixed-wing drone (UAV), an autonomous mobile transport robot (AMR), a self-propelled capsule endoscope, an underwater drone (AUV), a self-propelled vacuum cleaner, or a small flying object or autonomous robot in space. However, the mobile robot 120 may be any mobile robot, with no particular limitations on its locomotion method (wheels, tracks, bipedal, multi-legged, propellers, thrust engine) or location (ground, air, underwater, space). The mobile robot 120 includes a movement instruction acquisition unit 311, a movement control unit 312, and a movement unit 313. The movement instruction acquisition unit 311 acquires a target position from the control device 110. The movement control unit 312 controls the movement unit 313 to move to the target position. The movement unit 313 includes components for moving the mobile robot 120, such as a drive source and propellers or wheels. The mobile robot 120 also includes various sensors, such as an inertial sensor, a GPS device, an ultrasonic sensor, and a camera. Furthermore, the mobile robots 120 can communicate with each other via wireless communication, enabling cooperative control with other mobile robots 120.

[0043] The LLM 130 is realized by a computer executing a program, and as an example, is realized by a cloud computing system. The LLM 130 uses a large amount of text data as training data and is trained using deep learning technology so that it can be adapted to various natural language processing tasks. The LLM 130 performs natural language processing tasks using, for example, a deep learning model called a Transformer. Examples of the LLM 130 include GPT-4o, GPT-4, GPT-3.5, Mixtral, Mistral, LLaMA, Gemini, and Cloude3.

[0044] [Processing] Figure 5 is a flowchart illustrating the flow of processing performed by the control device 110 according to this embodiment. Figures 6A to 8B are diagrams illustrating examples of prompts that the control device 110 sends to the LLM 130 and examples of outputs that the LLM 130 sends to the control device 110. The flow of processing performed by the control device 110 will be described below with reference to Figures 5 to 8B.

[0045] In step S401, the user prompt acquisition unit 301 acquires information from the user that specifies the shape to be expressed by the mobile robot 120. The information that specifies the shape is input as text in one example, and is input as voice, gesture, or image information in other examples. Examples of the information that specifies the shape include "circle," "A-shape," and "tree."

[0046] In step S402, the prompt generation and transmission unit 303 acquires a system prompt from the system prompt storage unit 302, combines it with the information (user prompt) acquired in step S401, and generates a prompt (input information) to be transmitted to the LLM .

[0047] 6A shows an example of a prompt generated by the prompt generating and transmitting unit 303. A prompt 500 includes a system prompt 501 and a user prompt 504, and the system prompt 501 includes task definition information 502 and an input / output example 503.

[0048] The task definition information 502 is information describing in natural language the task to be performed by the LLM 130. As an example, the task definition information 502 describes the content of a task, such as determining the target positions of multiple mobile robots from shape-identifying information included in the user prompt 504. As shown in FIG. 6A , the task information includes the role of the LLM 130, input information, details of the processing content, and an output format. Note that the task information shown in FIG. 6A is merely an example; specifying the task content in more detail allows the LLM 130 to perform the intended processing. For example, the task definition information 502 may include instructions such as not adding unnecessary lines, not placing robots (boxes) inside shapes unless instructed to the contrary, ensuring that the center of the shape roughly coincides with the center of the overall structure, and maintaining an appropriate distance between robots (boxes) without touching or overlapping.

[0049] The input / output example 503 describes an example of an input and an example of an output corresponding to this input. In this input / output example 503, a combination of nine coordinates representing a circle is given as an example of an output in response to the input of "target shape: circle" and "number of boxes: 9". Note that the number of examples may be one (one-shot) or two or more (few-shot). Also, the input / output example 503 may not be necessary (zero-shot).

[0050] The task definition information 502 and the input / output example 503 are stored in advance as system prompts in the system prompt storage unit 302. The prompt generation / transmission unit 303 may use the system prompts (task definition information 502 and input / output example 503) stored in the system prompt storage unit 302 as is, or may use them after partially modifying them.

[0051] The user prompt 504 is generated based on information acquired by the user prompt acquisition unit 301 from the user. The user prompt acquisition unit 301 may acquire and use the user prompt 504 itself shown in FIG. 6A from the user, or may generate the user prompt 504 by processing the information received from the user. In this example, the task is to determine the coordinates of boxes to represent a target shape using a given number of boxes (mobile robots), so the user prompt includes a "target shape" and a "number of boxes." The number of boxes may not be explicitly specified by the user, and the LLM 130 may determine an appropriate number.

[0052] While FIG. 6A illustrates an example of a prompt for specifying a target shape using text, the target shape may also be specified using information in a format other than text. For example, the information specifying the target shape may be image information or audio information. FIG. 7 illustrates an example of a prompt for specifying a target shape using image and audio data. The prompt in this example is similar in that it includes task definition information 502, an example input / output 503, and a user prompt 504. However, in addition to an example input / output 503A for when text data is input, the example input / output 503 also includes an example input / output 503B for when image data is input, and an example input / output 503C for when audio data is input. The user prompt 504 includes any of text data, image data, and audio data as information for specifying the target shape. Note that the prompt illustrated in FIG. 7 is an example that can accommodate all of text data, image data, and audio data. For example, if only image data is handled as input, the example input / output 503 may include only the example input / output 503B for image data.

[0053] The prompt generation and transmission unit 303 generates a series of information including task definition information 502, an input / output example 503, and a user prompt 504 as a prompt.

[0054] In step S403, the prompt generation and transmission unit 303 transmits the generated prompt to the LLM 130 and obtains a response (output information) to the prompt from the LLM 130. Fig. 6B is a diagram showing an example of a response 510 from the LLM 130. Fig. 6C is a diagram plotting the coordinates included in the response 510, and it can be seen that the outline of a triangle is expressed by a set of target coordinates.

[0055] In step S404, the target position determination unit 304 determines a target position of the mobile robot 120 based on the response 510. Specifically, the target position determination unit 304 parses the response 510 to obtain a set of target positions.

[0056] In step S405, the movement instruction unit 305 generates and transmits a control command to the mobile robot 120 to move to the target position acquired in step S404. When the movement instruction acquisition unit 311 acquires the movement instruction, the mobile robot 120 notifies the movement control unit 312 of the target position, and the movement control unit 312 controls the movement unit 313 to move to the target position.

[0057] The control method for the mobile robot 120 is not particularly limited, but for example, control using a control policy learned by multi-agent reinforcement learning (MARL) can be adopted. As an example, in order to ensure flexibility in robot control, a cooperation policy that takes cooperation with other mobile robots 120 into consideration is learned without assigning a fixed target position to each mobile robot 120. As an example, the MARL algorithm can be configured as follows.

[0058] The observed values ​​and the action space are defined as follows. In this embodiment, the robot system is an autonomous distributed system, and each agent (robot) acts based on local observations. The observed value z of each agent i is ｉ is the velocity (v ｉ ｘ , v ｉ ｙ ) and position (x ｉ , y ｉ ), the positions of the k closest other agents, and the k closest target positions. The agent's behavior is either moving in one of four directions (up, down, left, right) or stopping in place.

[0059] The reward function is designed to achieve pattern formation and collision avoidance. The reward rp for pattern formation is calculated by the following equation: ｉ It is defined as the negative sum of the distances between the agent and its nearest neighbor. This formulation allows the agents to reach all target positions and form the desired pattern.

[0060] In addition, to prevent collisions, a penalty is imposed when the distance between any two agents is equal to or less than the radius of the agent. The penalty rc is defined as follows: In addition, function 1 Ａ is an indicator function whose value is 1 if a given conditional expression is true and 0 otherwise.

[0061] The final reward function is r = r ｐ +αr ｃ Here, the coefficient α is a coefficient for balancing the importance of the two goals of pattern formation and collision avoidance, and is determined appropriately.

[0062] Reinforcement learning is performed by simulation, but sim-to-real transfer may be performed to apply the policy to a real mobile robot 120. Differences between the simulation and the real world include, for example, that in the simulation, a policy is learned using a velocity offset, whereas in reality, the robot is position-controlled, and that in reality, the robot is differentially actuated and non-holonomic, whereas the simulation environment is holonomic.

[0063] These discrepancies are mitigated by the following approach: First, the velocity offsets of the policy are converted into position offsets by multiplying them by a constant coefficient, and then the target relative position of each robot is converted into the corresponding position offset at each time step. Furthermore, to address the mismatch between holonomic and nonholonomic behavior, a constraint is introduced to prevent the robot from continuously moving to reach the exact target position. For example, when the robot receives a new target position, it moves for a predetermined time (e.g., 0.1 seconds) and then stops moving. This prevents the robot from reaching the target position in an overly direct manner.

[0064] The mobile robot 120 may control its movement using a method other than reinforcement learning, or the control device 110 may determine control parameters for movement and notify the mobile robot 120 of the control parameters.

[0065] Since reinforcement learning acquires behavior that maximizes reward, various route plans are possible depending on the reward design, and route plans that take human preferences into account can also be performed. Specifically, multiple route candidates are presented to the reinforcement learning module, and a human is asked to rank them, and the reward is learned based on the rankings. By performing reinforcement learning based on the reward function determined in this way, it is possible to learn route plans that humans prefer.

[0066] [Advantageous Effects] By using the LLM 130 to specify the position coordinates of each mobile robot when forming a group shape using the mobile robots 120, it is expected that the design time will be significantly reduced. In addition, since control can be performed by human language instructions, even those without specialized knowledge can carry out the design.

[0067] In the above description, one target shape is given and the mobile robot 120 is controlled to form that target shape. In one embodiment, multiple target shapes may be given and the mobile robot 120 may be controlled to change the shape.

[0068] FIG. 8A shows an example of the output 601 of the LLM 130, including a user prompt for changing a shape. The system prompt 501 (task definition information 502 and input / output example 503) can be modified as needed to accommodate changes in the task content. In this example, eight mobile robots are assumed to form a circular pattern and then a triangular pattern. The control device 110 requests output of a set of target position coordinates for the mobile robots for each of the circular and triangular patterns, as well as target position coordinates (intermediate target positions) for intermediate shapes during transformation. In this example, the output is in the form of a list containing three lists corresponding to Shape 1, Intermediate Shape, and Shape 2, respectively, but the output format is not particularly limited. FIG. 8B illustrates output coordinates from the LLM 130. Target coordinate group 611 corresponds to the output of "Shape 1" and is a group of coordinates that causes the mobile robot 120 to form a circle. Target coordinate group 613 corresponds to the output of "Shape 2" and is a group of coordinates that causes the mobile robot 120 to form a triangle. The target coordinate group 612 corresponds to the output of the "intermediate shape" and is a coordinate group that represents the state during the transition from Shape 1 (circle) to Shape 2 (triangle).

[0069] Although the LLM 130 is requested to output only one set of intermediate shapes here, the LLM 130 may be requested to output multiple intermediate shapes. Also, although the transition from Shape 1 to Shape 2, i.e., the transition between two shapes, is handled here, transitions between three or more shapes can be processed in the same way.

[0070] Although the above description illustrates an example in which the mobile robot 120 forms a pattern on a two-dimensional plane, the mobile robot 120 may also form a pattern in three-dimensional space. In this case, the target position determination unit 304 determines the target position coordinates of each mobile robot 120 as three-dimensional coordinates. For example, the task definition information 502 includes an instruction to the LLM 130 to output position coordinates for forming a specified pattern in three-dimensional space, and the input / output example 503 includes a list of three-dimensional coordinates. The prompt generation / transmission unit 303 generates the prompt 500 using the task definition information 502 and the input / output example 503.

[0071] In the above embodiment, the LLM 130 transmits a set of target position coordinates to the control device 110. However, the LLM 130 may transmit a line drawing of the target shape to the control device 110, and the control device 110 may determine the target position coordinates of the mobile robot 120 from this line drawing. In this case, the LLM 130 uses the Vision-Language Model so that it can handle line drawings (images). In this modification, the target position determination unit 304 extracts coordinates equal to the number of mobile robots 120 from the line drawing obtained from the LLM 130 and sets these as the target position coordinates of the mobile robots 120. In this case, it is possible to prevent bias in the distances between the mobile robots 120, and to prioritize corners and areas with high curvature as the target positions so as to appropriately represent the shape.

[0072] Other Embodiments The above-described embodiment is merely an example, and the present disclosure can be implemented with appropriate modifications within the scope of the gist thereof.

[0073] <Note> A control device (110) for controlling a plurality of mobile robots (120), comprising: target position determination means (303, 304) for providing input information (500) including information specifying a shape to be expressed by the plurality of mobile robots to a language model (130), obtaining output information (510) related to a target position of each of the plurality of mobile robots for expressing the shape from the language model, and determining the target position of each of the plurality of mobile robots based on the output information; and output means (305) for outputting the target positions to the plurality of mobile robots for movement of the plurality of mobile robots.

[0074] 100: Robot system 110: Control device 120: Mobile robot 130: LLM 301: User prompt acquisition unit 302: System prompt storage unit 303: Prompt generation and transmission unit 304: Target position determination unit 305: Movement instruction unit 311: Movement instruction acquisition unit 312: Movement control unit 313: Movement unit

Claims

1. A control device for controlling a plurality of mobile robots, comprising: a target position determination means for providing input information including information specifying a shape to be represented by the plurality of mobile robots to a large-scale language model, obtaining output information related to a target position of each of the plurality of mobile robots for representing the shape from the large-scale language model, and determining the target position of each of the plurality of mobile robots based on the output information; and an output means for outputting the target positions to the plurality of mobile robots for movement of the plurality of mobile robots.

2. The control device according to claim 1, wherein the output information includes the target position of each of the mobile robots.

3. The control device according to claim 2, characterized in that the input information includes task definition information that describes in natural language the task of determining the target positions of each of the multiple mobile robots from the information that specifies the shape.

4. The control device according to claim 1, wherein the input information includes example information including at least one pair of an example of information specifying the shape and an example of corresponding output information.

5. The control device according to claim 1, characterized in that the output information includes a line drawing corresponding to the shape, and the target position determination means determines the target positions of each of the plurality of robots from the line drawing.

6. The control device according to claim 1, wherein the information specifying the shape is information written in a natural language.

7. The control device according to claim 1, wherein the information specifying the shape is image information.

8. The control device described in claim 1, characterized in that the input information includes information specifying a first shape and information specifying a second shape, the target positions include a first target position for expressing the first shape, a second target position for expressing the second shape, and at least one intermediate target position when transitioning from the first shape to the second shape, and the output means outputs the first target position, the intermediate target position, and the second target position to the multiple mobile robots.

9. A mobile robot comprising: a control means for acquiring a target position for movement from a control device; and a control means for controlling the robot to move to said target position; and said target position is determined based on output information obtained by providing input information, including information specifying a shape to be expressed by the mobile robot, to a large-scale language model.

10. The mobile robot according to claim 9, wherein the control means controls movement using a control policy that has been learned to allow the robot to reach a target position while cooperating with other mobile robots and avoiding collisions.

11. A control method for controlling a plurality of mobile robots, comprising the steps of: providing input information to a large-scale language model, the input information including information specifying a shape to be represented by the plurality of mobile robots; obtaining output information related to a target position of each of the plurality of mobile robots for representing the shape from the large-scale language model, and determining the target position of each of the plurality of mobile robots based on the output information; and instructing the plurality of mobile robots to move to the target position.

12. A program for causing a computer to function as each means of the control device according to any one of claims 1 to 8.

13. A program for causing a computer to execute each step of the control method according to claim 11.

Images