Drawing generation device, drawing generation method, and computer program

Abstract

The drawing generation device disclosed herein includes a generation unit that inputs a prompt including operation sequence information written in natural language indicating the operation sequence of multiple control target robots into a language model and causes the language model to generate drawing information showing a diagram representing the operation sequence of the multiple control target robots, and a memory unit that stores the drawing information.

Description

[Technical Field]

[0001] The present disclosure relates to a drawing generation device, a drawing generation method, and a computer program. [Background technology]

[0002] In recent years, the use of autonomously driven automated guided vehicles (AGVs) has been put to practical use for transporting cargo within facilities such as factories and warehouses. A control technology for a guidance system that drives an AGV to a predetermined destination along a predetermined rail track laid within a facility is disclosed, for example, in Patent Document 1. In addition, an autonomous driving technology that controls driving to a destination by estimating the vehicle's own position and creating an environmental map is described, for example, in Patent Document 2. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-305837 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-093811 Summary of the Invention [Problem to be solved by the invention]

[0004] When operating multiple robots, including the automated guided vehicle described in Patent Document 1, they are required to exchange signals with each other and synchronize. In this case, the design may require the designer to create a diagram (e.g., a sequence diagram) that shows the operation sequence of the multiple robots. In particular, the more robots there are, the more complex the diagram becomes. This can place an excessive burden on the designer. This is not limited to multiple robots of the same type, but also applies when creating a diagram that shows the operation sequence of multiple robots of different types.

[0005] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and its purpose is to provide a drawing generation device, a drawing generation method, and a computer program that can easily generate a diagram showing the operation sequence of multiple controlled robots. [Means for solving the problem]

[0006] According to the present disclosure, there is provided a drawing generation device comprising: a generation unit that inputs a prompt including operation sequence information written in natural language indicating the operation sequence of multiple control target robots into a language model, and causes the language model to generate drawing information showing a diagram representing the operation sequence of the multiple control target robots; and a memory unit that stores the drawing information.

[0007] According to the present disclosure, a drawing generation method is provided, which includes the steps of inputting a prompt containing operation sequence information written in natural language indicating the operation sequence of multiple control target robots into a language model, causing the language model to generate drawing information showing a diagram representing the operation sequence of the multiple control target robots, and acquiring the drawing information.

[0008] According to the present disclosure, a computer program is provided that causes a computer to execute the steps of inputting a prompt including operation sequence information written in natural language indicating the operation sequence of multiple control target robots into a language model, causing the language model to generate drawing information showing a diagram representing the operation sequence of the multiple control target robots, and acquiring the drawing information. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a drawing generation device, a drawing generation method, and a computer program that can easily generate a diagram showing the operation sequence of multiple robots to be controlled. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of a drawing generation system according to a first embodiment of the present disclosure. [Figure 2] 1 is a block diagram showing an example of the configuration of a drawing generation device according to a first embodiment. [Figure 3] FIG. 2 is a diagram for explaining an example of natural language information according to the first embodiment. [Figure 4] 10 is a flowchart showing an example of a prompt according to the first embodiment. [Figure 5] FIG. 2 is a diagram showing an example of a sequence diagram according to the first embodiment. [Figure 6] 1 is a flowchart showing an example of a drawing generation method according to the first embodiment. [Figure 7] FIG. 10 is a block diagram showing an example of the configuration of a drawing generation device according to a second embodiment of the present disclosure. [Figure 8] FIG. 10 is a block diagram showing an example configuration of a learning device according to a third embodiment of the present disclosure. [Figure 9] FIG. 10 is a block diagram showing an example of the configuration of a drawing generation device according to a fourth embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted. In the drawings, a database device may be referred to as a "DB."

[0012] (Embodiment 1) A robot system SYS according to a first embodiment of the present disclosure will be described with reference to FIGS. 1 to 6. FIG. 1 is a block diagram showing a configuration example of the robot system SYS. As shown in FIG. 1, the robot system SYS includes a learning device 100, a generative AI (Artificial Intelligence) device 200, a drawing generation device 300, a robot management device 400, a prompt database device 500, a drawing database device 600, a plurality of robots RB, at least one piece of industrial equipment EQ, a management system MG, and at least one user terminal UT. The robot system SYS may also include a constraint database device 700.

[0013] The learning device 100 is a device for learning and constructing a language model TM. The learning device 100 trains an untrained or pre-trained language model TM with multiple training datasets so as to generate drawing information (hereinafter referred to as "drawing information 601") indicating a diagram showing the movement order of multiple robots RB when a prompt (hereinafter referred to as "prompt P2") is input. The prompt P2 includes action order information written in natural language indicating the movement order of multiple robots RB. In this specification, the action order may include not only the order of the actions but also the content of the actions. The training dataset includes the training prompt P2 and the drawing information 601. In other words, the learning device 100 generates a trained language model TM by having the untrained or pre-trained language model TM learn the relationship between the training prompt P2 and the drawing information 601. In this specification, "learning" refers to, for example, "machine learning."

[0014] As an example, the learning device 100 generates a language model TM by fine-tuning a pre-trained large language model (LLM) or a vision language model (VLM). Fine-tuning is performed by, for example, supervised learning, reinforcement learning, self-supervised learning, or a combination of two or more of them. For example, in the case of supervised learning, the learning prompt P2 is the explanatory variable (input), and the learning drawing information 601 is the objective variable (output). The machine learning model of the LLM or VLM is, for example, a deep neural network with a Transformer as its basic structure.

[0015] The generative AI device 200 stores the language model TM constructed by the learning device 100. The language model TM is, for example, an LLM. The language model TM may also be, for example, a VLM. The generative AI device 200 is, for example, a server. The language model TM is a trained model. The language model TM is a computer program. Note that the learning device 100 may update the parameters of the language model TM by retraining the language model TM.

[0016] The drawing generation device 300 inputs a prompt P2 including operation sequence information written in natural language indicating the operation sequence of multiple robots RB into the language model TM, and causes the language model TM to generate drawing information 601 indicating a diagram showing the operation sequence of multiple robots RB (hereinafter referred to as a "sequence diagram SD").

[0017] The prompt database device 500 stores a plurality of prompts P2. The prompts P2 are generated by the diagram generating device 300. The plurality of prompts P2 may include prompts created by a person via an input device. Note that, for example, one prompt P2 is information corresponding to one sequence diagram SD.

[0018] The drawing database device 600 stores a plurality of pieces of drawing information 601. The drawing information 601 is drawing information generated by a language model TM. The plurality of pieces of drawing information 601 may include drawing information created by a person via an input device. Note that, for example, one piece of drawing information 601 corresponds to one sequence diagram SD.

[0019] The constraint database device 700 stores one or more pieces of constraint information 701. The constraint information 701 indicates constraint conditions when multiple robots RB operate. Typically, the constraint conditions are conditions that indicate constraints on the operations between the robots RB. When the constraint database device 700 stores multiple pieces of constraint information 701, for example, the multiple pieces of constraint information 701 indicate different constraint conditions from each other.

[0020] A constraint condition is, for example, a condition that indicates a mutually exclusive relationship regarding the simultaneous execution of operations among multiple robots RB. For example, if robot RB1 is an unmanned transport robot, robot RB2 is a work robot, and robot RB1 transports robot RB2, the constraint condition is "while robot RB1 is moving, robot RB2 is stopped. While robot RB2 is operating, robot RB1 is stopped."

[0021] The constraint condition may include, for example, a condition indicating a constraint between the plurality of robots RB and a management device (for example, the robot management device 400 or the management system MG). For example, the constraint condition is that the plurality of robots RB operate according to instructions from the management device.

[0022] The constraint conditions may include, for example, conditions indicating constraints on the physical space in which the multiple robots RB operate. The conditions indicating constraints on the physical space include, for example, conditions indicating one or more constraints on the workspace of the robot RB, its positional relationship with other equipment, obstacles on the work path, and a minimum distance to ensure safety. The minimum distance to ensure safety indicates, for example, the minimum physical distance that must be maintained between the robots RB or between a human and a robot RB.

[0023] The constraint conditions may include, for example, conditions indicating constraints on the performance of the plurality of robots RB. The conditions indicating constraints on the performance include, for example, conditions indicating one or more constraints on the operating range, maximum speed, acceleration, payload, battery capacity, operating accuracy, and sensor performance of the robot RB.

[0024] The constraint conditions may include, for example, conditions indicating constraints on the operation of the plurality of robots RB. The conditions indicating constraints on the operation include, for example, conditions indicating one or more constraints among the operating time periods of the robots RB, priority rules, safety regulations, maintenance requirements, and constraints on energy consumption.

[0025] The robot management device 400 manages multiple robots RB. The robot management device 400 includes, for example, a PLC (Programmable Logic Controller) and / or an FMS (Fleet Management System). The robot management device 400 communicates with the multiple robots RB and transmits instruction information to each robot RB. The instruction information includes information instructing the robot RB to start an operation to be performed. The multiple robots RB operate according to the instruction information. In addition, the robot management device 400 receives, from each robot RB, start notification information notifying that the operation instructed by the instruction information has started, and completion notification information notifying that the operation instructed by the instruction information has been completed.

[0026] The multiple robots RB may be the same type of robot or different types of robots. The type of robot RB is not particularly limited. Furthermore, there may be two or more types of robots RB. The robot RB corresponds to an example of a "robot to be controlled" in the present disclosure.

[0027] The robot RB1 is, for example, an unmanned transport robot that transports an object to be transported. The robot system SYS may include one or more robots RB1. The object to be transported is, for example, a cart, a workpiece, a package, or a robot (for example, a transport robot such as a conveyor or a working robot).

[0028] An automated guided robot is an automated guided vehicle used to transport objects such as various manufacturing parts and luggage in, for example, manufacturing factories and logistics warehouses. The automated guided vehicle is, for example, an automatic guided vehicle (AGV) or an autonomous mobile robot (AMR). As an example, the automated guided vehicle has an autonomous driving mode and / or a guided driving mode. The guided driving mode is a driving mode in which the automated guided vehicle moves along real or virtual guidelines. The autonomous driving mode is a driving mode in which the automated guided vehicle can move in areas where no guidelines are provided by estimating its own position. The autonomous driving mode is realized, for example, by a simultaneous localization and mapping (SLAM) function. The automated guided vehicle may also travel while towing an object to be transported.

[0029] The robot RB2 is, for example, a working robot. The working robot is, for example, an articulated robot. The working robot may be, for example, a collaborative robot. A collaborative robot is a robot that performs work in collaboration with a person. The robot system SYS may include one or more robots RB2.

[0030] The industrial equipment EQ refers to machines or devices used for the purpose of manufacturing, processing, transporting, or controlling products in business facilities such as factories or warehouses. The industrial equipment EQ is, for example, but is not limited to, a conveyor or an elevator. The industrial equipment EQ may also include a robot.

[0031] The management system MG manages the industrial equipment EQ and the robot system SYS. The management system MG includes, for example, a manufacturing execution system (MES) and / or a warehouse management system (WMS). The management system MG may transmit instruction information to the industrial equipment EQ and multiple robots RB, and may receive start notification information and completion notification information from the industrial equipment EQ and each robot RB. In this case, the management system MG corresponds to an example of a "management device" in the present disclosure.

[0032] The user terminal UT is a terminal device operated by a user. The user terminal UT is, for example, a personal computer, a tablet terminal, or a smartphone. The user inputs information written in natural language indicating the actions of multiple robots RB (hereinafter referred to as "natural language information P1") via the user terminal UT. The natural language information P1 is transmitted to the drawing generation device 300. The drawing generation device 300 generates a prompt P2 based on the natural language information P1.

[0033] The learning device 100, the generating AI device 200, the drawing generating device 300, the robot management device 400, the prompt database device 500, the drawing database device 600, the constraint database device 700, the robot RB, the user terminal UT, the industrial equipment EQ, and the management system MG are connected to a network NW and can communicate with each other. The network NW includes, for example, the Internet, a closed network, a public telephone network, a LAN (Local Area Network), and a short-range wireless network.

[0034] Fig. 2 is a block diagram showing an example configuration of the drawing generation device 300. As shown in Fig. 2, the drawing generation device 300 includes a processing unit 310, a communication unit 340, and a storage unit 350. The drawing generation device 300 may also include an input unit 320 and an output unit 330.

[0035] The input unit 320 is an input device for inputting various types of information to the processing unit 310. For example, the input unit 320 is a keyboard and pointing device, or a touch panel.

[0036] The output unit 330 outputs various types of information. The output unit 330 includes, for example, a display unit that displays the various types of information. The display unit is, for example, a liquid crystal display or an organic electroluminescence display.

[0037] The communication unit 340 is connected to the network NW. The communication unit 340 communicates with various devices connected to the network NW. The communication unit 340 is a communication device that performs communication according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol conforming to Ethernet (registered trademark), the Internet Protocol Suite, and / or a protocol conforming to a short-range wireless communication standard.

[0038] The processing unit 310 executes various processes (various calculations). The processing unit 310 controls the input unit 320, the output unit 330, the communication unit 340, and the storage unit 350. The processing unit 310 includes one or more processors. The processor is a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a digital signal processor (DSP), or an application specific integrated circuit (ASIC). The processor may be operated by a computer program or by hardwired logic.

[0039] The storage unit 350 includes one or more storage devices and stores data and computer programs. The storage unit 350 includes, for example, a main storage device such as a semiconductor memory and an auxiliary storage device such as a semiconductor memory or a storage drive. The storage drive is, for example, a hard disk drive or a solid-state drive. The storage unit 350 may be, for example, a non-transitory computer-readable storage medium.

[0040] The processing unit 310 includes a receiving unit 311 and a generating unit 312. The processing unit 310 may also include an updating unit 313. Specifically, the processor of the processing unit 310 executes a computer program stored in the storage device of the storage unit 150, thereby functioning as the receiving unit 311, the generating unit 312, and the updating unit 313.

[0041] The hardware configurations of the learning device 100, the generating AI device 200, the robot management device 400, the prompt database device 500, the drawing database device 600, the constraint database device 700, the user terminal UT, and the processing unit, memory unit, communication unit, input unit, and output unit of the management system MG are the same as the hardware configurations of the processing unit 310, memory unit 350, communication unit 340, input unit 320, and output unit 330, respectively.

[0042] As shown in Fig. 2, the receiving unit 311 receives natural language information P1 input by a user via the user terminal UT. The natural language information P1 is information written in a natural language that indicates the order of operations of multiple robots RB. The data format of the natural language information P1 is, for example, text format. The storage unit 350 stores the natural language information P1.

[0043] The generation unit 312 inputs a prompt P2 including operation sequence information written in a natural language indicating the operation sequence of the multiple robots RB into the language model TM, and causes the language model TM to generate drawing information 601 indicating a sequence diagram SD showing the operation sequence of the multiple robots RB. In this case, the generation unit 312 typically generates the prompt P2 based on the natural language information P1 accepted by the acceptance unit 311, and inputs the generated prompt P2 into the language model TM by transmitting the generated prompt P2 to the generation AI device 200 via the communication unit 340. The data format of the prompt P2 is, for example, text format. The storage unit 350 stores the prompt P2.

[0044] For example, the generation unit 312 optimizes the natural language information P1 received by the reception unit 311 for processing by the language model TM. Optimization means processing the natural language information P1 so that the quality of the drawing information 601 generated by the language model TM is higher than when the natural language information P1 received by the reception unit 311 is directly input to the language model TM as a prompt P2. Optimization may include, for example, one or more of organizing or structuring the natural language information P1, or supplementing or expanding information for the natural language information P1.

[0045] For example, the generation unit 312 may input the natural language information P1 received by the reception unit 311 directly to the language model TM as the prompt P2.

[0046] The format of the drawing information 601 is not particularly limited, and may be a format of image data showing the sequence diagram SD itself, a format of code information for drawing the sequence diagram SD on the user terminal UT (e.g., code information based on Mermaid notation), or an editable format (e.g., vector image format, CAD format, PowerPoint (registered trademark) format, etc.). It is preferable that the generation unit 312 include the format of the drawing information 601 to be generated by the language model TM in the prompt P2.

[0047] The generation AI device 200 receives the prompt P2 sent from the drawing generation device 300 and inputs it into the language model TM. As a result, the language model TM generates drawing information 601 indicating the sequence diagram SD. The generation AI device 200 then sends the drawing information 601 to the drawing generation device 300. The drawing generation device 300 receives the drawing information 601 sent from the generation AI device 200. The memory unit 350 stores the drawing information 601.

[0048] Fig. 3 is a diagram showing an example of natural language information P1 received by the receiving unit 311 from the user terminal UT. As shown in Fig. 3, the natural language information P1 indicates the order of operation of the multiple robots RB1 and RB2 in accordance with instructions from the robot management device 400. The natural language information P1 preferably includes constraint information 701a indicating constraint conditions under which the multiple robots RB1 and RB2 operate.

[0049] Fig. 4 is a diagram showing an example of a prompt P2 generated by the generation unit 312 based on the natural language information P1 of Fig. 3. As shown in Fig. 4, the generation unit 312 generates the prompt P2 by optimizing the natural language information P1 for processing by the language model TM. The prompt P2 indicates the order in which the multiple robots RB1 and RB2 should move in accordance with instructions from the robot management device 400. The prompt P2 preferably includes constraint information 701b indicating constraints on the movements of the multiple robots RB1 and RB2.

[0050] In the example of Fig. 4, prompt P2 describes the existence of the robot management device 400 and robots RB1 and RB2, the details of their respective operations (robot RB1 moves between point X and point Y. robot RB2 performs task X1 at point X and task Y1 at point Y), and the chain of command (robots RB1 and RB2 move in response to instructions from the robot management device 400). It also clearly states the sequence requirements and constraints of "repeatedly performing task X1 and task Y1."

[0051] The generation unit 312 generates the prompt P2 in Fig. 4 by identifying the related robots RB1, RB2 and the robot management device 400, explaining the roles and actions to be performed by each robot RB1, RB2, clearly indicating the chain of command to the robots RB1, RB2, specifying the sequence to be repeatedly executed, and describing constraints, based on the natural language information P1 in Fig. 3. The generation unit 312 can automatically generate a sequence diagram SD that represents the order of actions of the multiple robots RB1, RB2 by inputting the prompt P2 into the language model TM.

[0052] 5 shows an example of a sequence diagram SD obtained from drawing information 601 generated by inputting prompt P2 in FIG. 4 into the language model TM. As shown in FIG. 5, the sequence diagram SD represents the order of operations of multiple robots RB in chronological order. The sequence diagram SD is created based on the notation of sequence diagrams in UML (Unified Modeling Language), for example. However, the notation is not particularly limited.

[0053] The sequence diagram SD describes the robot management device 400 and the robots RB1 and RB2. The sequence diagram SD describes that the robot RB1 is an unmanned transport robot that moves between point X and point Y, and that the robot RB2 is an articulated robot that performs task X1 at point X and task Y1 at point Y. The sequence diagram SD also describes that the robots RB1 and RB2 operate in response to instructions from the robot management device 400.

[0054] The sequence diagram SD has three vertical lines (dashed lines) that correspond to the robot management device 400, robot RB1, and robot RB2. The horizontal arrows represent the sending and receiving of messages (instructions, notifications) between the robot management device 400 and robots RB1 and RB2. The initial state (all robots RB1 and RB2 are waiting) is also annotated at the top of the sequence diagram SD. Furthermore, the sequence diagram SD shows a sequence with a loop structure that repeats a set of tasks, where task X1 and task Y1 are considered to be one set.

[0055] The sequence diagram SD is created to satisfy the constraints included in the prompt P2 in Fig. 4. That is, in the sequence diagram SD, when the robot RB1 is moving, the robot RB2 is not moving, and when the robot RB2 is moving, the robot RB1 is not moving. Furthermore, the robots RB1 and RB2 operate according to instructions from the robot management device 400.

[0056] As described above with reference to FIGS. 3 to 5, according to the first embodiment, by inputting a prompt P2 based on natural language information P1 provided by a user into the language model TM, it is possible to automatically generate drawing information 601 representing a sequence diagram SD. In this way, the user can obtain a sequence diagram SD by describing the operation sequence of the robots RB1 and RB2 and the robot management device 400 in natural language. In other words, it is possible to easily generate a sequence diagram SD representing the operation sequence of multiple robots RB1 and RB2. For example, even when the number and / or types of robots RB are large and the sequence is complex, it is possible to reduce the burden on the user associated with creating a sequence diagram SD. Furthermore, for example, it is possible to reduce the dependency of the quality of the sequence diagram SD on the user's experience and skill, thereby making it possible to generate sequence diagrams SD of consistent quality.

[0057] In particular, the sequence diagram SD represents the order of operations of multiple robot management devices 400 in response to instructions from the robot management device 400. Therefore, it is possible to visualize the workflow in an actual operational environment centered around the robot management device 400.

[0058] For example, the sequence diagram SD indicated by the drawing information 601 generated by the language model TM may include information on the time schedules of multiple robots RB in addition to the order of operations. In this case, the sequence diagram can be considered as a time chart. The time chart indicates, for example, the start time, end time, and required time of each robot RB's operation. This makes it possible to express time constraints in the time chart. Furthermore, since it is possible to check at a glance the time overlap when multiple robots RB operate simultaneously, it is also useful for visualizing resource contention. Furthermore, since it is possible to visually grasp waiting time and idle time, it becomes easier to evaluate efficiency.

[0059] Fig. 6 is a flowchart showing an example of a drawing generation method according to the first embodiment. The drawing generation method is executed by a drawing generation device 300. As shown in Fig. 6, the drawing generation method includes steps S1 to S7. Specifically, a computer program stored in a storage unit 350 causes a processing unit 310 to execute each step included in the drawing generation method. The processing unit 310 corresponds to an example of a "computer" in the present disclosure.

[0060] As shown in FIG. 6, first, in step S1, the accepting unit 311 accepts natural language information P1 input by the user via the user terminal UT.

[0061] Next, in step S2, the generator 312 generates a prompt P2 based on the natural language information P1.

[0062] Next, in step S3, the update unit 313 stores the prompt P2 in the prompt database device 500, thereby updating the prompt database device 500.

[0063] Next, in step S4, the generation unit 312 transmits the prompt P2 to the generation AI device 200 and inputs it into the language model TM, thereby causing the language model TM to generate drawing information 601 indicating the sequence diagram SD.

[0064] Next, in step S5, the generation unit 312 receives the drawing information 601 generated by the language model TM from the generation AI device 200, thereby acquiring the drawing information 601.

[0065] Next, in step S6, the update unit 313 stores the drawing information 601 in the drawing database device 600, thereby updating the drawing database device 600.

[0066] Next, in step S7, the update unit 313 transmits to the learning device 100 (FIG. 1) information indicating that the prompt database device 500 and the drawing database device 600 have been updated (hereinafter referred to as "update notification information"), and the drawing generation method then ends.

[0067] The learning device 100 (FIG. 1) updates the parameters of the language model TM by re-learning the language model TM using the newly stored prompt P2 and drawing information 601 as a learning data set. Re-learning performed every time update notification information is received is sometimes referred to as "sequential re-learning." According to the first embodiment, update notification information is automatically transmitted from the drawing generating device 300 to the learning device 100, allowing for automatic sequential re-learning.

[0068] The learning device 100 may, for example, periodically retrain the language model TM, or may retrain the language model TM when a predetermined number of update notification information items have been received. Such retraining may be referred to as "batch retraining." When batch retraining is performed, the prompt P2 added to the prompt database device 500 after the previous retraining and the drawing information 601 added to the drawing database device 600 after the previous retraining are used as the learning data set.

[0069] (Embodiment 2) A robot system SYS according to a second embodiment of the present disclosure will be described with reference to Figures 1 and 7. The second embodiment differs from the first embodiment mainly in that constraint information 701 is automatically applied to the sequence diagram SD. The following mainly describes the differences between the second embodiment and the first embodiment.

[0070] 7 is a block diagram showing an example of the configuration of a drawing generating device 300 according to embodiment 2. As shown in FIG.

[0071] The receiving unit 311 receives natural language information P1, which is information written in a natural language indicating the order of operations of the robots RB and is input by a user via the user terminal UT. For example, the generating unit 312 generates a prompt P2 based on the natural language information P1.

[0072] The information acquisition unit 314 acquires constraint information 701 indicating constraint conditions when the plurality of robots RB operate from the constraint database device 700. The constraint database device 700 corresponds to an example of the "storage device" of the present disclosure.

[0073] The generation unit 312 acquires the drawing information 601 generated by the language model TM based on the input of the prompt P2 and the constraint information 701 acquired by the information acquisition unit 314 from the constraint database device 700. In this way, in the second embodiment, the constraint information 701 is automatically acquired from the constraint database device 700 by the information acquisition unit 314 and input to the language model TM. Therefore, the user does not need to input constraint conditions as natural language information P1. As a result, the input burden on the user can be further reduced. Note that, because the constraint information 701 is input to the language model TM, the constraint information 701 may be considered as a type of prompt.

[0074] As an example, a process using RAG (Retrieval-Augmented Generation) is used to make the language model TM generate drawing information 601. This example will be explained below.

[0075] The information acquisition unit 314 searches for one or more pieces of constraint information 701 related to the natural language information P1 from among the pieces of constraint information 701 stored in the constraint database device 700. In this example, the information acquisition unit 314 functions as a search unit (search engine). The information acquisition unit 314 outputs the constraint information 701 obtained as the search result to the generation unit 312.

[0076] For example, the constraint database device 700 may store constraint vector information indicating a numerical vector obtained by vectorizing the constraint information 701 in association with the constraint information 701. Then, the information acquisition unit 314 generates natural language vector information indicating a numerical vector obtained by vectorizing the natural language information P1. Furthermore, the information acquisition unit 314 identifies the constraint information 701 related to the natural language information P1 by calculating the similarity (e.g., cosine similarity) between the constraint vector information and the natural language vector information. That is, the information acquisition unit 314 identifies the constraint information 701 related to the natural language information P1 based on the semantic content of the natural language information P1, and outputs the identified constraint information 701 to the generation unit 312 as a search result.

[0077] Then, the generation unit 312 generates a prompt P2 based on the natural language information P1 and the constraint information 701. For example, the generation unit 312 may include in the prompt P2 information obtained by optimizing the natural language information P1 or the natural language information P1 and the constraint information 701. Note that, for example, the generation unit 312 may analyze the constraint information 701 and generate the prompt P2 based on the analysis result and the natural language information P1.

[0078] Then, the generation unit 312 inputs the prompt P2 into the language model TM. As a result, the language model TM generates drawing information 601. The generation unit 312 acquires the drawing information 601 from the language model TM.

[0079] However, the language model TM may interpret the prompt P2 and generate request information for the information acquisition unit 314 based on the interpretation result. The request information is information requesting the transmission of additional information to be referenced for generating the drawing information 601. The generation AI device 200 transmits the request information to the information acquisition unit 314. Then, the information acquisition unit 314 searches the constraint database device 700 or another database device for additional information corresponding to the request information and transmits the additional information to the language model TM. The language model TM generates the drawing information 601 based on the prompt P2 by referring to the additional information. Then, the generation unit 312 acquires the drawing information 601 from the language model TM. Note that the language model TM may generate the drawing information 601 after multiple exchanges between the information acquisition unit 314 and the language model TM. Exchanges between the information acquisition unit 314 and the language model TM may be performed via the generation unit 312. The information acquisition unit 314 may also be provided as a computer (e.g., a server) separate from the drawing generation device 300.

[0080] The above describes an example in which RAG is used. Even when RAG is used, prompt P2 (constraint information and prompt) including constraint information 701 is input to the language model TM, and the language model TM generates drawing information 601 while referring to additional information as necessary.

[0081] As another example, a process using an MCP (Model Context Protocol) server causes the language model TM to generate drawing information 601. This example will be described below.

[0082] In this example, the information acquisition unit 314 functions as an MCP server. Therefore, the information acquisition unit 314 is provided as an MCP server separate from the drawing generation device 300.

[0083] The generation unit 312 generates a prompt P2 based on the natural language information P1. The generation unit 312 then inputs the prompt P2 to the language model TM. As a result, the language model TM interprets the prompt P2 and generates request information for the drawing generation device 300 based on the interpretation result. The request information is information that requests the transmission of constraint information 701 to be referenced for generating the drawing information 601. Note that the prompt P2 may include an instruction to generate the drawing information 601 by referencing the constraint information 701.

[0084] The generation AI device 200 transmits the request information to the information acquisition unit 314. Then, the information acquisition unit 314 searches the constraint database device 700 for constraint information 701 corresponding to the request information, and transmits the constraint information 701 to the language model TM. The language model TM references the constraint information 701 and generates drawing information 601 based on the prompt P2. Then, the generation unit 312 acquires the drawing information 601 from the language model TM.

[0085] As in the case of using RAG, the language model TM may generate request information requesting additional information. Then, the information acquisition unit 314 searches for additional information corresponding to the request information from the constraint database device 700 or another database device, and transmits the additional information to the language model TM. The language model TM generates drawing information 601 based on the prompt P2 by referring to the additional information. Then, the generation unit 312 acquires the drawing information 601 from the language model TM. Note that the language model TM may generate the drawing information 601 after multiple exchanges between the information acquisition unit 314 and the language model TM.

[0086] The above describes an example in which an MCP server is used. Even when an MCP server is used, the language model TM generates drawing information 601 by inputting constraint information 701 and prompt P2 to the language model TM, while referencing additional information as necessary.

[0087] 7, in the second embodiment, the constraint information 701 is automatically acquired from the constraint database device 700 and reflected in the generation of the drawing information 601 by the language model TM. Therefore, the user does not need to input constraint conditions as natural language information P1, which further reduces the input burden on the user.

[0088] Furthermore, according to the second embodiment, it is possible to prevent the user from making erroneous instructions, such as forgetting to input constraint conditions or inputting inappropriate constraint conditions. For example, if constraint information 701 indicating standard constraint conditions according to the type of robot RB and the work content is stored in advance in the constraint database device 700, the language model TM can automatically refer to the constraint information 701 without the user explicitly specifying the constraint conditions.

[0089] Furthermore, in the second embodiment, for example, the constraint conditions indicated by the constraint information 701 include a condition indicating a mutually exclusive relationship regarding the simultaneous execution of actions between two or more robots RB, and a condition indicating a constraint between each robot RB and the robot management device 400. In this case, for example, it is possible to automatically instruct the language model TM that while one robot RB is operating, the other robot should stop operating, and that each robot RB should operate according to instructions from the robot management device 400.

[0090] (Embodiment 3) A robot system SYS according to a third embodiment of the present disclosure will be described with reference to Figures 1 and 8. The third embodiment differs from the first embodiment mainly in that the language model TM learns the change history of the drawing information 601. The following mainly describes the differences between the third embodiment and the first embodiment.

[0091] 8 is a diagram illustrating the operation of the learning device 100 of the robot system SYS according to the third embodiment. As shown in Fig. 8, the learning device 100 uses a learning prompt P2 and drawing information 601 as a learning data set, and trains an untrained or pre-trained language model TM. In this case, the learning data set includes information indicating the change history of the learning drawing information 601 (hereinafter referred to as "change history information 602").

[0092] In the third embodiment, a drawing database device 600 performs version management of drawing information 601. The drawing database device 600 stores change history information 602 in addition to the drawing information 601 generated by the language model TM. Specifically, the drawing database device 600 records the changes as the change history information 602 every time the drawing information 601 is updated.

[0093] 8, the learning device 100 includes a processing unit 110, a communication unit 140, and a storage unit 150. The hardware configurations of the processing unit 110, the communication unit 140, and the storage unit 150 are similar to the hardware configurations of the processing unit 310, the communication unit 340, and the storage unit 350 of the drawing generation device 300 in FIG.

[0094] The processing unit 110 includes an acquisition unit 111 and a learning unit 112. Specifically, the processor of the processing unit 110 executes a computer program stored in the storage device of the storage unit 150, thereby functioning as the acquisition unit 111 and the learning unit 112.

[0095] The acquisition unit 111 acquires a training prompt P2 from the prompt database device 500, and acquires training drawing information 601 and change history information 602 from the drawing database device 600. The storage unit 150 stores the prompt P2, the drawing information 601, and the change history information 602 as a training data set.

[0096] The learning unit 112 accesses the generation AI device 200 and constructs an untrained or pre-trained language model TM by training the training data set. Thus, according to the third embodiment, by including the change history information 602 of the drawing information 601 in the training data set, the language model TM can learn dynamic and chronological change information of the drawing information 601. As a result, the language model TM can learn the judgment criteria and improvement trends in the process of changing the drawing information 601. Therefore, the accuracy of the obtained language model TM is further improved, and more reliable drawing information 601 can be generated.

[0097] Specifically, the learning unit 112 uses a learning dataset (prompt P2, drawing information 601, and change history information 602) to train the language model TMB with a learning algorithm. The learning algorithm is, for example, supervised learning. In this case, the prompt P2 is basically an explanatory variable (input), and the drawing information 601 is a target variable (output). The change history information 602 is also a learning target, and the language model TM learns not only the relationship between the prompt P2 and the drawing information 601, but also the change pattern of the drawing information 601 from the change history information 602. For example, the change history information 602 may be used in reinforcement learning when fine-tuning a pre-trained language model TM. Furthermore, the learning algorithm is not limited to supervised learning and reinforcement learning, and may be, for example, self-supervised learning.

[0098] Furthermore, in the third embodiment, the training data set may include, in addition to the training drawing information 601 and the change history information 602, information on the reason for the change indicated by the change history of the training drawing information 601 (hereinafter, "change reason information 603"). In this case, the drawing database device 600 stores the change reason information 603 in addition to the drawing information 601 and the change history information 602.

[0099] The change reason information 603 is, for example, information input by the person who modifies the drawing information 601. The change reason information includes, for example, one or more of the reason for changing the drawing information 601, the purpose of the change, the problem solved by the change, and an explanation of the background of the change. The drawing database device 600 has a version control system, and the change history information 602 and the change reason information 603 are managed by the version control system.

[0100] The acquisition unit 111 acquires a training prompt P2 from the prompt database device 500 and acquires training drawing information 601, change history information 602, and change reason information 603 from the drawing database device 600. The training unit 112 then trains the language model TM with a training data set including the training prompt P2, drawing information 601, change history information 602, and change reason information 603. As described above, according to the third embodiment, by training the change reason information 603, the resulting language model TM can understand the intention and judgment criteria behind the change and generate drawing information 601 that is more appropriate and explainable depending on the situation. For example, the language model TM generated by training the change reason information 603 can infer the intention of the change. Therefore, flexible decisions can be made even in new situations, and highly reliable drawing information 601 can be generated even in new situations.

[0101] For example, the change reason information 603 may include information such as "the safety distance between the robots was insufficient, so it was corrected," "the operation sequence was inconsistent, so it was corrected," or "it was added to accommodate specific work conditions." By learning the change reason information 603, when a similar situation is included in a new prompt P2, the language model TM can generate more appropriate drawing information 601 from scratch by referring to the past correction history.

[0102] Furthermore, by learning the change reason information 603, the language model TM can also generate explanatory information (hereinafter referred to as "explanatory information EX") that explains the generated drawing information 601. The explanatory information EX indicates the basis on which the language model TM came to generate the drawing information 601. The explanatory information EX is displayed, for example, in text format. For example, the explanatory information EX includes an explanation of "why the drawing information 601 was generated in this way." In this way, a language model TM that is explainable AI (XAI) can be provided.

[0103] Specifically, the learning unit 112 uses a learning dataset (prompt P2, drawing information 601, change history information 602, and change reason information 603) to train the language model TMB with a learning algorithm. The learning algorithm is, for example, supervised learning. In this case, the prompt P2 is basically an explanatory variable (input), and the drawing information 601 is a target variable (output). The change history information 602 and the change reason information 603 are also used as learning targets, and the language model TMB learns not only the relationship between the prompt P2 and the drawing information 601, but also the change patterns and reasons for the changes in the drawing information 601 from the change history information 602 and the change reason information 603. For example, the change history information 602 and the change reason information 603 may be used in reinforcement learning when fine-tuning a pre-trained language model TM. Alternatively, the learning algorithm is not limited to supervised learning and reinforcement learning, and may be, for example, self-supervised learning.

[0104] As described above with reference to Fig. 8, according to the third embodiment, by including the change history information 602 in the training data set, the language model TM can learn the modification patterns and improvement trends of the drawing information 601. Furthermore, by including the change reason information 603, the language model TM can also understand the intentions and judgment criteria behind the changes. This allows the language model TM to generate more reliable drawing information 601 in response to the input prompt P2, taking into account the past modification history and the reasons for the changes.

[0105] Furthermore, the knowledge and experience of the person who modified the drawing information 601 (e.g., a skilled engineer) is reflected in the language model TM through the change history information 602 and the change reason information 603, making it possible to effectively utilize that expert knowledge. This improves the quality of the drawing information 601 generated by the language model TM.

[0106] (Embodiment 4) A robot system SYS according to a fourth embodiment of the present disclosure will be described with reference to Figures 1 and 9. The fourth embodiment differs from the first embodiment mainly in that the language model TM refers to the change history of the drawing information 601. The following mainly describes the differences between the fourth embodiment and the first embodiment.

[0107] Fig. 9 is a block diagram showing an example of the configuration of a drawing generation device 300 according to embodiment 4. As shown in Fig. 9, the configuration of the drawing generation device 300 of embodiment 4 is similar to the configuration of the drawing generation device 300 of embodiment 2 shown in Fig. 7.

[0108] 9, the receiving unit 311 receives natural language information P1, which is information written in a natural language indicating the order of operations of the robots RB and is input by a user via the user terminal UT. For example, the generating unit 312 generates a prompt P2 based on the natural language information P1.

[0109] The information acquisition unit 314 acquires the change history information 602 from the drawing database device 600. The change history information 602 includes information indicating the change history of the drawing information 601. The drawing database device 600 corresponds to an example of the "storage device" of the present disclosure.

[0110] The generation unit 312 acquires drawing information 601 generated by the language model TM based on the input of the prompt P2 and the change history information 602 acquired by the information acquisition unit 314 from the drawing database device 600. In this way, in the fourth embodiment, the change history information 602 is automatically acquired from the drawing database device 600 by the information acquisition unit 314 and input to the language model TM. This allows the language model TM to generate higher quality drawing information 601 by utilizing knowledge obtained from the past change history information 602. In addition, the fourth embodiment can achieve the same effects as the third embodiment. Note that, because the change history information 602 is input to the language model TM, the change history information 602 may be considered as a type of prompt.

[0111] As another example, the information acquiring unit 314 may acquire the change history information 602 and the change reason information 603 from the drawing database device 600. The change reason information 603 includes information related to the reason for the change indicated by the change history of the drawing information 601.

[0112] The generation unit 312 acquires the drawing information 601 generated by the language model TM based on the prompt P2 and the change history information 602 and the change reason information 603 acquired by the information acquisition unit 314 from the drawing database device 600. In this way, in the fourth embodiment, the change history information 602 and the change reason information 603 are automatically acquired from the drawing database device 600 by the information acquisition unit 314 and input to the language model TM. This allows the language model TM to utilize knowledge obtained from the past change history information 602. In addition, by utilizing the change reason information 603, the language model TM can understand not only the change pattern but also the intention and judgment criteria behind the changes. As a result, the language model TM can generate higher quality drawing information 601. In addition, the fourth embodiment achieves the same effects as the third embodiment. Note that, because the change history information 602 and the change reason information 603 are input to the language model TM, the change history information 602 and the change reason information 603 may be considered as a type of prompt.

[0113] As an example, a language model TM generates drawing information 601 through a process using RAG (Retrieval-Augmented Generation). This example will be described below. While the following describes an example in which change history information 602 and change reason information 603 are used, the same applies when only the change history information 602 is used, and the following description should be interpreted to refer to only the change history information 602.

[0114] The information acquisition unit 314 searches the drawing database device 600 for change history information 602 and change reason information 603 related to the natural language information P1. In this example, the information acquisition unit 314 functions as a search unit (search engine). The information acquisition unit 314 outputs the change history information 602 and change reason information 603 obtained as search results to the generation unit 312.

[0115] For example, the drawing database device 600 may store change history vector information indicating a numerical vector obtained by vectorizing the change history information 602 in association with the change history information 602. Then, the information acquisition unit 314 generates natural language vector information indicating a numerical vector obtained by vectorizing the natural language information P1. Furthermore, the information acquisition unit 314 identifies the change history information 602 related to the natural language information P1 by calculating the similarity (e.g., cosine similarity) between the change history vector information and the natural language vector information. That is, the information acquisition unit 314 identifies the change history information 602 related to the natural language information P1 based on the semantic content of the natural language information P1. Then, the information acquisition unit 314 identifies change reason information 603 corresponding to the identified change history information 602. Furthermore, the information acquisition unit 314 outputs the identified change history information 602 and change reason information 603 to the generation unit 312 as search results.

[0116] The generation unit 312 then generates a prompt P2 based on the natural language information P1, the change history information 602, and the reason for change information 603. For example, the generation unit 312 includes in the prompt P2 information obtained by optimizing the natural language information P1 or the natural language information P1, and the change history information 602 and the reason for change information 603. Note that the generation unit 312 may, for example, analyze the change history information 602 and the reason for change information 603 and generate the prompt P2 based on the analysis results and the natural language information P1.

[0117] Then, the generation unit 312 inputs the prompt P2 into the language model TM. As a result, the language model TM generates drawing information 601. The generation unit 312 acquires the drawing information 601 from the language model TM.

[0118] However, the language model TM may interpret the prompt P2 and generate request information for the information acquisition unit 314 based on the interpretation result. The request information is information requesting the transmission of additional information to be referenced for generating the drawing information 601. The generation AI device 200 transmits the request information to the information acquisition unit 314. Then, the information acquisition unit 314 searches for additional information corresponding to the request information from the drawing database device 600 or another database device and transmits the additional information to the language model TM. The language model TM generates the drawing information 601 based on the prompt P2 by referring to the additional information. Then, the generation unit 312 acquires the drawing information 601 from the language model TM. Note that the language model TM may generate the drawing information 601 after multiple exchanges between the information acquisition unit 314 and the language model TM. Exchanges between the information acquisition unit 314 and the language model TM may be performed via the generation unit 312. The information acquisition unit 314 may also be provided as a computer (e.g., a server) separate from the drawing generation device 300.

[0119] The above has been an example of using RAG. Even when RAG is used, prompt P2 (change history information, change reason information, and prompt) including change history information 602 and change reason information 603 is input to the language model TM, and the language model TM generates drawing information 601 while referring to additional information as necessary.

[0120] As another example, drawing information 601 is generated in a language model TM through a process using an MCP (Model Context Protocol) server. This example will be described below. While the following describes an example in which change history information 602 and change reason information 603 are used, the same applies when only change history information 602 is used, and the following description should be interpreted to refer to only change history information 602.

[0121] In this example, the information acquisition unit 314 functions as an MCP server. Therefore, the information acquisition unit 314 is provided as an MCP server separate from the drawing generation device 300.

[0122] The generation unit 312 generates a prompt P2 based on the natural language information P1. Then, the generation unit 312 inputs the prompt P2 to the language model TM. As a result, the language model TM interprets the prompt P2 and generates request information for the drawing generation device 300 based on the interpretation result. The request information is information that requests transmission of change history information 602 and change reason information 603 to be referenced for generating drawing information 601. Note that the prompt P2 may include an instruction to generate drawing information 601 by referencing the change history information 602 and change reason information 603.

[0123] The generation AI device 200 transmits the request information to the information acquisition unit 314. Then, the information acquisition unit 314 searches the drawing database device 600 for change history information 602 and change reason information 603 corresponding to the request information, and transmits the change history information 602 and the change reason information 603 to the language model TM. The language model TM references the change history information 602 and the change reason information 603 and generates drawing information 601 based on the prompt P2. Then, the generation unit 312 acquires the drawing information 601 from the language model TM.

[0124] As in the case of using RAG, the language model TM may generate request information requesting additional information. Then, the information acquisition unit 314 searches for additional information corresponding to the request information from the drawing database device 600 or another database device, and transmits the additional information to the language model TM. The language model TM generates drawing information 601 based on the prompt P2 by referring to the additional information. Then, the generation unit 312 acquires the drawing information 601 from the language model TM. Note that the language model TM may generate the drawing information 601 after multiple exchanges between the information acquisition unit 314 and the language model TM.

[0125] The above describes an example in which an MCP server is used. Even when an MCP server is used, the language model TM generates drawing information 601 by inputting change history information 602, change reason information 603, and prompt P2 into the language model TM, while referring to additional information as necessary.

[0126] As described above with reference to Fig. 9, according to the fourth embodiment, by inputting the change history information 602 into the language model TM, the language model TM can understand the modification patterns and improvement trends of the drawing information 601. Furthermore, by inputting the change reason information 603 into the language model TM, the language model TM can also understand the intentions and judgment criteria behind the changes. This allows the language model TM to generate more reliable drawing information 601 in response to the input prompt P2, taking into account the past modification history and the reasons for the changes.

[0127] Furthermore, the knowledge and experience of the person who modified the drawing information 601 (e.g., a skilled engineer) is reflected in the language model TM through the change history information 602 and the change reason information 603, making it possible to effectively utilize that expert knowledge. This improves the quality of the drawing information 601 generated by the language model TM.

[0128] (Embodiment 5) A robot system SYS according to a fifth embodiment of the present disclosure will be described with reference to Figures 1 and 2. The fifth embodiment differs from the first embodiment mainly in that a prompt P2 is generated based on a specific condition. The following mainly describes the differences between the fifth embodiment and the first embodiment.

[0129] As shown in Fig. 2, the receiving unit 311 receives natural language information P1, which is information written in a natural language indicating the order of operations of multiple robots RB and is input by a user via a user terminal UT. The generating unit 312 generates a prompt P2 based on the natural language information P1 and a specific condition. The specific condition may be, for example, a condition stored in advance in the storage unit 350, or a condition specified by the user via the user terminal UT. The specific condition may also be included in the natural language information P1.

[0130] The specific condition includes, for example, an optimization condition and / or an error recovery condition regarding the operation of the multiple robots RB. The optimization condition includes, for example, at least one of generating drawing information 601 so as to minimize the total movement distance of the multiple robots RB, generating drawing information 601 so as to minimize the total working time of the multiple robots RB, generating drawing information 601 so as to minimize the total power consumption of the multiple robots RB, generating drawing information 601 so as to maximize the operation rate of the multiple robots RB, and generating drawing information 601 so as to minimize the total standby time of the multiple robots RB. The error recovery condition includes, for example, generating drawing information 601 so as to include an operation sequence indicating a recovery scenario for the multiple robots RB in the event of an error.

[0131] The generation unit 312 generates the prompt P2 based on the specific condition, and the language model TM generates drawing information 601 that satisfies the specific condition based on the prompt P2. For example, the generation unit 312 includes information obtained by optimizing the natural language information P1 or the natural language information P1 and the specific condition in the prompt P2.

[0132] As described above, according to the fifth embodiment, the generation unit 312 generates the prompt P2 based on the natural language information P1 and the specific conditions, which allows the user to easily obtain the drawing information 601 that satisfies the specific conditions, such as the optimization conditions and the error recovery conditions, from the language model TM.

[0133] Furthermore, according to the fifth embodiment, the user can have the language model TM generate the diagram information 601 showing the sequence diagram SD that satisfies specific conditions such as optimization conditions and error recovery conditions, simply by inputting the basic operation sequence as the natural language information P1. This reduces the burden on the user.

[0134] For example, manually creating a sequence diagram SD that satisfies the optimization conditions requires complex calculations and trial and error. In contrast, in the fifth embodiment, a sequence diagram SD that satisfies the optimization conditions can be easily obtained by inputting a prompt P2 that includes the optimization conditions into the language model TM.

[0135] For example, a sequence diagram SD that includes a recovery scenario in the event of an error is important for improving the robustness and reliability of a system. According to the fifth embodiment, by inputting a prompt P2 that includes an error recovery condition into the language model TM, it is possible to easily generate a comprehensive sequence diagram SD that includes not only the main workflow but also recovery operations for various error situations. This makes it possible to consider countermeasures for potential problems in advance at the system design stage, thereby improving safety and efficiency in actual operation.

[0136] Furthermore, multiple specific conditions can be combined. For example, by simultaneously specifying "minimizing total work time" (optimization condition) and "including an error recovery scenario" (error recovery condition), an efficient yet robust sequence diagram SD can be generated. While such complex optimization is extremely difficult to achieve by manual design, the fifth embodiment uses the language model TM to easily generate diagram information 601 that satisfies both the optimization condition and the error recovery condition.

[0137] (Embodiment 6) A robot system SYS according to a sixth embodiment of the present disclosure will be described with reference to Figures 1 and 2. The sixth embodiment differs from the first embodiment mainly in that it detects an inconsistency in the operation order of multiple robots RB2 and generates drawing information 601 that resolves the inconsistency. The following mainly describes the differences between the sixth embodiment and the first embodiment.

[0138] As shown in FIG. 2 , the receiving unit 311 receives natural language information P1, which is information written in natural language indicating the operation sequence of multiple robots RB and is input by a user. The generating unit 312 generates a prompt P2 based on the natural language information P1. In this case, the generating unit 312 includes information instructing the language model TM to detect an inconsistency in the operation sequence of the multiple robots RB and generate drawing information 601 that resolves the inconsistency in the prompt P2. As a result, in the sixth embodiment, the language model TM can automatically detect potential problems contained in the user's natural language information P1 and generate drawing information 601 that resolves the problems. In other words, the user can obtain high-quality drawing information 601 in which the inconsistency in the operation sequence is resolved simply by concisely describing the complex operation sequence of multiple robots RB in natural language. This allows potential problems to be identified and resolved at an early stage of system design.

[0139] "Inconsistencies in the operation sequences of multiple robots RB" refer to one or more of the following: inconsistencies (logical inconsistencies, temporal inconsistencies) regarding the operation sequences of multiple robots RB, missing operations, resource conflicts, safety issues, and efficiency issues. A logical inconsistency occurs when operations that cannot be performed simultaneously are simultaneously instructed, or when the preconditions for the operations are not met. A temporal inconsistency occurs when the operation sequence is physically impossible (for example, performing an inspection before installing a part). A resource conflict occurs when multiple robots RB try to use the same resource (for example, workspace or parts) at the same time. Safety issues include the risk of collisions between robots RB and the issue of safe distances when working collaboratively with humans. Missing operations occur when a process necessary for completing a task is omitted. Efficiency issues occur when unnecessary waiting time or redundant operations are included.

[0140] The language model TM analyzes the content of the natural language information P1 based on the prompt P2, detects an inconsistency in the order of actions, and generates drawing information 601 in which the inconsistency has been resolved. In this case, the language model TM may include explanatory information EZ regarding the detected inconsistency and / or a solution in the drawing information 601. The explanatory information EZ allows the user to understand the inconsistency contained in the natural language information P1 and use this information to improve the system design. The user can also evaluate the appropriateness of the corrections made by the language model TM.

[0141] Furthermore, for example, the drawing information 601 and the description information EZ stored in the drawing database device 600 by the update unit 313 may be used as a new training data set for retraining the language model TM, thereby enabling the language model TM to continuously improve its ability to detect and resolve inconsistencies.

[0142] For example, the sixth embodiment is more effective when combined with the third embodiment. In the third embodiment, by including the change history information 602 and / or the change reason information 603 in the training dataset, the language model TM can understand the intention and judgment criteria behind the change. By utilizing such learned knowledge, the language model TM can effectively detect potential inconsistencies contained in the natural language information P1 and generate drawing information 601 that resolves the inconsistencies based on past modification patterns. Furthermore, by combining the sixth embodiment with the third embodiment, the content of the explanatory information EZ can be made more accurate and enriched.

[0143] For example, the sixth embodiment is more effective when combined with the fourth embodiment. In the fourth embodiment, the information acquisition unit 314 acquires change history information 602 and / or change reason information 603 related to the natural language information P1, and the change history information 602 and / or change reason information 603 are input to the language model TM. Therefore, the language model TM generates the drawing information 601 using information obtained from the past revision history and the revision reasons. By combining the mechanism of the fourth embodiment with the inconsistency detection and inconsistency resolution functions of the sixth embodiment, more accurate inconsistency detection and resolution becomes possible. Furthermore, by combining the sixth embodiment with the fourth embodiment, the content of the explanation information EZ can be made more accurate and enriched.

[0144] (Embodiment 7) A robot system SYS according to a seventh embodiment of the present disclosure will be described with reference to Figures 1 and 2. The seventh embodiment differs from the first embodiment mainly in that the language model TM modifies previously generated drawing information 601. The following mainly describes the differences between the seventh embodiment and the first embodiment.

[0145] 2 receives correction information written in a natural language indicating corrections to a sequence diagram SD that was previously generated. The corrections include, for example, changes, additions, and / or deletions. The correction information includes information for identifying the drawing information 601 indicating the sequence diagram SD to be corrected.

[0146] The generation unit 312 then corrects the prompt P2 that was input to the language model TM when generating a previously generated sequence diagram SD (the sequence diagram SD to be corrected) based on the correction information, inputs the corrected prompt P2 (hereinafter referred to as "prompt P20") to the language model TM, and causes the language model TM to correct the previously generated drawing information 601. The language model TM corrects the previously generated drawing information 601 based on the prompt P20, and outputs the corrected drawing information 601 (hereinafter referred to as "drawing information 6010"). The generation unit 312 acquires the drawing information 6010 from the language model TM.

[0147] According to the seventh embodiment, the user can efficiently modify complex drawing information 601 simply by specifying the modification content in natural language. In particular, modifying a sequence diagram SD usually requires dedicated tools and specialized knowledge, but the seventh embodiment enables modification with intuitive instructions in natural language. This effectively reduces the burden on the user involved in modifying the drawing information 601.

[0148] Specifically, the generation unit 312 obtains the previously generated drawing information 601 to be corrected from the drawing database device 600, and also obtains the prompt P2 that was input to the language model TM when generating the drawing information 601 from the prompt database device 500.

[0149] The drawing database device 600 stores drawing information 601 that was previously generated. The prompt database device 500 stores a prompt P2 that was used when generating the drawing information 601 that was previously generated. The drawing database device 600 and the prompt database device 500 associate the drawing information 601 with the prompt P2. Therefore, the generation unit 312 can identify and acquire the prompt P2 that corresponds to the drawing information 601 to be corrected.

[0150] The generation unit 312 modifies the acquired prompt P2 based on the modification information to generate a modified prompt P20. The modifications made by the generation unit 312 are one or more of direct modification, condition addition, and additional instruction. Direct modification indicates that the changes specified by the modification information are directly reflected in the prompt P2. Condition addition indicates that new conditions and / or constraints are added to the prompt P2 based on the modification information. An additional instruction indicates that the content of the original prompt P2 is left as is and the modification information is added as is.

[0151] The generation unit 312 inputs the corrected prompt P20 into the language model TM, causing the language model TM to generate corrected drawing information 6010. In this case, the generation unit 312 may include the drawing information 601 to be corrected in the prompt P20. This allows the language model TM to efficiently make corrections while referring to the drawing information 601 to be corrected.

[0152] The updating unit 313 may store the modified prompt P20 in the prompt database device 500, and may store the modified drawing information 6010 in the drawing database device 600. In this case, the updating unit 313 may overwrite the original prompt P2 and drawing information 601 with the prompt P20 and drawing information 6010, or may add the prompt P20 and drawing information 6010 as new versions. In the latter case, the updating unit 313 also records the associations between the original prompt P2 and the modified prompt P20, and between the original drawing information 601 and the modified drawing information 6010.

[0153] Furthermore, the update unit 313 may record, as change history information, the correction information, the corrected prompt P20, and the corrected drawing information 6010. As described in the third and fourth embodiments, the change history information can be used as a learning dataset for the language model TM.

[0154] The update unit 313 transmits update notification information to the learning device 100. Upon receiving the update notification information, the learning device 100 can re-learn the language model TM using the newly stored corrected prompt P20, corrected drawing information 6010, and correction information as a training data set. This allows the language model TM to learn prompt correction patterns and enable it to make more appropriate corrections when a similar correction request is received in the future.

[0155] As described above, according to the seventh embodiment, the user can easily modify complex drawing information 601 simply by specifying the modification content in natural language. Furthermore, by recording the modification information, the prompt P20 after the modification, and the modified drawing information 6010, the change history of the drawing information 601 can be managed. This makes it easy to track the evolution of the design and to later refer to the reason for a particular modification.

[0156] For example, the seventh embodiment is particularly effective when combined with the third embodiment. In the third embodiment, the change history information 602 and the change reason information 603 are included in the training data set, so that the language model TM can understand the intention and the criteria behind the change. Therefore, in the seventh embodiment, the language model TM can generate drawing information 6010 that has been more appropriately corrected by utilizing the intention and the criteria behind the change.

[0157] For example, the seventh embodiment is particularly effective when combined with the fourth embodiment. In the fourth embodiment, the language model TM generates the drawing information 601 by referring to the change history information 602 and / or the change reason information 603. Therefore, in the seventh embodiment, the language model TM can generate the drawing information 6010 that has been corrected more appropriately by utilizing the change history information 602 and / or the change reason information 603.

[0158] For example, the seventh embodiment is particularly effective when combined with the sixth embodiment. In this case, inconsistencies are automatically detected and resolved during the correction process. For example, if a user's correction instruction may cause a new inconsistency, the language model TM can detect the inconsistency and propose a solution. This further improves the quality and efficiency of the correction work.

[0159] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0160] For example, two or more of the first to seventh embodiments may be combined.

[0161] The devices or configurations described in this specification may be realized as a single device, or may be realized by multiple devices (e.g., cloud servers) some or all of which are connected via a network.

[0162] The series of processes performed by the devices described herein may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the processing units 110 and 310 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium storing such a computer program may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a network, without using a recording medium.

[0163] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0164] The following configurations also fall within the technical scope of the present disclosure.

[0165] (Item 1) a generation unit that inputs a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of control target robots into a language model, and causes the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of control target robots; a storage unit that stores the drawing information.

[0166] (Item 2) the generation unit acquires the drawing information generated by the language model in response to input of the prompt and constraint information acquired from a storage device by an information acquisition unit; Item 2. The drawing generation device according to item 1, wherein the constraint information indicates constraint conditions under which the plurality of robots to be controlled operate.

[0167] (Item 3) The diagram shows an operation sequence of the plurality of control target robots in accordance with an instruction from a management device, The drawing generation device described in item 2, wherein the constraint conditions include conditions indicating a mutually exclusive relationship regarding the simultaneous execution of operations between two or more of the control target robots, and conditions indicating constraints between each of the control target robots and a management device.

[0168] (Item 4) a receiving unit that receives natural language information that indicates a sequence of operations of the plurality of control target robots and is input by a user via a user terminal, the generating unit generates the prompt based on the natural language information and a specific condition; 3. The drawing generation device according to item 1 or 2, wherein the specific condition includes at least one of generating the drawing information so as to minimize a total movement distance of the plurality of control target robots, generating the drawing information so as to minimize a total working time of the plurality of control target robots, generating the drawing information so as to minimize a total power consumption of the plurality of control target robots, generating the drawing information so as to maximize an operating rate of the plurality of control target robots, generating the drawing information so as to minimize a total standby time of the plurality of control target robots, and generating the drawing information so as to include an operation sequence indicating a recovery scenario for the plurality of control target robots in the event of an error.

[0169] (Item 5) a receiving unit that receives natural language information that indicates a sequence of operations of the plurality of control target robots and is input by a user, the natural language information being written in a natural language; The generation unit generating the prompt based on the natural language information; 3. The drawing generation device according to item 1 or 2, wherein the prompt includes information that instructs the language model to detect an inconsistency in the order of movements of the plurality of robots to be controlled and to generate drawing information that resolves the inconsistency.

[0170] (Item 6) a receiving unit that receives correction information written in a natural language indicating correction details for the previously generated drawing, The generation unit correcting the prompt input to the language model when generating the previously generated diagram based on the correction information; 3. The drawing generating device according to item 1 or 2, wherein the modified prompt is input to the language model, and the drawing information is modified by the language model.

[0171] (Item 7) the language model is constructed by training a training dataset; 3. The drawing generating device according to item 1 or 2, wherein the learning data set includes a change history of drawing information.

[0172] (Item 8) 8. The drawing generating device according to item 7, wherein the learning dataset includes information regarding the reasons for changes indicated by the change history of the drawing information.

[0173] (Item 9) the generation unit acquires the drawing information generated by the language model by inputting the prompt and change history information acquired from a storage device by an information acquisition unit; 3. The drawing generating device according to item 1 or 2, wherein the change history information includes information indicating a change history of drawing information.

[0174] (Item 10) the generation unit acquires the drawing information generated by the language model in response to input of the prompt, and change history information and change reason information acquired by an information acquisition unit from a storage device; the change history information includes information indicating a change history of the drawing information, 3. The drawing generating device according to item 1 or 2, wherein the change reason information includes information about the reason for the change indicated by the change history.

[0175] (Item 11) 3. The drawing generating device according to item 1 or 2, wherein the drawing represents an operation sequence of the plurality of control target robots in accordance with instructions from a management device.

[0176] (Item 12) inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; and acquiring the drawing information.

[0177] (Item 13) On the computer, inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; and acquiring the drawing information. [Industrial Applicability]

[0178] The present disclosure provides a drawing generation device, a drawing generation method, and a computer program, and has industrial applicability. [Explanation of symbols]

[0179] 100 Learning device, 300 Drawing generation device, 311 Reception unit, 312 Generation unit, 313 Update unit, 314 Information acquisition unit, 400 Robot management device, 500 Prompt database device, 600 Drawing database device (storage device), 700 Constraint database device (storage device), TM Learning model, RB, RB1, RB2 Robot, EQ Industrial equipment, MG Management system, NW Network

Claims

1. a generation unit that inputs a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of control target robots into a language model, and causes the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of control target robots; a storage unit that stores the drawing information, the language model is constructed by training a training dataset; A drawing generation device, wherein the learning dataset includes a change history of drawing information.

2. A receiving unit is further provided for receiving natural language information which is information written in a natural language indicating the order of operations of the plurality of control target robots and which is information input by a user via a user terminal; the prompt is generated based on the natural language information; the generation unit acquires the drawing information generated by the language model in response to input of the prompt and constraint information acquired from a storage device by an information acquisition unit; the constraint information indicates constraint conditions under which the plurality of control target robots operate; the storage device stores a plurality of pieces of constraint information; The drawing generation device according to claim 1 , wherein the information acquisition unit searches for and acquires, from the storage device, the constraint information related to the natural language information or the constraint information based on an interpretation result of the operation sequence information.

3. The diagram shows an operation sequence of the plurality of control target robots in accordance with an instruction from a management device, The drawing generation device according to claim 2, wherein the constraint conditions include a condition indicating a mutually exclusive relationship regarding the simultaneous execution of operations between two or more of the controlled robots, and a condition indicating a constraint between each of the controlled robots and a management device.

4. a receiving unit that receives natural language information that indicates a sequence of operations of the plurality of control target robots and is input by a user via a user terminal, the generating unit generates the prompt based on the natural language information and a specific condition; 2. The drawing generation device according to claim 1, wherein the specific conditions include at least one of generating the drawing information so that the total movement distance of the multiple control target robots is shortest, generating the drawing information so that the total working time of the multiple control target robots is shortest, generating the drawing information so that the total power consumption of the multiple control target robots is minimized, generating the drawing information so that the operating rate of the multiple control target robots is maximized, generating the drawing information so that the total waiting time of the multiple control target robots is minimized, and generating the drawing information so that it includes an operation sequence that indicates a recovery scenario for the multiple control target robots in the event of an error.

5. a receiving unit that receives natural language information that indicates a sequence of operations of the plurality of control target robots and is input by a user, the natural language information being written in a natural language; The generation unit generating the prompt based on the natural language information; The drawing generating device according to claim 1 , wherein the prompt includes information instructing the language model to detect an inconsistency in the order of movements of the plurality of robots to be controlled and to generate drawing information that resolves the inconsistency.

6. a receiving unit that receives correction information written in a natural language indicating correction details for the previously generated drawing, The generation unit correcting the prompt input to the language model when generating the previously generated diagram based on the correction information; The drawing generating device according to claim 1 , wherein the modified prompt is input to the language model, and the drawing information is modified by the language model.

7. The drawing generating device according to claim 1 or 2, wherein the learning data set includes information on reasons for changes indicated by a change history of the drawing information.

8. 3. The drawing generating device according to claim 1, wherein the drawing represents an operation sequence of the plurality of robots to be controlled in accordance with an instruction from a management device.

9. A generation unit that inputs a prompt including operation sequence information written in a natural language indicating the operation sequence of a plurality of control target robots into a language model, and causes the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of control target robots; a storage unit for storing the drawing information; a receiving unit that receives natural language information that indicates a sequence of operations of the plurality of control target robots and is written in a natural language and is input by a user via a user terminal; the prompt is generated based on the natural language information; the generation unit acquires the drawing information generated by the language model by inputting the prompt and change history information acquired from a storage device by an information acquisition unit; the change history information includes information indicating a change history of the drawing information, the storage device stores a plurality of pieces of change history information; The information acquisition unit searches for and acquires, from the storage device, the change history information related to the natural language information or the change history information based on the interpretation result of the operation sequence information.

10. A generation unit that inputs a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of control target robots into a language model, and causes the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of control target robots; a storage unit for storing the drawing information; a receiving unit that receives natural language information that indicates a sequence of operations of the plurality of control target robots and is written in a natural language and is input by a user via a user terminal; the prompt is generated based on the natural language information; the generation unit acquires the drawing information generated by the language model in response to input of the prompt, and change history information and change reason information acquired by an information acquisition unit from a storage device; the change history information includes information indicating a change history of the drawing information, the change reason information includes information regarding the reason for the change indicated by the change history; the storage device stores a plurality of pieces of change history information and a plurality of pieces of change reason information; The information acquisition unit searches for and acquires from the storage device the change history information and the reason for change information related to the natural language information, or the change history information and the reason for change information based on the interpretation result of the operation sequence information.

11. inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; acquiring the drawing information; the language model is constructed by training a training dataset; A drawing generation method in which the learning dataset includes a change history of drawing information.

12. A step of inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; acquiring the drawing information; natural language information that is information written in a natural language and indicates a sequence of operations of the plurality of control target robots is received from a user terminal; the prompt is generated based on the natural language information; In the step of acquiring, the drawing information generated by the language model is acquired by inputting the prompt and change history information acquired by an information acquisition unit from a storage device; the change history information includes information indicating a change history of the drawing information, the storage device stores a plurality of pieces of change history information; A drawing generation method in which the information acquisition unit searches for and acquires, from the storage device, the change history information related to the natural language information or the change history information based on the interpretation result of the operation sequence information.

13. A step of inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; acquiring the drawing information; natural language information that is information written in a natural language and indicates a sequence of operations of the plurality of control target robots is received from a user terminal; the prompt is generated based on the natural language information; In the acquiring step, the drawing information generated by the language model is acquired by inputting the prompt, and change history information and change reason information acquired by an information acquisition unit from a storage device; the change history information includes information indicating a change history of the drawing information, the change reason information includes information regarding the reason for the change indicated by the change history; the storage device stores a plurality of pieces of change history information and a plurality of pieces of change reason information; A drawing generation method in which the information acquisition unit searches for and acquires from the storage device the change history information and the reason for change information related to the natural language information, or the change history information and the reason for change information based on the interpretation result of the operation sequence information.

14. On the computer, inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; acquiring the drawing information; the language model is constructed by training a training dataset; A computer program, wherein the learning data set includes a change history of drawing information.

15. A computer comprising: inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; acquiring the drawing information; the prompt is generated based on natural language information received from a user terminal; the natural language information is information written in a natural language that indicates an order of operations of the plurality of control target robots, In the step of acquiring, the drawing information generated by the language model is acquired by inputting the prompt and change history information acquired by an information acquisition unit from a storage device; the change history information includes information indicating a change history of the drawing information, the storage device stores a plurality of pieces of change history information; The information acquisition unit searches for and acquires, from the storage device, the change history information related to the natural language information or the change history information based on the interpretation result of the operation sequence information.

16. A computer comprising: inputting a prompt including operation sequence information written in a natural language indicating an operation sequence of a plurality of robots to be controlled into a language model, and causing the language model to generate drawing information indicating a diagram showing the operation sequence of the plurality of robots to be controlled; acquiring the drawing information; the prompt is generated based on natural language information received from a user terminal; the natural language information is information written in a natural language that indicates an order of operations of the plurality of control target robots, In the acquiring step, the drawing information generated by the language model is acquired by inputting the prompt, and change history information and change reason information acquired by an information acquisition unit from a storage device; the change history information includes information indicating a change history of the drawing information, the change reason information includes information regarding the reason for the change indicated by the change history; the storage device stores a plurality of pieces of change history information and a plurality of pieces of change reason information; The information acquisition unit searches for and acquires from the storage device the change history information and the reason for change information related to the natural language information, or the change history information and the reason for change information based on the interpretation result of the operation sequence information.

Images