Layout generation device

Abstract

A layout generation device according to one embodiment stores a plurality of items of sample data in which a machine model, a tool model, a workpiece model, and a peripheral equipment model are arranged in a virtual space. Upon acquiring a layout outline of the machine model, the tool model, the workpiece model, and the peripheral equipment model input in a natural language, the layout generation device transmits, to a generative AI device, a prompt instructing generation of the tool model, the workpiece model, and the peripheral equipment model and generation of layout information of the generated tool model, the generated workpiece model, and the generated peripheral equipment model, and the machine model, on the basis of the sample data and the layout outline. The layout generation device receives a response from the generative AI device, and upon acquiring the layout information, arranges and displays each model in the virtual space on the basis of the layout information.

Description

Layout generation device

[0001] An embodiment of the present invention relates to a technique for laying out a system including industrial machines.

[0002] There is a known technique in which a model that three-dimensionally represents a robot system including a robot equipped with a tool, a workpiece, and peripheral devices is arranged and displayed on a screen, and then a simulation of the operation program of the robot is performed.

[0003] Japanese Patent Application Laid-Open No. 2009-274148

[0004] Beginners who have never actually handled a robot or have never simulated an operation program of a robot do not know what procedures to follow to layout and simulate a robot system. Furthermore, since knowledge of robots and software is required to introduce simulation software, it is not easy.

[0005] Also, even for skilled persons having knowledge of robots and software, in order to layout a robot system and simulate an operation program, each operation has to be performed in accordance with a predetermined procedure, which requires a lot of man-hours.

[0006] A layout generation device according to an embodiment stores a plurality of sample data in which a machine model, a tool model, a workpiece model, and a peripheral device model are arranged in a virtual space. When acquiring an outline of the layout of the machine model, the tool model, the workpiece model, and the peripheral device model input in natural language, the layout generation device, based on the sample data and the outline of the layout, generates the tool model, the workpiece model, and the peripheral device model, and transmits a prompt instructing to generate layout information of the machine model and the generated tool model, workpiece model, and peripheral device model to a generation AI device. When the layout generation device receives a response from the generation AI device and acquires the layout information, it arranges and displays each model on a virtual space based on the layout information.

[0007] This is a block diagram showing the functional configuration of the layout generation device in the embodiment. This is a diagram illustrating the outline of the layout obtained in the embodiment. This is a diagram illustrating the outline of the layout with optimization instructions added in the embodiment. This is a flowchart showing the flow of the layout generation method in the embodiment. This is a diagram illustrating the response from the generation AI device in the embodiment. This is a diagram illustrating the models of tools, workpieces, and peripheral devices generated in the embodiment. This is a diagram illustrating a preview screen in the embodiment showing each model placed in a virtual space. This is a diagram illustrating the procedure for adjusting the generated layout information in the embodiment.

[0008] The following describes embodiments of the present invention. The layout generation device of this embodiment receives an outline of the layout of a system including industrial machinery in natural language, and obtains layout information by inputting appropriate prompts to a generation AI device using a large-scale language model (LLM), and presents it to the user. The type of industrial machinery is not limited, but in this embodiment, as an example, a system including one or more robots applied to applications such as handling, deburring, and arc welding is targeted.

[0009] Figure 1 is a block diagram showing the functional configuration of the layout generation device 1 in an embodiment. The layout generation device 1 is an information processing device (computer) equipped with a control unit 10 (processor) and a storage unit 20 (memory). The layout generation device 1 is also connected to the AI ​​generation device via communication, either directly or via a network.

[0010] The control unit 10 comprises a schematic acquisition unit 11, a transmission unit 12, a reception unit 13, a model placement unit 14, a display unit 15, a teaching unit 16, and an execution unit 17. These functional units operate when the control unit 10 reads and executes a predetermined program stored in the storage unit 20.

[0011] The storage unit 20 includes various software, including programs for realizing the functions of this embodiment, as well as a sample data storage unit 21. The sample data storage unit 21 is a reference database that stores multiple sample data sets, including models (for example, CAD models) that represent the shapes of robots (industrial machinery), tools operated by the robot, workpieces to be operated on, and peripheral equipment in a virtual space, and layout information in which these models are placed in the virtual space.

[0012] Numerous sample data sets, each with different types, numbers, and placement positions for each model, are prepared in advance and classified according to the application to which the robot is applied. Furthermore, layout information generated by this embodiment may be added to the sample data.

[0013] The outline acquisition unit 11 acquires outlines of the layouts of robot models, tool models, work models, and peripheral equipment models input in natural language. The outline acquisition unit 11 may also accept requests in natural language to adjust the layouts of the robot models, tool models, work models, and peripheral equipment models displayed by the display unit 15. The outlines of the layouts may be input as text or as voice.

[0014] The layout outline entered here may include, for example, the following information: (1) the type and number of robot models; (2) the type (hand, deburring tool, welding torch, etc.), shape, or file name of the tool models; (3) the shape (cylinder, cuboid, etc.), size, or file name of the workpiece models; (4) the type (container, tray, conveyor, machine tool, etc.), shape, or file name of the peripheral equipment models; (5) positional information including specific numerical values ​​or abstract expressions (farther away, closer, etc.) indicating the position, spacing, and orientation of the robot models, tool models, workpiece models, and peripheral equipment models; (6) information on the application to which the robot is applied (handling, deburring, arc welding, etc.).

[0015] Figure 2 illustrates an example of a layout obtained in the embodiment. For example, if the user is a beginner, they input an outline of what they will do with the robot into the layout generation device 1, without using technical terms. In this case, the layout outline is expected to be either (A) without including specific information such as the file name, type, or shape of the robot model to be used, or just a simple sentence (B).

[0016] On the other hand, if the user is an expert, it is expected that they will use a CAD model procured in advance. In this case, a summary (C) specifying the file name, etc., will be entered. Alternatively, it may include more detailed layout information (D). Furthermore, even experts may not have a CAD model prepared during the conceptual stage of the robot system, or the specific workpiece, tools, peripheral equipment, etc., may not have been decided.

[0017] The transmitting unit 12 generates prompts instructing the AI ​​generation device to generate tool models, work models, and peripheral device models that are suitable for the sample data and layout outline, and to generate layout information for the robot model and the generated tool models, work models, and peripheral device models, and transmits these prompts to the AI ​​generation device. Furthermore, if the outline acquisition unit 11 receives an additional request regarding layout adjustment, the transmitting unit 12 generates prompts instructing the AI ​​generation device to modify the layout information based on this request, and transmits these prompts to the AI ​​generation device.

[0018] Search-Augmented Generator (RAG) technology may be applied to referencing sample data. That is, the sample data to be referenced may be extracted from the sample data storage unit 21 based on similarity to the layout overview, etc. In this case, the similarity may be calculated after removing extraneous information from the text input as the layout overview, or after extracting important words (frequently occurring words, words registered in a predetermined dictionary, etc.). Furthermore, the generating AI device may have undergone additional training (fine-tuning) using a dataset previously extracted from sample data.

[0019] The transmission unit 12 may add instructions to the prompt to optimize the layout. Specifically, for example, an instruction to arrange the elements as follows may be added to the layout outline and sent to the generating AI as a prompt.

[0020] Figure 3 illustrates the schematic of a layout with added optimization instructions in an embodiment. Layout schematic (E) indicates an arrangement that does not cause interference with the robot model. Layout schematic (F) indicates an arrangement where the movement required of the robot falls within the robot's operational range. Layout schematic (G) indicates an arrangement where there is a predetermined margin in the robot's axis range during robot operation. Layout schematic (H) indicates an arrangement that minimizes the cycle time during robot operation.

[0021] For example, when optimizing (F) to (H) above, the robot's motion program may be generated by the AI ​​generation device, or a sample motion program may be input to the AI ​​generation device as a prompt.

[0022] The receiving unit 13 receives a response from the generating AI device and acquires layout information.

[0023] The model placement unit 14 places the robot model, tool model, work model, and peripheral equipment model in the virtual space based on the layout information.

[0024] The display unit 15 displays robot models, tool models, work models, and peripheral equipment models arranged in the virtual space.

[0025] The teaching unit 16 teaches an action program to simulate actual movements using a robot model placed in a virtual space.

[0026] The execution unit 17 executes an operation program in a virtual space and simulates the robot's movements. At this time, the outline acquisition unit 11 may accept further requests regarding layout adjustments based on the information provided by the execution unit 17.

[0027] Figure 4 is a flowchart showing the flow of the layout generation method in the embodiment. In step S1, the outline acquisition unit 11 acquires outlines of the layouts of the robot model, tool model, work model, and peripheral equipment model that are input in natural language.

[0028] In step S2, the transmission unit 12 extracts sample data that is related to (similar to) the outline of the acquired layout.

[0029] In step S3, the transmission unit 12 sends the extracted sample data along with a layout outline as a prompt to the generation AI device.

[0030] In step S4, the receiving unit 13 receives the layout information generated from the generation AI device.

[0031] In step S5, the outline acquisition unit 11 receives a request from the user and determines whether or not layout adjustment is necessary. If the determination is YES, the process moves to step S6; if the determination is NO, the process moves to step S7.

[0032] In step S6, the transmission unit 12 adds a request to the layout outline, and the process returns to step S3.

[0033] In step S7, the model placement unit 14 places the robot model, tool model, work model, and peripheral equipment model in the virtual space based on the layout information.

[0034] In step S8, the display unit 15 displays the robot model, tool model, work model, and peripheral equipment model arranged in the virtual space.

[0035] In step S9, the teaching unit 16 teaches the robot's motion program in a virtual space.

[0036] In step S10, the execution unit 17 executes the operation program in the virtual space and simulates the robot's movements.

[0037] Figure 5 illustrates the response from the generation AI device in the embodiment. The layout information includes the application (e.g., handling), the file names of the CAD data for the robot, tools, workpieces, and peripherals that constitute the system, and their respective location information.

[0038] Here, each model shows the use of existing CAD data, but it may also be newly generated based on the layout outline.

[0039] Figure 6 illustrates the models of tools, workpieces, and peripherals generated in the embodiment. For example, if the layout outline specifies that the application is handling, the AI ​​generation device will generate a new model of a hand (handling tool) based on the CAD data of the hand included in the sample data.

[0040] If the user requests a change to the outputted hand model, a prompt to generate a new model is sent to the generation AI device, and this process is repeated until the user approves. The timing of model generation can also be designed as needed; for example, the generated model may be used from the initial generation of layout information, the sample data model may be used with the layout information initially, or several model candidates may be presented from the sample data, and the generation AI device may be instructed to generate a new model if the user determines that there is no suitable model.

[0041] Similarly, if the layout outline includes a description of the workpiece, or if a conveyor is described as peripheral equipment, CAD data may be generated as a new model based on the relevant sample data.

[0042] Figure 7 illustrates a preview screen showing the placement of each model in the virtual space according to the embodiment. In this example, two robots, tools attached to each robot, workpieces in baskets, trays for arranging the workpieces, a camera for detecting the workpieces, and a conveyor belt for moving the trays are placed in the virtual space.

[0043] FIG. 8 is a diagram illustrating a procedure for adjusting the layout information generated in the embodiment. When an adjustment of the layout is requested from the user with respect to the preview (FIG. 7) of the generated layout information, this request is transmitted to the generation AI device as a prompt, and new layout information is presented. If there is a further request, an instruction to the generation AI device is repeated. When the user determines that there is no problem and inputs to that effect, the layout information is finalized.

[0044] According to the embodiments illustrated so far, the layout generation device 1 receives a layout outline in natural language and generates layout information from sample data by instructing the generation AI device. At this time, based on the layout outline and the sample data, by newly generating a tool model, a work model, and a peripheral device model, the simulation of the robot system can be executed by an appropriate model that meets the user's purpose.

[0045] In addition, the user can construct a layout of an appropriate robot system in the virtual space in an interactive manner by referring to the generated layout information, adjusting the layout outline as necessary, and repeating the generation of new layout information.

[0046] As a result, for example, a beginner can create a layout of a robot system in the virtual space without requiring special knowledge, and a skilled person can create a layout of a robot system in the virtual space without performing operations that were conventionally required, thus significantly reducing the man-hours.

[0047] Furthermore, the layout generation device 1 instructs the generation AI device to optimize the layout, such as an arrangement that can avoid interference, an arrangement that fits within the operable range of the robot, an arrangement that has a predetermined or more margin in the axis range, and an arrangement that minimizes the cycle time, and can generate appropriate layout information.

[0048] As described above, the embodiments of the present invention have been described in detail, but the present invention is not limited to the individual embodiments described above. These embodiments can be variously added, replaced, changed, partially deleted, etc., without departing from the gist of the invention or without departing from the idea and spirit of the present invention derived from the content described in the claims and its equivalents. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited thereto. The same applies when numerical values or mathematical formulas are used in the description of the above-described embodiments.

[0049] Regarding the above-described embodiments, the following supplementary notes are further disclosed. (Supplementary Note 1) A sample data storage unit (21) that stores a plurality of sample data in which a machine model, a tool model, a work model, and a peripheral device model are arranged in a virtual space, A schematic acquisition unit (11) that acquires a schematic of the layout of the machine model, the tool model, the work model, and the peripheral device model input in natural language, and based on the sample data and the schematic of the layout, so as to generate the tool model, the work model, and the peripheral device model, and a prompt that instructs to generate layout information of the machine model and the generated tool model, work model, and peripheral device model is transmitted to the generation AI device. A receiving unit (13) that receives a response from the generation AI device and acquires the layout information, and a model arrangement unit (14) that arranges the machine model, the tool model, the work model, and the peripheral device model on a virtual space based on the layout information, and a display unit (15) that displays the machine model, the tool model, the work model, and the peripheral device model arranged on the virtual space. A layout generation device (1) comprising:

[0050] (Note 2) The layout generation device (1) described in (Note 1), wherein the outline acquisition unit (11) receives requests in natural language for adjusting the layout of the displayed machine model, tool model, work model, and peripheral device model, and the transmission unit (12) transmits a prompt to the generation AI device instructing it to modify the layout information based on the requests.

[0051] (Note 3) The layout generation device (1) according to (Note 1) or (Note 2), wherein the transmitting unit (12) adds an instruction in the prompt to generate the layout information that does not interfere with the machine model.

[0052] (Note 4) The layout generation device (1) according to any one of (Note 1) to (Note 3), wherein the transmitting unit (12) adds an instruction in the prompt to generate the layout information such that the operation required of the machine falls within the operating range of the machine.

[0053] (Note 5) The layout generation device (1) according to any one of (Note 1) to (Note 4), wherein the transmitting unit (12) adds an instruction in the prompt to generate the layout information that has a predetermined margin in the axis range when the machine is in operation.

[0054] (Note 6) The layout generation device (1) according to any one of (Note 1) to (Note 5), wherein the transmitting unit (12) adds an instruction in the prompt to generate the layout information that has the shortest cycle time.

[0055] (Note 7) The layout outline is a layout generating device (1) as described in any of (Note 1) to (Note 6), including the type and number of machine models.

[0056] (Note 8) The layout outline includes information relating to the type or shape of the tool model, information relating to the type or shape of the work model, or information relating to the type or shape of the peripheral equipment model, as described in any of (Note 1) to (Note 7) of the layout generating apparatus (1).

[0057] (Note 9) The layout outline includes positional information of the machine model, the tool model, the workpiece model, or the peripheral equipment model, as described in any of (Note 1) to (Note 8) of the layout generating apparatus (1).

[0058] (Note 10) The layout outline is a layout generating device (1) as described in any of (Note 1) to (Note 9), which includes information about the application to which the machine is applied.

[0059] (Note 11) A layout generation device (1) according to any one of (Note 1) to (Note 10), comprising: a teaching unit (16) for teaching the operation program of a machine; and an execution unit (17) for simulating the operation program in the virtual space.

[0060] (Note 12) The outline of the above layout is input by voice to the layout generation device (1) described in any of (Note 1) to (Note 11).

[0061] (Note 13) The sample data is classified according to the application to which the machine is applied, as described in any of (Note 1) to (Note 12) of the layout generation device (1).

[0062] 1 Layout generation device 10 Control unit 11 Outline acquisition unit 12 Transmission unit 13 Receiving unit 14 Model placement unit 15 Display unit 16 Teaching unit 17 Execution unit 20 Storage unit 21 Sample data storage unit

Claims

1. A layout generation device comprising: a sample data storage unit that stores multiple sample data sets in which machine models, tool models, work models, and peripheral equipment models are arranged in a virtual space; a schematic acquisition unit that acquires schematic layouts of the machine models, tool models, work models, and peripheral equipment models input in natural language; a transmission unit that transmits a prompt to a generation AI device instructing it to generate the tool models, work models, and peripheral equipment models based on the sample data and the schematic layouts, and to generate layout information for the machine models, as well as the generated tool models, work models, and peripheral equipment models; a reception unit that receives a response from the generation AI device and acquires the layout information; a model placement unit that arranges the machine models, tool models, work models, and peripheral equipment models in a virtual space based on the layout information; and a display unit that displays the machine models, tool models, work models, and peripheral equipment models arranged in the virtual space.

2. The layout generation device according to claim 1, wherein the outline acquisition unit receives requests in natural language for adjusting the layout of the displayed machine model, tool model, work model, and peripheral device model, and the transmission unit transmits a prompt to the generation AI device instructing it to modify the layout information based on the requests.

3. The layout generation device according to claim 1 or 2, wherein the transmitting unit adds an instruction in the prompt to generate the layout information that does not interfere with the machine model.

4. The layout generation device according to any one of claims 1 to 3, wherein the transmitting unit adds an instruction in the prompt to generate the layout information such that the operation required by the machine falls within the operating range of the machine.

5. The layout generation device according to any one of claims 1 to 4, wherein the transmitting unit adds an instruction in the prompt to generate the layout information having a predetermined margin in the axis range when the machine is in operation.

6. The layout generation device according to any one of claims 1 to 5, wherein the transmitting unit adds an instruction in the prompt to generate the layout information that has the shortest cycle time.

7. The layout generation apparatus according to any one of claims 1 to 6, wherein the outline of the layout includes the type and number of the machine models.

8. The layout generation apparatus according to any one of claims 1 to 7, wherein the layout outline includes information relating to the type or shape of the tool model, information relating to the type or shape of the work model, or information relating to the type or shape of the peripheral device model.

9. The layout generation apparatus according to any one of claims 1 to 8, wherein the outline of the layout includes positional information on which the machine model, the tool model, the workpiece model, or the peripheral equipment model are arranged.

10. The layout generation apparatus according to any one of claims 1 to 9, wherein the outline of the layout includes information about the application to which the machine is applied.

11. A layout generation device according to any one of claims 1 to 10, comprising: a teaching unit for teaching a machine operation program; and an execution unit for simulating the operation program in the virtual space.

12. The layout generation device according to any one of claims 1 to 11, wherein the outline of the layout is input by voice.

13. The layout generation apparatus according to any one of claims 1 to 12, wherein the sample data is classified according to the application to which the machine is applied.

Images