Simulation device and program

Abstract

A simulation device according to an embodiment comprises a generation unit. The generation unit executes, in a virtual space, a first simulation indicating an operation in which a second virtual object indicating a robot holds a first virtual object indicating an object which is placed at a predetermined place and is conveyed by the robot. The generation unit fixes the first virtual object to a hand of the second virtual object on the basis of the result of the first simulation. The generation unit generates, by performing a second simulation that causes the second virtual object to perform an operation in which the first virtual object is conveyed from the predetermined place, a first operation path indicating an operation of the second virtual object in which the first virtual object is conveyed from the predetermined place in such a manner that the first virtual object and the second virtual object do not interfere with at least one of the second virtual object and a third virtual object indicating an obstacle.

Description

Simulation device and program

[0001] The present disclosure relates to a simulation device and a program.

[0002] There is a technology called offline simulation that uses a virtual robot in a virtual space to simulate the robot's movements. There is also a technology that automatically generates the robot's movement path by operating a virtual robot.

[0003] International Publication No. 2023 / 188407

[0004] A robot may transport a workpiece while holding it in its hand. In reality, when a robot performs such an operation, the hand does not have a workpiece before it holds it. However, after the hand holds the workpiece, the workpiece is fixed to the hand and moves together with the hand.

[0005] In conventional technology, when generating a motion path for transporting a workpiece, a simulation is performed to ensure that the robot does not interfere with obstacles. However, as mentioned above, in reality, a hand holds a workpiece. Therefore, when a real robot is operated along a motion path generated using conventional technology, there is a possibility that the workpiece held by the robot will interfere with obstacles. For these reasons, there is a need to generate a motion path for a robot that does not allow the workpiece held by the robot to interfere with the robot or obstacles.

[0006] The problem that the embodiments of the present invention aim to solve is to provide a simulation device and a program that can generate a robot movement path that prevents a workpiece held by the robot from interfering with the robot or an obstacle.

[0007] According to an embodiment, a simulation device includes a generation unit. The generation unit executes a first simulation in a virtual space, which shows an action of a second virtual object representing a robot holding a first virtual object representing an object placed at a predetermined location to be transported by the robot. The generation unit fixes the first virtual object to a hand of the second virtual object based on a result of the first simulation. The generation unit executes a second simulation in which the second virtual object performs an action of transporting the first virtual object from the predetermined location, thereby generating a first movement path showing an action of the second virtual object transporting the first virtual object from the predetermined location such that the first virtual object and the second virtual object do not interfere with the second virtual object and a third virtual object representing an obstacle.

[0008] 1 is a block diagram showing an example of a configuration of a main part of a robot system according to an embodiment and components included in the robot system, and a flowchart showing an example of processing by a processor in FIG. 1. A perspective view showing an example of a virtual space.

[0009] A robot system according to an embodiment will be described below with reference to the drawings. Note that the scale of each part in the drawings used in the following description of the embodiments may be changed as appropriate. Furthermore, the drawings used in the following description of the embodiments may omit the configuration for the sake of explanation. Furthermore, the same reference numerals indicate similar elements in the drawings and throughout this specification. Furthermore, "based on XX" in this application means "based on at least XX" and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX after calculation or processing. "XX" is any element (for example, any information).

[0010] FIG. 1 is a block diagram showing an example of a main configuration of a robot system 1 according to an embodiment and the components included in the robot system 1. Note that each component of the device may be built-in or external. The robot system 1 has a function of automatically generating a delivery path for the robot 200 through offline simulation. The delivery path indicates the operating path along which the robot 200 takes a workpiece from a container and transports it. The robot system 1 is also a system that controls the robot 200. The robot system 1 includes, as an example, a simulation device 100, a robot 200, a three-dimensional sensor 300, and a control device 400. Note that the robot system 1 may include only some of these. Note that while FIG. 1 shows one of each device, the number of each device is not limited.

[0011] The simulation device 100, the robot 200, the three-dimensional sensor 300, and the control device 400 are connected to, for example, a network NW. The network NW is typically a communication network including a private network such as an intranet. The network NW is typically a communication network including a local area network (LAN). The network NW may be a communication network including the Internet. The network NW may be a communication network including a wide area network (WAN). Furthermore, the network NW may be a wireless line or a wired line, or may be a combination of wireless and wired lines. Furthermore, the network NW may be a communication network including a dedicated line or a public mobile phone network. Note that the connection and communication between the devices may not be via the network NW.

[0012] The simulation device 100 is a device that performs an offline simulation of the robot 200. The simulation device 100 is, for example, a general-purpose device such as a personal computer (PC) or a server. Alternatively, the simulation device 100 may be a device dedicated to offline simulation. The simulation device 100 is also a device that controls the robot 200. The simulation device 100 includes, for example, a processor 101, a read-only memory (ROM) 102, a random-access memory (RAM) 103, an auxiliary storage device 104, an input device 105, a display device 106, and a communication interface 107. A bus 108 and the like connect these components. The simulation device 100 is an example of a simulation device.

[0013] The processor 101 is the central part of a computer that performs various calculations and processes, such as calculations and controls, necessary for the operation of the simulation apparatus 100. The processor 101 is, for example, a central processing unit (CPU), a microprocessing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 101 may be a combination of several of these. The processor 101 may also be a combination of these with a hardware accelerator or the like. The processor 101 controls each component to realize various functions of the simulation apparatus 100 based on programs such as firmware, system software, and application software stored in the ROM 102 or the auxiliary storage device 104. The processor 101 also executes the processes described below based on the programs. Note that some or all of the programs may be incorporated into the circuitry of the processor 101. The processor 101 is an example of a generating unit.

[0014] The ROM 102 and RAM 103 are main storage devices of the computer centered around the processor 101. The ROM 102 is a non-volatile memory used exclusively for reading data. The ROM 102 stores, for example, firmware among the above programs. The ROM 102 also stores data used by the processor 101 when performing various processes.

[0015] The RAM 103 is a memory used for reading and writing data. The RAM 103 is used as a work area for storing data that is temporarily used when the processor 101 performs various processes. The RAM 103 is typically a volatile memory.

[0016] The auxiliary storage device 104 is an auxiliary storage device of a computer centered around the processor 101. The auxiliary storage device 104 is, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), or a flash memory. The auxiliary storage device 104 stores, for example, system software and application software among the above programs. The auxiliary storage device 104 also stores data used by the processor 101 when performing various processes, data generated by the processes in the processor 101, various setting values, and the like.

[0017] The application software stored in the auxiliary storage device 104 includes software that enables offline simulation.

[0018] The input device 105 accepts operations by an operator of the simulation apparatus 100. The input device 105 is, for example, a keyboard, a keypad, a touchpad, a mouse, a controller, etc. The input device 105 may also be a device for voice input.

[0019] The display device 106 displays a screen for notifying various pieces of information to the operator of the simulation apparatus 100, etc. The display device 106 is, for example, a display such as a liquid crystal display or an organic electroluminescence (EL) display. A touch panel can also be used as the input device 105 and the display device 106. That is, a display panel included in the touch panel can be used as the display device 106, and a pointing device for touch input included in the touch panel can be used as the input device 105.

[0020] The communication interface 107 is an interface for the simulation device 100 to communicate via a network NW or the like.

[0021] The bus 108 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged among the various components of the simulation device 100 .

[0022] The robot 200 is, for example, a manipulator or a robot arm, or a robot equipped with these. The robot 200 is, for example, an articulated robot. The robot 200 includes, for example, a drive unit 201 and a hand 202. The robot 200 transports a workpiece W.

[0023] FIG. 1 also shows the workpiece W and obstacles OBS1 and OBS2 as elements related to the transportation of the workpiece W. Obstacle OBS1 is a container that holds the workpiece W. Obstacle OBS1 is, for example, a rectangular box-like shape with an open top. Alternatively, obstacle OBS1 may have other shapes. The workpiece W is, for example, piled up loosely or in a heap inside obstacle OBS1. One or more workpieces W are contained in obstacle OBS1. The inside of obstacle OBS1 is an example of a predetermined location. Obstacle OBS2 is, for example, equipment or the like.

[0024] The robot 200 includes one or more driving units 201. The driving units 201 are parts that are driven by a motor such as a servo motor. The driving units 201 are driven to rotate around a drive shaft, for example. The robot 200 changes the position and posture of the hand 202 by driving the driving units 201.

[0025] The robot 200 is equipped with a hand 202 at the tip of a manipulator. The hand 202 holds a workpiece W or the like. The hand 202 holds the workpiece W by gripping, suction, or by other methods. The workpiece W is an object that the robot 200 carries.

[0026] The three-dimensional sensor 300 is a sensor that detects the position of the surface of an object such as the workpiece W and the obstacle OBS. Obstacle OBS1 and obstacle OBS2 are examples of obstacle OBS. The three-dimensional sensor 300 can use various methods such as LIDAR (light detection and ranging), a stereo camera, other optical methods, an ultrasonic sensor, or radar. The three-dimensional sensor 300 detects the position of the surface of an object, and therefore can measure not only the position of the object but also the shape of the object.

[0027] The control device 400 communicates with the robot 200. The control device 400 is a device that controls the robot 200 using this communication.

[0028] The operation of the robot system 1 according to the embodiment will be described below with reference to Fig. 2 and other figures. Note that the content of the processing in the following description of the operation is an example, and various processing that can obtain similar results can be used as appropriate. Fig. 2 is a flowchart showing an example of processing by the processor 101 of the simulation device 100. The processor 101 executes the processing of Fig. 2 based on a program stored in, for example, the ROM 102 or the auxiliary storage device 104.

[0029] In step ST11 of Fig. 2, the processor 101 of the simulation device 100 determines whether or not to generate a movement path for the robot 200. The processor 101 determines to execute a simulation of the movement program when, for example, there is an input instructing to generate a movement path. If the processor 101 does not determine to generate a movement path for the robot 200, it determines No in step ST11 and repeats the processing of step ST11. On the other hand, if the processor 101 determines to generate a movement path for the robot 200, it determines Yes in step ST11 and proceeds to step ST12.

[0030] In step ST12, processor 101 determines whether the route to be generated is a payout route. If the route to be generated is a payout route, processor 101 determines Yes in step ST12 and proceeds to step ST13.

[0031] In step ST13, the processor 101 uses the three-dimensional sensor 300 to measure the position of each workpiece W and each obstacle OBS. The positions may be positions in a predetermined coordinate space, or may be relative positions with respect to the robot 200 or another predetermined reference. The processor 101 also acquires the position and orientation of the robot 200. The processor 101 acquires the position and orientation from the robot 200 or the control device 400, for example.

[0032] In step ST14, the processor 101 detects each workpiece W using the three-dimensional sensor 300.

[0033] In step ST15, the processor 101 generates a virtual space S as shown in FIG. 3. FIG. 3 is a perspective view showing an example of the virtual space S. The virtual space S is a virtual space simulated by the processor 101. The virtual space S includes a plurality of 3D (three-dimensional) objects OBJ. The 3D objects OBJ are 3D virtual objects in the virtual space S. The processor 101 generates the virtual space S by arranging each 3D object OBJ in the virtual space S so that the positional relationship between the 3D objects OBJ is the same as that of objects in real space. The processor 101 arranges each 3D object OBJ using the measurement results of step ST13. Here, the objects in real space refer to the robot 200, the workpiece W, the obstacle OBS, etc. The processor 101 uses a position measured in advance as the position of the robot 200. Alternatively, the processor 101 may also measure the position of the robot 200 using a three-dimensional sensor 300.

[0034] FIG. 3 shows a virtual workpiece OBJ1, a virtual robot OBJ2, a virtual obstacle OBJ3-1, and a virtual obstacle OBJ3-2 as 3D objects OBJ.

[0035] The virtual workpiece OBJ1 is a virtual workpiece W. The virtual workpiece OBJ1 is an example of one or more fourth virtual objects that represent an object placed at a predetermined location.

[0036] The virtual robot OBJ2 is a virtual robot 200. The virtual robot OBJ2 includes a virtual hand OBJ21. The virtual hand OBJ21 is a virtual hand 202. The virtual robot OBJ2 is an example of a second virtual object representing a robot.

[0037] The virtual obstacle OBJ3-1 is the virtual obstacle OBS1. The virtual obstacle OBJ3-2 is the virtual obstacle OBS2. The shapes of the virtual workpiece OBJ1, the virtual obstacle OBJ3-1, and the virtual obstacle OBJ3-2 are shapes measured by the three-dimensional sensor 300.

[0038] The inside of the virtual obstacle OBJ3-1 is the inside of a container. The inside of the virtual obstacle OBJ3-1 is an example of a predetermined location. Each of the virtual obstacle OBJ3-1 and the virtual obstacle OBJ3-2 is an example of a third virtual object representing an obstacle.

[0039] In step ST16, if there are multiple virtual workpieces OBJ1, the processor 101 determines which virtual workpiece OBJ1 to take out. For example, the processor 101 determines to take out a pre-specified virtual workpiece OBJ1. Alternatively, the processor 101 determines the virtual workpiece OBJ1 to take out based on the measurement results of step ST13. For example, the processor 101 determines the virtual workpiece OBJ1 that is considered to be the easiest to take out as the virtual workpiece OBJ1 to take out. The virtual workpiece OBJ1 determined here will be referred to as the "target workpiece" hereinafter. The target workpiece is an example of a first virtual object that represents an object placed in a predetermined location and transported by a robot.

[0040] In step ST17, the processor 101 performs a simulation in the virtual space S until the virtual robot OBJ2 holds the target workpiece. The movement path of the robot 200 for picking up the target workpiece (hereinafter referred to as the "holding path") is, for example, a predetermined path. The position and orientation of the robot 200 at the start of the simulation are, for example, the position and orientation acquired in step ST13. However, the position and orientation of the robot 200 at the start of the simulation do not have to be the position and orientation acquired in step ST13.

[0041] The simulation performed in step ST17 is an example of a first simulation showing an operation in which a second virtual object representing a robot holds a first virtual object representing an object placed at a predetermined location and transported by the robot in virtual space. As described above, by performing the processing of step ST17, the processor 101 functions as an example of a generation unit that performs the first simulation.

[0042] In step ST18, the processor 101 determines whether the virtual robot OBJ2 interferes with an obstacle in the simulation of step ST17. Here, interfering with an obstacle refers to the virtual robot OBJ2 coming into contact with a 3D object OBJ other than the target workpiece. It is preferable that interfering with an obstacle also includes contact of a portion of the virtual robot OBJ2 other than the virtual hand OBJ21 with the target workpiece. It is also preferable that interfering with an obstacle also includes contact of the virtual hand OBJ2 with the target workpiece in a case other than when the virtual hand OBJ21 and the target workpiece come into contact when the virtual hand OBJ2 is holding the target workpiece. It is also preferable that interfering with an obstacle also includes contact of a portion of the virtual robot OBJ2 with a portion other than the portion of the virtual robot OBJ2. Examples of 3D objects OBJ other than the target workpiece include a virtual workpiece OBJ1 other than the target workpiece, virtual obstacle OBJ3-1, and virtual obstacle OBJ3-2. If the processor 101 determines that the robot 200 will interfere with an obstacle, it judges Yes in step ST18 and proceeds to step ST19.

[0043] In step ST19, the processor 101 generates a new holding path. The new holding path is a holding path that does not cause virtual robot OBJ2 to interfere with obstacles. The processor 101 finds a holding path that does not cause obstacles by, for example, performing simulations multiple times while changing conditions. The processor 101 sets the found holding path as the generation result. Alternatively, the processor 101 may use other known methods to generate a holding path that does not cause obstacles. The processor 101 changes the holding path of virtual robot OBJ2 to the newly generated holding path.

[0044] The holding path generated in step ST19 is an example of a second movement path indicating a movement in which the second virtual object does not interfere with the third virtual object. As described above, by performing the processes of steps ST18 and ST19, the processor 101 functions as an example of a generation unit that generates a second movement path when the result of the first simulation indicates that the second virtual object interferes with the third virtual object.

[0045] After processing step ST19, processor 101 proceeds to step ST20. If robot 200 does not interfere with an obstacle, processor 101 determines No in step ST18 and proceeds to step ST20.

[0046] In step ST20, the processor 101 simulates the virtual robot OBJ2 to move along a predetermined holding path. If the holding path was changed in step ST19, this holding path is the changed holding path. The processor 101 then simulates the posture of the target workpiece OBJ11 to be held by the virtual hand OBJ21. The processor 101 then attaches the target workpiece OBJ11 to the virtual hand OBJ21 in this posture in the virtual space S. That is, the processor 101 fixes the target workpiece OBJ11 to the virtual hand OBJ21. This causes the target workpiece to follow the movement of the virtual hand OBJ21. FIG. 4 shows an example of the target workpiece fixed to the virtual hand OBJ21. FIG. 4 is a perspective view showing an example of the virtual space S. FIG. 4 shows the target workpiece OBJ11 and the virtual workpiece OBJ12. The virtual workpiece OBJ12 is the virtual workpiece OBJ1 other than the target workpiece OBJ11.

[0047] When attaching the virtual workpiece OBJ1 to the virtual hand OBJ21 as the target workpiece OBJ11, the processor 101 may choose not to perform physics calculations on the virtual workpiece OBJ1 that was originally inside the virtual obstacle OBJ3-1. In this case, the processor 101 places the target workpiece OBJ11 anew in the virtual space and fixes it to the virtual hand OBJ21. Note that "removal" means deleting the virtual workpiece OBJ1 from the virtual space S or setting the virtual workpiece OBJ1 so that physics calculations are not performed, thereby preventing physics calculations from being performed on the virtual workpiece OBJ1. A 3D object OBJ that does not perform physics calculations does not interfere with other 3D objects OBJ.

[0048] Alternatively, when attaching the virtual workpiece OBJ1 to the virtual hand OBJ21 as the target workpiece OBJ11, the processor 101 may fix the virtual workpiece OBJ1, which was originally inside the virtual obstacle OBJ3-1, to the virtual hand OBJ21.

[0049] The processor 101 does not need to perform the simulation up to the holding of the target workpiece in step ST20. In this case, the processor 101 uses the results of the simulation performed in step ST17 or step ST19 instead of the simulation.

[0050] The target workpiece OBJ11 is an example of a first virtual object. The virtual workpiece OBJ12 is an example of a virtual object that represents an object placed in a predetermined location other than the first virtual object.

[0051] As described above, by performing the processing of step ST20, the processor 101 functions as an example of a generation unit that fixes the first virtual object to the hand of the second virtual object based on the results of the first simulation.

[0052] Furthermore, the processor 101 functions as an example of a generation unit that executes a first simulation on a second movement path by performing a simulation on the holding path changed in step ST19. Furthermore, when the holding path is changed in step ST19, the processor 101 functions as an example of a generation unit that fixes a first virtual object to the hand based on the result of the first simulation executed on the second movement path by performing the processing of step ST20.

[0053] Furthermore, the processor 101 functions as an example of a generation unit that does not perform physical calculations on the virtual work OBJ1, newly places the target work OBJ11 in the virtual space, and fixes it to the virtual hand OBJ21, thereby not performing physical calculations on a fourth virtual object corresponding to an object to be transported from the specified location, among one or more fourth virtual objects representing objects placed at a specified location, and places a first virtual object in the virtual space and fixes the placed first virtual object to the hand.

[0054] In step ST21, the processor 101 generates a delivery path for the target workpiece OBJ11 by performing a simulation in the virtual space S. The delivery path is a path for moving the target workpiece OBJ11 to a predetermined position. The simulation starts, for example, when the virtual robot OBJ2 holds the target workpiece OBJ11. The processor 101 generates a delivery path such that the virtual robot OBJ2 and the target workpiece OBJ11 fixed to the virtual hand OBJ21 do not interfere with obstacles. Here, interference with an obstacle refers to the virtual robot OBJ2 and the target workpiece OBJ11 coming into contact with a 3D object OBJ other than the target workpiece OBJ11. The 3D object OBJ other than the target workpiece OBJ11 is, for example, the virtual workpiece OBJ12, virtual obstacle OBJ3-1, and virtual obstacle OBJ3-2. It is also preferable that interference with an obstacle also includes contact of a portion of the virtual robot OBJ2 other than the virtual hand OBJ21 with the target workpiece OBJ11. It is also preferable that interference with an obstacle also includes contact of a portion of the virtual robot OBJ2 with a portion other than that portion of the virtual robot OBJ2. It is also possible that interference with an obstacle also includes contact of a portion of the virtual hand OBJ21 other than the portion holding the target workpiece OBJ11 with the target workpiece OBJ11. The processor 101 finds a transfer path that does not interfere with obstacles, for example, by performing a simulation multiple times while changing conditions. The processor 101 regards the found transfer path as the generation result. Alternatively, the processor 101 may use other known methods to generate a transfer path that does not interfere with obstacles.

[0055] The payout path generated in step ST21 is an example of a first movement path that indicates the movement of the second virtual object to transport the first virtual object from the predetermined location so that the first virtual object and the second virtual object do not interfere with a third virtual object that indicates an obstacle. As described above, by performing the processing of step ST21, the processor 101 functions as an example of a generation unit that generates a first movement path by performing a second simulation in which the second virtual object performs the movement of transporting the first virtual object from the predetermined location.

[0056] On the other hand, if the route to be generated is not a payout route, the processor 101 judges No in step ST12 and proceeds to step ST22.

[0057] In step ST22, processor 101 generates a movement path that is not a payout path. Processor 101 generates the movement path by, for example, a conventional method.

[0058] After processing step ST21 or step ST22, the processor 101 proceeds to step ST23.

[0059] In step ST23, the processor 101 instructs the communication interface 107 to transmit the automatically generated movement path to the control device 400. Upon receiving this transmission instruction, the communication interface 107 transmits the movement path to the control device 400. The control device 400 controls the robot 200 to operate the robot 200 according to the received movement path. When the processor 101 proceeds from step ST21 to step ST23, the processor 101 transmits the determined holding path and the removal path generated in step ST21 to the control device 400. As a result, the robot 200 holds the workpiece W, removes it from the obstacle OBS1, and transports it. After processing step ST23, the processor 101 returns to step ST11. When the processor 101 proceeds from step ST22 to step ST23, the processor 101 transmits the movement path generated in step ST22 to the control device 400.

[0060] According to the robot system 1 of the embodiment, the simulation device 100 simulates the operation of the virtual robot OBJ2 holding the virtual workpiece OBJ1. Then, based on the simulation, the simulation device 100 of the embodiment fixes the target workpiece OBJ11 to the virtual hand OBJ21. This allows the simulation device 100 of the embodiment to fix the target workpiece OBJ11 to the virtual hand OBJ21 in an accurate position and orientation.

[0061] Furthermore, according to the robot system 1 of the embodiment, the simulation device 100 performs a simulation with the target workpiece OBJ11 fixed to the virtual hand OBJ21. Then, the simulation device 100 of the embodiment generates a transfer path that does not interfere with obstacles based on the simulation. By performing a simulation with the target workpiece OBJ11 fixed to the virtual hand OBJ21, the simulation device 100 of the embodiment can generate a transfer path that does not interfere with obstacles for the target workpiece OBJ11 and the virtual robot OBJ2. Furthermore, by using the transfer path, the simulation device 100 of the embodiment can operate the robot 200 so that the robot 200 and the workpiece W do not interfere with obstacles.

[0062] Furthermore, according to the robot system 1 of the embodiment, the simulation device 100 generates a delivery path that does not interfere with the robot 200 to which the target workpiece OBJ11 is fixed and the target workpiece OBJ, and the robot 200. This allows the simulation device of the embodiment to more accurately generate a delivery path that does not interfere with obstacles.

[0063] It is considered that the more randomly the workpieces W are stacked, the more different the positions and shapes at which the workpieces W are held will be each time. Therefore, the simulation device 100 of the embodiment is suitable for a situation in which the workpieces W are stacked randomly.

[0064] Furthermore, according to the robot system 1 of the embodiment, if the simulation results show that the virtual robot OBJ2 will interfere with an obstacle when holding the virtual workpiece OBJ1, the simulation device 100 generates a holding path that allows the virtual robot OBJ2 to hold the virtual workpiece OBJ1 without interfering with the obstacle. The simulation device 100 of the embodiment then uses the generated holding path to simulate the operation of the virtual robot OBJ2 holding the virtual workpiece OBJ1, and fixes the target workpiece OBJ11 to the virtual hand OBJ21 based on the simulation. This allows the simulation device 100 of the embodiment to prevent the virtual robot OBJ2 from interfering with an obstacle and to fix the target workpiece OBJ11 to the virtual hand OBJ21 in an accurate position and orientation.

[0065] Furthermore, according to the robot system 1 of the embodiment, the simulation device 100 does not perform physical calculations on the virtual workpiece OBJ1, which is the target workpiece. Then, the simulation device 100 of the embodiment fixes the newly placed target workpiece OBJ11 to the virtual hand OBJ21. This prevents the simulation device 100 of the embodiment from erroneously determining that the target workpiece OBJ11 will interfere with the virtual workpiece OBJ1, which is the target workpiece.

[0066] Furthermore, according to the robot system 1 of the embodiment, the simulation device 100 regards the virtual workpiece OBJ12 as a type of obstacle. As a result, the simulation device 100 of the embodiment can generate a delivery path in which the target workpiece OBJ11 and the virtual robot OBJ2 do not interfere with the virtual workpiece OBJ12.

[0067] Furthermore, the virtual workpiece OBJ1 and the workpiece W may be placed inside a container. According to the robot system 1 of the embodiment, the simulation device 100 regards the container as a type of obstacle. This allows the simulation device 100 of the embodiment to generate a delivery path that prevents the target workpiece OBJ11 and the virtual robot OBJ2 from interfering with the container.

[0068] The above embodiment can be modified as follows.

[0069] The robot 200 and the virtual robot OBJ2 may be able to hold multiple workpieces W or multiple virtual workpieces OBJ1. In this case, the processor 101 may set the multiple virtual workpieces OBJ1 as target workpieces OBJ11. When there are multiple target workpieces OBJ11, the processor 101 fixes all of the target workpieces OBJ11 to the virtual hand OBJ21.

[0070] The virtual space S may include a 3D object OBJ that is an integrated set of multiple virtual workpieces OBJ1. The processor 101 may determine the virtual workpiece OBJ1 included in the 3D object OBJ as the target workpiece. In this case, the processor 101 separates the target workpiece OBJ11 from the 3D object OBJ and divides it into two 3D objects OBJ.

[0071] The simulation apparatus 100 may display the virtual space S on the display device 106. The simulation apparatus 100 may display a simulation performed in the virtual space S on the display device 106.

[0072] At least one of the position and shape of at least one of the virtual obstacles OBJ3-1 and OBJ3-2 may not be measured by the three-dimensional sensor 300, but may be registered in the simulation device 100 in advance.

[0073] In the above embodiment, the workpiece W and the virtual workpiece OBJ1 are placed inside a container. However, the workpiece W and the virtual workpiece OBJ1 may be placed in a predetermined location other than inside a container.

[0074] At least one of the simulation device 100 and the control device 400 may be mounted on the robot 200. The simulation device 100 and the control device 400 may be integrated. The three-dimensional sensor 300 may be mounted on the robot 200.

[0075] Each device in the embodiment may be composed of a plurality of devices, and each device in the embodiment may be realized using cloud computing.

[0076] The processor 101 may implement some or all of the processes implemented by the programs in the above embodiments by a hardware circuit configuration.

[0077] A program for implementing the processes of the embodiments may be transferred in a state where it is stored in a non-transitory computer-readable storage medium within the device. However, the device may also be transferred without the program stored therein. The program may then be transferred separately and written to the device. In this case, the program may be transferred by, for example, recording it on a removable non-transitory computer-readable storage medium or by downloading it via a network such as the Internet or a local area network (LAN).

[0078] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0079] The following additional notes are provided regarding the above-described embodiment and modifications.

[0080] [Supplementary Note 1] In a virtual space (S), a first simulation is executed showing an action in which a second virtual object (OBJ2) representing a robot (200) holds a first virtual object (OBJ1, OBJ11), the first virtual object (OBJ1, OBJ11) representing an object (W) placed at a predetermined location and transported by the robot (200), and based on the result of the first simulation, the first virtual object (OBJ1, OBJ11) is fixed to a hand (OBJ21) of the second virtual object (OBJ21); a generation unit (101) that performs a second simulation in which the second virtual object (OBJ21) performs an action of transporting the first virtual object (OBJ1, OBJ11) from the predetermined location, thereby generating a first movement path indicating the movement of the second virtual object (OBJ21) to transport the first virtual object (OBJ1, OBJ11) from the predetermined location so that the first virtual object (OBJ1, OBJ11) and the second virtual object (OBJ21) do not interfere with at least one of the second virtual object (OBJ21) and third virtual objects (OBJ3-1, OBJ3-2) representing obstacles (OBS1, OBS2).

[0081] [Supplementary Note 2] The simulation device (100) according to Supplementary Note 1, wherein the generation unit (101): when a result of the first simulation shows that the second virtual object (OBJ21) interferes with at least one of the second virtual object (OBJ21) and the third virtual objects (OBJ3-1, OBJ3-2), generates a second movement path indicating the movement of the second virtual object (OBJ21) without interfering with the second virtual object (OBJ21) and the third virtual objects (OBJ3-1, OBJ3-2), executes the first simulation using the second movement path, and fixes the first virtual object (OBJ1, OBJ11) to the hand (OBJ21) based on a result of the first simulation executed using the second movement path.

[0082] [Supplementary Note 3] The simulation device (100) according to Supplementary Note 1, wherein the generation unit (101) prevents physical calculation of one or more fourth virtual objects (OBJ1) representing an object (W) placed at the predetermined location, the fourth virtual object (OBJ1) corresponding to an object (W) to be transported from the predetermined location, places the first virtual objects (OBJ1, OBJ11) in the virtual space (S), and fixes the placed first virtual objects (OBJ1, OBJ11) to the hand (OBJ21).

[0083] [Supplementary Note 4] The simulation device (100) according to Supplementary Note 1, wherein the generation unit (101) generates a virtual object (OBJ12) representing an object (W) placed at the predetermined location other than the first virtual object (OBJ1, OBJ11), the virtual object (OBJ12) being a type of the obstacle (OBS1, OBS2).

[0084] [Supplementary Note 5] The simulation device (100) according to Supplementary Note 1, wherein the predetermined location is inside a container (OBS1, OBJ3-1), and the container (OBS1, OBJ3-1) is one type of the obstacle (OBS1, OBS2).

[0085] [Supplementary Note 6] A processor (101) included in a simulation device (100) executes a first simulation showing an action of a second virtual object (OBJ21) representing the robot (200) holding a first virtual object (OBJ1, OBJ11) representing an object (W) placed at a predetermined location (OBS1, OBJ3-1) and carried by the robot (200) in a virtual space (S), and fixes the first virtual object (OBJ1, OBJ11) to a hand (OBJ21) of the second virtual object (OBJ21) based on a result of the first simulation; a generating unit (101) that generates a first movement path indicating a movement of the second virtual object (OBJ21) that transports the first virtual object (OBJ1, OBJ11) from the predetermined place (OBS1, OBJ3-1) so that the first virtual object (OBJ1, OBJ11) and the second virtual object (OBJ21) do not interfere with at least one of the second virtual object (OBJ21) and third virtual objects (OBJ3-1, OBJ3-2) that represent obstacles (OBS1, OBS2).

[0086] 1 Robot system 100 Simulation device 101 Processor 102 ROM 103 RAM 104 Auxiliary storage device 105 Input device 106 Display device 107 Communication interface 108 Bus 200 Robot 201 Drive unit 202 Hand 300 Three-dimensional sensor 400 Control device OBJ1, OBJ12 Virtual work OBJ2 Virtual robot OBJ3-1, OBJ3-2 Virtual obstacle OBJ11 Target work OBJ21 Virtual hand OBS1, OBS2 Obstacle W Work

Claims

1. A simulation device including a generation unit that performs a first simulation in a virtual space showing an action of a second virtual object representing a robot holding a first virtual object, the first virtual object representing an object placed at a predetermined location to be transported by the robot, and performs a second simulation based on a result of the first simulation in which the first virtual object is fixed to the hand of the second virtual object and the second virtual object performs an action of transporting the first virtual object from the predetermined location, thereby generating a first movement path showing the movement of the second virtual object transporting the first virtual object from the predetermined location such that the first virtual object and the second virtual object do not interfere with at least one of the second virtual object and a third virtual object representing an obstacle.

2. The simulation device according to claim 1, wherein the generation unit generates a second movement path indicating the movement of the second virtual object that does not interfere with the second virtual object or the third virtual object when the result of the first simulation shows that the second virtual object interferes with at least one of the second virtual object and the third virtual object, executes the first simulation using the second movement path, and fixes the first virtual object to the hand based on the result of the first simulation executed using the second movement path.

3. The simulation device according to claim 1, wherein the generation unit prevents physical calculation of one or more fourth virtual objects representing an object placed at the predetermined location, the fourth virtual object corresponding to an object to be transported from the predetermined location, places the first virtual object in the virtual space, and fixes the placed first virtual object to the hand.

4. The simulation device according to claim 1, wherein the generation unit generates a virtual object representing an object placed at the predetermined location other than the first virtual object, which is a type of the obstacle.

5. A simulation device according to any one of claims 1 to 4, wherein the predetermined location is inside a container, and the container is one type of the obstacle.

6. A program that causes a processor included in a simulation device to function as a generation unit that performs a first simulation in a virtual space showing an action of a second virtual object representing a robot holding a first virtual object representing an object placed at a predetermined location to be transported by the robot, and based on the result of the first simulation, performs a second simulation in which the second virtual object fixes the first virtual object to the hand of the second virtual object and causes the second virtual object to transport the first virtual object from the predetermined location, thereby generating a first movement path showing the movement of the second virtual object to transport the first virtual object from the predetermined location such that the first virtual object and the second virtual object do not interfere with at least one of the second virtual object and a third virtual object representing an obstacle.

Images