Simulation device, computer readable storage medium, and simulation method

Abstract

The present invention relates to a simulation device, a computer-readable storage medium, and a simulation method. A structure is provided that can comprehensively simulate the behavior of a system that applies certain external forces to a workpiece. The simulation device includes: a first simulator that calculates the behavior of one or more workpieces arranged in a virtual space; a measurement processing unit that performs measurement processing on a virtual image generated by photographing the virtual space using a virtual camera arranged at an arbitrary viewpoint position in the virtual space; a second simulator that calculates the behavior of a robot that transports the workpiece arranged in the virtual space based on an operation instruction generated according to the measurement result of the workpiece by the measurement processing unit; and an image generation unit that generates an image that visualizes the virtual space.

Description

Technical Field

[0001] The present technology relates to a simulation device, a computer-readable storage medium, and a simulation method capable of inferring the behavior of a workpiece arranged in a virtual space. Background Art

[0002] Simulation using a computer is used in various technical fields. By using such simulation, various preliminary studies can be performed even when there is no real device. As an example of using such simulation in factory automation (FA), for example, Japanese Patent Publication No. 2016-042378 (Patent Document 1) discloses a comprehensive simulation of a mechanical system including a visual sensor in a real space corresponding to a virtual shooting unit.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-042378 Summary of the invention

[0006] Problems to be solved by the invention

[0007] The comprehensive simulation disclosed in Patent Document 1 is to virtually photograph a virtual workpiece arranged on a virtual conveyor by a virtual visual sensor. The position of the virtual workpiece is uniquely determined by the movement of the virtual conveyor. In reality, there are not only systems that transport workpieces by conveyors, but also systems that change the position of workpieces by certain actuators. Patent Document 1 does not disclose any simulation related to such a system.

[0008] One object of the present technology is to provide a structure that can comprehensively simulate the behavior of a system that applies certain external forces to a workpiece.

[0009] Technical means of solving problems

[0010] A simulation device in one embodiment includes: a first simulator that calculates the behavior of one or more workpieces configured in a virtual space; a measurement processing unit that performs measurement processing on a virtual image generated by photographing the virtual space using a virtual camera configured at an arbitrary viewpoint position in the virtual space; a second simulator that calculates the behavior of a robot that transports the workpieces configured in the virtual space based on an operation instruction generated according to the measurement results of the workpiece by the measurement processing unit; and an image generation unit that generates an image that visualizes the virtual space.

[0011] According to the present disclosure, it is possible to realize a simulation that associates the following processes: calculating the behavior of one or more workpieces arranged in a virtual space; performing measurement processing on a virtual image; and calculating the behavior of a robot that transports the workpiece arranged in the virtual space based on an operation instruction generated based on the measurement result of the workpiece. Thus, it is possible to realize a simulation that reproduces the behavior of each workpiece.

[0012] In the disclosure, the simulation device may also include: a third simulator that calculates the behavior of a workpiece transport device that applies an external force to one or more workpieces whose behavior is calculated by the first simulator. According to the present disclosure, the behavior of the workpiece transport device that determines the external force applied to each workpiece can be reproduced, so the behavior of each workpiece can be reproduced more accurately.

[0013] In the disclosure, the simulation device may also include: a virtual camera unit that manages a virtual camera configured in a virtual space and generates a virtual image according to the viewpoint position and field of view of the virtual camera. According to the disclosure, after configuring the virtual camera at any position in the virtual space, image measurement can be reproduced, so it is also possible to achieve evaluation with different positions of the virtual camera.

[0014] In the above disclosure, the first simulator may calculate and update the position and posture of each workpiece at each predetermined time step. According to the present disclosure, the behavior of the workpiece can be reproduced at each time step.

[0015] In the disclosure, the first simulator can also respond to a change operation of the position or characteristic of any workpiece at any time, and continue to calculate the behavior of one or more workpieces according to the changed position or characteristic. According to the disclosure, arbitrary trial and error such as making the behavior of a specific workpiece different can be easily realized.

[0016] According to another embodiment of the present technology, a simulation program for causing a computer to perform simulation is provided. The simulation program causes the computer to perform the following steps: calculating the behavior of one or more workpieces arranged in a virtual space; performing measurement processing on a virtual image generated by photographing the virtual space using a virtual camera arranged at an arbitrary viewpoint position in the virtual space; calculating the behavior of a robot that transports the workpieces arranged in the virtual space based on an operation instruction generated based on a measurement result obtained by performing the measurement processing; and generating an image that visualizes the virtual space.

[0017] According to the present disclosure, it is possible to realize a simulation that associates the following processes: calculating the behavior of one or more workpieces arranged in a virtual space; performing measurement processing on a virtual image; and calculating the behavior of a robot that transports the workpiece arranged in the virtual space based on an operation instruction generated based on the measurement result of the workpiece. Thus, it is possible to realize a simulation that reproduces the behavior of each workpiece.

[0018] According to another embodiment of the present technology, a simulation method executed in a computer is provided. The simulation method includes the following steps executed by the computer: calculating the behavior of one or more workpieces arranged in a virtual space; performing measurement processing on a virtual image generated by photographing the virtual space using a virtual camera arranged at an arbitrary viewpoint position in the virtual space; calculating the behavior of a robot that transports the workpieces arranged in the virtual space based on an operation instruction generated according to a measurement result obtained by executing the measurement processing; and generating an image that visualizes the virtual space.

[0019] According to the present disclosure, it is possible to realize a simulation that associates the following processes: calculating the behavior of one or more workpieces arranged in a virtual space; performing measurement processing on a virtual image; and calculating the behavior of a robot that transports the workpiece arranged in the virtual space based on an operation instruction generated based on the measurement result of the workpiece. Thus, it is possible to realize a simulation that reproduces the behavior of each workpiece.

[0020] Effects of the Invention

[0021] According to the present technology, it is possible to provide a structure that can comprehensively simulate the behavior of a system that applies a certain external force to a workpiece. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram showing an application example of the simulation device of this embodiment.

[0023] Figure 2 This is an external view showing an example of a workpiece transport system serving as a simulation target of the simulation device of the present embodiment.

[0024] Figure 3 It is used to realize Figure 2 The schematic diagram shown is used to explain the control function of the workpiece transport system.

[0025] Figure 4 This is a schematic diagram showing an example of a hardware configuration for realizing a simulation device according to the present embodiment.

[0026] Figure 5 This is a schematic diagram showing an example of a functional configuration of a simulation device for realizing the present embodiment.

[0027] Figure 6 FIG. 1 is a diagram showing an example of a display screen showing a virtual space that visualizes the simulation results executed by the simulation device according to the present embodiment.

[0028] Figure 7 This is a diagram showing an example of a temporal change of an object representing a workpiece in a simulation result executed by the simulation device of the present embodiment.

[0029] Figure 8 This is a diagram showing an example of a display screen related to image measurement in a simulation result executed by the simulation device according to the present embodiment.

[0030] Fig. 9 This is a diagram showing an example of a physical simulation executed by the simulation device according to this embodiment.

[0031] Fig.10 This is a flowchart showing the flow of processing executed in the simulation device of this embodiment.

[0032] Fig.11 This is a flowchart showing the flow of processing related to parameter adjustment using the simulation device according to the present embodiment.

[0033] Explanation of symbols

[0034] 1: Simulation device

[0035] 2: Workpiece transport system

[0036] 4: Artifacts

[0037] 10: Workpiece transport device

[0038] 12: Workpiece supply department

[0039] 14: Workpiece conveying unit

[0040] 16: Transport surface

[0041] 18: Feed controller

[0042] 20: Image measuring device

[0043] 22: Camera

[0044] 24: Artifact model information

[0045] 30: Robot

[0046] 32: Adsorption part

[0047] 40: Robot controller

[0048] 50: Conveyor

[0049] 100: Information processing device

[0050] 102: Processor

[0051] 104: Main memory

[0052] 105: Virtual Space Information

[0053] 106: Operation unit

[0054] 108: Output unit

[0055] 110: Network interface

[0056] 112: Optical drive

[0057] 114: Recording Media

[0058] 116: Local communication interface

[0059] 118: Internal bus

[0060] 120: Storage

[0061] 122: Physical Simulation Program

[0062] 124: Physical simulation parameters

[0063] 126: Image measurement program

[0064] 128: Image measurement parameters

[0065] 130: Robot Simulation Program

[0066] 132: Robot parameters

[0067] 134: Comprehensive Program

[0068] 150: Virtual space information management module

[0069] 152: Workpiece transport simulator

[0070] 154: Artifact Behavior Simulator

[0071] 156: Virtual Camera Module

[0072] 158: Image measurement module

[0073] 160: Robot Simulator

[0074] 162: Artifact Tracking Module

[0075] 164: Visualization Tools

[0076] 200, 210, 230: Display screen

[0077] 202: Virtual Space

[0078] 204, 206, 208: Object

[0079] 220: Virtual Image

[0080] 222: Identification results

[0081] 224: Setting Window

[0082] 232: Cursor DETAILED DESCRIPTION

[0083] The embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, the same or corresponding parts in the drawings are denoted by the same reference numerals and their description will not be repeated.

[0084] <A. Application examples>

[0085] First, an example of a scenario to which the present invention is applicable will be described.

[0086] The simulation device 1 of this embodiment can also comprehensively simulate the behavior of a system that applies certain external forces to a workpiece. Typically, it can be applied to the following application, that is, applying external forces by a workpiece conveying device as described later to convey the workpiece, performing measurement processing on a virtual image obtained by photographing the conveyed workpiece, and picking up the workpiece by a robot based on the results of the measurement processing.

[0087] Figure 1 Schematic diagram showing an application example of the simulation device 1 of this embodiment. Figure 1 The simulation device 1 includes virtual space information 105, which defines the virtual space and objects arranged in the virtual space. The content of the virtual space information 105 is updated appropriately by calculating the behavior of each object in the virtual space.

[0088] The simulation device 1 includes a workpiece behavior simulator 154 that calculates the behavior of one or more workpieces arranged in the virtual space. The workpiece behavior simulator 154 corresponds to the first simulator. The workpiece behavior simulator 154 estimates the behavior of each workpiece in the virtual space based on the workpiece parameters.

[0089] The simulation device 1 includes an image measurement module 158 that performs measurement processing on a virtual image generated by photographing the virtual space using a virtual camera arranged at an arbitrary viewpoint position in the virtual space. The image measurement module 158 is equivalent to a measurement processing unit. That is, the image measurement module 158 is a visual sensor that takes the virtual space as an object.

[0090] The simulation device 1 includes a robot simulator 160 that calculates the behavior of a robot that transports a workpiece arranged in a virtual space based on an operation command generated based on a measurement result of the workpiece by the image measurement module 158. The robot simulator 160 corresponds to a second simulator.

[0091] The simulation device 1 includes a visualizer 164 that generates an image that visualizes the virtual space. The visualizer 164 corresponds to an image generation unit.

[0092] By linking these components and modules to each other, even if the actual system does not exist, it is possible to accurately infer the behavior of the workpiece and each device that may be generated when the workpiece is processed in the actual system.

[0093] <B. Example of applicable system>

[0094] Next, an example of a system to be simulated by the simulation device 1 according to the present embodiment will be described.

[0095] Figure 2 2 is an external view showing an example of a workpiece transport system 2 that is a simulation target of the simulation device 1 according to the present embodiment. Figure 2 The workpiece transport system 2 includes a workpiece transport device 10 , an image measuring device 20 connected to a camera 22 , a robot 30 , and a conveyor 50 .

[0096] The workpiece conveying device 10 includes a workpiece supply unit 12 and a workpiece conveying unit 14. The workpiece supply unit 12 holds a plurality of workpieces 4 in a storage unit, and vibrates periodically or according to certain events, thereby supplying a part or all of the workpieces held in the storage unit to a conveying surface 16 of the workpiece conveying unit 14. The workpiece conveying unit 14 can also vibrate periodically or according to certain events to apply external force to the workpieces 4 arranged on the conveying surface 16, thereby changing the arrangement position and posture of each workpiece 4.

[0097] The camera 22 is arranged so that at least a part of the conveying surface 16 of the workpiece conveying unit 14 enters the field of view. The vision measuring device 20 performs measurement processing on the image acquired by the camera 22, thereby measuring the position and posture of the workpiece 4 arranged on the conveying surface 16. The workpiece conveying unit 14 vibrates periodically or in response to an event, thereby appropriately changing the position and posture of each workpiece 4, and improving the accuracy of appropriately measuring the workpiece 4 by the vision measuring device 20.

[0098] The robot 30 adsorbs any workpiece 4 measured by the adsorption unit 32 disposed at the front end of the arm based on the measurement result (position and posture) of the workpiece 4 by the vision measuring device 20. Then, the robot 30 transports the adsorbed (picked) workpiece 4 to the conveyor 50 and arranges (places) it at a predetermined position on the conveyor 50. The operation of the robot 30 in the workpiece transport system 2 is also called a pick-up and place operation.

[0099] The conveying surface of the conveyor 50 moves basically at a constant speed, and the workpiece 4 arranged on the conveying surface is conveyed to a processing step on the downstream side at a constant speed.

[0100] Figure 3 It is used to realize Figure 2 The control function of the workpiece transport system 2 shown in FIG. Figure 3 The workpiece conveying system 2 has an image measuring device 20 connected to a camera 22. The image measuring device 20 performs measurement processing on an image captured by the camera 22 based on preset workpiece model information 24, thereby outputting workpiece measurement results (including measurement results of the position and posture of each workpiece).

[0101] The workpiece transport system 2 further includes a robot controller 40. The robot controller 40 acquires the position and posture of the workpiece to be picked up based on the workpiece measurement result from the vision measuring device 20, and gives an operation command to the robot 30 based on the acquired information.

[0102] The workpiece conveying system 2 further includes a feed controller 18. The feed controller 18 drives the workpiece supply unit 12 as a vibration generating actuator regularly or according to certain events. Similarly, the feed controller 18 drives the workpiece conveying unit 14 as a vibration generating actuator regularly or according to certain events.

[0103] As a typical example, the simulation device 1 of this embodiment will be described below to reproduce the Figure 2 and Figure 3 The operation or behavior of each device of the workpiece transport system 2 and the workpiece 4 will be described.

[0104] <C. Hardware structure>

[0105] Next, an example of the hardware configuration of the simulation device 1 according to the present embodiment will be described.

[0106] Figure 4 is a schematic diagram showing an example of a hardware configuration for realizing the simulation device 1 of the present embodiment. Figure 4 The information processing device 100 shown is implemented by executing necessary programs.

[0107] The information processing device 100 includes the following as main components: a processor 102 that executes an operating system (OS) and various programs described later; a main memory 104 that provides a work area for storing data required for executing programs in the processor 102; an operation unit 106 (operation receiving unit) that receives user operations such as a keyboard or a mouse; an output unit 108 that outputs processing results such as a display, various indicators, and a printer; a network interface 110 that connects to a network; an optical drive 112; a local communication interface 116 that communicates with an external device; and a storage 120. These components are connected to each other via an internal bus 118 or the like so that data can be communicated.

[0108] The information processing device 100 has an optical drive 112 that reads various programs from a computer-readable recording medium 114 such as an optical recording medium (eg, a digital versatile disc (DVD)) that stores computer-readable programs non-disposably and installs the programs in a storage device 120 or the like.

[0109] Various programs executed by the information processing device 100 may be installed via the computer-readable recording medium 114, or may be installed by downloading from a server device on the network, etc. In addition, the program related to the evaluation of functional safety of this embodiment may also be implemented in the form of a part of a module provided by the OS.

[0110] The storage 120 includes, for example, a hard disk drive (HDD) or a flash solid state drive (SSD), and stores programs executed by the processor 102. Specifically, the storage 120 includes a physical simulation program 122, an image measurement program 126, a robot simulation program 130, and an integrated program 134 as simulation programs for realizing the simulation of the present embodiment.

[0111] The physical simulation program 122 calculates the behavior of the workpiece transport device 10 and the behavior of the workpiece 4 affected by the workpiece transport device 10. The physical simulation program 122 is given a physical simulation parameter 124, which includes a parameter that specifies the behavior of the workpiece transport device 10 and a parameter that specifies the weight or shape of the workpiece 4. The value of the physical simulation parameter 124 can also be appropriately changed by user operation or the integrated program 134.

[0112] Computer Aided Design (CAD) data of the workpiece transport device 10 and the workpiece 4 may be used as the physical simulation parameters 124. By using the CAD data, the behavior of the actual device can be reproduced more accurately.

[0113] The image measurement program 126 performs substantially the same measurement processing as the image measuring device 20 on the virtually captured image reflecting the position and posture of each workpiece 4 calculated by the physical simulation program 122. The image measurement program 126 is given image measurement parameters 128 including parameters required for the measurement processing in the image measuring device 20.

[0114] The robot simulation program 130 functions as a simulator for reproducing the behavior of the robot 30. The robot simulation program 130 reproduces the picking and placing operation of the robot 30 on the workpiece 4 based on the measurement results calculated by executing the image measurement program 126. The robot simulation program 130 is given a robot parameter 132 including parameters required for reproducing the behavior of the robot 30.

[0115] The integrated program 134 executes processing for linking the physical simulation program 122, the image measurement program 126, and the robot simulation program 130. Specifically, the integrated program 134 typically generates and updates the virtual space information 105 describing the state of the object in the virtual space on the main memory 104. The physical simulation program 122, the image measurement program 126, and the robot simulation program 130 refer to the virtual space information 105 to execute the processing of each simulation, and reflect the necessary information in the execution results to the virtual space information 105. Through the function provided by the integrated program 134, the behavior and processing of the workpiece conveying system 2 including the workpiece conveying device 10, the image measurement device 20, and the robot 30 are reproduced.

[0116] Figure 4 , an example of realizing the simulation device 1 using a single information processing device 100 is shown, but a plurality of information processing devices may be associated to realize the simulation device 1. In this case, the information processing device 100 may be used to execute a portion of the processing required to realize the simulation device 1, and the remaining processing may be executed by a server (cloud) on a network or the like.

[0117] Figure 4This shows an example of implementing the simulation device 1 by executing one or more programs by the processor 102, but a dedicated integrated circuit (ASIC) or a field programmable gate array (FPGA) may be used to install part of the processing and functions required to implement the simulation device 1.

[0118] <D. Functional structure>

[0119] Next, an example of the functional configuration of the simulation device 1 according to the present embodiment will be described. Figure 5 1 is a schematic diagram showing an example of a functional structure of the simulation device 1 for realizing the present embodiment. Typically, Figure 5 Each functional structure shown is realized by the processor 102 of the information processing device 100 executing a program (physical simulation program 122 , image measurement program 126 , robot simulation program 130 , integrated program 134 , etc.).

[0120] Reference Figure 5 The simulation device 1 includes a virtual space information management module 150, a workpiece transport simulator 152, a workpiece behavior simulator 154, a virtual camera module 156, an image measurement module 158, a robot simulator 160, a workpiece tracking module 162 and a visualization tool 164 as its functional structure.

[0121] The virtual space information management module 150 is implemented by executing the integrated program 134 ( Figure 4 ) is implemented by managing virtual space information 105, which specifies information such as the position and posture of each object in the simulated virtual space.

[0122] The workpiece transport simulator 152 is equivalent to a third simulator that calculates the behavior of the workpiece transport device 10, and the workpiece transport device 10 applies an external force to one or more workpieces 4 whose behavior is calculated by the workpiece behavior simulator 154. The workpiece transport simulator 152 is implemented by executing the physical simulation program 122, and reproduces the behavior of the workpiece transport device 10 through the physical simulation parameters 124. More specifically, the workpiece transport simulator 152 reproduces the behavior of the workpiece supply unit 12 and the workpiece transport unit 14 that constitute the workpiece transport device 10. The information on the behavior of the workpiece transport device 10 calculated by the workpiece transport simulator 152 is reflected in the virtual space information 105.

[0123] The workpiece behavior simulator 154 is equivalent to a first simulator that calculates the behavior of one or more workpieces 4 arranged in the virtual space. Specifically, the workpiece behavior simulator 154 is implemented by executing the physical simulation program 122, and calculates and updates the position and posture of each workpiece 4 arranged in the virtual space at each specified time step based on the information of the behavior of the workpiece conveying device 10 simulated by the workpiece conveying simulator 152. When the workpiece behavior simulator 154 calculates the position and posture of each workpiece 4 at each time step, it calculates the weight, position, posture, moving speed, moving direction, etc. of each workpiece 4 at each time step. The position and posture of each workpiece 4 are calculated based on the workpiece parameters arbitrarily set by the user or the like. The information on the position and posture of the workpiece 4 calculated by the workpiece behavior simulator 154 is reflected in the virtual space information 105.

[0124] The virtual camera module 156 is equivalent to a virtual camera unit, which manages the virtual camera configured in the virtual space and generates a virtual image according to the viewpoint position and field of view of the virtual camera. The virtual camera module 156 is implemented by executing the image measurement program 126, and generates a virtual image by photographing the virtual space using a virtual camera configured at an arbitrary viewpoint position in the virtual space. The user can configure the virtual camera at an arbitrary position, and can also arbitrarily change the field of view of the virtual camera.

[0125] The image measurement module 158 corresponds to a measurement processing unit that performs measurement processing on a virtual image generated by photographing a virtual space using a virtual camera arranged at an arbitrary viewpoint position in the virtual space. The image measurement module 158 is implemented by executing the image measurement program 126, performs measurement processing specified by the image measurement parameter 128 on the virtual image from the virtual camera module 156, and outputs the measurement result.

[0126] The robot simulator 160 corresponds to a second simulator that calculates the behavior of the robot 30 that carries the workpiece 4 arranged in the virtual space based on the operation command generated based on the measurement result of the workpiece by the measurement processing unit. More specifically, the robot simulator 160 is implemented by executing the robot simulation program 130, and reproduces the behavior of the robot 30 based on the measurement result from the image measurement module 158. The information on the behavior of the robot 30 calculated by the robot simulator 160 is reflected in the virtual space information 105.

[0127] The workpiece tracking module 162 is implemented by executing the robot simulation program 130, and tracks the workpiece 4 picked up and placed by the robot 30 in the virtual space based on the behavior information of the robot 30 from the robot simulator 160. The position information (workpiece position) of the workpiece 4 tracked by the workpiece tracking module 162 is reflected in the virtual space information 105.

[0128] The visualization tool 164 corresponds to an image generation unit that generates an image that visualizes the virtual space. The visualization tool 164 visualizes the behavior of each object (the workpiece transport device 10 , the robot 30 , the workpiece 4 , etc.) in the virtual space based on the virtual space information 105 managed by the virtual space information management module 150 .

[0129] pass Figure 5 The functions shown above are linked to each other, and the behavior of the system to be simulated (for example, the workpiece transport system 2 ) can be reproduced with high accuracy.

[0130] <E.Simulation example>

[0131] Next, an example of a simulation provided by the simulation device 1 of the present embodiment will be described.

[0132] Figure 6 2 is a diagram showing an example of a display screen 200 showing a virtual space that visualizes the simulation results executed by the simulation device 1 of the present embodiment. Figure 6 On the display screen 200, there is expressed a virtual space 202 in which a simulation is performed, and there are expressed an object 204 representing a robot configured in the virtual space 202, an object 206 corresponding to a plurality of workpieces, and an object 208 which simply represents the conveying surface 16 of the workpiece conveying device 10.

[0133] The objects 204, 206, and 208 change their positions and postures based on the position and posture information (time change) calculated at each time step by the simulation device 1. As a result, the objects 204, 206, and 208 appear to move in the virtual space 202.

[0134] Figure 7 This is a diagram showing an example of a temporal change of the object 206 representing the workpiece 4 in the simulation result executed by the simulation device 1 of the present embodiment. Figure 7 (a)~ Figure 7 In (c), a plurality of bolts and nuts are shown as the workpiece 4 . Figure 7 (a)~ Figure 7 In the visualized result shown in (c) , the position of the object 206 corresponding to each workpiece 4 changes over time.

[0135] In this embodiment, the position and posture of each workpiece 4 are calculated individually by physical simulation, and the influence of collision with other workpieces 4 is also reflected. The trajectory of each workpiece 4 will also change according to the mass of each workpiece 4 and the magnitude of the external force applied.

[0136] The results, for example, Figure 7 (a)~ Figure 7 As shown in (c), the workpieces 4 that are initially aggregated to a certain extent become scattered over time. In this way, the simulation device 1 of this embodiment calculates the behavior of the workpieces 4 individually by physical simulation, so that high-precision simulation can be achieved.

[0137] Figure 8 FIG. 1 is a diagram showing an example of a display screen related to image measurement in a simulation result executed by the simulation device 1 of the present embodiment. Figure 8 The display screen 210 includes an example of a virtual image 220 that is the measurement processing target of the image measurement module 158. In the virtual image 220, a recognition result 222 obtained by the measurement processing of the virtual image 220 is displayed.

[0138] The display screen 210 includes a setting window 224 for setting image measurement parameters 128 related to the measurement process. The user can also set arbitrary image measurement parameters 128 by operating the setting window 224. Regarding the details of the image measurement parameters 128, the parameters used in the general measurement process can be adopted, and therefore, each parameter will not be described in detail.

[0139] By viewing Figure 6 to Figure 8 The visualized results shown enable the user to confirm the behavior reproduced with high accuracy even if the system (eg, the workpiece transport system 2 ) that is the simulation target does not exist in reality.

[0140] <F. Physical simulation>

[0141] Next, the physical simulation employed in the simulation device 1 of the present embodiment will be described.

[0142] The simulation device 1 of this embodiment can calculate the time behavior of each workpiece 4 even when there are a plurality of workpieces 4. When there are a plurality of workpieces 4, the amount of calculation increases, so the amount of calculation can be reduced using a known algorithm.

[0143] As an example of such an algorithm for reducing the amount of computation, the Hierarchical Approximate Convex Decomposition (HACD) algorithm can be used. According to the HACD algorithm, by convexly decomposing the object to be the subject of physical simulation, the physical computation can be accelerated by simplifying the object without changing the shape of the object.

[0144] The physical simulation adopted in the simulation device 1 of the present embodiment can estimate the behavior of each workpiece 4. Furthermore, the position or characteristics of any workpiece 4 can be arbitrarily changed at any time.

[0145] Fig. 9 FIG. 1 is a diagram showing an example of a physical simulation performed by the simulation device 1 of the present embodiment. Fig. 9 On the display screen 230 shown in (a), the object 206 corresponding to any workpiece 4 can be selected by the cursor 232. By selecting any object 206, the entire simulation can also be temporarily stopped. Fig. 9 As shown in (b) of FIG. 1 , the selected object 206 is arranged at an arbitrary position using the cursor 232 , thereby being able to continue the simulation from the newly arranged position.

[0146] That is, the workpiece behavior simulator 154 (first simulator) responds to a change operation of the position or characteristic of any workpiece at any time, and continues to calculate the behavior of one or more workpieces 4 based on the changed position or characteristic.

[0147] In this way, in the physical simulation adopted by the simulation device 1 of this embodiment, the behavior of any workpiece 4 among multiple workpieces 4 can be changed at any time, so by configuring any workpiece 4 at any position in the virtual space, the behavior of each device can be confirmed in advance.

[0148] <G. Processing Flow>

[0149] Next, the flow of processing executed in the simulation device 1 of the present embodiment will be described.

[0150] Fig.10 : is a flowchart showing the process flow executed in the simulation device 1 of this embodiment. Typically, Fig.10 Each of the steps shown is realized by the processor 102 of the information processing device 100 executing a program (such as the physical simulation program 122 , the image measurement program 126 , the robot simulation program 130 , and the integrated program 134 ).

[0151] Reference Fig.10First, the information processing device 100 reads various settings specified by the user (physical simulation parameters 124, image measurement parameters 128, robot parameters 132, CAD data and other setting information) (step S100), and generates initial data required for simulation (step S102).

[0152] When the start of the simulation is instructed (YES in step S104 ), the information processing device 100 executes a process of calculating the behavior of one or more workpieces 4 arranged in the virtual space.

[0153] Specifically, the information processing device 100 calculates the state (position, speed, acceleration, posture, etc.) of the workpiece conveying device 10 at the next time step (step S106). Then, the information processing device 100 calculates the external force applied to each workpiece 4 based on the state of the workpiece conveying device 10 at the next time step (step S108), and calculates the state (position, speed, acceleration, posture, etc.) of each workpiece 4 based on the calculated external force (step S110).

[0154] Typically, the processing of steps S106 to S110 is provided by execution of the physics simulation program 122 .

[0155] Then, the information processing device 100 performs measurement processing on the virtual image generated by photographing the virtual space using a virtual camera configured at an arbitrary viewpoint position in the virtual space. More specifically, the information processing device 100 generates a virtual image by virtual photography of the virtual camera configured in the virtual space (step S112), and performs measurement processing on the generated virtual image (step S114). The measurement processing includes processing such as shape search (search) of the virtual image. Then, the information processing device 100 generates or updates instructions to the robot based on the measurement results obtained through the measurement processing (step S116).

[0156] The processing of step S114 and step S116 is provided by execution of the image measurement program 126 .

[0157] Then, the information processing device 100 performs the following processing, that is, based on the operation command generated according to the measurement result obtained by executing the measurement processing, the behavior of the robot 30 that transports the workpiece 4 arranged in the virtual space is calculated. More specifically, the information processing device 100 calculates the state (position, speed, acceleration, posture, etc.) of the robot 30 based on the command to the robot generated or updated in step S116 (step S118). At the same time, if the calculated state of the robot 30 is in a predetermined state (pickup position or placement position) (step S120), the information processing device 100 sets the target workpiece 4 to the pickup state or the placement state (step S122).

[0158] The processing of steps S116 to S122 is provided by the execution of the robot simulation program 130 .

[0159] Finally, the information processing device 100 generates and outputs an image that visualizes the virtual space based on the information reflecting the state of each device calculated in the above step (the virtual space information 105 that describes the state of objects in the virtual space) (step S124).

[0160] The information processing device 100 determines whether a predetermined end condition or stop condition is satisfied (step S126). If neither the predetermined end condition nor the stop condition is satisfied (No in step S126), the processing from step S106 onwards is repeated.

[0161] On the other hand, when a predetermined end condition or stop condition is satisfied (YES in step S126 ), information processing apparatus 100 ends the processing.

[0162] <H. Application examples of simulation devices>

[0163] By using the simulation device 1 of the present embodiment as described above, even when there is no actual device (for example, before the introduction of equipment or during construction), it is possible to adjust the optimal parameters in advance. Furthermore, it can also be used for analyzing the cause of the robot stopping due to factory errors or some problems.

[0164] Hereinafter, as an application example of the simulation device 1 of the present embodiment, an example of adjusting various parameters will be described. As the object of such parameter adjustment, the image measurement parameter 128 is assumed, but other parameters may also be the object of adjustment. Generally, regarding image measurement, the parameter value greatly affects the recognition result, so it is important to optimize the image measurement parameter 128.

[0165] As a method of adjusting such parameters, by repeatedly changing the parameter values ​​as appropriate and evaluating the results obtained by executing the simulation (such as the transfer rate or non-recognition rate of the workpiece 4), the parameters that obtain the best results can be set as the optimal values.

[0166] Fig.11 1 is a flowchart showing the process flow related to parameter adjustment using the simulation device 1 of this embodiment. Typically, Fig.11 Each of the steps shown is realized by the processor 102 of the information processing device 100 executing a program (such as the physical simulation program 122 , the image measurement program 126 , the robot simulation program 130 , and the integrated program 134 ).

[0167] Reference Fig.11The information processing device 100 accepts parameter candidates as adjustment objects and the setting of the parameter adjustment range of the adjustment objects (step S200). Then, the information processing device 100 sets one or more parameter values ​​to any value of the corresponding parameter adjustment range from the parameter candidates as adjustment objects accepted in step S200 (step S202).

[0168] Then, the information processing device 100 performs the following operation based on the parameters set in step S202 and other fixed parameters. Fig.10 The simulation shown (step S204). If the content of the simulation reaches a predetermined state (yes in step S206), the information processing device 100 stops the execution of the simulation and calculates an evaluation value for the simulation result (step S208).

[0169] Next, the information processing device 100 determines whether all values ​​of the corresponding parameter adjustment range have been selected for the parameter candidates accepted as adjustment objects in step S200 (step S210). That is, it is determined whether all the parameter candidates accepted as adjustment objects in step S200 have been set to all values ​​of the set parameter adjustment range, and the execution of the simulation has been completed.

[0170] If not all values ​​of the corresponding parameter adjustment range have been selected (No in step S210 ), the information processing device 100 repeats the processing from step S202 onwards.

[0171] On the other hand, if all values ​​of the corresponding parameter adjustment range have been selected (Yes in step S210), the information processing device 100 outputs the parameter value corresponding to the highest evaluation value among the evaluation values ​​calculated in step S208 as the adjustment result (step S212). Then, the parameter adjustment process ends.

[0172] Fig.11 The process shown in the flowchart is repeated a plurality of times depending on the number of parameter candidates to be adjusted and the magnitude of the parameter adjustment width for each parameter candidate.

[0173] By using this parameter adjustment process, even if an actual device does not exist, it is possible to perform parameter adjustment in advance.

[0174] <I. Notes>

[0175] The present embodiment described above includes the following technical ideas.

[0176] [Structure 1]

[0177] A simulation device, comprising:

[0178] The first simulator 154 calculates the behavior of one or more workpieces 4 arranged in the virtual space 202;

[0179] The measurement processing unit 158 ​​performs measurement processing on a virtual image 220 generated by photographing the virtual space using a virtual camera arranged at an arbitrary viewpoint position in the virtual space;

[0180] The second simulator 160 calculates the behavior of the robot 30 that transports the workpiece arranged in the virtual space based on the operation instruction generated according to the measurement result of the workpiece by the measurement processing unit; and the image generation unit 164 generates an image that visualizes the virtual space.

[0181] [Structure 2]

[0182] The simulation device according to structure 1 further includes:

[0183] The third simulator 152 calculates the behavior of the workpiece transport device 10 , and the workpiece transport device 10 applies an external force to the one or more workpieces 4 whose behavior is calculated by the first simulator.

[0184] [Structure 3]

[0185] The simulation device according to structure 1 or 2 further includes:

[0186] The virtual camera unit 156 manages the virtual camera arranged in the virtual space, and generates the virtual image according to the viewpoint position and the field of view of the virtual camera.

[0187] [Structure 4]

[0188] A simulation device according to any one of structures 1 to 3, wherein

[0189] The first simulator calculates and updates the position and posture of each workpiece at each predetermined time step.

[0190] [Structure 5]

[0191] A simulation device according to any one of structures 1 to 4, wherein

[0192] The first simulator responds to a change operation to a position or characteristic of any workpiece at any time, and continues to calculate the behavior of the one or more workpieces based on the changed position or characteristic.

[0193] [Structure 6]

[0194] A simulation program is a simulation program 122, 126, 130, 134 for causing a computer to perform a simulation, wherein the simulation program causes the computer 100 to perform:

[0195] Step S110, calculating the behavior of one or more artifacts configured in the virtual space;

[0196] Step S114, performing measurement processing on a virtual image 220 generated by photographing the virtual space using a virtual camera configured at an arbitrary viewpoint position in the virtual space;

[0197] Step S118, calculating a behavior of a robot that carries out conveyance of the workpiece arranged in the virtual space based on an operation instruction generated according to a measurement result obtained by executing the measurement process; and

[0198] Step S124, generating an image that visualizes the virtual space.

[0199] [Structure 7]

[0200] A simulation method is a simulation method executed in a computer 100, the simulation method comprising:

[0201] Step S110, calculating the behavior of one or more artifacts configured in the virtual space;

[0202] Step S114, performing measurement processing on a virtual image 220 generated by photographing the virtual space using a virtual camera configured at an arbitrary viewpoint position in the virtual space;

[0203] Step S118, calculating a behavior of a robot that carries out conveyance of the workpiece arranged in the virtual space based on an operation instruction generated according to a measurement result obtained by executing the measurement process; and

[0204] Step S124, generating an image that visualizes the virtual space.

[0205] <J. Advantages>

[0206] The simulation device 1 according to the present embodiment can also comprehensively simulate the behavior of a system that applies a certain external force to a workpiece.

[0207] The embodiments disclosed this time should be considered as illustrative in all aspects and not restrictive. The scope of the present invention is indicated by the claims rather than the description above, and is intended to include all modifications within the meaning and scope equivalent to the claims.

Claims

1. A simulation device, comprising: An artifact behavior simulator that calculates the behavior of one or more artifacts configured in a virtual space through physical simulation; A workpiece transport simulator that calculates the external force applied by the workpiece transport device to the one or more workpieces through the physical simulation based on the behavior of the workpiece transport device arranged in the virtual space, so as to change the arrangement position and posture of the one or more workpieces; a measurement processing unit that performs measurement processing on the virtual space; a robot simulator that calculates a behavior of a robot that carries out conveyance of the one or more workpieces arranged in the virtual space based on an operation instruction generated according to a measurement result of the one or more workpieces by the measurement processing unit; as well as The image generation unit generates an image that visualizes the virtual space.

2. The simulation device according to claim 1, wherein The workpiece transport simulator calculates the behavior of the workpiece transport device, and the workpiece transport device applies an external force to the one or more workpieces whose behavior is calculated by the workpiece behavior simulator. The workpiece behavior simulator calculates the behavior of the one or more workpieces according to the external force applied to the one or more workpieces.

3. The simulation device according to claim 1 or 2, wherein The workpiece behavior simulator calculates the position and posture of each workpiece at each specified time step and updates the position and posture.

4. The simulation device according to claim 1 or 2, wherein The workpiece behavior simulator responds to a change operation on the position or characteristic of any workpiece at any time, and continues to calculate the behavior of the one or more workpieces based on the changed position or characteristic.

5. A computer-readable storage medium comprising a simulation program for causing a computer to perform a simulation, wherein the simulation program causes the computer to perform the following steps: Calculating the behavior of one or more artifacts configured in the virtual space through physical simulation; Based on the behavior of the workpiece transport device arranged in the virtual space, the external force applied by the workpiece transport device to the one or more workpieces is calculated by the physical simulation to change the arrangement position and posture of the one or more workpieces; performing measurement processing on the virtual space; calculating a behavior of a robot that carries out conveyance of the one or more workpieces arranged in the virtual space based on an operation instruction generated according to a measurement result of the one or more workpieces obtained by executing the measurement process; as well as An image visualizing the virtual space is generated.

6. A simulation method is a simulation method executed in a computer, the simulation method comprising the following steps executed by the computer: Calculating the behavior of one or more artifacts configured in the virtual space through physical simulation; Based on the behavior of the workpiece transport device arranged in the virtual space, the external force applied by the workpiece transport device to the one or more workpieces is calculated by the physical simulation to change the arrangement position and posture of the one or more workpieces; performing measurement processing on the virtual space; calculating a behavior of a robot that carries out conveyance of the one or more workpieces arranged in the virtual space based on an operation instruction generated according to a measurement result of the one or more workpieces obtained by executing the measurement process; as well as An image visualizing the virtual space is generated.

Images