Simulation device and simulation program

Abstract

The present invention provides a simulation device capable of determining a proper illumination condition at a site in a short time. The simulation device simulates illumination, in a three-dimensional real space, of a machine system including a workpiece processed by a machine and the machine provided in a real space, as illumination, in a three-dimensional virtual space, of a machine system provided in the virtual space, and includes an acquisition unit, a setting unit, and a reproduction unit. The acquisition unit acquires information on brightness of a real space attention location in the three-dimensional real space, and the setting unit sets information on brightness of a virtual space attention location in the three-dimensional virtual space corresponding to the real space attention location on the basis of the acquired information on brightness of the real space attention location. The reproduction unit reproduces illumination of the machine system in the three-dimensional virtual space on the basis of the set information on brightness of the virtual space attention location.

Description

Simulation device and simulation program

[0001] The present disclosure relates to a simulation device and a simulation program.

[0002] In recent years, machine vision (robot vision) has been used in industrial machinery, industrial robots, and collaborative robots, and the industrial machinery, etc. are made to perform predetermined operations based on image information acquired by the machine vision.

[0003] For example, in a machine system using machine vision, the spatial position information of the workpiece (object) is determined based on image information captured by a camera (image capturing device) of a work area including the workpiece (object), and the machine is controlled to perform a specified process on the workpiece.

[0004] When capturing an image of a work area with a camera, it can be difficult to obtain image information of the work area with proper exposure due to factors such as illumination from lights (lighting fixtures), reflections of light from industrial machinery, surrounding equipment, ceilings, and walls, or the influence of the material of the workpiece and ambient light other than lights.

[0005] Conventionally, various proposals have been made as techniques for calculating conditions suitable for a site where an industrial machine or the like is installed.

[0006] JP 2017-045166 A JP 2011-186928 A

[0007] As described above, in order to find the proper (optimal) lighting conditions at each site, such as a factory where industrial machines are installed, it takes a long time for adjustment work, for example, because trial and error is repeated at the site. Furthermore, at the site, it is sometimes necessary to take into account the influence of natural light other than light, for example, and it is becoming increasingly difficult to find the proper lighting conditions at the site in a short time.

[0008] Therefore, there is a demand for a simulation device and a simulation program that can quickly determine appropriate lighting conditions on site. Note that, in this specification, for the sake of simplicity, a robot system using an industrial robot will be described as an example, but the application of the simulation device and the simulation program according to this embodiment is not limited to robot systems, and can be widely applied to various mechanical systems including industrial and consumer robots and other machines.

[0009] According to one embodiment of the present disclosure, there is provided a simulation device that simulates lighting in a three-dimensional real space of a machine system including a machine provided in a real space and a workpiece processed by the machine as lighting in a three-dimensional virtual space of a machine system provided in a virtual space, the simulation device including an acquisition unit, a setting unit, and a reproduction unit.

[0010] The acquisition unit acquires brightness information of a real-space point of interest in the three-dimensional real space, the setting unit sets brightness information of a virtual-space point of interest in the three-dimensional virtual space corresponding to the real-space point of interest based on the acquired brightness information of the real-space point of interest, and the reproduction unit reproduces the illumination of the mechanical system in the three-dimensional virtual space based on the brightness information of the set virtual-space point of interest.

[0011] FIG. 1 is a diagram schematically illustrating an example of a mechanical system to which a simulation device according to this embodiment is applied. FIG. 2 is a diagram for explaining an example of a robot system to which a simulation device according to this embodiment is applied. FIG. 3 is a functional block diagram of an example of a simulation device according to this embodiment. FIG. 4 is a diagram of a container in the robot system shown in FIG. 2 as viewed from above. FIG. 5 is a diagram (part 1) for explaining differences in lighting in the container shown in FIG. 4. FIG. 6 is a diagram (part 2) for explaining differences in lighting in the container shown in FIG. 4. FIG. 7 is a flowchart for explaining an example of processing in a first example of a simulation program according to this embodiment. FIG. 8 is a diagram for explaining an example of brightness adjustment processing in the simulation program shown in FIG. 7. FIG. 9 is a flowchart for explaining an example of processing in a second example of a simulation program according to this embodiment.

[0012] Hereinafter, examples of a simulation device and a simulation program according to the present embodiment will be described in detail with reference to the accompanying drawings. In each drawing, identical or similar components are assigned identical or similar reference numerals. Furthermore, the embodiments described below do not limit the technical scope and meaning of the terms of the invention described in the claims.

[0013] 1 is a diagram schematically illustrating an example of a mechanical system to which a simulation device according to this embodiment is applied, and shows, as an example of the mechanical system, a robot system using an industrial robot 1. Here, the robot system shown in Fig. 1 performs so-called "bulk picking," in which the robot (industrial robot) 1 picks out individual workpieces W from among a plurality of workpieces W placed randomly in a container (storage vessel) 4 in a three-dimensional real space (site) such as a factory illuminated by lights (lighting fixtures) 2.

[0014] In the robot system, for example, a robot 1 installed on a site such as a factory picks up a workpiece W based on three-dimensional image data from a three-dimensional imaging device (camera) 3 attached to a camera mount 40. That is, a hand (end effector) 12 is provided at the tip of an arm 11 of the robot 1, and is configured to pick up one workpiece W in order from among a plurality of workpieces W in a container 4. Here, the hand 12 is a suction hand that sucks the workpiece W by negative pressure, but various other types such as a grasping hand (pneumatic gripper) or a magnetic hand can also be used.

[0015] The three-dimensional imaging device 3 includes a first camera 31, a second camera 32, and a projector 33. Here, the first camera 31 and the second camera 32 function as a stereo camera, and the projector 33 projects a pattern of light, such as a striped pattern, onto the surfaces of the multiple workpieces W in the container 4. In Fig. 1, the lighting that illuminates the multiple workpieces W in the container 4 is a single light 2 provided on the ceiling above the container 4 (robot 1), but multiple lights may be provided on the ceiling, and a light whose illuminance and light distribution can be adjusted may also be used.

[0016] Fig. 2 is a diagram for explaining an example of a robot system to which the simulation device according to this embodiment is applied, and Fig. 3 is a functional block diagram of the example of the simulation device according to this embodiment. Fig. 2 corresponds to the robot system shown in Fig. 1 to which a robot control device 5 and a simulation device 6 are added. Furthermore, Fig. 4 is a diagram showing a container in the robot system shown in Fig. 2 as viewed from above.

[0017] The robot control device 5, for example, executes a robot control program stored in its storage unit, and controls the robot 1 to sequentially remove the workpieces W from the container 4 based on the three-dimensional image data from the three-dimensional imaging device 3. The simulation device 6 is a device that simulates the lighting at the site (three-dimensional real space) of the robot system including the robot (machine) 1 installed in real space and the workpieces W processed by the robot 1, as lighting in the three-dimensional virtual space of the robot system installed in virtual space. Here, the simulation device 6 is installed in a location away from the site, such as a factory, where the robot 1 is installed, but it can also be installed, for example, as a separate device near the robot control device 5, or installed within the robot control device 5.

[0018] 2, reference symbols OP1 to OP5 each indicate a point of interest in three-dimensional real space (real-space point of interest). That is, as shown in FIG. 4, in a view of the container 4 seen from above, point of interest OP1 indicates the upper left corner of the multiple workpieces W stored in the rectangular container 4, OP2 indicates the lower left corner, OP3 indicates the lower right corner, OP4 indicates the upper right corner, and OP5 indicates the center. Note that points of interest are not limited to these five portions OP1 to OP5, and can also be set to more portions (for example, the portion between each corner portion OP1 to OP4 and the center portion OP5, etc.).

[0019] Figures 5 and 6 are diagrams for explaining the difference in lighting in the container shown in Figure 4, with Figure 5(a) showing a case where the multiple workpieces W stored in the container 4 are dark overall (illuminance is lower than the appropriate value), and Figure 5(b) showing a case where the multiple workpieces W stored in the container 4 are bright overall (illuminance is higher than the appropriate value). Furthermore, Figure 6(a) shows a case where only a portion (upper right corner portion OP4) of the multiple workpieces W stored in the container 4 is dark, and Figure 6(b) shows a case where only a portion (upper right corner portion OP4) is bright.

[0020] According to the simulation device 6 of this embodiment, for example, brightness information of real-space points of interest OP1 to OP5 in a three-dimensional real space as shown in Figures 4 to 6 is reproduced as brightness information of virtual-space points of interest op1 to op5 in a three-dimensional virtual space, and is displayed on the display unit 64. The operator of the simulation device 6 refers to the brightness information of the virtual-space points of interest op1 to op5 displayed on the display unit 64 (virtual-space captured image), and operates the adjustment unit 65 to adjust the virtual-space captured image to the optimum (appropriate) one shown in Figure 4. The simulation device 6 will be described in detail below.

[0021] 3, the simulation device 6 includes an acquisition unit 61, a setting unit 62, a reproduction unit 63, a display unit 64, and an adjustment unit 65. The acquisition unit 61 acquires brightness information of real-space points of interest OP1 to OP5 in a three-dimensional real space, and the setting unit 62 sets brightness information of virtual-space points of interest op1 to op5 in a three-dimensional virtual space that correspond to the real-space points of interest OP1 to OP5, based on the brightness information of the real-space points of interest OP1 to OP5 acquired by the acquisition unit 61.

[0022] The reproduction unit 63 reproduces the lighting of the robot system (mechanical system) in the three-dimensional virtual space based on the brightness information of the virtual space attention points op1 to op5 set by the setting unit 62. The display unit 64 displays the lighting of the robot system in the three-dimensional virtual space reproduced by the reproduction unit 63, and the adjustment unit 65 adjusts the lighting of the robot system in the three-dimensional virtual space. Note that the adjustment unit 65 includes various input devices such as a mouse and a keyboard, and the operator adjusts the lighting by operating the input device while referring to the image displayed on the display unit 64. Needless to say, the display unit 64 can display not only an image related to the lighting of the robot system in the three-dimensional virtual space but also various other information.

[0023] When the brightness of the virtual space points of interest op1 to op5 is inappropriate, the adjustment unit 65 adjusts the lighting of the robot system in the three-dimensional virtual space so that the brightness of the virtual space points of interest op1 to op5 becomes appropriate. Furthermore, when the brightness of the virtual space points of interest op1 to op5 is insufficient, the adjustment unit 65 can also adjust by adding new lights (lighting fixtures) 21 and 22. The reproduction unit 63 reproduces the lighting of the robot system in the three-dimensional virtual space based on the brightness of the virtual space points of interest op1 to op5 that takes into account the adjustment by the adjustment unit 65. Here, the brightness information of the real space points of interest OP1 to OP5 may be information based on the results of a measurement of the brightness of the real space points of interest OP1 to OP5 by a worker (operator) at the work site using a measuring device.

[0024] Alternatively, the brightness information at the real-space points of interest OP1 to OP5 may be information based on the output of an image capturing device that captures an image including the real-space points of interest OP1 to OP5 at the site. Specifically, the brightness information at the real-space points of interest OP1 to OP5 may utilize, for example, a two-dimensional image captured by the first camera 31 or the second camera 32 of the three-dimensional imaging device 3. Of course, a dedicated camera may also be provided for acquiring brightness information at the real-space points of interest OP1 to OP5.

[0025] That is, the robot system is equipped with a camera that captures real-space captured images including brightness information for the real-space points of interest OP1 to OP5, and can utilize the real-space captured images captured by the camera. The reproduction unit 63 reproduces, in the three-dimensional virtual space, virtual-space captured images including brightness information for the virtual-space points of interest op1 to op5 corresponding to the brightness information for the real-space points of interest OP1 to OP5, based on the real-space captured images captured by the camera. The display unit 64 displays the virtual-space captured images reproduced by the reproduction unit 63. As described above, the operator adjusts the lighting of the robot system in the three-dimensional virtual space by operating an input device such as a mouse while referring to the image displayed on the display unit 64.

[0026] The acquisition unit 61 of the simulation device 6 can be configured to acquire real-space captured images online using, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a dedicated communication line. When the exposure of the virtual-space captured image reproduced by the reproduction unit 63 is inappropriate, the adjustment unit 65 can adjust the lighting of the robot system in the three-dimensional virtual space so that the exposure of the virtual-space captured image is appropriate. Here, when the exposure of the virtual-space captured image is insufficient, the adjustment unit 65 can adjust the lighting by adding new lighting fixtures (lights) 21, 22. That is, in the simulation device, the new lights 21, 22 are added to adjust the insufficient exposure of the virtual-space captured image and the virtual-space captured image is checked. Then, when the exposure of the virtual-space captured image becomes appropriate, the actual lights 21, 22 are attached to, for example, the support poles of the camera mount 40, corresponding to the lights 21, 22 added by the simulation device.

[0027] As described above, according to this embodiment, for example, a simulation device installed at a location away from the site (factory) where the robot 1 is installed can be used to consider and confirm the exposure of the virtual space captured image with the lights 21 and 22 added, and then the lights 21 and 22 can be attached to the support columns of the camera mount 40 at the actual site. That is, the simulation device according to this embodiment can determine appropriate lighting conditions in a short time, thereby reducing the need for repeated trial and error on site. Here, for example, if the lights 2 installed on the ceiling or the like are adjustable in illuminance and / or light distribution, at least one of the illuminance and light distribution of the lights 2 may be adjusted to optimize the exposure of the virtual space captured image, instead of adding new lights 21 and 22. Alternatively, for example, in a lighting system in which multiple lights 2 are arranged across the entire ceiling, and each light 2 illuminates a portion of multiple workpieces W stored in a container 4, the brightness of each corresponding light 2 can be adjusted to enable the camera on site to capture optimal images (real space captured images).

[0028] In the above, the brightness information at the real-space points of interest OP1 to OP5 may be brightness information obtained by adding, for example, the light irradiated from the light 2 and natural light other than the light 2 in the three-dimensional real space (site). Here, if the influence of brightness information based on natural light, such as brightness information based on natural light during daytime and nighttime hours or on sunny and cloudy days, can be measured separately, a simulation of brightness information based on the light irradiated from the light 2 can be performed with higher accuracy. Alternatively, a simulation can be performed by acquiring online real-space images captured by a camera under lighting that combines the light irradiated from the light 2 and natural light. For example, a measuring device that measures the brightness of natural light can be placed near the container 4, and the light 2 at the site can be adjusted to compensate for the influence of the measured natural light. It goes without saying that if the site where the robot 1 is installed is illuminated only by light irradiated from the light 2, the brightness information at the real-space points of interest OP1 to OP5 will be brightness information based on the light irradiated from the light 2.

[0029] In this way, the simulation device according to this embodiment makes it possible to, for example, consider on-site lighting adjustments in advance and quickly determine appropriate on-site lighting conditions. In other words, it is possible to achieve optimal lighting while significantly reducing the amount of work at the site, such as in a factory, where the robot 1 is installed.

[0030] 7 is a flowchart illustrating an example of processing in a first example of the simulation program according to this embodiment, illustrating an example in which an operator measures the brightness of a location of interest in a real space (three-dimensional real space) such as a factory where a robot 1 is installed. As shown in FIG. 7 , when an example of processing in the first example of the simulation program according to this embodiment starts (START), in step ST11, for example, an operator measures the brightness of a location of interest (real-space location of interest) in a factory (worksite) where the robot 1 is installed using a measuring device, and then the process proceeds to step ST12. Here, the real-space locations of interest OP1 to OP5 where the operator performs measurements are, as described above, for example, the four corners OP1 to OP4 and the center OP5 of a rectangular container 4 that contains multiple workpieces W.

[0031] In step ST12, the measured brightness of each of the real-space points of interest OP1 to OP5 is output to a file, and the process proceeds to step ST13, where the output file is read into the simulation device 6. For example, in step ST12, the brightness information (file) of each of the real-space points of interest OP1 to OP5 measured by the operator using a measuring device is output to and stored in a USB memory or the like, and the process proceeds to step ST13, where the brightness information of each of the real-space points of interest OP1 to OP5 stored in the USB memory or the like is read into the simulation device 6.

[0032] Next, the process proceeds to step ST14, where various adjustments (such as brightness adjustment and adding a light source to the real space) are made, and then the process proceeds to step ST15. Here, the various adjustments in step ST14 are made, for example, by an operator referring to brightness information of the virtual space captured image (virtual space points of interest op1 to op5) displayed on the display unit 64 and operating various input devices such as a mouse and keyboard to adjust the lighting of the robot system in the three-dimensional virtual space. Examples of such lighting adjustments include adjusting the light emitted from the light 2 installed on the ceiling, adding new lights (light sources) 21 and 22, or adjusting at least one of the illuminance and light distribution of the lights 2, 21, and 22. Various other adjustments are also made to obtain the optimal virtual space captured image (real space captured image) shown in FIG. 4 .

[0033] In step ST15, it is determined whether camera imaging is possible in the real space. If it is determined that camera imaging is not possible in the real space (NO), the process returns to step ST14 and various adjustments are made again. If it is determined in step ST15 that camera imaging is possible in the real space (YES), that is, that the optimal real-space captured image shown in Figure 4 can be obtained, the process proceeds to step ST16, where the camera imaging simulation is completed and an example of the processing in the first example of the simulation program according to this embodiment is ended (END). Note that the processing in step ST15 can be performed, for example, by connecting the camera (first camera 31 or second camera 32) of the three-dimensional imaging device 3 at the site (real space) of a factory or the like where the robot 1 is installed to the simulation device 6 via a LAN, WAN, or dedicated communication line, so that the real-space captured image captured by the camera can be acquired and confirmed online.

[0034] Fig. 8 is a diagram for explaining an example of brightness adjustment processing in the simulation program shown in Fig. 7, showing a case where adjustment is made to add new lights 21 and 22. For example, if the virtual space captured image displayed on the display unit 64 of the simulation device 6 is dark overall inside the container 4 corresponding to the real space captured image of Fig. 5(a), the lights 21 and 22 are attached to the support poles of the camera mount 40 at the actual site in order to brighten the entire dark inside of the container 4 and obtain the optimal real space captured image shown in Fig. 4.

[0035] That is, the operator recognizes that the entire interior of the container 4 is dark in the virtual space captured image displayed on the display unit 64 of the simulation device 6, and adjusts the virtual space captured image to brighten the entire interior of the container 4, thereby obtaining the optimal real space captured image shown in Fig. 4. The adjustment by the simulation device 6 is made, for example, by attaching lights 21, 22 to the supports of the camera mount 40 in the virtual space, in order to obtain a virtual space captured image that corresponds to the optimal real space captured image shown in Fig. 4. Based on the adjustment result by the simulation device 6, the operator attaches lights 21, 22 to the supports of the camera mount 40 in the three-dimensional real space in which the robot 1 is installed.

[0036] Here, the brightness adjustment process described above not only involves adding new lighting fixtures, but also includes various adjustments such as adjusting the illuminance and light distribution of lighting fixtures depending on the target system (robot system), workpiece, and site (factory) conditions. For example, as shown in FIG. 5(b), the brightness of light 2 that illuminates the entire interior of container 4 is dimmed. For example, as shown in FIG. 6(a), if only a portion (the upper right corner) of container 4 is dark, adjustment is made to brighten only that portion. Also, as shown in FIG. 6(b), if only a portion (the upper right corner) of container 4 is bright, adjustment is made to darken only that portion.

[0037] 9 is a flowchart for explaining an example of processing in a second example of the simulation program according to this embodiment. As shown in FIG. 9 , when the example of processing in the second example of the simulation program according to this embodiment starts (START), in step ST21, an image of the target area (real-space captured image) is acquired using on-site lighting, and the process proceeds to step ST22. Here, the target area refers to an area including the container 4 storing multiple workpieces W from which the robot 1 performs the workpiece removal process. As mentioned above, the image of the target area can be captured using a camera or the like of the three-dimensional imaging device 3. The image of the target area can be acquired by the acquisition unit 61 of the simulation device 6, for example, via a LAN, a WAN, or a dedicated communication line.

[0038] In step ST22, the brightness of the image acquired by the acquisition unit 61 is analyzed, and the process proceeds to step ST23, where the illumination is adjusted so that the image is appropriate based on the analyzed brightness. Here, the image (virtual space captured image) in step ST23 can be adjusted in various ways, similar to the various adjustments in step ST14 of Fig. 7 described above. Then, the process proceeds to step ST24, where it is determined whether processing is possible with the adjusted illumination.

[0039] If it is determined in step ST24 that processing is not possible with the adjusted lighting (NO), the process returns to step ST23, where the lighting is adjusted again so that the image is appropriate based on the analyzed brightness. If it is determined in step ST24 that processing is possible with the adjusted lighting (YES), the process proceeds to step ST25. In step ST25, based on the lighting adjustment results in step ST23 for which processing is determined to be possible in step ST24, the lighting at the site where the robot 1 is installed is adjusted, an image of the target area is acquired, and the process in the second example of the simulation program according to this embodiment is terminated (END). The images used in the determination process in step ST24 and the images of the target area acquired after adjusting the lighting at the site in step ST25 can be acquired online, for example, by connecting a camera at the site to the simulation device 6 via a LAN, WAN, or dedicated communication line, similar to the images used in the determination process in step ST15 of FIG. 7 .

[0040] The simulation program according to the present embodiment described above is executed, for example, by an arithmetic processing unit (MPU, CPU, etc.) of the simulation device 6. The simulation program according to the present embodiment may be provided by being recorded on a computer-readable non-transitory recording medium or non-volatile semiconductor memory, or may be provided via a wired or wireless communication line. Examples of the computer-readable non-transitory recording medium include optical disks such as CD-ROMs (Compact Disc Read Only Memory) and DVD-ROMs, and hard disk drives. Examples of the non-volatile semiconductor memory include PROMs (Programmable Read Only Memory) and flash memories. Furthermore, the program may be distributed from a server device via a wired or wireless LAN or WAN.

[0041] As described above in detail, the simulation device and simulation program according to this embodiment make it possible to determine appropriate lighting conditions on-site in a short time.

[0042] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0043] The following supplementary notes are further disclosed regarding the above-described embodiment and modified examples. [Supplementary Note 1] A simulation device (6) that simulates lighting in a three-dimensional real space of a machine system including a machine (1) provided in a real space and a workpiece (W) processed by the machine (1), as lighting in a three-dimensional virtual space of the machine system provided in a virtual space, comprising: an acquisition unit (61) that acquires brightness information of real-space points of interest (OP1 to OP5) in the three-dimensional real space; a setting unit (62) that sets brightness information of virtual-space points of interest (op1 to op5) in the three-dimensional virtual space that correspond to the real-space points of interest (OP1 to OP5) based on the acquired brightness information of the real-space points of interest (OP1 to OP5); and a reproduction unit (63) that reproduces lighting of the machine system in the three-dimensional virtual space based on the set brightness information of the virtual-space points of interest (op1 to op5). [Supplementary Note 2] The simulation device according to Supplementary Note 1, further comprising a display unit (64) that displays lighting of the mechanical system in the three-dimensional virtual space. [Supplementary Note 3] The simulation device according to Supplementary Note 2, further comprising an adjustment unit (65) that adjusts lighting of the mechanical system in the three-dimensional virtual space. [Supplementary Note 4] The simulation device according to Supplementary Note 3, wherein, when the brightness of the virtual space points of interest (op1 to op5) is inappropriate, the adjustment unit (65) adjusts the lighting of the mechanical system in the three-dimensional virtual space so that the brightness of the virtual space points of interest (op1 to op5) becomes appropriate. [Supplementary Note 5] The simulation device according to Supplementary Note 4, wherein, when the brightness of the virtual space points of interest (op1 to op5) is insufficient, the adjustment unit (65) adjusts by adding a new lighting device. [Supplementary Note 6] The simulation device according to Supplementary Note 4 or Supplementary Note 5, wherein the brightness information at the real-space points of interest (OP1 to OP5) is information based on a measurement result of measuring the brightness of the real-space points of interest (OP1 to OP5) in the three-dimensional real space.[Supplementary Note 7] The simulation device according to Supplementary Note 4 or Supplementary Note 5, wherein the brightness information at the real-space points of interest (OP1 to OP5) is information based on the output of an image capturing device that captures an image including the real-space points of interest (OP1 to OP5) in the three-dimensional real space. [Supplementary Note 8] The simulation device according to Supplementary Note 3, wherein the mechanical system includes a camera that captures a real-space captured image including brightness information at the real-space points of interest (OP1 to OP5), the reproduction unit (63) reproduces, in the three-dimensional virtual space, a virtual-space captured image that corresponds to the real-space captured image and includes brightness information at the virtual-space points of interest (op1 to OP5) that corresponds to the brightness information of the real-space points of interest (OP1 to OP5), and the display unit (64) displays the virtual-space captured image. [Supplementary Note 9] The simulation device according to Supplementary Note 8, wherein the acquisition unit (61) acquires the real-space captured image captured by the camera online. [Supplementary Note 10] The simulation device according to Supplementary Note 8 or Supplementary Note 9, wherein the adjustment unit (65), when an exposure of the virtual space captured image is inappropriate, adjusts lighting of the mechanical system in the three-dimensional virtual space so that the exposure of the virtual space captured image becomes appropriate. [Supplementary Note 11] The simulation device according to Supplementary Note 10, wherein the adjustment unit (65), when an exposure of the virtual space captured image is insufficient, adjusts by adding a new lighting device. [Supplementary Note 12] The simulation device according to Supplementary Note 10, wherein at least one of illuminance and light distribution of the lighting is adjustable, and when an exposure of the virtual space captured image is inappropriate, the adjustment unit (65) adjusts at least one of the illuminance and light distribution of the lighting so that the exposure of the virtual space captured image becomes appropriate. [Supplementary Note 13] The simulation device according to any one of Supplementary Note 3 to Supplementary Note 12, wherein the reproduction unit (63) reproduces the illumination of the mechanical system in the three-dimensional virtual space based on the brightness of the virtual space attention points (op1 to op5) taking into account the adjustment by the adjustment unit (65).[Supplementary Note 14] The simulation device according to any one of Supplementary Note 1 to Supplementary Note 13, wherein the brightness information at the real-space points of interest (OP1 to OP5) is brightness information obtained by adding together, in the three-dimensional real space, illumination light irradiated from a lighting fixture and natural light other than the illumination fixture. [Supplementary Note 15] A simulation program that simulates lighting in a three-dimensional real space of a mechanical system that includes a machine (1) provided in a real space and a workpiece (W) processed by the machine (1), as lighting in a three-dimensional virtual space of the mechanical system provided in a virtual space, the simulation program causing a calculation processing device to execute the following steps: acquiring brightness information of real-space points of interest (OP1 to OP5) in the three-dimensional real space; setting brightness information of virtual-space points of interest (op1 to op5) in the three-dimensional virtual space that correspond to the real-space points of interest (OP1 to OP5) based on the acquired brightness information of the real-space points of interest (OP1 to OP5); and reproducing lighting of the mechanical system in the three-dimensional virtual space based on the set brightness information of the virtual-space points of interest (op1 to op5).

[0044] REFERENCE SIGNS LIST 1 Industrial robot (machine) 2, 21, 22 Light (lighting fixture) 3 Three-dimensional imaging device 4 Container (storage vessel) 5 Robot control device 6 Simulation device 11 Arm 12 Hand (end effector) 31, 32 Camera 33 Pattern output unit 40 Camera stand 61 Acquisition unit 62 Setting unit 63 Reproduction unit 64 Display unit 65 Adjustment unit OP1 to OP5 Real space attention point (measurement point) op1 to op5 Virtual space attention point W Work (object)

Claims

1. A simulation device that simulates the illumination in the three-dimensional real space of a machine system including a machine provided in the real space and a workpiece processed by the machine as illumination in the three-dimensional virtual space of the machine system provided in the virtual space, the simulation device comprising: an acquisition unit that acquires brightness information of a real-space attention point in the three-dimensional real space; a setting unit that sets brightness information of a virtual-space attention point in the three-dimensional virtual space corresponding to the real-space attention point based on the acquired brightness information of the real-space attention point; and a reproduction unit that reproduces the illumination of the machine system in the three-dimensional virtual space based on the set brightness information of the virtual-space attention point.

2. The simulation device according to claim 1, further comprising a display unit that displays the illumination of the machine system in the three-dimensional virtual space.

3. The simulation device according to claim 2, further comprising an adjustment unit that adjusts the illumination of the machine system in the three-dimensional virtual space.

4. The adjustment unit of the simulation device according to claim 3 adjusts the illumination of the machine system in the three-dimensional virtual space so that the brightness of the virtual-space attention point becomes appropriate when the brightness of the virtual-space attention point is not appropriate.

5. The adjustment unit of the simulation device according to claim 4 adjusts to add a new lighting fixture when the brightness of the virtual-space attention point is insufficient.

6. The simulation device according to claim 4 or claim 5, wherein the brightness information at the real-space attention point is information based on a measurement result of measuring the brightness of the real-space attention point in the three-dimensional real space.

7. The simulation device according to claim 4 or claim 5, wherein the brightness information at the real-space attention point is information based on the output of an image capturing device that captures an image including the real-space attention point in the three-dimensional real space.

8. The mechanical system includes a camera that captures a real-space captured image including brightness information of the real-space target location. The reproduction unit reproduces, in the three-dimensional virtual space, a virtual-space captured image corresponding to the real-space captured image and including brightness information of a virtual-space target location corresponding to the brightness information of the real-space target location. The display unit displays the virtual-space captured image. The simulation device according to claim 3.

9. The acquisition unit acquires, online, the real-space captured image captured by the camera. The simulation device according to claim 8.

10. When the exposure of the virtual-space captured image is not appropriate, the adjustment unit adjusts the illumination of the mechanical system in the three-dimensional virtual space so that the exposure of the virtual-space captured image becomes appropriate. The simulation device according to claim 8 or claim 9.

11. When the exposure of the virtual-space captured image is insufficient, the adjustment unit adjusts to add a new lighting fixture. The simulation device according to claim 10.

12. At least one of the illuminance and the light distribution of the illumination is adjustable. When the exposure of the virtual-space captured image is not appropriate, the adjustment unit adjusts at least one of the illuminance and the light distribution of the illumination so that the exposure of the virtual-space captured image becomes appropriate. The simulation device according to claim 10.

13. The reproduction unit reproduces the illumination of the mechanical system in the three-dimensional virtual space based on the brightness of the virtual-space target location considering the adjustment by the adjustment unit. The simulation device according to any one of claims 3 to 12.

14. The brightness information at the real-space target location is brightness information obtained by adding irradiation light irradiated from a lighting fixture and natural light other than the lighting fixture in the three-dimensional real space. The simulation device according to any one of claims 1 to 13.

15. A simulation program that simulates the illumination in the three-dimensional real space of a machine system including a machine provided in the real space and a workpiece processed by the machine as illumination in the three-dimensional virtual space of the machine system provided in the virtual space, the simulation program causing an arithmetic processing unit to: acquire brightness information of a real-space attention point in the three-dimensional real space; set brightness information of a virtual-space attention point in the three-dimensional virtual space corresponding to the real-space attention point based on the acquired brightness information of the real-space attention point; and reproduce the illumination of the machine system in the three-dimensional virtual space based on the set brightness information of the virtual-space attention point.

Images