Simulation system, program, simulation method, and simulation system construction apparatus

Abstract

Provided is a simulation system for simulating the operation of a cell including a robot and a device, the simulation system comprising: a virtual device for simulating the output of the device; and a virtual robot controller for simulating, in accordance with the output of the virtual device, the operation of a robot controller which controls the robot. The virtual device performs output on the basis of a plurality of output histories of the device.

Description

Simulation system, program, simulation method, and simulation system construction device

[0001] The present invention relates to a simulation system, a program, a simulation method, and a simulation system construction device.

[0002]

[0005] Patent Literature 1 describes a simulation device including a first simulator that simulates control of a first machine by a first controller, a second simulator that simulates control of a second machine by a second controller, and a simulation manager that controls the progress of the simulation by the first simulator and the progress of the simulation by the second simulator so as to correspond to the relationship between the progress of the control by the first controller and the progress of the control by the second controller.

[0006] Patent Literature 2 describes a simulation device that executes a simulation of a robot using a robot program, further using a file separate from the robot program, the separate file including instructions that set the state of a signal or an instruction that sets the value of a data register, which are written corresponding to a line of the robot program and are referenced by execution of the line of the robot program, and based on the instructions, changes the setting of the state of the signal or the value of the data register written corresponding to the executed line during execution of the simulation in synchronization with the executed line of the robot program. Patent Document 3 describes a control device for controlling a control target, the control device including a PLC engine that cyclically executes a program including sequence instructions, a robot control engine that controls a robot, an image processing engine that performs image processing on images from a camera, and a simulation module that is constructed according to user settings and simulates at least a part of the control target, the robot, and the camera. [Prior Art Documents] [Patent Document 1] Japanese Patent No. 7400104 [Patent Document 2] Japanese Patent Laid-Open No. 2014-144524 [Patent Document 3] Japanese Patent Laid-Open No. 2022-043871 General disclosure

[0003] According to one embodiment of the present invention, there is provided a simulation system. The simulation system may simulate the operation of a cell including a robot and a device. The simulation system may include a virtual device that simulates the output of the device. The simulation system may include a virtual robot controller that simulates the operation of a robot controller that controls the robot in accordance with the output of the virtual device. The virtual device may perform output based on a plurality of output histories of the device.

[0004] The virtual device may output different outputs for each output based on the multiple output histories. The simulation system may further include a re-planning unit that re-plans a program based on the operation of the robot controller based on the output of the virtual device and the results of a simulation by the virtual robot controller in response to the output. The re-planning unit may execute a simulation by the virtual robot controller in response to the output of the virtual device multiple times based on the modified program to confirm that no errors occur. The re-planning unit may change at least one of the selection of commands instructing robot operation in the program, the order of commands, adjustment of command arguments, command invocation logic, and command invocation timing, based on the output of the virtual device and the results of a simulation by the virtual robot controller in response to the output. The virtual device may output the output based on the multiple output histories so that the response time varies for each output. The output history of the device may include command history data indicating commands to the device and responses from the device at each time. The simulation system may further include a response time indexing unit that indexes multiple response times from input of a command to the device to output of a response from the device based on multiple command history data. The virtual device may perform the output so that the response time varies for each output based on the multiple response times indexed by the response time indexing unit. The virtual device that simulates the output of a robot hand may perform the output so that the response time varies for each output based on a history of multiple opening and closing operations of the robot hand. The output history of the device may include output data output by the device to the robot controller, and the virtual device may output different output data for each output based on the multiple output histories. The virtual device that simulates the output of a vision sensor that recognizes a workpiece may output different recognition results for each output based on multiple recognition results of the vision sensor.The simulation system may further include an image creation unit that creates a different image for each output based on the multiple recognition results of the vision sensor, and the virtual device may output the recognition results of the workpiece in the image. The virtual device may select and output a different output history from the multiple output histories for each output. The virtual device may select and output the output history in chronological order from the multiple time-series output histories. Two or more virtual devices may each select and output one output history at a corresponding timing from the multiple output histories of the devices to which the virtual devices respectively correspond. The virtual robot controller may perform a simulation based on the operation history of the robot controller, and the virtual device may select and output one output history from the multiple output histories corresponding to the timing of the operation history based on the virtual robot controller. The simulation system may further include an acquisition unit that acquires a trigger condition and an output target when the trigger condition is satisfied, and a creation unit that creates the virtual device that outputs the output target when the trigger condition is satisfied, based on the trigger condition and the output target acquired by the acquisition unit. The simulation system may further include a communication adjustment unit that adjusts a communication delay between the virtual device and the virtual robot controller based on a plurality of communication delay histories between the device and the robot controller.

[0005] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the simulation system.

[0006] According to one embodiment of the present invention, there is provided a simulation method for simulating the operation of a cell including a robot and a device. The simulation method may include an output step in which a virtual device that simulates the output of the device outputs based on a plurality of output histories of the device. The simulation method may also include a simulation step in which the operation of a robot controller that controls the robot is simulated in accordance with the output of the virtual device.

[0007] According to one embodiment of the present invention, there is provided a simulation system construction apparatus for constructing a simulation system that simulates the operation of a cell including a robot and a device. The simulation system construction apparatus may include a creation unit that creates a virtual device that simulates the output of the device, the virtual device providing output to a virtual robot controller that simulates the operation of a robot controller that controls the robot in accordance with the output of the virtual device, based on a plurality of output histories of the device.

[0008] According to one embodiment of the present invention, there is provided a simulation system for simulating the operation of a cell including a robot and a device. The simulation system may include a virtual device that simulates the output of the device. The simulation system may include a robot controller that controls the robot according to the output of the virtual device. The virtual device may output based on a plurality of output histories of the device.

[0009] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also be inventions.

[0010] 1 schematically illustrates an example of a simulation system 10. 1 schematically illustrates an example of the functional configuration of a simulator 100. 1 schematically illustrates an example of a virtual environment 101. 1 schematically illustrates an example of a processing flow in the simulation system 10. 1 schematically illustrates an example of a processing flow in the simulation system 10. 1 schematically illustrates an example of a processing flow in the simulation system 10. 1 schematically illustrates an example of a hardware configuration of a computer 1200 that functions as the simulation system 10 or the simulator 100.

[0011] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0012] 1 shows a schematic diagram of an example of a simulation system 10. The simulation system 10 simulates the operation of a cell 22 containing robots and devices in a real environment.

[0013] 1 illustrates a real environment in which a CC (Cell Controller) 20, an RC (Robot Controller) 30, a robot 40 which is a vertical articulated robot, a robot hand 51 and a vision sensor 52 disposed at the tip of the robot 40, a camera 53, a workpiece 60, a platform 72, a platform 74, a platform 76, and a DB 80 are disposed. The real environment targeted by the simulation system 10 is not limited to this, and the simulation system 10 can simulate the operation of a cell 22 including any type of industrial robot and any type of device.

[0014] The CC 20 controls the cell 22 according to a program. The CC 20 may control the coordination between tasks of equipment such as robots and devices, execution timing, task transitions, etc. The CC 20 may coordinate with a higher-level controller. The CC 20 may coordinate with an HMI (Human Machine Interface). The CC 20 may coordinate with an engineering tool capable of generating programs for controllers, robots, devices, etc. The CC 20 may be, for example, a programmable logic controller. The CC 20 may be connected to an RC 30. In the example shown in FIG. 1, two RCs 30 are connected to the CC 20. A device may be connected to the CC 20. In the example shown in FIG. 1, a camera 53 is connected to the CC 20.

[0015] The RC 30 controls the robot 40 in accordance with a program. The RC 30, for example, repeatedly executes segment processing, which operates the robot 40, the robot hand 51, and the vision sensor 52 in accordance with control commands, at a predetermined control period. In this embodiment, the segment processing may be processing assigned per cycle. The control command includes, for example, an execution command for a job program. The job program is an operation program including one or more operation commands in a time series. The job program may be prepared in advance and stored in the RC 30.

[0016] In the real environment illustrated in FIG. 1 , for example, the operation command for the robot 40 may include a target position and orientation of the tip of the robot 40 and a target movement speed of the tip to the target position and orientation. The segment processing may include calculating the target position and orientation of the tip for each control cycle, calculating operation target values ​​of each of multiple actuators for moving the tip to the calculated target position and orientation, and operating each of the multiple actuators according to the calculated operation target values. The operation command for the robot hand 51 may include, for example, a close command including a command to close the hand and a time or a speed for closing the hand. The operation command for the robot hand 51 may include a close confirmation command for confirming that the hand has been closed. The operation command for the robot hand 51 may include, for example, an open command including a command to open the hand and a time or a speed for opening the hand. The operation command for the robot hand 51 may include an open confirmation command for confirming that the hand has been opened. The operation command for the vision sensor 52 may include, for example, a recognition command for capturing an image of a workpiece and performing recognition.

[0017] 1 , for example, the CC 20 repeatedly executes, at a predetermined control period, segment processing that coordinates the operations of multiple robots 40. The segment processing includes, for example, receiving status information of the multiple robots 40 and workpieces 60 from multiple RCs 30, etc., identifying the robot 40 that should start operating based on the program and the status information, and transmitting a control command to the robot 40 to start the operation.

[0018] Here, an example of the operation of the cell 22 will be described using an example in which the first robot 40 grasps the workpiece 60 placed on the stage 72 and moves it onto the stage 74, and the second robot 40 grasps the workpiece 60 placed on the stage 74 and moves it to the stage 76. Note that, for the sake of explanation, a simple operation is shown as an example, but the operation of the cell 22 may be more complex.

[0019] The CC 20 may, for example, transmit a recognition command to a camera 53 that captures an image of the entire cell 22 to recognize the status of the cell 22. In response to the recognition command, the camera 53 may capture an image of the entire cell 22 and transmit the recognition result to the CC 20. The CC 20 may, for example, create or update status information of the cell 22 using the recognition result by the camera 53 and measurement results by other sensors in the cell 22.

[0020] When the CC 20 determines, based on the status information, that the workpiece 60 has been placed on the platform 72, it transmits a control command to the first RC 30, including a command to execute a job program for the workpiece 60. The first RC 30 controls the first robot 40, the first robot hand 51, and the first vision sensor 52 in accordance with the control command to move the workpiece 60 from the platform 72 to the platform 74. For example, the first RC 30 transmits to the first robot 40 an operation command including a target position, a target posture, and a target speed of the tip of the workpiece 60 relative to the platform 72. In response to receiving a completion response from the first robot 40, the first RC 30 transmits to the first vision sensor 52 an operation command including capturing an image of the workpiece 60 and recognizing it, and transmits to the first robot hand 51 an operation command including an instruction to close the hand and a time until closing, and a closure confirmation command. The first robotic hand 51 may start the operation of closing the hand in accordance with the operation command and may transmit a completion report to the first RC 30 in response to the hand being closed. The closure of the hand may be determined, for example, by a sensor that detects the closed end of the hand, which is built into the hand. In response to receiving the completion response from the first vision sensor 52 and the first robotic hand 51, the first RC 30 transmits to the first robot 40 an operation command including a target position, a target posture, and a target velocity of the tip of the hand relative to the platform 74. In response to receiving the completion response from the first robot 40, the first RC 30 transmits to the first robot hand 51 an operation command including an instruction to open the hand and a time until opening, as well as an open confirmation command. In response to receiving the completion response from the first robotic hand 51, the first RC 30 transmits to the first robot 40 an operation command including a target position, a target posture, and a target velocity of the tip of the hand relative to a default position.

[0021] When the CC 20 determines, based on the status information, that the workpiece 60 has been placed on the platform 74, it transmits a control command to the second RC 30, including an execution command for a job program for the workpiece 60. The second RC 30 controls the second robot 40, the second robot hand 51, and the second vision sensor 52 in accordance with the control command to move the workpiece 60 from the platform 74 to the platform 76. For example, the second RC 30 transmits to the second robot 40 an operation command including a target position, a target posture, and a target speed of the tip of the workpiece 60 relative to the platform 74. In response to receiving a completion response from the second robot 40, the second RC 30 transmits to the second vision sensor 52 an operation command including capturing an image of the workpiece 60 and recognizing it, and transmits to the second robot hand 51 an operation command including an instruction to close the hand and a time until closing, and a closure confirmation command. In response to receiving the completion response from the second vision sensor 52 and the second robot hand 51, the second RC 30 transmits to the second robot 40 an operation command including a target position, a target posture, and a target speed of the tip portion relative to the platform 76. In response to receiving the completion response from the second robot 40, the second RC 30 transmits to the second robot hand 51 an operation command including an instruction to open the hand and a time until opening, and an open confirmation command. In response to receiving the completion response from the second robot hand 51, the second RC 30 transmits to the second robot 40 an operation command including a target position, a target posture, and a target speed of the tip portion relative to a default position.

[0022] The robot 40 may operate according to control commands from the CC 20, and the CC 20 may also realize an autonomous distributed robot cell. That is, the CC 20 may function as an information bulletin board, and multiple RCs 30 may operate autonomously based on information from the information bulletin board. For example, the CC 20 may disclose the results of the operations of each of the multiple RCs 30 as status information, and the multiple RCs 30 may make decisions and perform operations based on the disclosed status information.

[0023] The DB (Database) 80 accumulates various data in the real environment. The DB 80 may accumulate the operation history of each device in the real environment. The DB 80 may accumulate measurement data from each device in the real environment. The DB 80 may accumulate feedback data from each device in the real environment. The DB 80 may accumulate measurement data measured by various sensors (not shown) placed in the real environment. As a specific example, the DB 80 may accumulate measurement data measured by a sensor that measures the movement of the robot 40 or a sensor that measures the movement of the robot hand 51. The DB 80 may accumulate the communication delay history of each device in the real environment.

[0024] The DB 80 may store command history data including a history of commands, instructions, responses, and the like exchanged within the cell 22. The command history data may include timestamps for each command, instruction, and response.

[0025] The DB 80 may accumulate the output history of the device. The output history includes, for example, the response time of the device to a command or instruction. The response time may be the time from when a command or instruction is input to the device until the device outputs a response. For example, the response time of a device is the time from when a command from the RC 30 is input to the device until the device outputs a response. As a specific example, the operation result of the robot hand 51 includes the response time from when an opening / closing command is input to the robot hand 51 until the robot hand 51 opens / closes the hand and transmits a response. Note that the opening / closing operation of the robot hand 51 is not limited to an operation of opening and closing the hand, but may be an operation of simply opening the hand or an operation of simply closing the hand. The output history includes, for example, output data output by the device. For example, the output data of a device is data output by the device to the RC 30. As a specific example, the output history of the vision sensor 52 includes the recognition result of the workpiece 60. The recognition result of the workpiece 60 may include at least one of the position, posture, and type of the recognized workpiece 60.

[0026] The DB 80 may store the output history of the device in association with environmental information indicating the environment of the device when the device produced the output. The environmental information may indicate the environment of the space in which the device is installed. The environmental information may be measured, for example, by a sensor placed in the space of the cell 22. The environment of the space may include the temperature of the space. The environment of the space may include the humidity of the space. The environmental information may include the operating period of the device. The operating period of the device may be the period of time since the device was newly installed and started operating.

[0027] The simulation system 10 may include a virtual environment 101 constructed by a simulator 100, which is a virtualized version of a real environment. The virtual environment 101 includes a VD (Virtual Device) 500 that simulates the output of a device. The virtual environment 101 includes a VRC (Virtual Robot Controller) 300 that simulates the operation of an RC 30. The virtual environment 101 includes a VR (Virtual Robot) 400 that simulates a robot 40. The virtual environment 101 may include a VCC (Virtual Cell Controller) 200 that simulates a CC 20. The simulator 100 may be an example of a simulation system construction device.

[0028] The VRC 300 may simulate the operation of the RC 30 according to the output of the VD 500. Simulating the operation of the RC 30 according to the output of the VD 500 may include simulating the operation of the RC 30 directly using the output of the VD 500, or may include simulating the operation of the RC 30 indirectly using the output of the VD 500. For example, the VRC 300 acquires the result of the output from the VD 500 being processed by the VCC 200, and simulates the operation of the RC 30 using the acquired result. As a specific example, the VRC 300 acquires the result of the VCC 200 using the output from the VD 500 for control, and uses the acquired result to control the VR 400, etc. The VCC 200 may simulate the operation of the CC 20 according to the output of the VRC 300. The VCC 200 may simulate the operation of the CC 20 according to the output of the VD 500.

[0029] The simulation system 10 may include a simulator 100. That is, the simulation system 10 may include the simulator 100 and a virtual environment 101 constructed by the simulator 100.

[0030] The VCC 200 may execute a program that controls segment processing based on production instruction information from a higher-level controller, similar to the CC 20. The program used by the VCC 200 may be created or configured by an engineering tool.

[0031] The VRC 300 may control the VR 400 in the same way that the RC 30 controls the robot 40. The programs used by the VRC 300 may be created or configured by the pendant software. The VRC 300 may be capable of creating programs that can be used directly in the robot 40.

[0032] [Output Based on Multiple Output Histories of Real Devices] Checking the operation of the VCC200 and VRC300 programs requires input and output from the devices. For example, by creating a virtual device that outputs a preset theoretical value in response to input from the VRC300, the operation of the cell 22 can be simulated, allowing the operation of the VCC200 and VRC300 programs to be checked. However, even when performing the same operation, the output of a real device may vary due to various factors. Nevertheless, using a virtual device that outputs a theoretical value makes it impossible to accurately simulate the real environment, and when the simulation results are reflected in the real device, unexpected errors or other unexpected results may occur. In contrast, the VD500 of this embodiment outputs based on multiple output histories of the real device being simulated. The VD500 may output based on multiple output histories of the real device, so that the output varies similarly to that of the real device. The VD500 may output using multiple output histories of the real device stored in the DB80.

[0033] By having the VD500 output based on multiple output histories of the actual device, the VD500's output can be made closer to the output of the actual device, contributing to improved simulation accuracy. By improving the accuracy of the simulation, the likelihood of expected operation being achieved when a program adjusted or created based on the simulation results is implemented in the actual device is increased. For example, when an adjustment is made to increase the throughput of cell 22 based on the simulation results, the expected increase in throughput can be achieved without unexpected errors occurring due to the adjustment. For example, even when the operation of cell 22 or the layout of cell 22 is changed based on the simulation results to achieve a variation, the target item can be processed appropriately without unexpected errors occurring.

[0034] The VD500 may output differently for each output based on multiple output histories of the actual device. The output of the actual device is often not the same for each output. By using multiple outputs from the actual device and outputting differently for each output, the VD500's output can be made closer to the output of the actual device, which contributes to improving the accuracy of the simulation.

[0035] [Response Time Variation] The VD500, for example, generates output data based on multiple output histories of a real device, such that the response time varies for each output. For example, the VD500 calculates multiple past response times from the multiple output histories and generates output data based on the calculated multiple past response times, such that the response time varies for each output. The output history may include past response times, and the VD500 may generate output data based on the multiple past response times included in the multiple output histories, such that the response time varies for each output. A real device may have different response times even when repeatedly executing the same operation. For example, a real device receives a command, performs an operation in accordance with the command, and outputs a response after the operation is completed. However, the response time may vary due to factors such as variations in the time required for the operation. In contrast, the VD500 does not respond with a preset fixed response time, but instead generates output data based on multiple output histories of the real device, such that the response time varies for each output. This allows the VD500 to reproduce the response time variation of the real device, thereby contributing to improved simulation accuracy.

[0036] For example, the VD500 simulating the output of the robot hand 51 outputs a response time that varies for each output based on the history of multiple opening and closing operations of the robot hand 51. For example, when the robot hand 51 repeatedly performs a predetermined task and repeatedly grasps a workpiece 60, the time it takes to close the hand may vary depending on the position, posture, and type of the workpiece 60 to be grasped, resulting in a different response time. In response to this, the VD500 does not respond with a preset fixed response time, but instead outputs a response time that varies for each output based on the history of past opening and closing operations of the actual robot hand 51. This allows the VD500 to reproduce the variability in the response time of the actual robot hand 51, thereby contributing to improved accuracy of the simulation. The same applies to other devices such as the vision sensor 52, the camera 53, and an AGV (Automatic Guided Vehicle). The VD500 may output a response time that varies for each output based on the history of multiple operations of the actual device being simulated.

[0037] [Variations in Output Data] The VD500 outputs different output data for each output, for example, based on multiple output histories of the real device. For example, the VD500 outputs different output data for each output, based on multiple past output data of the real device. The real device, for example, performs a predetermined operation to create output data and outputs the output data to the RC30. Even when the same operation is performed repeatedly, the output data output by the real device may vary due to various factors. In contrast, the VD500 does not output fixed output data that is set in advance, but rather outputs different output data for each output based on multiple past output data of the real device. This makes it possible to reproduce variations in the output data of the real device, contributing to improved accuracy of the simulation.

[0038] For example, the VD500, which simulates the output of the vision sensor 52, outputs a different recognition result each time based on multiple recognition results from the vision sensor 52. When the robot 40 repeatedly transports the workpiece 60, the vision sensor 52 repeatedly recognizes the workpiece 60. Even with such repetitive operations, the position (position, posture), type, etc., of the workpiece 60 recognized by the vision sensor 52 in the real environment may differ each time. In contrast, the VD500 does not output a fixed, preset recognition result, but rather outputs a different recognition result each time based on multiple past recognition results from the vision sensor 52 of the actual machine. This allows the VD500 to reproduce the variability in the recognition results of the vision sensor 52 of the actual machine, thereby contributing to improved accuracy of the simulation. The position, etc., of the workpiece 60 in the virtual environment 101 may be changed depending on the output from the VD500. In other words, if the VD500 causes variability in the recognition results of the workpiece 60, the position, etc. of the workpiece 60 in the virtual environment 101 is changed to accommodate the variability. In this way, the recognition results of the VD500 may be reflected in the variables for the operation of the VCC200 and VRC300, and may also be reflected in the placement of the subject workpiece 60. This allows the output by the VD500 to the VCC200 and VRC300 to vary in the same way as the output of the vision sensor 52 of the actual machine, and also allows the variation to be reflected in subsequent scenes where the workpiece 60 is handled, making it possible to execute the simulation normally.

[0039] [Variations Due to Selected Use of Output History] For example, the VD500 selects one output history from multiple output histories of a real device for each output and outputs it. For example, the VD500 selects one response time from multiple response times for each output and performs output at the selected response time. For example, the VD500 selects one output data from multiple output data for each output and outputs the selected output data. It is possible to model and use output variability from multiple output histories, but errors with the actual data may occur during the modeling process. If errors occur, the output of the VD500 may differ from the output of the real device, which may reduce the accuracy of the simulation. In contrast, the VD500 can directly use multiple output histories of the real device to generate output that is proven by the real device, thereby contributing to improved accuracy of the simulation.

[0040] [Using Output History in Chronological Order] The VD500 selects and outputs output history in chronological order from, for example, multiple chronological output histories of real devices. For example, the VD500 selects a response time in chronological order for each output from a list in which the response time histories of real devices are registered in chronological order, and performs output at the selected response time. For example, the VD500 selects output data in chronological order for each output from a list in which the output data histories of real devices are registered in chronological order, and outputs the selected output data.

[0041] It is possible to model output variability from multiple output histories. However, using a model will result in variability trends similar to those of the actual device, but the variability values ​​will be different each time. In contrast, by selecting and using output histories in chronological order from multiple output histories, the output of VD500 will be equivalent to that of the actual device. Under these conditions, programs such as VCC200 and VRC300 can be changed, enabling engineering verification of cell 22 without changing the VD500 conditions. Furthermore, because the VD500 can achieve the same variability each time, differences that occur before and after program changes to VCC200 and VRC300 can be treated as being caused by the programs, such as VCC200 and VRC300, making it easier to confirm the effectiveness of program changes.

[0042] Furthermore, the output variation of a real device may depend on a time series. For example, if a response time is delayed relative to a reference time, the delayed state may continue in some cases, while in other cases the delay may decrease the next time after the delay. As a specific example, if feedback control regarding the response time is performed, the delay may tend to decrease the next time after the delay. In contrast, using a model or randomly selecting from multiple output histories makes it impossible to reproduce the time series dependency. However, by having the VD500 select output histories in chronological order, the time series dependency of the variation can also be reproduced.

[0043] [Output of VD500 Based on Operation History of RC30] The VRC300 performs a simulation based on the operation history of the RC30, and the VD500 may select and output one output history from multiple output histories of the actual device that corresponds to the timing of the operation history on which the VRC300 is based. Table 1 below simply shows an example of the chronological relationship between the operation history of the RC30 and the output history of the device being simulated by the VD500. The VRC300 performs a simulation of an operation using operation history Ra, a simulation of an operation using operation history Rb, and a simulation of an operation using operation history Rc. When outputting an operation performed by the VRC300 using operation history Ra, the VD500 selects output history Da corresponding to the timing of operation history Ra and performs output using the output history Da. When outputting an operation performed by the VRC300 using operation history Rb, the VD500 selects output history Db corresponding to the timing of operation history Rb and performs output using the output history Db. When the VD500 performs output for an operation performed by the VRC300 using the operation history Rc, the VD500 selects an output history Dc corresponding to the timing of the operation history Rc and performs output using the output history Dc. Since there is thought to be a correlation between the variability in the operation of the RC30 and the variability in the device output, if the VD500 performs output using an output history other than the output history Da when performing output for an operation performed by the VRC300 using the operation history Ra, it may not be possible to reproduce the variability in the real environment. In contrast, the VD500 can reproduce the same variability as in the real environment by selecting and outputting one output history corresponding to the timing of the operation history based on the VRC300 from multiple output histories of the actual device, which can contribute to improving the accuracy of the simulation.

[0044]

[0045] [Output of Multiple VDs 500] Two or more VDs 500 may each select and output one output history at a corresponding timing from among a plurality of output histories of the actual devices to which the VDs 500 respectively correspond.

[0046] For example, the first VD 500 and the second VD 500, which respectively simulate the robot hand 51 and the vision sensor 52 arranged at the tip of one of the two robots 40 shown in FIG. 1, select and output one output history at a corresponding timing from the multiple output histories of the robot hand 51 and the vision sensor 52. Table 2 below simply shows an example of the chronological relationship between the output history of the vision sensor 52 and the output history of the robot hand 51. In the case where the output histories of the vision sensor 52 and the robot hand 51 are the example shown in Table 2 below, the first VD 500 and the second VD 500 first select and output the output history Aa and the output history Ba at a corresponding timing, then select and output the output history Ab and the output history Bb at a corresponding timing in chronological order, and then select and output the output history Ac and the output history Bc at a corresponding timing in chronological order.

[0047]

[0048] Also, for example, the first VD500 and second VD500 that respectively simulate the first robot hand 51 and first vision sensor 52 arranged at the tip of the first robot 40 of the two robots 40 shown in Figure 1, and the third VD500 and fourth VD500 that respectively simulate the second robot hand 51 and second vision sensor 52 arranged at the tip of the second robot 40, each select and output one output history at a corresponding timing from the multiple output histories of the first robot hand 51, first vision sensor 52, second robot hand 51, and second vision sensor 52.

[0049] If two or more VDs 500 corresponding to two or more related devices use output histories with different timings, the resulting simulation will have variations different from those in the actual environment, which may reduce the accuracy of the simulation. For example, in a case where a first device operates, then a second device operates, and then the first device operates, and then the second device operates again, using output histories that are unrelated to this order will result in variations different from those in the real environment. However, by selecting and using output histories in this order, it is possible to reproduce the same variations as in the real environment, which can contribute to improving the accuracy of the simulation.

[0050] [Output Variation Due to Probability Distribution, etc.] The VD500 may perform output using a probability distribution created based on multiple output histories. Using this probability distribution, values ​​included in the multiple output histories and values ​​not included in the multiple output histories are output according to the occurrence probability of each value in the multiple output histories. By performing a simulation in which the VD500 performs output using this probability distribution, it is possible to confirm events that occur only 0.1% or 0.01% of the time in the probability distribution. This makes it possible, for example, to easily generate events that are difficult to generate in an actual machine during equipment startup. The VD500 may also perform output using a learning model that is created by machine learning using multiple output histories and that outputs in line with the trends of the multiple output histories.

[0051] [Output Variation Using Minimum and Maximum Values] The VD 500 may generate a different output for each output by randomly generating a value for each output within the range from the minimum to the maximum value of multiple output histories. As a specific example, the VD 500 may identify the minimum and maximum values ​​of multiple response times and randomly generate a response time for each output within the range from the minimum to the maximum value.

[0052] [Manual Setting of Variation] The VD500 may perform output according to a manually set output. For example, the simulator 100 may provide the user with a setting screen on which the output for each output can be set, and accept the setting of the output for each output. The VD500 performs output for each output set by the user.

[0053] [Use of Output History of Actual Device in the Same Environment as Simulation] The VD 500 may perform output using, from among a plurality of output histories of the actual device, a plurality of output histories whose associated environmental information corresponds to the simulation environment.

[0054] For example, the VD500 performs output using, of the multiple output histories, multiple output histories whose associated environmental information matches or approximates the environment of the space to be simulated. As a specific example, when a simulation is performed by specifying the temperature and humidity in the cell 22, the VD500 performs output using, of the multiple output histories of the actual device, multiple output histories whose associated environmental information matches or approximates the specified temperature and humidity. This makes it easier to reproduce the output variation when the actual device operates in the same environment as the environment of the space to be simulated, contributing to improved accuracy of the simulation.

[0055] Furthermore, for example, the VD500 performs output using, from among the multiple output histories, multiple output histories whose associated environmental information corresponds to the operating period of the device to be simulated. As a specific example, when simulating a brand new device, the VD500 performs output using, from among the multiple output histories, multiple output histories whose associated environmental information indicates that the device is brand new. Brand new may mean, for example, that the operating period is within one year. As a specific example, when simulating a five-year-old device, the VD500 performs output using, from among the multiple output histories, multiple output histories whose associated environmental information indicates that the device is five years old. This makes it easier to reproduce the output variation of a brand new device when simulating a brand new device, and easier to reproduce the output variation of a five-year-old device when simulating a five-year-old device, thereby contributing to improved accuracy of the simulation.

[0056] [Another Example of Simulation System 10] The simulation system 10 may include a VD500 that simulates the output of a real device and an RC30 that controls the robot 40 in accordance with the output of the VD500. In this case, the simulator 100 may create only the VD500. By creating a VD500 that outputs based on multiple output histories of the real device in response to commands from the RC30 of the real device, it is possible to simulate input / output between the RC30 and the device in a manner that is close to a real environment. Furthermore, in order to simulate input / output between the RC30 and various devices, it is possible to perform simulations without actually preparing the devices.

[0057] 2 shows an example of the functional configuration of the simulator 100. The simulator 100 includes a storage unit 102, an acquisition unit 104, a creation unit 106, a history reference unit 108, a response time calculation unit 110, a simulation execution unit 112, an image creation unit 114, a communication adjustment unit 116, an application unit 118, a re-planning unit 120, a presentation unit 122, and a change acceptance unit 124. It is not essential that the simulator 100 include all of these units.

[0058] The storage unit 102 stores various types of data. For example, some or all of the data stored in the DB 80 is copied to the storage unit 102.

[0059] The acquisition unit 104 acquires various types of data, such as data input by a user of the simulator 100. The storage unit 102 stores the data acquired by the acquisition unit 104.

[0060] The acquisition unit 104 may acquire data for creating the virtual environment 101. For example, the acquisition unit 104 acquires layout data of the cell 22. The layout data of the cell 22 may include the arrangement of each device in the cell 22. The acquisition unit 104 may acquire data for simulating the CC 20. The data for simulating the CC 20 may include data indicating the function of the CC 20, a program for the CC 20, etc. The acquisition unit 104 may acquire data for simulating the RC 30. The data for simulating the RC 30 may include data indicating the function of the RC 30, a program for the RC 30, etc. The acquisition unit 104 may acquire data for simulating the robot 40. The data for simulating the robot 40 may include data indicating the function of the robot 40, a program for the robot 40, etc. The acquisition unit 104 may acquire data for simulating a device. For example, the acquisition unit 104 acquires a trigger condition and an output target when the trigger condition is satisfied.

[0061] The creation unit 106 creates the virtual environment 101 using the data stored in the storage unit 102. For example, the creation unit 106 creates a VCC 200, a VRC 300, a VR 400, a VD 500, etc., and lays them out to create the virtual environment 101.

[0062] The creating unit 106 creates a VD 500 that outputs the output target when the trigger condition is satisfied, based on the trigger condition and the output target acquired by the acquiring unit 104 .

[0063] The history reference unit 108 references the output history of the actual device that is simulated by the VD 500 created by the creation unit 106. The creation unit 106 may configure the VD 500 to perform output based on the multiple output histories referenced by the history reference unit 108.

[0064] For example, when referring to a response time, if DB 80 stores response times of actual devices, history reference unit 108 refers to the response times stored in DB 80. If response times of actual devices are stored in storage unit 102, history reference unit 108 may refer to the response times stored in storage unit 102.

[0065] If the DB 80 does not store response times for actual devices but instead stores command history data indicating commands to and responses from devices at each time, the response time indexing unit 110 may index multiple response times from multiple command history data. For example, the response time indexing unit 110 identifies a start time at which the state of a command to a device satisfies a start condition and an end time at which the state of a response from the device satisfies an end condition, and indexes the time from the start time to the end time as the response time. Table 3 below shows a simplified example of multiple command history data for the robot hand 51. The time series column shows a simplified timestamp, with larger values ​​indicating earlier time. The execution command column indicates the on / off status of an open / close command from the RC 30 to the robot hand 51. The completion response column indicates the on / off status of a completion response from the robot hand 51 to the RC 30. If the opening / closing command is a command to close the hand, the time when the execution command field changes from 0 to 1 indicates the time when the command is input from the RC 30 to the robot hand 51 and the hand starts to close, and the time when the completion response field changes from 0 to 1 indicates the time when the hand finishes closing and the robot hand 51 outputs a response to the RC 30. The response time indexing unit 110 may index the time from the time when the execution command field changes from 0 to 1 to the time when the completion response field changes from 0 to 1 as the response time. In this way, by having the response time indexing unit 110 index the response time from the command history data, it is possible to prepare a VD 500 with appropriate output variations simply by preparing the command history data of the actual device, thereby reducing the burden of preparing advance data.

[0066]

[0067] For example, when referring to output data, if DB 80 stores output data of a real device, history reference unit 108 refers to the output data stored in DB 80. If output data of a real device is stored in storage unit 102, history reference unit 108 may refer to the output data stored in storage unit 102.

[0068] The creation unit 106 may use multiple output histories referenced by the history reference unit 108 to create a list of outputs for each output by the VD 500. For example, if the VD 500 performs the same operation three times and outputs each time, with the response times varying, the creation unit 106 creates a list including the response times for the first, second, and third outputs. The response times for each output may be the response times themselves; for example, the first response time is 2.9 seconds, the second response time is 3.0 seconds, and the third response time is 3.1 seconds. The response times for each output may be the difference from the default response time; for example, if the default response time is 3.0 seconds, the first response time is -0.1 seconds, the second response time is 0 seconds, and the third response time is +0.1 seconds. The VD 500 may use this list to create the first, second, and third responses.

[0069] The simulation execution unit 112 executes a simulation using the virtual environment 101 created by the creation unit 106. The simulation execution unit 112 simulates the operation of the cell 22 by operating the VCC 200, VRC 300, VR 400, and VD 500 included in the virtual environment 101. By having the VD 500 perform output based on multiple output histories of real devices, the output of the VD 500 can be made closer to the real environment, and the simulation execution unit 112 can execute a highly accurate simulation. A user of the simulator 100 can consider program changes based on the operation of each device in the virtual environment 101 while referring to the simulation results by the simulation execution unit 112.

[0070] The simulation execution unit 112 may execute a simulation using the virtual environment 101 created by the creation unit 106 in a state where a real environment does not exist. In this case, the creation unit 106 may create a VD 500 that outputs according to a manually set output. By executing a simulation using such a virtual environment 101, the simulation execution unit 112 can simulate the operation of the entire cell 22 and verify the feasibility of the cell 22, such as interference and cycle time, before constructing the real environment. After the real environment is set up, data in the real environment is accumulated in the DB 80, and the creation unit 106 constructs the virtual environment 101, which is a virtual version of the real environment.

[0071] The image creation unit 114 creates various images in the virtual environment 101. For example, the image creation unit 114 creates images that are the recognition results of the workpiece 60 by the VD 500 that simulates the vision sensor 52. The image creation unit 114 creates a different image for each output, for example, based on multiple recognition results of the workpiece 60 by the vision sensor 52. The recognition results of the workpiece 60 by the vision sensor 52 may include the position, posture, and type of the workpiece 60, and may also include the image of the workpiece 60 itself. The VD 500 may output the recognition results of the workpiece 60 in the image. By the VD 500 outputting the recognition results of the workpiece 60 in the image that is created differently for each output, a different recognition result is output for each output.

[0072] The communication adjustment unit 116 adjusts communication between each device in the virtual environment 101. The communication adjustment unit 116 may adjust communication between each device in the virtual environment 101 based on communication delay histories between each device in the real environment, which are stored in the DB 80. For example, the communication adjustment unit 116 adjusts the communication delay between the VRC 300 and the VD 500 based on multiple communication delay histories between the RC 30 and the device. The communication adjustment unit 116 may adjust the communication delay between the VRC 300 and the VD 500 based on the multiple communication delay histories so that the communication delay between the VRC 300 and the VD 500 varies similarly to that in the real environment. For example, the communication adjustment unit 116 selects one communication delay history that is different for each communication from the multiple communication delay histories, and reflects the selected communication delay history in the communication between the VRC 300 and the VD 500. The communication adjustment unit 116 may adjust the communication delay between the VRC 300 and the VD 500 so that it varies similarly to a real environment by using a probability distribution, a learning model, or minimum and maximum values ​​of a communication delay history. Just as the communication adjustment unit 116 adjusts the communication delay between the VRC 300 and the VD 500 based on multiple communication delay histories between the RC 30 and the device, the communication adjustment unit 116 may adjust the communication delay between the VCC 200 and the VD 500 based on multiple communication delay histories between the CC 20 and the device. By varying the output of the VD 500 in a similar manner to a real environment and varying the communication delay between the VD 500 and the VRC 300 and the communication delay between the VD 500 and the VCC 200 in a similar manner to a real environment, it is possible to perform a simulation that is closer to a real environment.

[0073] The application unit 118 applies to the real environment the program changes of each device in the virtual environment 101 that were examined through the simulation by the simulation execution unit 112. For example, the application unit 118 applies the program changes of the VRC 300 to the RC 30. For example, the application unit 118 applies the program changes of the VCC 200 to the CC 20.

[0074] The re-planning unit 120 re-plans a program based on the operation of the equipment in the real environment, based on the result of the simulation by the simulation executing unit 112. The program re-planned by the re-planning unit 120 may be applied to the equipment in the real environment by the application unit 118.

[0075] For example, the re-planning unit 120 re-plans the program based on which the RC 30 operates, based on the output of the VD 500, which varies based on multiple output histories of the actual device, and the results of a simulation by the VRC 300 corresponding to that output. By performing a simulation using the VD 500, whose output varies based on multiple output histories of the actual device, it is possible to realize a simulation that takes into account possible variations in device output, and by using the results of that simulation to re-plan the program for the RC 30, it is possible to create a program that can achieve the expected improvements without causing unexpected errors when the program is applied to the RC 30.

[0076] The re-planning unit 120 changes at least one of the selection of commands instructing the robot's operation in the program, the order of the commands, adjustment of command arguments, the logic for calling the commands, and the timing for calling the commands, based on, for example, the output of the VD500 and the results of a simulation by the VRC300 based on the program in response to the output.

[0077] For example, if an error occurs in a simulation performed while varying the output of the VD500, the re-planning unit 120 modifies the program to avoid the error. As a specific example, if an error occurs during the robot 40's operation, such as a collision with an obstacle, the re-planning unit 120 modifies the teaching points of the robot 40 to prevent the collision. The re-planning unit 120 may execute multiple simulations using the VRC300 according to the output of the VD500 based on the modified program to confirm that no errors occur. When the output of the VD500 is fixed, even if it is confirmed that no errors occur, the possibility of errors occurring due to variations in the device output in a real environment cannot be denied. However, the VD500 according to this embodiment can reproduce output variations that may actually occur, thereby reducing the possibility of unexpected errors occurring after applying the modified program to a real environment.

[0078] The re-planning unit 120 may re-plan a program based on the operation of CC20 based on the output of VD500, which varies based on multiple output histories of the actual device, and the results of a simulation by VRC300 in response to that output, just as the re-planning unit 120 re-plans a program based on the operation of CC20 based on the output of VD500, which varies based on multiple output histories of the actual device, and the results of a simulation by VCC200 in response to that output.

[0079] The presentation unit 122 presents information regarding the program replanning by the replanning unit 120 to the user. The presentation unit 122 may present the information to the user by displaying the information on a display or transmitting it to a communication terminal owned by the user. For example, the presentation unit 122 presents to the user a program change proposal for the RC 30, which is a program that has been confirmed to be error-free after multiple simulations have been performed by the VRC 300 in accordance with the output of the VD 500 based on the modified program. For example, the presentation unit 122 presents to the user a program change proposal for the CC 20, which is a program that has been confirmed to be error-free after multiple simulations have been performed by the VCC 200 in accordance with the output of the VD 500 based on the modified program. The change acceptance unit 124 accepts a program change instruction from a user who has viewed the program change proposal. For example, the change acceptance unit 124 accepts a change instruction to change the program of the RC 30 in accordance with the program change proposal presented by the presentation unit 122. In response to the change instruction received by the change receiving unit 124, the application unit 118 may apply the changed program to the RC 30. The change receiving unit 124 may, for example, receive a change instruction to change the program of the CC 20 in accordance with the program change proposal presented by the presentation unit 122. In response to the change receiving unit 124 receiving the change instruction, the application unit 118 may apply the changed program to the CC 20.

[0080] Fig. 3 schematically illustrates an example of the virtual environment 101. The virtual environment 101 illustrated in Fig. 3 includes a VCC 200, a VRC 300, a VR 400, a V robot hand 510 that simulates the robot hand 51, a V vision sensor 520 that simulates the vision sensor 52, a V camera 530 that simulates the camera 53, a V workpiece 600, a V stand 720, a V stand 740, a V stand 760, and a VDB (Virtual Database) 800.

[0081] The VCC 200 controls the cells of the virtual environment 101 according to a program. The VRC 300 controls the VR 400 according to a program. The V robot hand 510 outputs based on multiple output histories of the robot hand 51. The V robot hand 510 may output so as to reproduce the variability in the output of the robot hand 51. The V vision sensor 520 outputs based on multiple output histories of the vision sensor 52. The V vision sensor 520 may output so as to reproduce the variability in the output of the vision sensor 52. The V camera 530 outputs based on multiple output histories of the camera 53. The V camera 530 may output so as to reproduce the variability in the output of the camera 53. The VDB 800 accumulates various data within the virtual environment 101.

[0082] The simulation execution unit 112 may execute a simulation using the virtual environment 101. For example, the simulation execution unit 112 executes a simulation in which a first VR 400 grasps a V workpiece 600 placed on a V stand 720 and moves it onto the V stand 740, and a second VR 400 grasps the V workpiece 600 placed on the V stand 740 and moves it to the V stand 760. In this simulation, each VD 500 outputs data to reproduce variations in the target real machine based on multiple output histories of the target real machine. For example, the V camera 530 varies the response time and recognition results based on multiple output histories of the camera 53. For example, the V vision sensor 520 varies the recognition results and response time of the V workpiece 600 based on multiple output histories of the vision sensor 52. For example, the V robot hand 510 varies the response time based on multiple output histories of the robot hand 51.

[0083] When the V vision sensor 520 outputs a recognition result for the V workpiece 600 based on multiple output histories of the vision sensor 52, the simulation execution unit 112 may reflect the recognition result in the V workpiece 600. For example, the simulation execution unit 112 may change the placement of the V workpiece 600 so that it matches the placement indicated by the recognition result output by the V vision sensor 520. This makes it possible to vary the output from the V vision sensor 520 to the VRC 300 in accordance with the variability in the output from the vision sensor 52 to the RC 30, and to change the placement of the target V workpiece 600 in accordance with that variability. This allows the variability to be reflected in subsequent handling of the V workpiece 600, contributing to improved accuracy of the simulation.

[0084] 4 shows an example of a process flow in the simulation system 10. Here, an example of a process flow in which the simulator 100 creates a VD 500 that simulates a target device will be described.

[0085] In step (sometimes abbreviated as S) 102, the acquisition unit 104 acquires a trigger condition and an output target when the trigger condition is satisfied. For example, the acquisition unit 104 acquires, as the trigger condition, receipt of a command from the VRC 300, and acquires, as the output target, a response to be output by executing an operation in accordance with the command.

[0086] In S104, the history reference unit 108 references multiple output histories of the target device. In S106, the creation unit 106 creates a VD 500 that outputs the output target when the trigger condition acquired by the acquisition unit 104 in S102 is satisfied. The creation unit 106 creates a VD 500 that performs output based on the multiple output histories referenced by the history reference unit 108 in S104. The creation unit 106 uses the multiple output histories, for example, to create a list including outputs for each output of the VD 500, and sets the VD 500 to perform output in accordance with the list.

[0087] 5 shows an example of a processing flow in the simulation system 10. Here, an example of a processing flow will be described in which the VD 500 receives output settings based on a plurality of output histories, executes an operation according to an instruction from the VRC 300, and executes a task of outputting to the VRC 300 a predetermined number of times.

[0088] In S202, the VD 500 receives output settings from the creation unit 106. The creation unit 106 sets the output of the VD 500 so that the output of the VD 500 varies for each output. Here, the creation unit 106 sets the VD 500 so that output is performed using a list including multiple chronological output histories of the target device.

[0089] In S204, the VD 500 receives a command from the VRC 300. In S206, the VD 500 performs output to the VRC 300 in accordance with the output setting received in S202. The VD 500 performs output using the first output history in the list.

[0090] If the task has ended (YES in S208), the processing ends, and if the task has not ended (NO in S208), the processing returns to S204. In S204, the VD 500 receives a command from the VRC 300. In S206, the VD 500 performs output using the second output history in the list. In this way, the VD 500 selects and uses output history in order from the first output history in the list for each output. As a result, the output of the VD 500 varies for each output. Note that while FIG. 5 describes a case in which the VD 500 performs an operation in response to a command from the VRC 300 and performs an output to the VRC 300, the same may also be true for a case in which the VD 500 performs an operation in response to a command from the VCC 200 and performs an output to the VCC 200.

[0091] 6 shows an example of a process flow in the simulation system 10. Here, the re-planning unit 120 re-plans a program based on the operation of the VRC 300 while causing the simulation execution unit 112 to execute a simulation in which the same task is repeatedly executed in the virtual environment 101.

[0092] In S302, the simulation execution unit 112 starts executing the task. In the simulation, the VRC 300 operates according to the program, the VR 400 operates according to commands from the VRC 300, and the VD 500 performs output based on multiple output histories of the actual device in response to the commands from the VRC 300. The output of the VD 500 varies for each output.

[0093] If no error occurs during the progress of the task (NO in S304) and the task is completed (YES in S306), the process proceeds to S310. If an error occurs during the progress of the task (YES in S304), the process proceeds to S308.

[0094] In S308, the re-planning unit 120 re-plans the program for the VRC 300. The re-planning unit 120 re-plans the program for the VRC 300 so as to avoid the error that has occurred.

[0095] In S310, the re-planning unit 120 determines whether or not to end the simulation. If it is determined not to end the simulation, the process returns to S302, and if it is determined to end the simulation, the process proceeds to S312. The re-planning unit 120 determines to end the simulation if the task has been executed a predetermined number of times without an error occurring. For example, if the task has been completed without an error occurring, the re-planning unit 120 counts up the number of times, and if an error occurs, the re-planning unit 120 resets the number of times.

[0096] If the program for the VRC 300 has been re-planned between the start and end of the simulation (YES in S312), the process proceeds to S314; if not, the process terminates. In S314, the application unit 118 applies the final re-planned program for the VRC 300 to the RC 30. While FIG. 6 illustrates the case in which the re-planning unit 120 re-plans a program based on the operation of the VRC 300, the re-planning unit 120 may similarly re-plan a program based on the operation of the VCC 200. That is, in the simulation in S302, the VCC 200 operates according to the program, and the VD 500 outputs, in response to commands from the VCC 200, based on multiple output histories of the actual device. In S308, the re-planning unit 120 re-plans the program for the VCC 200, and in S314, the application unit 118 applies the final re-planned program for the VCC 200 to the CC 20. The re-planning unit 120 may execute the re-planning of the program for the VCC 200 and the re-planning of the program for the VRC 300 together.

[0097] 7 schematically illustrates an example of a hardware configuration of a computer 1200 that functions as the simulation system 10 or simulator 100. A program installed on the computer 1200 may cause the computer 1200 to function as the virtual environment 101. A program installed on the computer 1200 may cause the computer 1200 to function as the simulator 100. Such a program may be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein. Each of the VCC 200, VRC 300, VR 400, and VD 500 may be realized by one or more CPUs 1212.

[0098] The computer 1200 includes a CPU 1212, a RAM 1214, and a graphics controller 1216, all of which are interconnected by a host controller 1210. The computer 1200 also includes a communications interface 1222, a storage device 1224, and input / output units such as a DVD drive and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240. The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 and displays the image data on a display device 1218. The communications interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212. Information processes written in the programs are read by the computer 1200 and cause the programs to cooperate with the various types of hardware resources described above.

[0099] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of a device responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0100] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium with instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. The computer-readable instructions may be provided locally or via a wide area network (WAN) such as a local area network (LAN), the Internet, or the like to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus or programmable circuitry, such that the processor or programmable circuitry executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, central processing units, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like. A computer may comprise one processor or multiple processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0101] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0102] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0103] 10 Simulation system, 20 CC, 22 Cell, 30 RC, 40 Robot, 51 Robot hand, 52 Vision sensor, 53 Camera, 60 Work, 72, 74, 76 Unit, 80 DB, 100 Simulator, 101 Virtual environment, 102 Memory unit, 104 Acquisition unit, 106 Creation unit, 108 History reference unit, 110 Response time calculation unit, 112 Simulation execution unit, 114 Image creation unit, 116 Communication adjustment unit, 118 Application unit, 120 Re-planning unit, 122 Presentation unit, 124 Change acceptance unit, 200 VCC, 300 VRC, 400 VR, 500 VD, 510 V robot hand, 520 V vision sensor, 530 V camera, 600 V work, 720, 740, 760 V-unit, 800 VDB, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. A simulation system for simulating the operation of a cell including robots and devices, A virtual device that simulates the output of the aforementioned device, A virtual robot controller that simulates the operation of the robot controller that controls the robot according to the output of the virtual device, and Equipped with, The virtual device outputs based on a plurality of output histories of the device. Simulation system.

2. The simulation system according to claim 1, wherein the virtual device performs a different output each time based on the plurality of output histories.

3. A replanning unit replans the program based on the operation of the robot controller, based on the output of the virtual device and the results of the simulation by the virtual robot controller corresponding to that output. The simulation system according to claim 1 or 2, further comprising the above.

4. The simulation system according to claim 3, wherein the replanning unit, based on the modified program, performs a simulation multiple times using the virtual robot controller according to the output of the virtual device to confirm that no errors occur.

5. The simulation system according to claim 3, wherein the replanning unit modifies at least one of the following in the program based on the output of the virtual device and the results of a simulation by the virtual robot controller based on the program corresponding to the output: the selection of commands to instruct the robot's movement, the order of commands, the adjustment of command arguments, the logic for calling commands, and the timing for calling commands.

6. The simulation system according to claim 2, wherein the virtual device performs the output such that the response time differs for each output based on the plurality of output histories.

7. The output history of the device includes command history data showing commands to the device and responses from the device at each time point. A response time calculation unit determines multiple response times from multiple instruction history data, from the input of an instruction to the device to the output of a response by the device. Furthermore, The simulation system according to claim 6, wherein the virtual device performs the output such that the response time differs for each output based on the plurality of response times determined by the response time calculation unit.

8. The simulation system according to claim 6, wherein the virtual device for simulating the output of a robot hand performs the output such that the response time differs for each output based on the history of multiple opening and closing operations of the robot hand.

9. The output history of the device includes the output data that the device outputs to the robot controller. The simulation system according to claim 1 or 2, wherein the virtual device outputs different output data for each output based on the plurality of output histories.

10. The simulation system according to claim 9, wherein the virtual device that simulates the output of a vision sensor for recognizing a workpiece outputs a different recognition result each time it outputs based on a plurality of recognition results of the vision sensor.

11. Image creation unit that creates a different image each time it outputs based on the multiple recognition results of the vision sensor. Furthermore, The simulation system according to claim 10, wherein the virtual device outputs the recognition result of the workpiece in the image.

12. The simulation system according to claim 1 or 2, wherein the virtual device selects and outputs one different output history from the plurality of output histories for each output.

13. The simulation system according to claim 12, wherein the virtual device selects and outputs the output history in chronological order from the plurality of output histories in chronological order.

14. The simulation system according to claim 12, wherein two or more virtual devices each select and output one output history of a corresponding timing from the plurality of output histories of the device to which each virtual device corresponds.

15. The virtual robot controller performs a simulation based on the operation history of the robot controller. The simulation system according to claim 12, wherein the virtual device selects and outputs one output history from the plurality of output histories that corresponds to the timing of the operation history based on the virtual robot controller.

16. An acquisition unit that acquires a trigger condition and the output target when the trigger condition is met, Based on the trigger condition and output target acquired by the acquisition unit, a creation unit creates a virtual device that outputs the output target when the trigger condition is met. The simulation system according to claim 1 or 2, further comprising the above.

17. A communication adjustment unit adjusts the communication delay between the virtual device and the virtual robot controller based on a plurality of communication delay histories between the device and the robot controller. The simulation system according to claim 1 or 2, further comprising the above.

18. A program for causing a computer to function as the simulation system described in claim 1 or 2.

19. A simulation method for simulating the operation of a cell including robots and devices, A virtual device that simulates the output of the aforementioned device has an output stage in which it outputs based on a plurality of output histories of the aforementioned device, A simulation step in which the operation of the robot controller that controls the robot is simulated according to the output of the virtual device. A simulation method comprising the following features.

20. A simulation system construction apparatus for constructing a simulation system that simulates the operation of a cell including robots and devices, A creation unit that creates a virtual device that simulates the output of the aforementioned device, and a virtual robot controller that simulates the operation of a robot controller that controls the robot according to the output of the virtual device, and that outputs based on a plurality of output histories of the aforementioned device. A device for constructing a simulation system, equipped with the necessary components.

21. A simulation system for simulating the operation of a cell including robots and devices, A virtual device that simulates the output of the aforementioned device, A robot controller that controls the robot in accordance with the output of the virtual device. Equipped with, The virtual device outputs based on a plurality of output histories of the device. Simulation system.