Robot system, link application, robot simulation method, and command application manufacturing method

Abstract

This robot system comprises: a physical simulator capable of calculating, in a simulation space having a virtual robot that is a 3D model of a robot, a physical phenomenon at least with respect to the virtual robot; a controller capable of calculating the behavior of a robot of a type corresponding to the virtual robot; and a link application that links the physical simulator with the controller.

Description

Robot system, cooperation application, robot simulation method, and manufacturing method of command application

[0001] The present disclosure relates to a robot system, a cooperation application, a robot simulation method, and a manufacturing method of a command application.

[0002] Patent Document 1 discloses a system including a plurality of local controllers that respectively control a plurality of local devices including a robot, and a simulation unit that simulates operations to be executed by the plurality of local controllers on the plurality of local devices based on model information of the plurality of local devices.

[0003] Japanese Unexamined Patent Application Publication No. 2021-086393

[0004] The present disclosure provides a system that can easily deploy the results of simulation to an actual robot.

[0005] A robot system according to an aspect of the present disclosure includes a physical simulator capable of calculating at least physical phenomena for a virtual robot, which is a 3D model of a robot, in a simulation space having the virtual robot, a controller capable of calculating the behavior of a robot of a model corresponding to the virtual robot, and a cooperation application that performs cooperation between the physical simulator and the controller.

[0006] Another aspect of this disclosure relates to a collaborative application that facilitates collaboration between a physical simulator and a controller, wherein the physical simulator calculates at least physical phenomena for a virtual robot in a simulation space having a virtual robot which is a 3D model of a robot, the controller calculates the behavior of a robot of a model corresponding to the virtual robot, and the collaborative application passes the calculation results of the robot's behavior from the controller to the physical simulator so that the physical simulator drives the virtual robot based on the calculation results of the robot's behavior by the controller, and passes the calculation results of the physical phenomena from the physical simulator to the controller so that the controller calculates the behavior based on the calculation results of the physical phenomena by the physical simulator.

[0007] A robot simulation method relating to yet another aspect of this disclosure involves a physical simulator that calculates at least the physical phenomena for a virtual robot in a simulation space having a virtual robot which is a 3D model of a robot; a controller that calculates the behavior of a robot of a model corresponding to the virtual robot; and a collaborative application that facilitates communication between the physical simulator and the controller.

[0008] A method for manufacturing a command application relating to yet another aspect of this disclosure includes: generating commands using a command application; calculating the robot's behavior based on the commands using a controller; driving a virtual robot, which is a 3D model of the robot, in a simulation space using a physical simulator based on the calculation results of the robot's behavior, and calculating at least physical phenomena for the virtual robot; and adjusting parameters for generating commands in the command application using a parameter adjustment unit based on at least one of the calculation results of the physical phenomena and the calculation results of the robot's behavior.

[0009] According to this disclosure, it is possible to provide a system that makes it easy to apply the results of simulations to actual robots.

[0010] This is a schematic diagram illustrating the configuration of a robot system. This is a timing chart illustrating the progress of calculations in a simulation. This is another timing chart showing another example of the progress of calculations in a simulation. This is a diagram illustrating a modified version of the robot system. This is a block diagram illustrating the hardware configuration of the simulation device. This is a flowchart illustrating the robot simulation procedure. This is a flowchart showing another example of the robot simulation procedure.

[0011] The embodiments will be described in detail below with reference to the drawings. In the description, the same elements or elements having the same function will be denoted by the same reference numeral, and redundant descriptions will be omitted.

[0012] [Robot System] The robot system 1 shown in Figure 1 is a system in which a robot 10 performs operations on a workpiece 21, for example, in a production facility. Examples of operations on the workpiece 21 include grasping the workpiece 21 and moving it to another location, combining multiple workpiece 21 parts, applying paint to the surface of the workpiece 21, welding the workpiece 21, polishing the workpiece 21, and applying sealing material to the surface of the workpiece 21.

[0013] The robot system 1 comprises a robot 10, a controller 110, and a simulation device 200. The robot 10 is, for example, an industrial vertical articulated robot, which performs operations on the workpiece 21 in real space RS.

[0014] For example, robot 10 has a base 11, an arm 12, and an end effector 13. The base 11 is fixed to a floor or ceiling surface, etc. The arm 12 is a serial link mechanism having multiple links 15 connected by multiple joints 14. The arm 12 may be a so-called SCARA type. The arm 12 is not limited to a serial link type, but may also be a parallel link type. The end effector 13 is attached to the tip of the arm 12 and is a part that acts directly on the workpiece 21, and is replaceable depending on the work content.

[0015] The controller 110 controls the robot 10 to perform tasks on the workpiece 21 according to a pre-prepared program. For example, the controller 110 calculates command values ​​that determine the movement of the robot 10 (for example, command values ​​for the angle or torque of each joint 14) according to the program, obtains feedback values ​​that represent the state of the robot 10 (for example, feedback values ​​for the angle or torque of each joint 14), and drives the robot 10 so that the feedback values ​​follow the command values, repeating this process at a predetermined control cycle. The simulation device 200 is a device that verifies the program by simulation in order to operate the actual robot 10 according to the program.

[0016] Industrial robots like Robot 10 have been widely used in many manufacturing industries as an essential element for automating production lines. However, in recent years, manufacturing has shifted from mass production to high-mix, low-volume production to meet diverse customer needs. Along with this change, complex and diverse tasks that are difficult to handle with conventional, uniform automation technologies still rely on human labor. To solve these unautomated challenges, AI (artificial intelligence) technology, particularly AI models pre-trained in a simulation environment, can be used to enable robots to perform complex tasks. However, when attempting to apply AI models trained in a simulation environment to real-world robots, a major obstacle known as the "sim-to-real gap" arises. This gap is caused by physical phenomena that cannot be fully considered in simulations, as well as errors in the robot's own movements. Therefore, even ambitious AI models that show good results in simulations often fail to function as expected in actual production environments, preventing their practical application.

[0017] Therefore, the simulation device 200 has a physical simulator 210, a controller 220, and a collaborative application 230 as functional components (hereinafter referred to as "functional blocks"). The robot system 1 has a physical simulator 210, a controller 220, and a collaborative application 230. The physical simulator 210 calculates physical phenomena for at least the virtual robot V10 in the simulation space VS. The simulation space VS has a virtual robot V10, which is a 3D model of the robot 10. The controller 220 calculates the behavior of the robot 10 of a model corresponding to the virtual robot V10. The collaborative application 230 coordinates between the physical simulator 210 and the controller 220.

[0018] Physical phenomena other than the behavior of the robot 10 (forces exerted by the robot 10 on surrounding objects, forces received by the robot 10 from surrounding objects, the effect of gravity, images obtained by the camera, etc.) can be calculated by the physical simulator 210 without the actual robot being operated. A separate controller 220 can calculate the behavior of the robot 10, which is close to that of the actual robot and reflects the characteristics of the controller 220. By combining the results of the robot 10 behavior calculation by the controller 220 and the results of the physical phenomenon calculation by the physical simulator 210 using the linked application 230, trial and error with the actual robot in real space RS can be simulated with high accuracy. As a result, the barrier (gap between simulation and reality) to deploying the system configuration and programs constructed by simulation to real space RS is reduced. Therefore, a system can be provided that makes it easy to deploy the results of the simulation to the actual robot 10. With this system, the system configuration and programs can be quickly refined by simulation without performing trial and error with the actual robot, and the results can be smoothly deployed to the actual robot 10.

[0019] According to the simulation device 200, by closely linking the advanced simulation technology of the physical simulator with the high-precision robot control technology of the robot controller, a synergistic effect between the two can be extracted, reducing the gap between the simulation and the real world. The physical simulator can reproduce a variety of scenes close to the real world, such as the shape, texture, and physical properties of the workpiece (object to be worked on), and even the state in which they are placed randomly (randomly stacked). However, if the behavior of the virtual robot operating in that simulation space deviates from the behavior of the real robot, it will not be possible to accurately reproduce the movements planned in the simulation (for example, the suction position and posture of the workpiece, the movement trajectory, etc.) with the real robot. Therefore, the simulation device 200 is configured to accurately reflect the robot's operating characteristics (for example, drive limits and response characteristics, etc.) of the robot controller in the simulation. This makes it possible to formulate an action plan that the robot can execute at the simulation stage.

[0020] For example, it is possible to verify in advance whether a certain motion trajectory will exceed the robot's drive limits and generate an error (alarm), and to calculate the optimal trajectory that does not generate errors. By combining a physical simulator that can reproduce the diversity of physical phenomena and environments with a robot controller that governs the accuracy and autonomy of the robot's movements, it is possible to provide a "Sim-to-Real" system in which a real-world robot can seamlessly and accurately reproduce a series of tasks created in the simulation (e.g., picking up bulk-stacked products, retrying actions in case of gripping failure, etc.) with little to no additional adjustments (engineering) required on-site. With such a high-precision Sim-to-Real environment, it is possible to combine the "judgment ability" acquired by combining AI with the robot's inherent "work ability," promote robot autonomy, and provide a powerful platform for evolving robotics solutions. In future manufacturing sites that require response to diverse environments and work objects, and where constraints such as time, place, and cost make verification in the real environment increasingly difficult, it becomes possible to make the most of the simulation environment and expand the application area of ​​automation.

[0021] "Physical phenomena" are distinguished from kinematic simulations, which merely calculate changes in geometric positional relationships. The calculation of physical phenomena includes, at a minimum, dynamic simulations that calculate mechanical interactions, or optical simulations that calculate optical phenomena. "Physical phenomena for the virtual robot V10" include, for example, mechanical interactions such as the forces the virtual robot V10 exerts on an object it contacts and the forces it receives from that object. It may also include the effects of gravity on each axis and end effector of the virtual robot V10 due to its own mass and the mass of the object it grasps. Furthermore, physical phenomena also include optical phenomena. For example, the effect of light sources in the simulation space on the virtual robot V10, such as the way images are captured by a camera mounted on the virtual robot V10 and the generation of shadows. Physical phenomena are not limited to these and may include various phenomena depending on the simulation target. The physical simulator 210, for example, has rendering technology and a physics calculation engine that can construct a photorealistic 3D environment and perform high-fidelity simulations based on the physical laws of rigid bodies, soft bodies, fluids, etc. By using such a physical simulator 210, it is possible to generate a virtual environment that is extremely close to the real world and reproduce physical phenomena within it with high accuracy.

[0022] "Robot behavior" broadly refers to information that defines the operation of robot 10. This includes both commands that define the operation of robot 10 as a target (e.g., target joint angles and end-effector positions) and the operation of robot 10 that is estimated to actually occur in response to those commands (e.g., joint angles and end-effector positions as responses to the commands). Specific examples of robot behavior include command values ​​for the angles of each joint 14, estimated feedback values ​​for the command values ​​of the angles of each joint 14, command values ​​for the motor torque of each joint 14, estimated feedback values ​​for the command values ​​of the torque of each joint 14, command values ​​for the position and orientation of the tip of arm 12 (or end effector 13), estimated feedback values ​​for the position and orientation of the tip of arm 12, command values ​​for the force generated by arm 12, and estimated feedback values ​​for the command values ​​of the force generated by arm 12. "Estimated feedback values" are predicted values ​​of the actual feedback values ​​calculated considering the response characteristics of robot 10 to the command values ​​(e.g., response delay, gain, inertia, etc. of the servo system).

[0023] The controller 220 may be a virtual controller that simulates the controller 110. A virtual controller is a software program that calculates the behavior of the robot 10 using the same algorithm as the control program of the controller 110 that controls the robot 10 in real space RS. In this case, the controller 220 may calculate command values ​​that define the movement of the robot 10 as the behavior of the robot 10, similar to the controller 110. Alternatively, the controller 110 itself may be used as the controller 220. When the controller 110 is used as the controller 220, the actual robot 10 is not driven (kept offline), and the command output destination of the controller 110 is switched to the virtual robot V10 of the physical simulator 210 via the linked application 230, allowing the robot 10 to participate in the simulation.

[0024] By configuring the controller 220 not to be part of the physical simulator 210, and instead linking the controller 220 and the physical simulator 210 via a linking application 230, the physical simulator 210 can be developed independently of the controller 220 (e.g., improved accuracy, expanded model types). Furthermore, the controller 220 can also be developed independently of the physical simulator 210. The linking application 230 may be an application (program) independent of the physical simulator 210 and the controller 220, or it may be an add-in program integrated into the physical simulator 210 or the controller 220. Even if the physical simulator 210 has its own robot motion generation engine, the simulation device 200 does not use that engine, but instead drives the virtual robot V10 based solely on the behavior generated by the external controller 220. Even if the physical simulator 210 and the controller 220 are programs running on the same computer, they are treated as functionally independent and separate applications. The linking application 230 plays the role of mediating data between these two independent applications and enabling coordinated operation. To fulfill this role, the collaborative application 230 has data conversion functions, synchronization functions, API call functions, and the like.

[0025] The collaborative application 230 may pass the calculation results of the robot 10's behavior from the controller 220 to the physical simulator 210 so as to drive the virtual robot V10 in the physical simulator 210 based on the calculation results of the robot 10's behavior by the controller 220, and may also pass the calculation results of physical phenomena from the physical simulator 210 to the controller 220 so as to cause the controller 220 to calculate the behavior based on the calculation results of physical phenomena by the physical simulator 210. This allows the virtual robot V10 in the physical simulator 210 to move more closely to the robot 10 in the real space RS, thereby reducing the difference between the simulation and reality. Causing the controller 220 to calculate the behavior includes the collaborative application 230 instructing the controller 220 to start the calculation (for example, an action that can be expressed as triggering the start of processing or kicking off processing).

[0026] The collaborative application 230 may convert the data received from the controller 220 into a format that the physical simulator 210 can interpret and pass it to the physical simulator 210. For example, the collaborative application 230 may convert the data into a format that the physical simulator 210 can interpret and write it to a memory area referenced by the physical simulator 210. The collaborative application 230 may also convert the data received from the physical simulator 210 into a command that conforms to the specifications of an API (Application Programming Interface) provided by the other party and call that API (API call). With this configuration, it becomes unnecessary to make special modifications to the general-purpose physical simulator 210 for cooperation with the controller 220, and the independence between the physical simulator 210 and the controller 220 can be increased.

[0027] The physical simulator 210, having received the calculation results of the robot 10's behavior, reproduces the robot 10's behavior in the simulation space VS using a virtual robot V10, and calculates the physical phenomena that occur in the simulation space VS as a result. Examples of physical phenomena that occur in the simulation space VS include the position, velocity, acceleration of each part of the virtual robot V10, or external forces that V10 receives from the outside. For example, if the physical simulator 210 receives command values ​​for the joint angles of each joint of the robot 10 as a calculation result of the robot 10's behavior, it moves each joint of the virtual robot V10 to the angles specified by those command values. Then, as a result of that movement, it calculates physical phenomena such as the reaction force that occurs when the virtual robot V10 comes into contact with the virtual workpiece V21, and the visual changes in the image captured by the virtual camera V17.

[0028] The collaborative application 230 may convert the data received from the physical simulator 210 into a format that the controller 220 can interpret and pass it to the controller 220. For example, it may write the virtual feedback data calculated by the physical simulator 210 to a memory area that the controller 220 references to obtain feedback values ​​from the actual device's sensors. Alternatively, it may call an API for feedback input provided by the controller 220. This configuration eliminates the need to make special modifications to the controller 220 (for example, modifications to incorporate different functions from the controller 110) for collaboration with the physical simulator 210, thereby increasing the independence between the physical simulator 210 and the controller 220.

[0029] The physical simulator 210 may calculate virtual feedback data as a result of calculating physical phenomena. The virtual feedback data may be feedback generated by the virtual robot V10 due to physical phenomena caused by the behavior of the virtual robot V10. The collaborative application 230 may pass the virtual feedback data from the physical simulator 210 to the controller 220. The controller 220 may calculate the behavior of the robot 10 based on the virtual feedback data. Physical phenomena can be easily reflected in the calculation of the behavior of the robot 10.

[0030] "Virtual feedback data" refers to data simulated in the simulation space VS, which is the feedback data acquired by the controller 110 when the actual robot 10 is controlled in the real space RS. Examples of feedback data include torque feedback values, force feedback values, and images of the workpiece 21 captured by a camera.

[0031] The simulation space may have virtual sensors corresponding to the sensors of the real space RS. The physical simulator 210 may calculate virtual feedback data including the detection results from the virtual sensors.

[0032] In the illustrated example, the robot 10 further includes a force sensor 16 and a camera 17 as examples of sensors for the real-world RS. The force sensor 16 detects the force acting on the tip of the arm 12. An example of the force sensor 16 is a load cell. The force sensor 16 may also be a six-axis force sensor that detects the force along each of the three orthogonal axes and the torque around each axis.

[0033] Camera 17 is fixed to the tip of arm 12 and photographs the area around arm 12 from a viewpoint linked to the tip of arm 12. When camera 17 is pointed towards workpiece 21, camera 17 photographs workpiece 21. Camera 17 is a digital camera having, for example, an optical system such as a lens and an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and may be capable of acquiring color images, depth images, or both.

[0034] The simulation space VS includes a virtual sensor V16 corresponding to the force sensor 16 and a virtual camera V17 corresponding to the camera 17. The physical simulator 210 calculates the force acting on the virtual sensor V16 in the simulation space VS as a result of detection by the virtual sensor V16. The physical simulator 210 also calculates the image of the simulation space VS captured by the virtual camera V17 as a result of detection by the virtual camera V17. The linked application 230 passes the detection results from the virtual sensor V16 and virtual camera V17 from the physical simulator 210 to the controller 220. For example, the linked application 230 receives the detection results from the virtual sensor V16 and virtual camera V17 from the physical simulator 210 and writes the received detection results to a memory area that the controller 220 references to acquire the detection results from the force sensor 16 and camera 17.

[0035] The controller 220 calculates the behavior of the robot 10 based on the recorded detection results. For example, based on the detection results (force and torque) of the virtual sensor V16, the controller 220 calculates the behavior to correct the position and orientation of the tip of the arm 12 so that the robot 10 does not apply excessive force to the workpiece 21, or presses it with a constant force (force control). In addition, based on the detection results (image) of the virtual camera V17, the controller 220 recognizes the position and orientation of the workpiece 21 and calculates the behavior to accurately position the tip of the arm 12 according to the recognition results (visual servo).

[0036] The simulation space VS may have a virtual work V21. The virtual work V21 corresponds to the work 21 that the robot 10 works on in the real space RS. The physical simulator 210 may calculate virtual feedback data generated in the virtual robot V10 due to physical phenomena occurring between the virtual work V21 and the virtual robot V10 as a result of the behavior of the virtual robot V10. The accuracy of the simulation can be further improved by including the influence that the robot 10 receives from the work 21. For example, the physical simulator 210 may calculate the force detected by the virtual sensor V16 when the virtual work V21 and the virtual robot V10 are in contact, or it may calculate images taken of the virtual work V21 by a virtual camera V17 that is linked to the behavior of the virtual robot V10.

[0037] The simulation device 200 may further include a command application 240 as a functional block. The command application 240 generates commands for the controller 220 based on physical phenomena and outputs them to the controller 220. The controller 220 calculates the behavior of the robot 10 based at least on the commands generated by the command application 240. By adjusting the command application 240 through simulation, which facilitates the deployment of the results to the actual robot 10 (for example, by reinforcement learning), the adjusted command application 240 can be smoothly deployed to the actual robot 10.

[0038] The command application 240 may be a model that is adjusted by machine learning (e.g., reinforcement learning by simulation) to generate commands to the controller 220 based on physical phenomena. For example, if the physical phenomenon is the result of detection by the virtual camera V17, the command application 240 may be an image recognition / pose estimation model that determines the workpiece to be gripped, the optimal gripping position and gripping posture of the workpiece from an image of multiple randomly stacked virtual workpieces V21, and generates a command to move the end effector 13 to that position and posture. Alternatively, if the physical phenomenon is the result of detection by the virtual sensor V16, the command application 240 may be a model that generates a command to determine the next action (e.g., adjust the force, try a different location, etc.) according to the detected force. The commands generated by the command application 240 are not particularly limited in type, as long as they can be interpreted by the controller 220. For example, they may be a position command indicating a target position, a speed command indicating a target speed, or a command to set the operating mode or parameters of the controller 220. The command application 240 may be an application (program) independent of the physical simulator 210 and the controller 220, or it may be an add-in program integrated into the physical simulator 210 or the controller 220.

[0039] The adjusted command application 240 is used by the controller 110 to control the robot 10. For example, a copy of command application 240, command application 130, is implemented in the real-world RS. For example, command application 130 is incorporated into a computing device 120 that can communicate with the controller 110. Command application 130 may also be incorporated directly into the controller 110. Command application 130 generates commands based on physical phenomena in the real-world RS (for example, images captured by the camera 17), and the controller 110 controls the robot 10 based on the commands generated by command application 130.

[0040] The collaborative application 230 may cause the controller 220 to calculate the behavior of the robot 10 based on the commands generated by the command application 240 and the calculation results of the physical phenomena. The accuracy of the simulation can be improved by including both the influence of the calculation results of the physical phenomena on the behavior of the robot 10 via commands and the direct influence of the calculation results of the physical phenomena on the behavior of the robot 10.

[0041] The physical phenomena used to generate commands by the command application 240 and the physical phenomena used to calculate the behavior of the robot 10 by the controller 220 may be different from or the same. For example, the command application 240 may generate commands based on the detection results from the virtual camera V17, and the controller 220 may calculate the behavior of the robot 10 based on the generated commands and the detection results from the virtual sensor V16.

[0042] The simulation device 200 may further include a parameter adjustment unit 250 as a functional block. The parameter adjustment unit 250 adjusts the parameters for generating commands in the command application 240 based on at least one of the calculation results of physical phenomena and the calculation results of the behavior of the robot 10 based on commands generated by the command application 240. The controller 220 calculates the behavior of the robot 10 based on the commands output from the command application 240 with adjusted parameters. The command application 240 can be easily adjusted by adjusting the parameters. The simulation device 200 may repeat the calculation of physical phenomena by the physical simulator 210, the calculation of commands by the command application 240, the calculation of the behavior of the robot 10 by the controller 220, and the adjustment of parameters by the parameter adjustment unit 250 until at least one of the calculation results of physical phenomena and the calculation results of the behavior of the robot 10 reaches a desired state.

[0043] The parameter adjustment unit 250 may adjust parameters according to a user input based on at least one of the calculation result of a physical phenomenon and the calculation result of the behavior of the robot 10 based on the command generated by the command application 240. The parameters to be adjusted may include not only numerical values but also command data that defines the movement path of the robot 10 itself. When the command application 240 is a model adjusted by machine learning, parameter adjustment includes the act of re-learning or additional learning of the model using the simulation result as teacher data. By using the result obtained in the simulation as a feedback value in the backpropagation method, learning can be performed more accurately and in a shorter time.

[0044] The simulation device 200 may further include a load generation unit 260 as a functional block. The load generation unit 260 generates information on the load on the robot 10 based on at least one of the calculation result of a physical phenomenon and the calculation result of the behavior of the robot 10 based on the command generated by the command application 240. The parameter adjustment unit adjusts the parameters based on the information on the load on the robot 10. The controller 220 calculates the behavior of the robot 10 based on the command output from the command application 240 whose parameters have been adjusted. Considering the load on the robot 10, the command application 240 can be adjusted, for example, so as not to overload it. Therefore, the adjusted command application 240 can be more smoothly deployed to the actual machine of the robot 10.

[0045] The load on the robot 10 includes not only mechanical loads such as torque or force applied to each joint of the robot 10, but also geometric loads such as range of motion utilization, which indicates what percentage of the range of motion of each joint is being used. The mechanical load is calculated by the physical simulator 210. The geometric load is calculated by the controller 220. As an example of the geometric load, the controller 220 may calculate a determination result of whether the angle of each joint exceeds the range of motion. The controller 220 may also calculate a determination result of whether the movement of the robot 10 constitutes a segment overrun in the trajectory plan (inability to move within a specified time), whether the acceleration limit has been reached, or whether the velocity limit has been reached.

[0046] The physical simulator 210 may calculate the state of the virtual work V21 after physical phenomena caused by the driving of the virtual robot V10 have occurred. The parameter adjustment unit 250 may adjust the parameters based on the state of the virtual work V21. Considering the state of the virtual work V21, the command application 240 can be adjusted, for example, to bring the state of the virtual work closer to the target. This allows the adjusted command application 240 to be deployed more smoothly to the actual robot 10. Examples of the state of the virtual work V21 include the position, orientation, size, shape, temperature, color, etc.

[0047] The linked application 230 may pass the calculation results of the robot 10's behavior by the controller 220 to the physical simulator 210 at a timing corresponding to the progress of the physical phenomenon calculations by the physical simulator 210, so that the relationship between the timing of the occurrence of physical phenomena in the simulation space and the timing of the robot 10's behavior approaches the relationship between the timing of the occurrence of physical phenomena in the real space RS and the timing of the robot 10's behavior. By reflecting the calculation results of the robot 10's behavior in the behavior of the virtual robot V10 in the simulation in accordance with the progress of the physical phenomenon calculations by the physical simulator 210, the accuracy of the calculation results of physical phenomena can be improved.

[0048] For example, when the physical simulator 210 and the controller 220 perform calculations on independent time axes, the cooperation application 230 may refer to a time stamp or calculation cycle number assigned to one calculation result, etc., and adjust the data transfer timing so that the calculation result is used at the corresponding time or cycle of the other. As another example, it is also possible to cause the controller 220 to execute calculations in synchronization with the progress of the simulation time axis of the physical simulator 210. This includes simply making the progress speeds of the time axes in the calculations of both match. In these timing adjustment methods, calculations such as predicting communication delays (time lags) associated with data transfer, response delays of the robot 10 to commands from the controller 110 (e.g., servo delays), etc., and compensating for those delays may also be included.

[0049] The cooperation application 230 may cause the controller 220 to calculate the behavior of the robot 10 in synchronization with the elapsed time in the simulation space. By reflecting the calculation result of the physical phenomenon in the calculation result of the behavior of the robot 10, the accuracy of the calculation result of the behavior of the robot 10 can be improved.

[0050] The cooperation application 230 may repeatedly transfer the calculation result of the physical phenomenon from the physical simulator 210 to the controller 220, cause the controller 220 to calculate the behavior of the robot 10 at the elapsed time corresponding to the calculation result of the physical phenomenon based on the calculation result of the physical phenomenon, and transfer the calculation result of the behavior of the robot 10 from the controller 220 to the physical simulator 210. By sequentially reflecting the calculation result of the physical phenomenon in the calculation result of the behavior of the robot 10 and sequentially reflecting the calculation result of the behavior of the robot 10 in the calculation result of the physical phenomenon, the accuracy of the simulation can be further improved.

[0051] Figure 2 is a timing chart illustrating the progress of calculations in the physical simulator 210 and the controller 220, respectively. In the illustrated example, the physical simulator 210 completes the calculations of physical phenomena at times vt1, vt2, vt3, vt4, and vt5 in the simulation space at times t1, t2, t3, t4, and t5 in the real space, respectively. Even if the intervals between times vt1, vt2, vt3, vt4, and vt5 are uniform in the simulation space, the calculation time will vary depending on the load, so the intervals between times t1, t2, t3, t4, and t5 may be non-uniform as shown in the figure.

[0052] For example, at time t1, the collaborative application 230 receives the calculation results of a physical phenomenon, which are associated with information representing time vt1 (e.g., a timestamp or cycle number), from the physical simulator 210 to the controller 220. Based on the received calculation results of the physical phenomenon, the controller 220 calculates the behavior of the robot 10 at time VT1, which corresponds to time vt1. Since the computational load on the controller 220 is far smaller than that on the physical simulator 210, the calculation by the controller 220 is completed at time T1, which is only slightly delayed from time t1. The collaborative application 230 receives the calculation results of the robot 10's behavior from the controller 220 to the physical simulator 210. Based on the calculation results of the robot 10's behavior, the physical simulator 210 drives the virtual robot V10 in the simulation space VS, and based on the results of driving the virtual robot V10, completes the calculation of the physical phenomenon at time vt2 at time t2.

[0053] At time t2, the collaborative application 230 receives the calculation result of a physical phenomenon, which is associated with information representing time vt2, from the physical simulator 210 to the controller 220. Based on the received calculation result of the physical phenomenon, the controller 220 calculates the behavior of the robot 10 at time VT2, which corresponds to time vt2. The collaborative application 230 receives the calculation result of the robot 10's behavior by the controller 220 from the controller 220 to the physical simulator 210. The same process is repeated at subsequent times t3, t4, and t5.

[0054] Thus, even if there is variation in the calculation completion time by the physical simulator 210, synchronization between the time corresponding to the calculation result of the physical phenomenon and the time corresponding to the calculation result of the robot 10's behavior can be maintained by using information representing the elapsed time in the simulation space VS. The collaborative application 230 may also cause the controller 220 to calculate the behavior of the robot 10 over the elapsed time based on the calculation result of the physical phenomenon acquired previously and the calculation result of the physical phenomenon acquired newly. This can further improve the accuracy of the calculation result of the robot 10's behavior. For example, the collaborative application 230 may calculate the behavior of the robot 10 from time VT1 to time VT2 based on the calculation result of the physical phenomenon received at time t1, and then calculate the behavior of the robot 10 at time VT2 based on that calculation result and the calculation result of the physical phenomenon received at time t2.

[0055] The collaborative application 230 may instruct the controller 220 to calculate the behavior of the robot 10 between the elapsed time corresponding to the calculation result of the previously acquired physical phenomenon and the elapsed time corresponding to the calculation result of the newly acquired physical phenomenon, based on at least the calculation result of the previously acquired physical phenomenon. For example, if the period from time VT1 to time VT2 is more than twice the control cycle described above, the controller 220 may sequentially calculate the behavior of the robot 10 for each control cycle based on at least the calculation result of the physical phenomenon at time vt1. By increasing the calculation cycle by the controller 220, the accuracy of the calculation result of the robot 10's behavior can be further improved. The collaborative application 230 may calculate the physical phenomenon for each control cycle by interpolation (e.g., linear interpolation) between the calculation result of the physical phenomenon received at time t1 and the calculation result of the physical phenomenon received at time t2, and then instruct the controller 220 to calculate the behavior of the robot 10 for each control cycle based on the physical phenomenon for each control cycle.

[0056] The collaborative application 230 may repeatedly transfer the calculation results of the robot 10's behavior from the controller 220 to the physical simulator 210, and have the physical simulator 210 calculate the physical phenomena at the elapsed time corresponding to the calculation results of the robot 10's behavior. By having the physical simulator 210 sequentially calculate the physical phenomena caused by the robot 10's behavior, the accuracy of the simulation can be further improved.

[0057] Figure 3 is a timing chart showing another example of the progress of calculations in the physical simulator 210 and the controller 220, respectively. For example, at time T1, the collaborative application 230 receives the calculation result of the behavior of the robot 10, which is associated with information representing time VT1 (e.g., a timestamp or cycle number), from the controller 220 to the physical simulator 210. Based on the received calculation result of the behavior of the robot 10, the physical simulator 210 drives the virtual robot V10 in the simulation space VS, and calculates the physical phenomenon at time vt1 corresponding to time VT1 based on the result of driving the virtual robot V10. Since the calculation load in the physical simulator 210 is much larger than that in the controller 220, the calculation by the physical simulator 210 is completed at time t1, which is significantly delayed from time T1. The collaborative application 230 receives the calculation result of the physical phenomenon from the physical simulator 210 to the controller 220. The controller 220 completes the calculation of the robot 10's behavior at time VT2 at time T2, based on the calculation results of the physical phenomenon.

[0058] At time T2, the linked application 230 receives the calculation result of the robot 10's behavior, which is associated with information representing time VT2, from the controller 220 to the physical simulator 210. Based on the received calculation result of the robot 10's behavior, the physical simulator 210 drives the virtual robot V10 in the simulation space VS, and based on the result of driving the virtual robot V10, it calculates the physical phenomenon at time vt2 corresponding to time VT2. The same process is repeated at subsequent times T3, T4, and T5.

[0059] Figure 2 shows an example in which the calculations of the physical simulator 210 are synchronized with the elapsed time that the controller 220 is calculating, and Figure 3 shows an example in which the calculations of the controller 220 are synchronized with the elapsed time that the physical simulator 210 is calculating. The collaborative application 230 may be configured to combine these synchronizations to synchronize the calculations of the physical simulator 210 and the controller 220 bidirectionally.

[0060] As shown in Figure 4, the simulation device 200 may further include an activation unit 241 and an activation unit 242 as functional blocks. The activation unit 241 activates the physical simulator 210. For example, the activation unit 241 activates the physical simulator 210 if the activation key entered by the user matches the unique key of the physical simulator 210 (hereinafter referred to as the "first key"). If the activation key entered by the user does not match the first key, the activation unit 241 does not activate the physical simulator 210. If the physical simulator 210 is not activated, it does not perform calculations of physical phenomena; if it is activated, it performs calculations of physical phenomena.

[0061] The activation unit 242 activates the controller 220 using a different activation method than that used for the physical simulator 210. Activating using a different method includes activating the controller 220 based on a unique key for the controller 220 (hereinafter referred to as the "second key") that is different from the unique key for the physical simulator 210 (the first key described above). For example, the activation unit 242 activates the controller 220 if the activation key entered by the user matches the second key. If the activation key entered by the user does not match the second key, the activation unit 242 does not activate the controller 220.

[0062] If the controller 220 is not activated, it will not output at least some of the calculation results of the robot 10's behavior. If the controller 220 is activated, it will output at least some of the calculation results of the robot 10's behavior. By differentiating the activation method of the controller 220 from that of the physical simulator 210, it is possible to prevent the leakage of internal information (algorithms, calculation results, etc.) in the controller 220 that has been developed through simulation. This enhanced protection allows for an increase in the types of controllers 220 that can be linked to the physical simulator 210, thereby promoting the development of the simulation space.

[0063] Not outputting calculation results includes not only cases where the controller 220 does not perform the calculation itself, but also cases where the calculation is performed but the results are not output to external parties (such as the linked application 230 or the physical simulator 210). Furthermore, even if the linked application 230 receives the calculation results from the controller 220, blocking it from passing them to the physical simulator 210 is also included in not outputting calculation results. Not outputting at least some calculation results does not only include cases where all calculation results are not output, but also cases where only specific important calculation results, such as torque command values ​​that are essential for reproducing the behavior of the robot 10, are not output.

[0064] The difference in activation methods between the physical simulator 210 and the controller 220 includes cases where the activation algorithms themselves are different for the physical simulator 210 and the controller 220. Furthermore, the difference in activation methods between the physical simulator 210 and the controller 220 also includes cases where the controller 220 requires activation, while the physical simulator 210 does not. In this case, the simulation device 200 does not have an activation unit 241.

[0065] Figure 5 is a block diagram illustrating the hardware configuration of the simulation device 200. As shown in Figure 5, the simulation device 200 has a circuit 290. The circuit 290 includes a processor 291, a memory 292, a storage 293, and a communication circuit 294. The storage 293 stores a program for configuring the simulation device 200 with functional blocks such as a physical simulator 210, a controller 220, and a collaborative application 230. The storage 293 includes one or more non-volatile storage devices. Examples of non-volatile storage devices include hard disk drives, flash memory, optical disk drives, and read-only memory.

[0066] Memory 292 temporarily stores programs loaded from storage 293. Memory 292 includes one or more volatile storage devices. Examples of volatile storage devices include random access memory. The processor 291 executes the programs loaded into memory 292, causing the simulation device 200 to configure the aforementioned functional blocks. The processor 291 includes one or more computing devices. Examples of computing devices include a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The communication circuit 294 communicates with an external device (e.g., controller 110) based on a request from the processor 291.

[0067] The configuration of the simulation device 200 is an example and can be changed. For example, the simulation device 200 may be divided into multiple devices that can communicate with each other. At least one of the multiple functional blocks that make up the simulation device 200 may be made up of dedicated logic circuits such as ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0068] [Procedure for constructing a command application] As an example of a method for constructing a command application that includes a robot simulation method, the procedure for constructing a command application executed by the simulation device 200 is illustrated. This procedure includes: generating commands using the command application 240; calculating the behavior of the robot 10 based on the commands using the controller 220; driving a virtual robot V10, which is a 3D model of the robot 10, in the simulation space using the physical simulator 210 based on the calculation results of the robot 10's behavior, and calculating at least the physical phenomena for the virtual robot V10; and adjusting the parameters for generating commands in the command application 240 using the parameter adjustment unit, based on at least one of the calculation results of the physical phenomena and the calculation results of the robot 10's behavior.

[0069] Furthermore, the robot simulation method included in this procedure includes: using a physical simulator 210 to calculate at least the physical phenomena for the virtual robot V10, which is a 3D model of the robot 10, in a simulation space; using a controller 220 to calculate the behavior of a robot 10 of a model corresponding to the virtual robot V10; and using a collaborative application 230 to coordinate between the physical simulator 210 and the controller 220.

[0070] As shown in Figure 6, the simulation device 200 first executes steps S01 and S02. In step S01, the physical simulator 210 calculates the initial physical phenomena in the simulation space VS before the virtual robot V10 starts operating. In step S02, the collaborative application 230 obtains information representing the time of occurrence of the physical phenomena in the simulation space VS and the calculation results of the physical phenomena from the physical simulator 210, and passes the information representing the time of occurrence to the controller 220 and the command application 240, including the calculation results of the physical phenomena. In some cases, the command application 240 can directly obtain the calculation results of the physical phenomena from the physical simulator 210. For example, the command application 240 may directly obtain images captured by the virtual camera V17 from the physical simulator 210. In this case, the collaborative application 230 may notify the command application 240 that the calculation of the physical phenomena by the physical simulator 210 is complete, and the command application 240 may obtain the calculation results of the physical phenomena from the physical simulator 210 in response to the notification.

[0071] Next, the simulation device 200 executes steps S03, S04, and S05. In step S03, the command application 240 generates a command for the controller 220 based on the calculation result of the received physical phenomenon. In step S04, the controller 220 calculates the behavior of the robot 10 at the time the physical phenomenon occurs, based on the command generated by the command application 240 and the calculation result of the received physical phenomenon. In step S05, the cooperation application 230 passes the calculation result of the robot 10's behavior by the controller 220 to the physical simulator 210.

[0072] Next, the simulation device 200 executes steps S06 and S07. In step S06, the physical simulator 210 drives the virtual robot V10 based on the calculation results of the robot 10's behavior that it has received, and calculates the physical phenomena at the next time. In step S07, the linked application 230 checks whether the virtual robot V10 has completed its work in the simulation space VS. For example, the linked application 230 checks whether the controller 220 has completed the execution of the program described above.

[0073] In step S07, if it is determined that the virtual robot V10's work in the simulation space VS is not complete, the simulation device 200 returns to step S02. Thereafter, the processes from steps S02 to S07 are repeated, and calculations of physical phenomena and behaviors are executed sequentially.

[0074] In step S07, if it is determined that the work of the virtual robot V10 in the simulation space VS has been completed, the simulation device 200 executes step S08. In step S08, the parameter adjustment unit 250 checks whether it is necessary to adjust the parameters of the command application 240 based on a comparison of the execution results of the work of the virtual robot V10 in the simulation space VS with predetermined evaluation criteria. For example, if the load information (joint torque, etc.) generated by the load generation unit 260 exceeds an allowable value, or if the final position, orientation, shape, size, etc. of the virtual work V21 calculated by the physical simulator 210 deviates from the target by a predetermined error, the parameter adjustment unit 250 determines that parameter adjustment is necessary.

[0075] If the simulation device 200 determines in step S08 that it is necessary to adjust the parameters of the command application 240, it executes step S09. In step S09, the parameter adjustment unit 250 adjusts the parameters of the command application 240 based on the evaluation results. For example, if the load exceeds the allowable value, the parameters are adjusted to reduce the operating speed of the robot 10. Also, if the state of the virtual work V21 deviates from the target, the reward in reinforcement learning is redesigned and the model is retrained to reduce the deviation.

[0076] Subsequently, the simulation device 200 returns to step S01. Thereafter, the simulation and parameter adjustment cycle is repeated until the performance of the command application 240 meets the evaluation criteria. In step S08, if it is determined that there is no need to adjust the parameters of the command application 240, the simulation device 200 completes the construction of the command application 240. The constructed command application 240 is installed in the computing device 120 as command application 130 and used for controlling the robot 10 by the controller 110 and the computing device 120.

[0077] Figure 7 is a flowchart showing a modified version of the command application. While the procedure shown in Figure 6 was an example of a physics simulator-led process in which the controller 220 operates triggered by the completion of calculations by the physics simulator 210, the procedure shown in Figure 7 is an example of a controller-led process in which the physics simulator 210 operates triggered by calculations by the controller 220. The simulation device 200 first executes steps S11 and S12. In step S11, the physics simulator 210 calculates the physical phenomena in the simulation space VS before the virtual robot V10 starts operating. In step S12, the collaborative application 230 passes the calculation results of the physical phenomena by the physics simulator 210 from the physics simulator 210 to the controller 220 and the command application 240. The command application 240 may also directly obtain the calculation results of the physical phenomena from the physics simulator 210.

[0078] Next, the simulation device 200 executes steps S13, S14, and S15. In step S13, the command application 240 generates a command for the controller 220 based on the calculation results of the received physical phenomena. In step S14, the controller 220 calculates the behavior of the robot 10 based on the command generated by the command application 240 and the calculation results of the received physical phenomena. In step S15, the collaboration application 230 obtains from the controller 220 information representing the occurrence time of the behavior of the virtual robot V10 in the simulation space VS and the calculation results of the robot 10's behavior, and passes the information representing the occurrence time to the physical simulator 210, including it in the calculation results of the robot 10's behavior.

[0079] Next, the simulation device 200 executes steps S16 and S17. In step S16, the physical simulator 210 drives the virtual robot V10 based on the calculation results of the robot 10's behavior that it has received, and calculates the physical phenomenon at the specified time of occurrence. In step S17, the linked application 230 confirms whether the virtual robot V10 has completed its work in the simulation space VS.

[0080] If, in step S17, the simulation device 200 determines that the work of the virtual robot V10 in the simulation space VS is not yet complete, it returns to step S12. Thereafter, the processes from steps S12 to S17 are repeated, and calculations of physical phenomena and behaviors are executed sequentially.

[0081] In step S17, if it is determined that the virtual robot V10 has completed its work in the simulation space VS, the simulation device 200 executes step S18. In step S18, the parameter adjustment unit 250 checks whether it is necessary to adjust the parameters of the command application 240 based on a comparison of the execution results of the virtual robot V10's work in the simulation space VS with predetermined evaluation criteria.

[0082] If the simulation device 200 determines in step S18 that it is necessary to adjust the parameters of the command application 240, it executes step S19. In step S19, the parameter adjustment unit 250 adjusts the parameters of the command application 240 based on the evaluation results.

[0083] Subsequently, the simulation device 200 returns to step S11. Thereafter, the simulation and parameter adjustment cycle is repeated until the performance of the command application 240 meets the evaluation criteria. In step S18, if it is determined that there is no need to adjust the parameters of the command application 240, the simulation device 200 completes the construction of the command application 240.

[0084] [Summary] (1) A robot system 1 comprising: a physical simulator 210 capable of calculating physical phenomena for at least the virtual robot V10 in a simulation space VS having a virtual robot V10 which is a 3D model of robot 10; a controller 220 capable of calculating the behavior of a robot 10 of a model corresponding to the virtual robot V10; and a cooperation application 230 that coordinates between the physical simulator 210 and the controller 220. Physical phenomena other than the behavior of robot 10 (forces that robot 10 exerts on surrounding objects, forces that robot 10 receives from surrounding objects, the effect of gravity, images obtained by a camera, etc.) can be calculated by the physical simulator 210 without the operation of the actual robot. The controller 220, separate from the physical simulator 210, can calculate the behavior of robot 10 that is close to that of the actual robot, reflecting the characteristics within the controller 220. By combining the results of the robot 10 behavior calculation by the controller 220 and the results of the physical phenomenon calculation by the physical simulator 210 using the cooperation application 230, trial and error with the actual robot in the real space RS can be simulated with high accuracy. Therefore, the barriers (gap between simulation and reality) to deploying the system configuration and programs constructed through simulation to the real-world RS are reduced. Consequently, a system can be provided that makes it easier to deploy the results of the simulation to the actual robot 10. With this system, the system configuration and programs can be quickly refined through simulation without the need for trial and error with the actual robot, and the results can be smoothly deployed to the actual robot 10.

[0085] (2) The robot system 1 as described in (1), wherein the collaborative application 230 passes the calculation results of the robot 10's behavior from the controller 220 to the physical simulator 210 so that the controller 220 drives the virtual robot V10, and passes the calculation results of physical phenomena from the physical simulator 210 to the controller 220 so that the controller 220 calculates the behavior based on the calculation results of physical phenomena by the physical simulator 210. The virtual robot V10 in the physical simulator 210 can be made to move more closely to the robot 10 in the real space RS, thereby reducing the difference between the simulation and reality. With a configuration in which the controller 220 and the physical simulator 210 are mediated by the collaborative application 230, rather than the controller 220 being part of the physical simulator 210, the physical simulator 210 can be developed independently of the controller 220 (for example, improved accuracy, expansion of model types, etc.). Furthermore, the controller 220 can also be developed independently of the physical simulator 210.

[0086] (3) The linked application 230 transmits the results of the calculation of the robot 10's behavior by the controller 220 to the physical simulator 210 at a timing corresponding to the progress of the calculation of physical phenomena by the physical simulator 210, so that the relationship between the timing of the occurrence of physical phenomena in the simulation space VS and the timing of the occurrence of the robot 10's behavior approaches the relationship between the timing of the occurrence of physical phenomena in the real space RS and the timing of the occurrence of the robot 10's behavior. This is the robot system 1 as described in (2). By reflecting the calculation results of the robot 10's behavior in the behavior of the virtual robot V10 in the simulation in accordance with the progress of the calculation of physical phenomena by the physical simulator 210, the accuracy of the calculation results of physical phenomena can be improved.

[0087] (4) The linked application 230 causes the controller 220 to calculate the behavior of the robot 10 in synchronization with the elapsed time in the simulation space VS, as described in (3) of the robot system 1. By reflecting the calculation results of physical phenomena in the calculation results of the behavior of the robot 10, the accuracy of the calculation results of the behavior of the robot 10 can be improved.

[0088] (5) The linked application 230 repeatedly transfers the calculation results of physical phenomena from the physical simulator 210 to the controller 220, causes the controller 220 to calculate the behavior of the robot 10 at the elapsed time corresponding to the calculation results of the physical phenomena based on the calculation results of the physical phenomena, and transfers the calculation results of the behavior of the robot 10 from the controller 220 to the physical simulator 210, thereby providing the robot system 1 described in (4). By sequentially reflecting the calculation results of physical phenomena in the calculation results of the behavior of the robot 10, and sequentially reflecting the calculation results of the behavior of the robot 10 in the calculation results of physical phenomena, the accuracy of the simulation can be further improved.

[0089] (6) The robot system 1 described in (5) is configured such that the linked application 230 causes the controller 220 to calculate the behavior of the robot 10 over time based on the calculation result of the previously acquired physical phenomenon and the calculation result of the newly acquired physical phenomenon. The accuracy of the calculation result of the behavior of the robot 10 can be further improved.

[0090] (7) The robot system 1 as described in (5) or (6), wherein the linked application 230 causes the controller 220 to calculate the behavior of the robot 10 during the elapsed time corresponding to the calculation result of the previously acquired physical phenomenon and the elapsed time corresponding to the calculation result of the newly acquired physical phenomenon, based on the calculation result of the physical phenomenon acquired at least once. By increasing the calculation period by the controller 220, the accuracy of the calculation result of the behavior of the robot 10 can be further improved.

[0091] (8) The robot system 1 described in any one of (4) to (7), wherein the linked application 230 repeatedly transmits the calculation results of the robot 10's behavior from the controller 220 to the physical simulator 210, and causes the physical simulator 210 to calculate the physical phenomena at the elapsed time corresponding to the calculation results of the robot 10's behavior. The accuracy of the simulation can be further improved by having the physical simulator 210 sequentially calculate the physical phenomena caused by the robot 10's behavior.

[0092] (9) The robot system 1 according to any one of (2) to (8), wherein the physical simulator 210 calculates virtual feedback data, which is feedback generated in the virtual robot V10 due to physical phenomena caused by the behavior of the virtual robot V10, as a result of the calculation of physical phenomena, the linked application 230 passes the virtual feedback data from the physical simulator 210 to the controller 220, and the controller 220 calculates the behavior of the robot 10 based on the virtual feedback data. Physical phenomena can be easily reflected in the calculation of the behavior of the robot 10.

[0093] (10) The robot system 1 as described in (9), wherein the simulation space VS has virtual sensors V16 and V17 corresponding to the sensors of the real space RS, and the physical simulator 210 calculates virtual feedback data including the detection results from the virtual sensors V16 and V17. Physical phenomena can be more easily reflected in the calculation of the behavior of the robot 10.

[0094] (11) The robot system 1 according to (9) or (10), wherein the simulation space VS has a virtual work V21 that corresponds to the workpiece 21 that the robot 10 works on in the real space RS, and the physical simulator 210 calculates virtual feedback data generated in the virtual robot V10 due to physical phenomena that occur between the virtual work V21 and the virtual robot V10 as a result of the behavior of the virtual robot V10. The accuracy of the simulation can be further improved to include the influence that the robot 10 receives from the workpiece 21.

[0095] (12) The robot system 1 according to any one of (2) to (11), further comprising an activation unit 242 that activates the controller 220 by an activation method different from that of the physical simulator 210, wherein the controller 220 does not output calculation results of at least some of the behavior of the robot 10 if it is not activated, but outputs calculation results of at least some of the behavior of the robot 10 if it is activated. By making the activation method of the controller 220 different from the activation method of the physical simulator 210, it is possible to prevent leakage of internal information (algorithms, calculation results, etc.) of the controller 220 that has been developed through simulation. Enhanced protection allows for an increase in the types of controllers 220 that can be linked to the physical simulator 210, thereby promoting the development of the simulation space VS.

[0096] (13) The robot system 1 according to any one of (1) to (12), further comprising a command application 240 that generates commands to the controller 220 based on physical phenomena and outputs them to the controller 220, wherein the controller 220 calculates the behavior of the robot 10 based on commands generated by at least the command application 240. By adjusting the command application 240 through simulations that facilitate the deployment of the results to the actual robot 10, the adjusted command application 240 can be smoothly deployed to the actual robot 10.

[0097] (14) The robot system 1 described in (13) wherein the collaborative application 230 causes the controller 220 to calculate the behavior of the robot 10 based on the commands generated by the command application 240 and the calculation results of physical phenomena. The accuracy of the simulation can be improved by including both the influence that the calculation results of physical phenomena have on the behavior of the robot 10 via commands and the direct influence that the calculation results of physical phenomena have on the behavior of the robot 10.

[0098] (15) The robot system 1 according to (13), further comprising a parameter adjustment unit 250 that adjusts the parameters for generating commands in the command application 240 based on at least one of the calculation results of physical phenomena and the calculation results of the behavior of the robot 10 based on commands generated by the command application 240, wherein the controller 220 calculates the behavior of the robot 10 based on the commands output from the command application 240 with adjusted parameters. The command application 240 can be easily adjusted by adjusting the parameters.

[0099] (16) The robot system 1 according to (15), further comprising a load generation unit that generates load information on the robot 10 based on at least one of the calculation results of physical phenomena and the calculation results of the behavior of the robot 10 based on commands generated by the command application 240, a parameter adjustment unit 250 that adjusts parameters based on the load information on the robot 10, and a controller 220 that calculates the behavior of the robot 10 based on commands output from the command application 240 with adjusted parameters. The command application 240 can be adjusted to take into account the load on the robot 10, for example, so as not to make the load excessive. As a result, the adjusted command application 240 can be deployed to the actual robot 10 more smoothly.

[0100] (17) The robot system 1 as described in (15) or (16), wherein the simulation space VS has a virtual workpiece corresponding to the workpiece 21 that the robot 10 will work on in the real space RS, the physical simulator 210 calculates the state of the virtual workpiece after physical phenomena caused by the driving of the virtual robot V10 occur, the parameter adjustment unit 250 adjusts the parameters based on the state of the virtual workpiece, and the controller 220 calculates the behavior of the robot 10 based on the commands output from the command application 240 with adjusted parameters. The command application 240 can be adjusted, for example, to bring the state of the virtual workpiece closer to the target, taking into account the state of the virtual workpiece. Therefore, the adjusted command application 240 can be deployed to the actual robot 10 more smoothly.

[0101] (18) A collaborative application 230 that performs cooperation between a physical simulator 210 and a controller 220, wherein the physical simulator 210 calculates physical phenomena for at least the virtual robot V10 in a simulation space VS having a virtual robot V10 which is a 3D model of the robot 10, the controller 220 calculates the behavior of a robot 10 of a model corresponding to the virtual robot V10, and the collaborative application 230 passes the calculation results of the robot 10's behavior from the controller 220 to the physical simulator 210 so that the physical simulator 210 drives the virtual robot V10, and passes the calculation results of the physical phenomena from the physical simulator 210 to the controller 220 so that the physical simulator 210 calculates the behavior based on the calculation results of the physical phenomena.

[0102] (19) A robot simulation method in which a physical simulator 210 calculates physical phenomena for at least the virtual robot V10 in a simulation space VS having a virtual robot V10 which is a 3D model of the robot 10, a controller 220 calculates the behavior of a robot 10 of a model corresponding to the virtual robot V10, and a coordinating application 230 coordinates the physical simulator 210 and the controller 220.

[0103] (20) A method for manufacturing a command application, comprising: generating commands using a command application 240; calculating the behavior of the robot 10 based on the commands using a controller 220; driving a virtual robot V10, which is a 3D model of the robot 10, in a simulation space VS using a physical simulator 210 based on the calculation results of the behavior of the robot 10, and calculating at least physical phenomena for the virtual robot V10; and adjusting the parameters for generating commands in the command application 240 using a parameter adjustment unit 250 based on at least one of the calculation results of the physical phenomena and the calculation results of the behavior of the robot 10.

[0104] Although embodiments have been described above, this disclosure is not necessarily limited to the embodiments described above, and various modifications are possible without departing from its essence.

[0105] 1...Robot system, RS...Real space, 10...Robot, 21...Workpiece, 210...Physical simulator, VS...Simulation space, V10...Virtual robot, V16, V17...Virtual sensors, 220...Controller, 230...Cooperation application, 240...Command application, 250...Parameter adjustment unit, 242...Activation unit.

Claims

1. A robot system comprising: a simulation space having a virtual robot which is a 3D model of a robot; a physical simulator capable of calculating physical phenomena for the virtual robot; a controller capable of calculating the behavior of a robot of a model corresponding to the virtual robot; and a cooperation application that coordinates between the physical simulator and the controller.

2. The robot system according to claim 1, wherein the collaborative application passes the calculation result of the robot's behavior from the controller to the physical simulator so as to drive the virtual robot based on the calculation result of the robot's behavior by the controller, and passes the calculation result of the physical phenomenon from the physical simulator to the controller so as to calculate the behavior based on the calculation result of the physical phenomenon by the physical simulator.

3. The robot system according to claim 2, wherein the linked application transmits the calculation results of the robot's behavior by the controller to the physical simulator at a timing corresponding to the progress of the calculation of the physical phenomenon by the physical simulator, so that the relationship between the timing of the occurrence of the physical phenomenon in the simulation space and the timing of the robot's behavior in the simulation space approaches the relationship between the timing of the occurrence of the physical phenomenon and the timing of the robot's behavior in the real space.

4. The robot system according to claim 3, wherein the linked application causes the controller to calculate the behavior of the robot in synchronization with the elapsed time in the simulation space.

5. The robot system according to claim 4, wherein the linked application repeatedly: transfers the calculation result of the physical phenomenon from the physical simulator to the controller; causes the controller to calculate the behavior of the robot at the elapsed time corresponding to the calculation result of the physical phenomenon based on the calculation result of the physical phenomenon; and transfers the calculation result of the robot's behavior from the controller to the physical simulator.

6. The robot system according to claim 5, wherein the linked application causes the controller to calculate the behavior of the robot over the elapsed time based on the calculation result of the physical phenomenon acquired previously and the calculation result of the physical phenomenon acquired recently.

7. The robot system according to claim 5, wherein the linked application causes the controller to calculate the behavior of the robot between the elapsed time corresponding to the calculation result of the physical phenomenon acquired previously and the elapsed time corresponding to the calculation result of the physical phenomenon newly acquired, based on the calculation result of the physical phenomenon acquired at least previously.

8. The robot system according to claim 4, wherein the collaborative application repeatedly transfers the calculation result of the robot's behavior from the controller to the physical simulator, and causes the physical simulator to calculate the physical phenomenon at the elapsed time corresponding to the calculation result of the robot's behavior.

9. The robot system according to claim 2, wherein the physical simulator calculates virtual feedback data, which is feedback generated in the virtual robot due to the physical phenomenon caused by the behavior of the virtual robot, as a result of the calculation of the physical phenomenon; the collaborative application passes the virtual feedback data from the physical simulator to the controller; and the controller calculates the behavior of the robot based on the virtual feedback data.

10. The robot system according to claim 9, wherein the simulation space has virtual sensors corresponding to sensors in the real space, and the physical simulator calculates the virtual feedback data including the detection results from the virtual sensors.

11. The robot system according to claim 9, wherein the simulation space has a virtual workpiece corresponding to a workpiece that the robot will work on in real space, and the physical simulator calculates the virtual feedback data generated by the virtual robot due to the physical phenomena that occur between the virtual workpiece and the virtual robot as a result of the behavior of the virtual robot.

12. The robot system according to claim 2, further comprising an activation unit that activates the controller by an activation method different from that of the physical simulator, wherein the controller does not output calculation results of at least a part of the robot's behavior if activation is not performed, but outputs calculation results of at least a part of the robot's behavior if activation is performed.

13. The robot system according to claim 1, further comprising a command application that generates commands to the controller based on the physical phenomena and outputs them to the controller, wherein the controller calculates the behavior of the robot based at least on the commands generated by the command application.

14. The robot system according to claim 13, wherein the coordinating application causes the controller to calculate the behavior of the robot based on the commands generated by the command application and the calculation results of the physical phenomena.

15. The robot system according to claim 13, further comprising a parameter adjustment unit that adjusts the parameters for generating the commands in the command application based on at least one of the calculation results of the physical phenomenon and the calculation results of the robot's behavior based on the commands generated by the command application, wherein the controller calculates the robot's behavior based on the commands output from the command application whose parameters have been adjusted.

16. The robot system according to claim 15, further comprising a load generation unit that generates load information for the robot based on at least one of the calculation results of the physical phenomenon and the calculation results of the robot's behavior based on the command generated by the command application, the parameter adjustment unit that adjusts the parameters based on the load information for the robot, and the controller that calculates the robot's behavior based on the command output from the command application whose parameters have been adjusted.

17. The robot system according to claim 15, wherein the simulation space has a virtual workpiece corresponding to a workpiece that the robot will work on in real space, the physical simulator calculates the state of the virtual workpiece after the physical phenomenon caused by the driving of the virtual robot occurs, the parameter adjustment unit adjusts the parameters based on the state of the virtual workpiece, and the controller calculates the behavior of the robot based on the commands output from the command application whose parameters have been adjusted.

18. A collaborative application for coordinating between a physical simulator and a controller, wherein the physical simulator calculates at least physical phenomena for the virtual robot in a simulation space having a virtual robot which is a 3D model of a robot; the controller calculates the behavior of a robot of a model corresponding to the virtual robot; the collaborative application passes the calculation results of the robot's behavior from the controller to the physical simulator so that the physical simulator drives the virtual robot based on the calculation results of the robot's behavior by the controller; and passes the calculation results of the physical phenomena from the physical simulator to the controller so that the controller calculates the behavior based on the calculation results of the physical phenomena by the physical simulator.

19. A robot simulation method comprising: using a physical simulator to calculate at least the physical phenomena relating to a virtual robot, which is a 3D model of a robot, in a simulation space; using a controller to calculate the behavior of a robot of a model corresponding to the virtual robot; and using a collaborative application to coordinate between the physical simulator and the controller.

20. A method for manufacturing a command application, comprising: generating commands using a command application; calculating the behavior of a robot based on the commands using a controller; driving a virtual robot, which is a 3D model of the robot, in a simulation space using a physical simulator based on the calculation results of the robot's behavior, and calculating at least the physical phenomena for the virtual robot; and adjusting the parameters for generating the commands in the command application using a parameter adjustment unit, based on at least one of the calculation results of the physical phenomena and the calculation results of the robot's behavior.

Images