Operation program generation system

Abstract

An operation program generation system (100) comprises: a server (30) that simulates an operation of a robot (10) by using an operation program (11) for the robot, and generates, from the simulation result, operation information including position information and posture information of the robot that do not depend on robot operation and robot control; and a computer (20) that generates an operation video of the robot in a set virtual space on the basis of the operation information.

Description

Motion program generation system

[0001] The present disclosure relates to a motion program generation system.

[0002] Conventionally, systems that simulate the motion of robots have been disclosed. For example, the system disclosed in Japanese Patent Laid-Open No. 2003-117863 includes a robot simulation device that simulates the motion of a robot based on the actual motion of the robot and peripheral equipment, and displays a video of the simulation result.

[0003] Japanese Patent Laid-Open No. 2003-117863

[0004] In the system of Japanese Patent Laid-Open No. 2003-117863, the robot simulation device simulates the motion of the robot based on the actual motion of the robot and peripheral equipment, and displays a video of the simulation result. For this reason, in the system of Japanese Patent Laid-Open No. 2003-117863, simulation of the robot's motion can only be performed after the actual robot has been introduced, making it difficult to check the robot's motion before introducing the robot. Furthermore, in order to check the video of the simulation result, the user needs to prepare in advance a robot simulation device capable of executing robot simulation, which makes it difficult for the user to easily check the video of the simulation result. Therefore, there is a demand for easily checking the motion of a robot before introducing the robot.

[0005] The present disclosure has been made to solve the above-described problems, and one object of the present disclosure is to provide a motion program generation system that allows easy checking of a robot's motion before the robot is introduced.

[0006] A motion program generation system according to one aspect of this disclosure comprises: a simulation execution unit that simulates the movement of a robot using motion programs that depend on robot movement and robot control; a motion information generation unit that generates motion information including robot position information and posture information that does not depend on robot movement and robot control from the simulation results by the simulation execution unit; and a video generation unit that generates video of the robot's movement in a set virtual space based on the motion information generated by the motion information generation unit.

[0007] The motion program generation system according to one aspect of this disclosure includes a simulation execution unit that simulates robot motion using motion programs that depend on robot motion and robot control, as described above. This allows the simulation execution unit to perform simulations based on motion programs generated before the robot is introduced. The system also includes a motion information generation unit that generates motion information including robot position and posture information that does not depend on robot motion and robot control from the simulation results of the simulation execution unit, and a video generation unit that generates video of the robot's motion in a set virtual space based on the motion information generated by the motion information generation unit. As a result, the video generation unit used by the user does not need to prepare a simulator for simulating motion programs that depend on robot motion and robot control. Furthermore, since the video of the motion is generated based on motion information including robot position and posture information that does not depend on robot motion and robot control, the user can easily obtain video of the simulation results. As a result, the robot's motion can be easily confirmed before the robot is introduced.

[0008] The operation program generation system disclosed herein allows for easy verification of the robot's operation before its introduction.

[0009] This is a block diagram showing an operation program generation system according to one embodiment. This is a diagram for explaining the execution of a simulation of the operation program generation system according to one embodiment. This is a diagram for explaining the generation of operation video of the operation program generation system according to one embodiment. This is a diagram for explaining the setting of a virtual space for the operation program generation system according to one embodiment.

[0010] Hereinafter, embodiments of this disclosure will be described based on the drawings.

[0011] As shown in Figure 1, the motion program generation system 100 is a system that generates the motion program 11 for the robot 10.

[0012] As shown in Figure 1, the operation program generation system 100 is a system for a user of the robot 10 to specify a task to be performed by the robot 10's operation and to obtain an operation program 11 for operating the robot 10.

[0013] As shown in Figure 1, the operation program generation system 100 comprises a computer 20 and a server 30 connected to the computer 20 via a network. The operation program generation system 100 also comprises a plurality of computers 40 connected to the server 30 via a network. The operation program generation system 100 also comprises a plurality of computers 50 connected to the server 30 via a network. The operation program generation system 100 also comprises a plurality of computers 60 connected to the server 30 via a network. The computer 20 is an example of the "image generation unit" in this disclosure. The server 30 is an example of the "simulation execution unit" and "operation information generation unit" in this disclosure.

[0014] As shown in Figure 1, robot 10 includes a social robot that provides services to people. Robot 10 includes, for example, a robot with one or more articulated arms, a robot capable of autonomous movement on wheels, a robot capable of walking on multiple legs, a flying drone robot, and a robot with a head. Robot 10 provides services such as customer service, product transport, cleaning, security, and guidance in commercial facilities, etc. Robot 10 also provides services such as assisting patients or residents in hospitals and nursing homes. Note that robot 10 is not limited to a social robot. Robot 10 may also be an industrial robot or a medical robot.

[0015] As shown in Figure 1, the computer 20 is used by the user of the robot 10. The computer 20 is, for example, a personal computer. The computer 20 includes a control unit 21, a display unit 22, and an operation input unit 23. The computer 20 may be used, for example, to control the operation of the robot 10. For example, the computer 20 is used to install the operation program 11 for the robot 10 into the robot 10. Note that a separate computer may be provided to control the operation of the robot 10. Note that the computer 20 may also be used by a coordinator who assists in the introduction of the robot 10 into a facility.

[0016] The control unit 21 includes a processor such as a CPU (Central Processing Unit), and memory such as ROM (Read Only Memory) and RAM (Random Access Memory). The control unit 21 also executes programs and performs processing.

[0017] The display unit 22 displays information. The display unit 22 includes, for example, a display such as an organic EL display or a liquid crystal display.

[0018] The operation input unit 23 accepts editing of the robot 10's operation program 11. The operation input unit 23 includes devices such as a keyboard and a mouse. Alternatively, the operation input unit 23 may be a touch panel provided on the display unit 22.

[0019] Computer 20 accepts task input to operate the robot 10 and perform tasks. In other words, computer 20 accepts input from the user using the robot 10 to select tasks to be performed by the robot 10. Computer 20 also stores information about the robot 10 used by the user. Note that task selection is not limited to input from the user and may be input from others.

[0020] Here, the robot 10's operation program 11 is edited using low-code editing. In other words, the robot 10's operation program 11 is generated and edited by selecting a task expressed in a way that is understandable even to a user unfamiliar with the robot 10's programming. Furthermore, once a task is selected by the user and the robot 10's operation is set, the server 30 generates the robot 10's operation program 11 using ROS (Robot Operating System) compatible code. Note that the creation and editing of the robot 10's operation program 11 is not limited to low-code; it may also be done using no-code methods without any code, or using code and constituent data. For example, the robot 10's operation program 11 may be created and edited directly using structured data in JSON format.

[0021] Furthermore, the skills that constitute the robot 10's operation program 11 are set to execute a task. In other words, a skill is a function that the robot 10 can perform. A task is a unit of work that the robot 10 will perform. Also, a task is independent of the robot 10's functions. The robot 10's operation program 11 is generated by combining multiple skills or by using a single skill in order to execute a desired task.

[0022] The information about robot 10 is used to select which robot 10 to operate and to select the skills to perform. In addition, depending on the task, if multiple robots 10 are required to perform subtasks that subdivide the task, the information about robot 10 is used when creating an operation program 11 for multiple robots 10 to cooperate in executing the task.

[0023] Here, the tasks selected by the computer 20 are expressed in a way that is understandable even to users unfamiliar with the robot 10's programming. For example, a task is a unit of work that the robot 10 will perform. Furthermore, a task is independent of the robot 10's functions.

[0024] Furthermore, tasks can be combined to create task sets. These created task sets are then made public and available to multiple users. Users can also create new task sets by combining tasks within a publicly available task set. Publicly available task sets are stored, for example, on server 30. Task sets may be created by users using robot 10 or by task set developers.

[0025] Server 30 is connected to computer 20 via a network. Server 30 generates the operation program 11 for robot 10. Server 30 also performs simulations of the operation program 11 for robot 10. Server 30 includes a control unit 31 and a storage unit 32. Server 30 is a cloud server located in the cloud. In other words, the simulation of the operation program 11 for robot 10 is performed on the cloud. Note that server 30 is not limited to a cloud server located in the cloud. For example, server 30 may be an on-premises server. Also, if it is an on-premises server, server 30 may be located at the business premises of the operator of the operation program generation system 100.

[0026] The control unit 31 includes a processor such as a CPU, and memory such as ROM and RAM. The control unit 31 also executes programs and performs processing.

[0027] The memory unit 32 stores information. For example, the memory unit 32 stores information uploaded to the server 30. The memory unit 32 stores information including multiple task information 32a, multiple skill information 32b, and multiple simulation programs 32c. The memory unit 32 includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0028] The server 30 generates an operation program 11 for the robot 10 based on setting at least one skill information 32b selected from a plurality of skill information 32b that are stored in advance and include elements of an operation program 11, in order to have the robot 10 execute the task of the task information 32a input by the computer 20.

[0029] Specifically, the user's computer 20 operates to select task information 32a from the server 30. The server 30 then sets at least one skill information 32b for the selected task information 32a. Based on the setting of skill information 32b for task information 32a, the server generates the robot operation program 11 for the robot 10.

[0030] The server 30 generates an operation program 11 based on setting at least one skill information 32b from a plurality of skill information 32b suitable for the type of robot 10 and the type of task.

[0031] Server 30 stores multiple skill information 32b created by multiple skill creators. Based on setting at least one skill information 32b selected from the multiple skill information 32b stored in Server 30, Server 30 generates an operation program 11.

[0032] The server 30 simulates the operation of the robot 10 using the robot operation program 11 via the simulation program 32c. Specifically, the server 30 simulates the robot 10 performing a task using the operation program 11 generated with the skill information 32b set. The operation program 11 is a program that can describe the robot's movements. In other words, the operation program 11 depends on the robot's movements and robot control. For example, the operation program 11 is a ROS-compatible program. That is, the simulator of the server 30 simulates the operation of the robot 10 using the ROS-compatible operation program 11. In other words, when performing a simulation or when making the robot 10 perform an operation, the ROS-compatible operation program 11 is executed. On the other hand, when editing the operation program 11, editing can be done using task information 32a and skill information 32b in a format other than ROS-compatible. Note that ROS is an example of a "predetermined operating system for robots" in this disclosure. The operation program 11 may also be a program compatible with a "predetermined operating system for robots" other than ROS.

[0033] Here, if a ROS-compatible simulator is installed on the computer 20, the simulation of the operation program 11 may be performed on the computer 20 without using the server 30. Also, if it is possible to edit the operation program 11 in a ROS-compatible format, a specialist user may edit the operation program 11 in a ROS-compatible format using the computer 20. This allows a specialist user to perform simulations using a ROS-compatible computer 20 with a ROS-compatible operation program 11, enabling more advanced simulations and program editing. In other words, it becomes possible to edit the operation program 11, including fine-grained control of the robot 10 that cannot be adjusted by editing task information 32a and skill information 32b, which are not in a ROS-compatible format.

[0034] In this embodiment, the server 30 generates motion information from the simulation results, including position information and posture information of the robot 10 that are independent of robot motion and robot control. For example, the server 30 generates motion information from the simulation results, including position information and posture information of the robot 10, that is compatible with formats other than ROS. In other words, the server 30 converts the simulation results into a format that is not compatible with ROS, and generates motion information including position information and posture information of the robot 10 so that it can be used outside of the simulator. The position information of the robot 10 includes coordinate data indicating the position of the robot 10 in the operating space. The posture information of the robot 10 includes angle data for each joint of the robot 10.

[0035] Here, information dependent on robot motion and robot control refers to information related to the control required to operate the robot 10, and includes, for example, information on the motion command values ​​used when operating the robot 10. When operating the robot 10, an operation program 11 containing ROS-compatible code is executed, and motion command values ​​generated by the execution of the operation program 11 are input to the robot 10. Such motion command values ​​are information that depends on the control method and control parameters of the robot 10. On the other hand, information independent of robot motion and robot control is information that allows the user to determine whether the operation was successful or unsuccessful from the motion video based on the simulation results. In other words, information independent of robot motion and robot control is information related to the operating state of the robot 10 obtained as a result of control by the execution of the operation program 11, and is information that can be acquired or used without presupposing the control method itself. For example, information independent of robot motion and robot control is not information that affects control and motion, such as which control method was used, what control command values ​​were used, or control delay, error, noise, etc., but rather information that has been virtually verified by simulation. Such information is obtained as a result of the operation program 11 being executed by the simulation program 32c and the robot 10 being virtually operated. Information independent of robot motion and robot control includes, for example, position information, posture information, movement amount, velocity information of the robot 10, and information on the operating status of movable parts such as arms. This information can be treated as information representing the results of the robot 10's operation, regardless of the type of control method employed, such as PID control or model predictive control.

[0036] Furthermore, the motion information may include position and orientation information of objects other than the robot 10. For example, the motion information may include position and orientation information of an object moved by the robot 10. In other words, when the robot 10 is to perform a transport operation, the motion information may include position and orientation information of the object being transported by the robot 10.

[0037] The position information of robot 10 is information about where robot 10 is located. The position information of robot 10 is expressed, for example, in a three-dimensional coordinate system (X, Y, Z) in the robot 10's operating space. The attitude information of robot 10 is information about the orientation of robot 10. The attitude information of robot 10 is expressed, for example, in three-axis rotation (Roll, Pitch, Yaw). The position information and attitude information of robot 10 can be obtained, for example, from the simulation execution log of robot 10.

[0038] Furthermore, the motion information may include position and orientation information of objects within the operating environment of the robot 10. For example, the motion information may include position and orientation information of movable objects such as furniture, tools, and devices within the operating environment of the robot 10.

[0039] The server 30 transmits motion information, including the position and posture information of the generated robot 10, to the computer 20 via the network. The server 30 also stores the motion information, including the position and posture information of the generated robot 10, in the storage unit 32. This makes it possible to use the motion information later by other devices such as the computer 20.

[0040] Furthermore, when the server 30 performs a simulation to sequentially execute multiple tasks set when generating the operation program 11, it synchronizes information about the currently running task with the simulation results. For example, the server 30 extracts the execution time from the simulation results and stores it in an external memory area of ​​the simulator. By synchronizing the operation information with the execution time, it becomes possible to obtain the currently running task. This makes it possible to display the content of the currently running task in the operation video based on the simulation results. In the case of tasks such as conversations, it is also possible to display the content of the conversation in the operation video based on the simulation results.

[0041] The computer 40 is connected to the server 30 via a network. The computer 40 is used by a skill creator who creates skill information 32b. The computer 40 is, for example, a personal computer. The computer 40 creates the skill information 32b in accordance with an operation performed by the skill creator. The computer 40 transmits the created skill information 32b to the server 30 in accordance with an operation performed by the skill creator. Note that the computer 40 may be a mobile terminal such as a tablet or a smartphone.

[0042] The skill information 32b that is created in the computer 40 and transmitted to the server 30 may be made public, may be made available only to limited parties, or may be kept private, depending on the selection made by the skill creator.

[0043] The computer 50 is connected to the server 30 via a network. The computer 50 is used by a simulation creator who creates a simulation program 32c that performs simulation using an operation program 11. The computer 50 is, for example, a personal computer. The computer 50 creates the simulation program 32c in accordance with an operation performed by the simulation creator. The computer 50 transmits the created simulation program 32c to the server 30 in accordance with an operation performed by the simulation creator.

[0044] The simulation program 32c that is created in the computer 50 and transmitted to the server 30 may be made public, may be made available only to limited parties, or may be kept private, depending on the selection made by the simulation creator.

[0045] The computer 60 is connected to the server 30 via a network. The computer 60 is used by a robot creator who creates robot elements. The computer 60 is, for example, a personal computer. The computer 60 creates an element of the robot 10 in accordance with an operation performed by the robot creator. The computer 60 transmits the created element of the robot 10 to the server 30 in accordance with an operation performed by the robot creator.

[0046] The elements of the robot 10 include, for example, device drivers and mechanical design information of the robot 10.

[0047] The elements of the robot 10 created in the computer 60 and transmitted to the server 30 may be made public, may be made partially public with limited disclosure recipients, or may be kept private, depending on the selection of the robot creator.

[0048] In the present embodiment, the computer 20 generates a motion video of the robot 10 in a virtual space set separately from the simulation, based on motion information including position information and posture information of the robot 10 among simulation results obtained by the server 30. That is, the computer 20 acquires part of the information from the simulation results performed by the server 30, and generates the motion video of the robot 10. Note that, similarly to the computer 20, the computers 40, 50, or 60 may also generate a motion video of the robot 10 in a virtual space set separately from the simulation, based on motion information including position information and posture information of the robot 10 among simulation results obtained by the server 30.

[0049] Specifically, the computer 20 determines the position of the robot 10 based on the position information acquired from the server 30, and arranges the robot 10 in the virtual space. Further, the computer 20 determines a posture such as a joint angle of the robot 10 arranged in the virtual space based on the posture information acquired from the server 30. Further, the computer 20 determines the position and posture of the robot 10 at predetermined time intervals, and moves and operates the robot 10 in the virtual space. The position information may be coordinate information indicating an absolute position in the virtual space, or may be a displacement from a previous position. Further, the posture information may be angle information indicating an absolute joint angle of the robot 10, or may be a displacement angle from a previous state. Furthermore, all of the actual robot 10, the computer 20, and the server 30 may be temporally synchronized, or part of them may not be synchronized. For example, before causing the actual robot 10 to execute an operation, a test run in which the operation is virtually executed may be performed in the server 30.

[0050] The simulation is performed, for example, when a user is considering introducing robot 10 into a facility, and the user or a coordinator assisting with the introduction wants to see a video of robot 10 in action.

[0051] Furthermore, the computer 20 is connected to the server 30 via a network and generates motion video based on the position and posture information of the robot 10 received from the server 30 and the information of the configured virtual space. The computer 20 generates motion video in general-purpose formats such as MPEG4, MPEG2, AVI, WMV, MOV, FLV, and AVCHD.

[0052] Furthermore, the computer 20 displays the generated motion video on the display unit 22.

[0053] Furthermore, the computer 20 receives instructions for generating and playing motion videos via the operation input unit 23. For example, the computer 20 executes an application to cause the server 30 to perform a simulation. In the application, the computer 20 receives instructions to execute the simulation, instructions to generate video footage based on the simulation, and instructions to play back the generated motion video. The computer 20 can also log the results and generate or play back video footage from any arbitrary timing.

[0054] Furthermore, the computer 20 accepts settings for the display quality of the motion video via the operation input unit 23. For example, the computer 20 accepts settings such as the bitrate for the robot 10's movements, the number of pixels in the video, and the level of detail of the virtual space as display quality settings. The computer 20 then generates and plays back the motion video based on the accepted display quality settings.

[0055] As shown in Figure 3, the server 30 acquires the position and orientation information of the robot 10, the operation confirmation information of the robot 10, and the physical contact information of the robot 10 through simulation. The server 30 then transmits some of the information acquired through the simulation to the computer 20 via the network. For example, the server 30 transmits the position and orientation information of the robot 10, the operation confirmation information of the robot 10, and the physical contact information of the robot 10 to the computer 20.

[0056] The computer 20 generates an operation video using pre-configured virtual space information, the position and posture information of the robot 10, the operation confirmation information of the robot 10, and the physical contact information of the robot 10, specifically the position and posture information of the robot 10. The virtual space information may be set based on the location information of the user's facility, or based on the location information of a sample facility. Sample facilities may be prepared for each type of facility, such as hospitals, commercial facilities, offices, and public facilities. Furthermore, multiple samples may be prepared for the same type of facility, depending on their size, such as large, medium, and small.

[0057] The computer 20 may send the motion video generated based on the simulation results to the server 30.

[0058] Server 30 stores the video footage generated by computer 20 for viewing. The video footage stored on server 30 may be made public, made publicly available to a limited audience, or kept private, at the creator's discretion.

[0059] In the example shown in Figure 2, the user operates the computer 20 and edits the operation program 11 while referring to the simulation results.

[0060] The user edits the operation program 11 by editing the skill information settings using low-code editing. The user then runs a simulation to verify the operation of the robot 10 with the edited operation program 11. The user then visually checks the simulation results and edits the operation program 11 to improve the operation of the robot 10.

[0061] Specifically, the target robot is pre-configured on the simulator on server 30. The configured target robot is then deployed to the simulator. In other words, the specification information of the target robot 10 is placed and deployed on the simulator on server 30. Then, the edited current operation program 11 is loaded into the simulator.

[0062] Server 30 uses a simulator module to simulate the operation of robot 10 based on the operation program 11 registered in the task sequence module and the robot information registered in the robot module. Server 30 performs the simulation using, for example, a robot simulator. Here, the robot simulator is used by experts such as robot engineers, and those unfamiliar with robots may find it difficult to interpret the simulation results or access necessary information. The robot simulator can, for example, verify the operation of robot 10 and confirm physical contact.

[0063] Therefore, the server 30 transmits to the computer 20 motion information, which includes position information including the robot's current position and posture information including the posture information of the robot's joints, from the simulation results.

[0064] Then, the computer 20 generates and displays a user-friendly digital twin video of the robot 10 in motion based on its position and orientation information. Here, the results shown by the simulator that performs the simulation of the robot 10 may be displayed in a professional appearance so that experts such as robot engineers can verify the robot 10's motion and physical contact. For example, a robot expert may use a robot model for collision verification to check whether objects have collided if the collision model and the actual model differ from each other. A robot expert may also check by looking at the coordinate information of the object's position. In this case, the simulation results that the expert checks will have a professional appearance. In this case, the simulation results will be difficult for non-robot experts to understand. On the other hand, the motion video of the robot 10 in a user-friendly virtual space, generated by the computer 20, will be easy for non-robot experts to understand as well. For example, a non-robot expert may want to check whether the robot's task was successful or unsuccessful, such as whether it picked up the correct item. In this case, a non-robot expert will want to check the robot's motion, such as which object was picked up and what the robot grasped, so the display will have a user-friendly appearance with information such as textures. For example, if the object is a box, even if it's the same box, it will have a visually clear appearance that allows us to distinguish between a cardboard box and a candy box, etc. In contrast, robotics experts want to check for collisions with conceptual boxes.

[0065] As shown in Figure 4, the operation space data used by the simulator and the operation space data used to display the operation video of the computer 20 may be created from common data.

[0066] For example, server 30 is provided with an area for managing spatial data. Spatial data is created and edited, for example, by a UI development vendor that performs spatial editing. Spatial data is created from CAD data, on-site photographs, etc. Spatial data includes, for example, spatial shape information, object position information such as desks and chairs, and floor plan information such as doors and windows. The spatial data is then converted into simulator spatial data for execution by the simulator and simplified display spatial data for displaying the simulation results on computer 20.

[0067] (Effects of the Embodiment) The motion program generation system 100 includes a server 30 that simulates the operation of the robot 10 using a motion program 11 that depends on robot motion and robot control. This allows the server 30 to perform simulations based on the motion program 11 generated before the robot 10 is introduced. The system includes a server 30 that generates motion information including position information and posture information of the robot 10 that does not depend on robot motion and robot control from the simulation results of the server 30, and a computer 20 that generates motion video of the robot 10 in a set virtual space based on the motion information generated by the server 30. This eliminates the need for the user's computer 20 to prepare a simulator for simulating the motion program 11 that depends on robot motion and robot control. Furthermore, since motion video is generated based on motion information including position information and posture information of the robot 10 that does not depend on robot motion and robot control, the user can easily obtain the simulation result video. As a result, the operation of the robot 10 can be easily confirmed before the robot 10 is introduced.

[0068] Computer 20 is connected to server 30 via a network and generates motion video based on the position and posture information of robot 10 received from server 30 and the information of the configured virtual space. This allows computer 20 to connect to server 30 remotely via the network, so that users can view the simulation results video from any location.

[0069] Computer 20 includes an operation input unit 23 that accepts editing of the robot 10's operation program 11 in a format that is not compatible with ROS. This allows the computer 20 to edit the robot 10's operation program 11 while checking the simulation results, making it easy to generate the robot 10's operation program 11 with high accuracy. In other words, while editing the robot 10's operation program 11 in a ROS-compatible format is possible for experts, it is difficult for non-expert users to understand and perform. Furthermore, non-expert users find it difficult to identify which parts of the robot 10's operation program 11 in a ROS-compatible format need to be modified based on the simulation results. On the other hand, as described above, users can edit the robot 10's operation program 11 in a format that is not compatible with ROS by editing skills, tasks, and sequences that are intuitively and clearly expressed, while checking the simulation results. This allows even non-expert users to easily modify the robot 10's operation program 11 based on the simulation results. In addition, simulations can be performed using an external server 30 even from a computer 20 that does not have ROS installed.

[0070] The computer 20 receives instructions for generating and playing motion videos via the operation input unit 23. This allows the computer 20 to easily execute simulations based on the robot 10's motion program 11. Furthermore, the computer 20 can easily play back videos based on the simulation results.

[0071] The computer 20 accepts the setting of the display quality of the motion video via the operation input unit 23. This allows the user to switch the display quality of the video, such as a simplified video display or a realistic video display, through operations from the computer 20. This allows the user to check the operation of the robot 10 with motion video of a display quality that suits them.

[0072] The server 30 acquires the position and orientation information of the robot 10, the operation confirmation information of the robot 10, and the physical contact information of the robot 10 through simulation. The computer 20 uses the position and orientation information of the robot 10 from the position and orientation information of the robot 10, the operation confirmation information of the robot 10, and the physical contact information of the robot 10 to generate an operation video. As a result, the computer 20 can generate a simplified simulation result video using the position and orientation information of the robot 10 from the simulation results, thereby suppressing an increase in the processing load on the computer 20.

[0073] A server 30 is provided to store the motion video generated by the computer 20 in a viewable format. This allows other users to view and review the generated motion video, making it easier for other users to consider introducing the robot 10.

[0074] Robot 10 includes a social robot that provides services to people. This makes it easy to verify the operation of the social robot when introducing a social robot that provides services to people.

[0075] (Variations) The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than the description of the embodiments above, and further includes all modifications (variations) in the sense and scope equivalent to the claims.

[0076] In the above embodiment, an example was shown in which the server 30 that generates the operation program 11 and the server 30 that performs the simulation are the same, but the disclosure is not limited to this. For example, the server 30 that generates the operation program 11 and the server 30 that performs the simulation may be different servers.

[0077] In the above embodiment, an example was shown in which the operation program 11 is generated by the server 30, but the disclosure is not limited thereto. For example, the operation program 11 may be generated by a computer 20 used by a user of the robot 10, or the operation program 11 may be generated using both the server 30 and the computer 20.

[0078] In the above embodiment, an example was shown in which the server 30 that stores task information and skill information is the same, but this disclosure is not limited to this. For example, the server that stores task information and the server that stores skill information may be separate.

[0079] In the above embodiment, an example was shown in which the server 30 that stores skill information and the simulation program is the same, but this disclosure is not limited to this. For example, the server that stores skill information and the server that stores the simulation program may be separate.

[0080] In the above embodiment, an example was shown in which a computer 20 for generating the operation program 11 and a robot 10 are provided separately, but the disclosure is not limited thereto. For example, the computer 20 may be provided on the robot 10. Also, if the computer 20 is provided on the robot 10, the robot 10 and the server 30 may communicate information directly.

[0081] In the above embodiment, an example was shown in which the computer for generating the operation program 11 and the computer for generating the robot's operation video in the set virtual space based on the operation information generated from the simulation results are a common computer 20. However, the disclosure is not limited to this. For example, the computer for generating the operation program 11 and the computer for generating the robot's operation video in the set virtual space based on the operation information generated from the simulation results may be separate computers.

[0082] In the above embodiment, an example was shown in which the robot 10 introduced into the facility is a social robot, but this disclosure is not limited to this. For example, the robot 10 introduced into the facility may be an industrial robot or a medical robot.

[0083] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, dedicated processors, integrated circuits, ASICs (Application Specific Integrated Circuits), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuits. In this disclosure, a circuit, unit, or means is hardware that performs the enumerated functions, or hardware programmed to perform the enumerated functions. The hardware may be hardware disclosed herein, or other known hardware that is programmed or configured to perform the enumerated functions. If the hardware is a processor, which is considered a type of circuit, then the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or the processor.

[0084] (Embodiments) The exemplary embodiments described above will be understood by those skilled in the art to be specific examples of the following embodiments.

[0085] (Aspect 1) An action program generation system comprising: a simulation execution unit that simulates the movement of a robot using an action program that depends on robot movement and robot control; an action information generation unit that generates action information including position information and attitude information of the robot that does not depend on robot movement and robot control from the simulation results of the simulation execution unit; and an image generation unit that generates an image of the robot's movement in a set virtual space based on the action information generated by the action information generation unit.

[0086] (Aspect 2) The motion program generation system according to aspect 1, wherein the video generation unit is connected to the motion information generation unit via a network, and generates motion video based on the position information and posture information of the robot received from the motion information generation unit and the information of the set virtual space.

[0087] (Aspect 3) The motion program generation system according to aspect 1 or aspect 2, wherein the video generation unit includes an operation input unit that accepts editing of the motion program in a format that does not correspond to a predetermined operation system for the robot.

[0088] (Aspect 4) The operation program generation system according to aspect 3, wherein the video generation unit receives instructions to generate and play the operation video from the operation input unit.

[0089] (Aspect 5) The operation program generation system according to aspect 3 or aspect 4, wherein the video generation unit receives the setting of the display quality of the operation video from the operation input unit.

[0090] (Aspect 6) An operation program generation system according to any one of aspects 1 to 5, wherein the simulation execution unit obtains the position information and posture information of the robot, the operation confirmation information of the robot, and the physical contact information of the robot by simulation, and the video generation unit generates the operation video using the position information and posture information of the robot from the position information and posture information of the robot, the operation confirmation information of the robot, and the physical contact information of the robot.

[0091] (Aspect 7) The operation program generation system according to any one of aspects 1 to 6, further comprising a server that stores the operation video generated by the video generation unit in a viewable format.

[0092] (Aspect 8) The motion program generation system according to any one of aspects 1 to 7, wherein the robot includes at least one of a social robot that provides services to people, an industrial robot, and a medical robot.

[0093] 10 Robot 11 Motion program 20 Computer (image generation unit) 23 Operation input unit 30 Server (simulation execution unit, motion information generation unit) 100 Motion program generation system

Claims

A simulation execution unit that simulates the robot's movements using motion programs that depend on robot motion and robot control, A motion information generation unit generates motion information, including robot position information and attitude information that are independent of robot movement and robot control, from the simulation results obtained by the simulation execution unit, An action program generation system comprising: an action information generation unit that generates video footage of the robot's actions in a set virtual space based on the action information generated by the action information generation unit; and a video generation unit. 　 The motion program generation system according to claim 1, wherein the video generation unit is connected to the motion information generation unit via a network, and generates motion video based on the position information and posture information of the robot received from the motion information generation unit and the information of the set virtual space. 　 The motion program generation system according to claim 1, wherein the video generation unit includes an operation input unit that accepts editing of the motion program in a format that is not compatible with a predetermined operating system for the robot. 　 The operation program generation system according to claim 3, wherein the video generation unit receives instructions to generate and play the operation video from the operation input unit. 　 The operation program generation system according to claim 3, wherein the video generation unit receives the setting of the display quality of the operation video from the operation input unit. 　 The simulation execution unit obtains the position information and orientation information of the robot, the operation confirmation information of the robot, and the physical contact information of the robot through the simulation. The motion program generation system according to claim 1, wherein the video generation unit generates motion video using the position information and posture information of the robot, the motion confirmation information of the robot, and the physical contact information of the robot, specifically the position information and posture information of the robot. 　 The operation program generation system according to claim 1, further comprising a server that stores the operation video generated by the video generation unit in a viewable format. 　 The operation program generation system according to claim 1, wherein the robot includes at least one of a social robot that provides services to people, an industrial robot, and a medical robot.

Images