Generation method, computer program, and generation system

Abstract

To provide technology capable of generating an action control program actualizing desired robot actions.SOLUTION: A generation method of generating an action control program for a robot comprises the processes of: displaying a plurality of parameters associated with human's actions; receiving a choice of simulation parameters as parameters for letting the robot simulate actions among the plurality of displayed parameters; and generating the action control program by referring to human's actions corresponding to the received simulation parameters.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] This disclosure relates to a technology for generating an operation control program for a robot.

Background Art

[0002] Conventionally, there has been known a technology for generating an operation control program that controls the operation of a robot by imitating a human's actual operation (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, when generating an operation control program by uniformly imitating all the operations of a human's actual operation for a robot, it may be impossible to generate an operation control program that realizes the desired operation of the robot. For example, depending on the place where the robot is used, the work content, the shape of the workpiece to be worked on, etc., when all of the human operations are uniformly imitated for the robot, the operation by the robot may not be suitable for the work scene for the workpiece.

Means for Solving the Problems

[0005] According to the first aspect of this disclosure, a generation method for generating an operation control program for a robot is provided. This generation method includes a step of displaying a plurality of parameters related to a human operation, a step of receiving a selection of imitation parameters, which are the parameters to be imitated by the robot, from among the displayed plurality of parameters, and a step of generating the operation control program by referring to the human operation corresponding to the received imitation parameters.

[0006] According to a second embodiment of the present disclosure, a computer program is provided that causes a processor to generate a robot motion control program. This computer program causes the processor to perform the following functions: a function to display a plurality of parameters relating to human motion on a display device; a function to accept the selection of imitation parameters from among the displayed plurality of parameters, which are the parameters to be imitated by the robot; and a function to generate the motion control program by referring to the human motion corresponding to the accepted imitation parameters.

[0007] A third embodiment of the present disclosure provides a generation system for generating a robot motion control program. This generation system includes a display control unit that causes a plurality of parameters relating to human motion to be displayed on a display device; a reception unit that receives the selection of imitation parameters from among the displayed plurality of parameters, which are the parameters to be imitated by the robot; and a program generation unit that generates the motion control program by referring to the human motion corresponding to the received imitation parameters. [Brief explanation of the drawing]

[0008] [Figure 1] An overall configuration diagram showing the generation system of this embodiment. [Figure 2] A diagram showing the detailed configuration of the generation system. [Figure 3] A diagram illustrating the initial parameter information. [Figure 4] A diagram used to explain human motion information. [Figure 5] A flowchart illustrating the robot's teaching process. [Figure 6] A flowchart illustrating the process of generating a robot motion control program. [Figure 7] A diagram illustrating the process of generating a motion control program. [Figure 8] A diagram showing a video demonstrating the operation. [Figure 9] The screen displayed on the display device after executing step S80. [Figure 10] Figure 6 shows the first flowchart illustrating the details of step S30. [Figure 11] Figure 6 shows a second flowchart illustrating the details of step S30. [Figure 12] Figure 6 shows a third flowchart illustrating the details of step S30. [Figure 13] A diagram showing the first example of a video for confirming operation and an image of the received response. [Figure 14] This figure shows a second example of a video for confirming operation and an image of the received data. [Figure 15] This figure shows a third example of a video for confirming operation and an image of the received data. [Figure 16] This figure shows a fourth example of a video for confirming operation and an image of the received data. [Figure 17] A diagram showing a fifth example of a video for confirming operation and an image of the received data. [Figure 18] This figure shows a sixth example of a video for confirming operation and an image of the received data. [Modes for carrying out the invention]

[0009] A. Embodiments: Figure 1 is an overall configuration diagram showing the generation system 1000 of this embodiment. The generation system 1000 is a system that generates a motion control program for the robot 100 by referring to the actions of a human 80 during motion teaching to the robot 100. In this embodiment, an example is described in which the motion control program for the robot 100 is generated by referring to the actions of a human 80 moving a workpiece WK on a workbench 90 from a first position to a second position. The generation system 1000 comprises a display device 10, a robot control device 20, a system control device 50, sensors 30, and a robot 100.

[0010] In this embodiment, the robot 100 is a single-arm robot used by attaching various end effectors 140 to the arm flange 120 at the tip of the arm 110.

[0011] The arm 110 has six joints J1 to J6. The joints J2, J3, and J5 are bending joints, and the joints J1, J4, and J6 are torsion joints. Each joint is provided with a servo motor and a position sensor. The servo motor generates a rotational output for driving each joint. The position sensor 160 detects the angular position of the output shaft of the servo motor. For the sake of easy understanding of the technology, in FIG. 1, the servo motor and the position sensor are not shown.

[0012] Various end effectors 140 for performing operations such as gripping and processing on the object are mounted on the arm flange 120 at the tip of the joint J6. In this specification, the object handled by the robot 100 is also referred to as a "workpiece".

[0013] The position near the tip of the arm 110 can be set as a tool center point. Hereinafter, the tool center point is referred to as "TCP". TCP is a position used as a reference for the position of the end effector 140. For example, a predetermined position on the rotation axis of the joint J6 can be set as TCP.

[0014] The robot 100 can arrange the end effector 140 at an arbitrary position and in an arbitrary posture within the movable range of the arm 110. A force detector 130 and an end effector 140 are installed on the arm flange 120. The end effector 140 is a gripper in this embodiment. The end effector 140 can hold the workpiece WK.

[0015] The force detector 130 is provided on the robot 100 and can measure an external force applied to the robot 100. Specifically, the force detector 130 is a six-axis sensor. The force detector 130 can detect the magnitudes of forces parallel to the x-axis, y-axis, and z-axis that are orthogonal to each other in the sensor coordinate system, which is its own coordinate system, and the magnitudes of torques around the three axes.

[0016] The coordinate system that defines the space in which robot 100 is installed is called the "robot coordinate system." The robot coordinate system is a three-dimensional orthogonal coordinate system defined by mutually orthogonal x and y axes on the horizontal plane and a z axis with the vertically upward direction being positive. The coordinate system shown in Figure 1 is the robot coordinate system. Any position in three-dimensional space can be represented by the position along the x, y, and z axes, and any orientation in three-dimensional space can be represented by the rotation angles around the x, y, and z axes. In this specification, "position" can mean both position and orientation. Also, in this specification, "force" can mean both force and torque.

[0017] The display device 10 displays various information such as characters and images. The display device 10 is, for example, a liquid crystal monitor. In this embodiment, the display device 10 is a monitor of a personal computer.

[0018] The sensor system 30 comprises a first imaging device 81, a second imaging device 82, and a pressure sensor 75. The sensor system 30 and the system control device 50 are connected by wire or wireless means, and data can be transmitted from the sensor system 30 to the system control device 50. The first imaging device 81 includes an image sensor that acquires images of the movements of a person 80 as the person 80 works on a workpiece WK over time, and a depth sensor that detects the distance to objects such as the person 80 and the workpiece WK, as well as the three-dimensional shape of the objects. The second imaging device 82 includes an image sensor that acquires images of the movements of a robot 100 as the robot 100 works on a workpiece WK over time, and a depth sensor that detects the distance to objects such as the robot 100 and the workpiece WK, as well as the three-dimensional shape of the objects. The data acquired or detected by the first imaging device 81 and the second imaging device 82 are transmitted to the system control device 50. The pressure sensor 75 is attached, for example, to the hand of a person 80 to detect the magnitude of the force with which the person 80 grips the workpiece WK.

[0019] The robot control device 20 controls the arm 110 and the end effector 140 by transmitting control commands to the robot 100. The system control device 50 generates an motion control program for the robot 100 based on data from the sensors 30 and reference parameters received (described later). The generated motion control program is transmitted to the robot control device 20 and stored in the robot control device 20's memory. The robot control device 200 and the system control device 50 may be a single device or separate devices.

[0020] Figure 2 shows the detailed configuration of the generation system 1000. The system control device 50 is a computer comprising a first processor 52 and a first storage device 71. The system control device 50 functions as a display control unit 53, a reception unit 54, a specification unit 55, a program generation unit 56, and a moving image generation unit 59 by executing various programs stored in the first storage device 71. The display control unit 53 controls the display operation of the display device 10 by transmitting various information to be displayed on the display device 10. The reception unit 54 accepts external input via an input interface such as a keyboard or mouse.

[0021] The identification unit 55 identifies the movements of the human 80 based on data such as detection results from the sensors 30. The identification unit 55 includes a finger identification unit 61, an object identification unit 62, a force identification unit 63, and a movement identification unit 64.

[0022] The finger identification unit 61 uses the captured image and depth information acquired from the first imaging device 81 to identify the position and orientation of the human 80's fingers at predetermined time intervals. For example, the finger identification unit 61 can identify the position and orientation of the fingers by attaching markers to the fingers of the human 80 to be identified and detecting the markers in the captured image. In other embodiments, the finger identification unit 61 may identify the position and orientation of the fingers using other methods such as pattern matching.

[0023] The object identification unit 62 uses the captured image and depth information acquired from the first imaging device 81 to identify the position and orientation of the workpiece WK at predetermined time intervals. For example, the object identification unit 62 performs image processing such as edge extraction on the captured image and identifies the workpiece WK from the extracted edges. Then, the object identification unit 62 uses the captured image and depth information to determine the position and orientation of the identified workpiece WK.

[0024] The force identification unit 63 determines the magnitude of the force applied by the human 80 to the workpiece WK by receiving detection signals from the pressure sensor 75 at predetermined time intervals.

[0025] The motion identification unit 64 identifies the movements of the human 80 using a predetermined coordinate system and physical quantities by arranging in chronological order (i) the position of the human 80's fingers identified by the finger identification unit 61, (ii) the position of the workpiece WK identified by the object identification unit 62, and (iii) the magnitude of the force, at predetermined time intervals. In this embodiment, the predetermined coordinate system is, for example, the camera coordinate system of the first imaging device 81. The identified movements of the human 80 are stored in the first storage device 71 as human motion information 74. The human motion information 74 includes parameters for operating the robot 100 and the contents of each parameter. Details of the human motion information 74 will be described later.

[0026] The first storage device 71 is composed of ROM, RAM, etc., and stores various programs executed by the first processor 52, as well as parameter initial information 72 and human motion information 74. The parameter initial information 72 is the parameters necessary for the operation of the robot 100 and their contents, and is used when the robot 100 is to be operated without referring to the contents of the parameters identified by the identification unit 55. Details of the parameter initial information 72 will be described later.

[0027] The program generation unit 56 generates a motion control program for the robot 100 based on the actions of the human 80 identified by the parameter initial information 72 and the identification unit 55. The program generation unit 56 includes an imitation teaching decision unit 57 and a robot motion conversion unit 58. The imitation teaching decision unit 57 determines, based on the imitation parameters received via the reception unit 54, whether to use the content of the parameters identified by the identification unit 55 or the content of the parameters in the parameter initial information 72 for each parameter used when generating the motion control program. The robot motion conversion unit 58 uses the content defined for each parameter determined by the imitation teaching decision unit 57 and generates a motion control program using a conversion program such as a conversion matrix predetermined by inverse kinematics.

[0028] The video generation unit 59 generates an action confirmation video PI that represents the actions of the human 80 identified by the identification unit 55 and the actions of the robot 100 when the action control program generated by the program generation unit 56 is executed. The action confirmation video PI is displayed by the display device 10 under the control of the display control unit 53. The action confirmation video PI has a human video PIA1 acquired by capturing the actions of the human 80 with the first imaging device 81, and a robot video PIA2 that represents the actions of the robot 100 simulated based on the video control program generated by the program generation unit 56. The robot video PIA2, which is a simulation video, is generated by the video generation unit 59. For the human video PIA1, the video image taken when the actions of the human 80 were captured by the first imaging device 81 for imitation teaching is used. In other embodiments, the robot video PIA2 may have a video image acquired by capturing the actions of the robot 100 when it executes the video control program generated by the program generation unit 56 with the second imaging device 82.

[0029] Figure 3 is a diagram illustrating the parameter initial information 72. The parameter initial information 72 defines several parameters for operating the robot 100 and the pre-set operation content of the robot 100 for each parameter. The parameters are "movement trajectory," "workpiece position," "workpiece gripping position," "force on workpiece," "movement speed," and "preparatory action." "Movement trajectory" is a parameter relating to the movement trajectory of the workpiece WK. "Workpiece position" is a parameter relating to the position of the workpiece WK before and after movement. "Force on workpiece" is a parameter relating to the magnitude of the force applied to the workpiece WK. "Movement speed" is a parameter relating to the movement speed of the workpiece WK. "Preparatory action" is a parameter relating to the action of the robot 100 performed before moving the workpiece WK.

[0030] When the program generation unit 56 generates a robot motion control program using the parameter initial information 72, the robot 100 operates as follows: Regarding the movement trajectory, the robot 100 operates so that the trajectory from the starting point before movement to the ending point after movement is the shortest possible trajectory. In other embodiments, the robot 100 may operate so that the movement trajectory is the trajectory that minimizes the movement of each joint of the robot 100, or any other predetermined trajectory. Regarding the position of the workpiece WK before and after movement, the robot 100 operates so that it is at a position corresponding to the parameter's movement trajectory. The position of the workpiece WK before movement may be predetermined. Regarding the gripping position in which the robot 100 grips the workpiece WK, the robot 100 operates so that the robot 100 grips the workpiece WK at a predetermined position. The predetermined position is, for example, the coordinate position of the center of the gripping surface of the workpiece WK. The robot 100 operates so that the magnitude of the force applied to the workpiece WK by gripping it is a predetermined initial value. The robot 100 operates so that the movement speed of the workpiece WK is the fastest possible movement speed for the robot's TCP. In the parameter initial information 72, no preparatory movements for the robot 100 are set. The contents of each parameter in the parameter initial information 72 are specified by various coordinate systems and physical quantities such as velocity. For example, the movement speed of the workpiece WK is specified by a physical quantity. By pre-defining relational expressions that show the relationships between various coordinate systems such as the robot coordinate system and the sensor coordinate system, and relational expressions that convert physical quantities into the control program for the robot 100, the contents of each parameter expressed in any coordinate system or physical quantity can be converted into the robot 100's motion control program.

[0031] Figure 4 is a diagram illustrating the human motion information 74. The human motion information 74 defines multiple parameters for operating the robot 100 and the content of the movements of the human 80 identified by the identification unit 55. Each parameter of the human motion information 74 differs from each parameter of the parameter initial information 72 shown in Figure 3 in that it targets the movements performed by the human 80 during motion teaching. In other words, each parameter of the parameter initial information 72 is a parameter related to the movements of the human 80. In the parameter initial information 72, the parameter "movement trajectory" is a parameter that indicates the movement trajectory of the workpiece WK. Also, in the parameter initial information 72, the parameter "workpiece position" is a parameter that indicates the position of the workpiece WK before and after movement when the human 80 moves the workpiece WK. Also, in the parameter initial information 72, the parameter "workpiece gripping position" is a parameter that indicates the gripping position in which the human 80 grips the workpiece WK. Furthermore, in the parameter initial information 72, the parameter "force on workpiece" indicates the magnitude of the force applied to the workpiece WK by the human 80 gripping the workpiece WK. Also in the parameter initial information 72, the parameter "movement speed" indicates the movement speed of the workpiece WK when the human 80 moves the workpiece WK. Also in the parameter initial information 72, the parameter "preparatory action" indicates the action performed by the human 80 before moving the workpiece WK. The contents of each parameter are determined by the identification unit 55 using physical quantities such as camera coordinate system and velocity.

[0032] As shown in Figure 2, the robot control device 20 is a computer comprising a second processor 22 and a second storage device 24. The second processor 22 controls the operation of the robot 100 by executing an operation control program 25 stored in the second storage device 24. The second storage device 24 is composed of RAM and ROM. The operation control program 25 generated by the program generation unit 56 of the system control device 50 is stored in the second storage device 24.

[0033] Figure 5 is a flowchart of the teaching process for the robot 100. First, in step S10, a human 80 with a pressure sensor 75 attached performs teaching actions for the robot 100 in front of the first imaging device 81, and the system control device 50 acquires detection information from the sensors 30. Specifically, in step S10, the system control device 50 acquires captured images and depth information from the first imaging device 81 and detection signals from the pressure sensor 75 at predetermined time intervals.

[0034] Next, in step S12, the identification unit 55 of the system control device 50 uses the acquired image, depth information, and detection signal to determine the position and shape of the human 80's fingers, the position of the workpiece WK, and the magnitude of the force applied to the workpiece WK by the human 80 at predetermined time intervals.

[0035] Next, in step S14, the identification unit 55 identifies the movements of the human 80, including preparatory movements, using the position and shape of the human 80's fingers, the position of the workpiece WK, and the magnitude of the force identified in step S12. The identified movements of the human 80 are stored in the first storage device 71 as human movement information 74. The magnitude of the force applied by the human 80 is defined as the content of the "force on the workpiece" parameter in the human movement information 74. Preparatory movements are defined, for example, by the human 80 performing a predetermined gesture before grasping the workpiece WK. For example, if the human 80 approaches the workpiece WK with their hands closed and then opens their hands when they reach a predetermined distance from the workpiece WK, the content of the gesture described above is defined in the human movement information 74 as a "preparatory movement". When the motion control program is generated by referencing the preliminary movements in the human motion information 74, the robot 100 will perform an action that mimics the preliminary movements specified in the human motion information 74. In other words, for the gripper, which is the end effector 140, the motion control program will be generated to maintain a closed state until it approaches the workpiece WK, and then open when it approaches a predetermined distance from the workpiece WK.

[0036] Figure 6 is a flowchart illustrating the process of generating the motion control program for the robot 100. Figure 7 is a diagram illustrating the process of generating the motion control program. Figure 7 shows the screen displayed on the display device 10. The generation process shown in Figure 6 may start automatically after the teaching process shown in Figure 5 is completed, or it may start when a start command is received from the user.

[0037] As shown in Figure 6, in step S20, the display control unit 53 of the system control device 50 displays a reception image IM that accepts instructions from the user in order to generate an operation control program. As shown in Figure 7, the reception image IM is an image that accepts the selection of an action to be imitated by the robot 100 from among the actions of the human 80 identified in step S14 of Figure 5. The reception image IM has an input explanation image IM0, a parameter explanation image IM1, a parameter selection image IM2, a confirmation image IM3, and a decision image IM4. The input explanation image IM0 is an image that prompts the user to input via the reception image IM. The parameter explanation image IM1 is an image that shows multiple parameters and is a text image that explains each parameter. Each parameter in the parameter explanation image IM1 corresponds to each parameter in the parameter initial information 72 and the human action information 74. Each parameter in the parameter explanation image IM1 is a parameter related to the actions of the human 80 and is a candidate parameter to be referenced, i.e., imitated, by the robot 100.

[0038] Parameter selection image IM2 is an image used to determine whether or not to select each of several parameters as an imitation parameter, which is a parameter that the robot 100 should imitate. Parameter selection image IM2 consists of images representing "YES" and "NO" placed next to each parameter in parameter description image IM1. The user uses an input device such as a mouse to select "YES" for parameters that the robot 100 should imitate the movements of human 80, and "NO" for parameters that the robot 100 should not imitate. Parameters for which "YES" is selected are selected as imitation parameters.

[0039] The confirmation image IM3 is an image for receiving the information selected by the parameter selection image IM2 via the reception unit 54. The user selects the confirmation image IM3 using an input interface such as a mouse, and the selection information selected by the parameter selection image IM2 is received by the reception unit 54. The received selection information is stored in the first storage device 71. That is, as in step S22, when the confirmation image IM3 is selected, the reception unit 54 accepts the selection of imitation parameters, which are the parameters to be imitated by the robot 100, from among the multiple parameters displayed in the parameter explanation image IM1. Also, when the confirmation image IM3 is selected, the video generation unit 59 generates the aforementioned motion confirmation video PI. The decision image IM4 is used to transfer the motion control program generated by the program generation unit 56 to the robot control device 20. When the decision image IM4 is selected, the motion control program generated by the program generation unit 56 is transferred to the robot control device 20.

[0040] Next, in step S30 shown in Figure 6, the program generation unit 56 generates an action control program by referring to the movements of the human 80 corresponding to the imitation parameters and the parameter initial information 72. Specifically, the program generation unit 56 identifies the content of the movements of the human 80 corresponding to the imitation parameters from the human movement information 74, and identifies the content of the movements of the parameters other than the imitation parameters from the parameter initial information 72. Then, the program generation unit 56 generates an action control program using the identified content of the movements and a transformation program such as a transformation matrix predetermined by inverse kinematics. As described above, in the process of generating the action control program in step S30, the parameter initial information 72 is used to generate the action control program for the parameters other than the imitation parameters among the multiple parameters.

[0041] Next, in step S70, the motion image generation unit 59 generates an motion confirmation motion image PI. Figure 8 shows the motion confirmation motion image PI. The motion confirmation motion image PI includes the aforementioned human motion image PIA1 and robot motion image PIA2, a human assistant image PIB1, and a robot assistant image PIB2. In the human motion image PIA1 and robot motion image PIA2 shown in Figure 8, for illustrative purposes, the movement trajectory of the workpiece WK is indicated by dotted arrows YA1 and YA2, and the movement speed of the workpiece WK is represented by the length of the dotted line. The longer the length of the dotted line, the slower the movement speed. The human assistant image PIB1 is an image that shows the gripping position of the workpiece WK and the magnitude of the gripping force of the workpiece WK, among the movements of the human 80 identified by the identification unit 55 during motion teaching. The human motion image PIA1 and robot motion image PIA2 also include images of the workbench 90.

[0042] The human-assisted image PIB1 has a first workpiece image IWK1 that mimics the workpiece WK, and a first point image D1a superimposed on the first workpiece image IWK1. The position and size of the first point image D1a within the first workpiece image IWK1 indicate the gripping position and the magnitude of the gripping force applied by the human 80 when gripping the workpiece WK during motion teaching. The larger the size of the first point image D1a, the greater the gripping force.

[0043] The robot assistance image PIB2 has a second workpiece image IWK2 that mimics the workpiece WK, and a second point image D1b superimposed on the second workpiece image IWK2. The position of the second point image D1b in the second workpiece image IWK2 indicates the gripping position where the robot 100 grips the workpiece WK. The size of the second point image D1b indicates the magnitude of the gripping force applied by the robot 100 to the workpiece WK. The larger the size of the second point image D1b, the greater the gripping force.

[0044] As shown in Figure 6, in step S80 following step S70, the display control unit 53 causes the generated operation confirmation video PI to be displayed on the display device 10. Figure 9 shows the screen displayed on the display device 10 after step S80 is executed. The display control unit 53 causes the display device 10 to display the operation confirmation video PI on the same screen as the reception image IM. The example shown in Figure 9 shows the case where all parameters are selected as imitation parameters that refer to the actions of human 80.

[0045] In step S80, following step S70 as shown in Figure 6, the program generation unit 56 determines in step S90 whether or not to transfer the motion control program generated in step S30 to the robot control device 20. In other words, in step S90, it is determined whether or not the motion control program generated in step S30 has been decided as the final program for controlling the movement of the robot 100. In step S90, if the decision image IM4 shown in Figure 9 is selected, the determination is "Yes", and the motion control program is transferred to the robot control device 20.

[0046] On the other hand, if the user wishes to modify the generated motion control program by referring to the motion confirmation video PI, they select either the "YES" or "NO" image from the parameter selection image IM2 again, and then select the confirmation image IM3. This causes steps S22 onward to be executed again. Alternatively, if the user wishes to modify the generated motion control program by referring to the motion confirmation video PI, the human 80 may perform the teaching action again to execute the teaching process shown in Figure 5.

[0047] Figure 10 is a first flowchart showing the details of step S30 shown in Figure 6. In step S32, the imitation teaching decision unit 57 determines whether "movement trajectory" has been selected as the imitation parameter. If the determination in step S32 is "Yes", the robot motion conversion unit 58 decides in step S36 to refer to the movement trajectory of the workpiece WK over time, where the parameter in the human motion information 74 is the content of "movement trajectory", when generating the motion control program. Then, in step S38, the robot motion conversion unit 58 generates a program for the movement trajectory of the workpiece WK among the motion control programs for the robot 100. Specifically, in step S38 after going through step S36, the robot motion conversion unit 58 generates a motion control program such that the movement trajectory of the workpiece WK due to the robot 100's movement becomes the movement trajectory defined in the human motion information 74.

[0048] On the other hand, if the determination in step S32 is "No", the robot motion conversion unit 58 decides in step S34 to refer to the content of the parameter "movement trajectory" in the parameter initial information 72 when generating the motion control program. Then, in step S38, if step S34 is passed, the robot motion conversion unit 58 generates a motion control program such that the movement trajectory of the workpiece WK due to the movement of the robot 100 becomes the movement trajectory defined in the parameter initial information 72.

[0049] In step S42, the imitation teaching determination unit 57 determines whether "work position" has been selected as the imitation parameter. If the determination in step S42 is "Yes", the robot motion conversion unit 58 decides in step S46 to refer to the position of the workpiece WK before and after movement, for which the parameter in the human motion information 74 is "work position", when generating the motion control program. Then, in step S48, the robot motion conversion unit 58 generates a program for the robot 100's motion control program that includes the position of the workpiece WK before and after movement. Specifically, in step S48, when step S46 is passed, the robot motion conversion unit 58 generates a motion control program such that the position of the workpiece WK before and after movement due to the robot 100's motion is the position before and after movement defined in the human motion information 74.

[0050] On the other hand, if the determination in step S42 is "No", the robot motion conversion unit 58 decides in step S44 to refer to the content of the parameter "work position" in the parameter initial information 72 when generating the motion control program. Then, in step S48, if step S44 is passed, the robot motion conversion unit 58 generates a motion control program such that the position of the work WK before and after movement by the robot 100 is the position specified in the parameter initial information.

[0051] Figure 11 is a second flowchart showing the details of step S30 shown in Figure 6. In step S52, the imitation teaching decision unit 57 determines whether "workpiece gripping position" has been selected as the imitation parameter. If the determination in step S52 is "Yes", the robot motion conversion unit 58 decides in step S56 to refer to the content of the parameter "gripping position" in the human motion information 74 when generating the motion control program. Then, in step S58, the robot motion conversion unit 58 generates a program for the gripping position in which the robot 100 grips the workpiece WK within the motion control program for the robot 100. Specifically, in step S58 when going through step S56, the robot motion conversion unit 58 generates a motion control program so that the gripping position of the workpiece WK by the robot 100's movement becomes the gripping position defined in the human motion information 74.

[0052] On the other hand, if the determination in step S52 is "No", the robot motion conversion unit 58 decides in step S54 to refer to the content of the parameter "workpiece gripping position" in the parameter initial information 72 when generating the motion control program. Then, in step S58, if step S54 is passed, the robot motion conversion unit 58 generates a motion control program such that the gripping position of the workpiece WK by the robot 100's movement, that is, the gripping position in which the robot 100 grips the workpiece WK, becomes the gripping position defined in the parameter initial information 72.

[0053] In step S62, the imitation teaching determination unit 57 determines whether "force on the workpiece" has been selected as the imitation parameter. If the determination in step S62 is "Yes", the robot motion conversion unit 58 decides in step S66 to refer to the content of the parameter "force on the workpiece" in the human motion information 74 when generating the motion control program. Then, in step S68, the robot motion conversion unit 58 generates a program for the force applied to the workpiece WK in the motion control program of the robot 100. Specifically, in step S68, if step S66 is passed, the robot motion conversion unit 58 generates an motion program such that the magnitude of the force used by the robot 100 to grip the workpiece WK is the magnitude of the force defined in the human motion information 74.

[0054] On the other hand, if the determination in step S62 is "No", the robot motion conversion unit 58 decides in step S64 to refer to the content of the parameter "work gripping position" in the parameter initial information 72 when generating the motion control program. Then, in step S68, if step S64 is passed, the robot motion conversion unit 58 generates a motion control program such that the magnitude of the force used by the robot 100 to grip the workpiece WK is the magnitude of the force specified in the parameter initial information 72.

[0055] Figure 12 is a third flowchart showing the details of step S30 shown in Figure 6. In step S72, the imitation teaching decision unit 57 determines whether "movement speed" has been selected as the imitation parameter. If the determination in step S72 is "Yes", the robot motion conversion unit 58 decides in step S76 to refer to the content of the parameter "movement speed" in the human motion information 74 when generating the motion control program. Then, in step S78, the robot motion conversion unit 58 generates a program for the movement speed of the workpiece WK in the motion control program of the robot 100. Specifically, in step S78 after going through step S76, the robot motion conversion unit 58 generates a motion control program so that the movement speed of the workpiece WK due to the robot 100's movement becomes the movement speed specified in the human motion information 74.

[0056] On the other hand, if the determination in step S72 is "No", the robot motion conversion unit 58 decides in step S74 to refer to the content of the parameter "movement speed" in the parameter initial information 72 when generating the motion control program. Then, in step S78, if step S74 is passed, the robot motion conversion unit 58 generates a motion control program such that the movement speed of the workpiece WK due to the movement of the robot 100 becomes the movement speed specified in the parameter initial information 72.

[0057] In step S82, the imitation teaching decision unit 57 determines whether "preparatory movement" has been selected as the imitation parameter. If the determination in step S82 is "Yes", the robot motion conversion unit 58 decides in step S86 to refer to the content of the "preparatory movement" parameter in the human motion information 74 when generating the motion control program. Then, in step S88, the robot motion conversion unit 58 generates a program for the preparatory movement in the motion control program of the robot 100. Specifically, in step S88, when step S86 has been passed, the robot motion conversion unit 58 generates a motion control program such that the robot 100's movement becomes the content specified in the human motion information 74 as the preparatory movement.

[0058] On the other hand, if the determination in step S82 is "No", the robot motion conversion unit 58 decides in step S84 to refer to the content of the parameter "preparatory action" in the parameter initial information 72 when generating the motion control program. Then, in step S88, if step S84 is passed, the robot motion conversion unit 58 generates a motion control program such that the preparatory action for the robot 100's movement is as specified in the parameter initial information 72. As shown in Figure 3, since the parameter initial information 72 specifies that no "preparatory action" is performed, in this embodiment, the gripper, which is the end effector 140, moves to approach the workpiece WK in an open state.

[0059] Figure 13 shows the first example of the motion confirmation video PI and reception image IM. It indicates that the image with single hatching among the parameter selection images IM2 has been selected. The same applies to the following figures. In Figure 13, among the multiple parameters, only "work position" is selected as a simulated parameter that refers to the movements of the human 80. In this case, in the robot video PIA2 displayed on the display device 10, the position of the work WK before and after movement is the position defined by the human motion information 74. In other words, the position of the work WK on the workbench 90 before and after movement in the human video PIA1 is the same as the position of the work WK on the workbench 90 before and after movement in the robot video PIA2.

[0060] Figure 14 shows a second example of the motion confirmation video PI and reception image IM. In Figure 14, among several parameters, only "movement trajectory" is selected as an imitation parameter that references the movements of the human 80. In this case, in the robot video PIA2 displayed on the display device 10, only the movement trajectory of the workpiece WK becomes the movement trajectory defined by the human motion information 74. In other words, the movement trajectory of the workpiece WK in the human video PIA1 and the movement trajectory of the workpiece WK in the robot video PIA2 are the same.

[0061] Figure 15 shows a third example of the motion confirmation video PI and the reception image IM. In Figure 15, among several parameters, only "workpiece gripping position" is selected as a simulated parameter that references the movements of the human 80. In this case, in the robot video PIA2 displayed on the display device 10, only the gripping position of the workpiece WK becomes the gripping position defined by the human motion information 74. That is, the position of the first point image D1a relative to the first workpiece image IWK1 and the position of the second point image D1b relative to the second workpiece image IWK2 become the same.

[0062] Figure 16 shows a fourth example of the motion confirmation video PI and reception image IM. In Figure 16, among several parameters, only "force on the workpiece" is selected as a simulated parameter that references the movements of the human 80. In this case, in the robot video PIA2 displayed on the display device 10, only the magnitude of the force applied to the workpiece WK is the magnitude of the force defined by the human motion information 74. That is, the size of the first point image D1a and the size of the second point image D1b are the same.

[0063] Figure 17 shows a fifth example of the motion confirmation video PI and reception image IM. In Figure 17, among several parameters, only "movement speed" is selected as an imitation parameter that references the movements of the human 80. In this case, in the robot video PIA2 displayed on the display device 10, only the movement speed of the workpiece WK due to the movements of the robot 100 becomes the movement speed of the workpiece WK defined by the human motion information 74. In other words, the movement speed of the workpiece WK in the human video PIA1 and the movement speed of the workpiece WK in the robot video PIA2 are the same.

[0064] Figure 18 shows a sixth example of the motion confirmation video PI and reception image IM. In Figure 18, among several parameters, only the "preparatory motion" is selected as an imitation parameter that refers to the motion of the human 80. In this case, in the robot video PIA2 displayed on the display device 10, only the preparatory motion among the motions of the robot 100 becomes the preparatory motion defined by the human motion information 74. In the robot video PIA2 shown in Figure 18, the gripper, which is the end effector 140, remains closed until it approaches the workpiece WK, and opens when it approaches a predetermined distance from the workpiece WK.

[0065] The user checks the motion confirmation video PI, which is an example of Figures 13 to 18 above, determines the parameters related to the movements of the human 80 that the robot 100 should imitate, and causes the system control device 50 to generate a motion control program.

[0066] According to the above embodiment, since it is possible to select imitation parameters to be imitated by the robot 100 from among multiple parameters related to the movements of the human 80, it is possible to generate an action control program that can accurately realize the desired robot movements. However, if the action control program is generated so that the movement trajectory of the workpiece WK due to the movements of the human 80 becomes the movement trajectory of the workpiece WK due to the movements of the robot 100, there may be cases where the movement is not easy for the robot 100 to operate. As a result, there is a risk that the desired movement speed of the workpiece WK cannot be achieved. Furthermore, the movements of the human 80 and the movements of the robot 100 are not necessarily performed in exactly the same place. Also, there may be cases where it is not necessary for the robot 100 to move the workpiece WK to the same position as the position after the workpiece WK was moved by the movements of the human 80.Therefore, as shown in Figure 7, by making it possible to select imitation parameters to be imitated by the robot 100 from among multiple parameters related to the movements of the human 80, the system control device 50 can generate an action control program that can accurately realize the desired robot 100 movements.

[0067] Furthermore, according to the above embodiment, as shown in Figure 4, multiple parameters are set that are assumed to be highly likely to be imitated by the robot 100. In other words, the multiple parameters are parameters corresponding to the basic movements that the robot 100 performs when moving the workpiece WK. Also, according to the above embodiment, as shown in Figures 10 to 12, the program generation unit 56 can generate an operation control program using the parameter initial information 72 for parameters other than the imitation parameters.

[0068] B. Other embodiments: B-1. Other Embodiments 1: In the above embodiment, as shown in Figure 4, there were six parameters, but there can be two or more, and other parameters may also be included.

[0069] C. Other forms: This disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms (aspects). The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure, or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate.

[0070] (1) According to a first embodiment of the present disclosure, a method for generating a robot motion control program is provided. This generation method comprises the steps of: displaying a plurality of parameters relating to human motion; receiving the selection of an imitation parameter from among the displayed plurality of parameters, which is the parameter to be imitated by the robot; and generating the motion control program by referring to the human motion corresponding to the received imitation parameter. According to this embodiment, since an imitation parameter to be imitated by the robot can be selected from among a plurality of parameters relating to human motion, a motion control program that can accurately realize the desired robot motion can be generated.

[0071] (2) In the above embodiment, the plurality of parameters may include at least two of the following: the position of the workpiece before and after movement of the workpiece handled by the robot; the movement trajectory of the workpiece; the gripping position in which the human grasps the workpiece; the movement speed of the workpiece; the force applied by the human to the workpiece; and the preparatory actions performed by the human before moving the workpiece. In this embodiment, the plurality of parameters may include at least two parameters corresponding to the basic actions performed by the robot when moving the workpiece.

[0072] (3) In the above embodiment, if the imitation parameters include the preparatory movements, the generation step may generate the motion control program so that the robot performs an action that imitates the preparatory movements. According to this embodiment, the robot can be made to imitate human preparatory movements as an action performed before moving the workpiece.

[0073] (4) In the above embodiment, the generation step may generate the motion control program using pre-set parameter initial information that defines the robot's operation for parameters other than the imitation parameters among the plurality of parameters. According to this embodiment, the motion control program can be generated using parameter initial information for parameters other than the pattern parameters.

[0074] (5) In the above embodiment, the display step may display a reception image that includes a parameter description image showing the plurality of parameters and a parameter selection image for determining whether or not to select each of the plurality of parameters as the imitation parameter. In this embodiment, the user can easily decide whether or not to select each of the plurality of parameters as the imitation parameter using the parameter selection screen.

[0075] This disclosure can also be implemented in various forms other than those described above. For example, it can be implemented in the form of a computer program that causes a processor to generate a robot motion control program, a non-transitory storage medium that stores the computer program, or a generation system that generates a robot motion control program. [Explanation of Symbols]

[0076] 10…Display device, 20…Robot control device, 22…Second processor, 24…Second memory device, 25…Motion control program, 30…Sensors, 50…System control device, 52…First processor, 53…Display control unit, 54…Reception unit, 55…Identification unit, 56…Program generation unit, 57…Imitation teaching determination unit, 58…Robot motion conversion unit, 59…Motion image generation unit, 61…Finger identification unit, 62…Object identification unit, 63…Force identification unit, 64…Motion identification unit, 71…First memory device, 72…Parameter initial information, 74…Human motion information, 75…Pressure sensor, 80…Human, 81…First imaging device, 82…Second imaging device, 100…Robot 110…Arm, 120…Arm flange, 130…Force detector, 140…End effector, 160…Position sensor, 200…Robot control device, 1000…Generation system, D1a…First point image, D1b…Second point image, IM…Reception image, IM0…Input explanation image, IM1…Parameter explanation image, IM2…Parameter selection image, IM3…Confirmation image, IM4…Decision image, IWK1…First work image, IWK2…Second work image, J1~J6…Joint, PI…Motion confirmation video, PIA1…Human video, PIA2…Robot video, PIB1…Human assistance video, PIB2…Robot assistance video

Claims

1. A method for generating a robot motion control program, A process of displaying multiple parameters related to human movement, A step of receiving the selection of an imitation parameter from among the displayed plurality of parameters, which is the parameter to be imitated by the robot for each of the plurality of parameters, A step of generating the motion control program by referring to the human actions corresponding to the received imitation parameters, A generation method comprising the steps of displaying on a display device a human motion image representing the human motion and a robot motion image representing the robot's motion simulated based on the motion control program.

2. A method for generating according to claim 1, The aforementioned multiple parameters are, The position of the workpiece before and after movement handled by the robot, The movement trajectory of the aforementioned workpiece and The gripping position in which the human being grips the workpiece, The moving speed of the aforementioned workpiece and The force applied by the person to the workpiece, Preliminary actions performed by the person before moving the workpiece, A method of generation that includes at least two of the following.

3. A method for generating according to claim 2, The generation step is a generation method that, if the imitation parameters include the preliminary movements, generates the motion control program so that the robot performs movements that imitate the preliminary movements.

4. A method for generating according to any one of claims 1 to 3, The generation step is a generation method in which, with respect to the parameters other than the imitation parameters among the plurality of parameters, the motion control program is generated using parameter initial information which is set in advance and defines the motion of the robot.

5. A method for generating according to any one of claims 1 to 4, The generation method includes a step of displaying a plurality of parameters, which involves displaying a reception image that includes a parameter description image showing the plurality of parameters and a parameter selection image that determines whether or not to select each of the plurality of parameters as the imitation parameter.

6. A computer program that causes a processor to execute the generation of a robot motion control program, A function that displays multiple parameters related to human movement on a display device, A function that accepts the selection of an imitation parameter from among the displayed multiple parameters, which is the parameter to be imitated by the robot for each of the multiple parameters, A function to generate the motion control program by referring to the human actions corresponding to the received imitation parameters, A computer program that causes the processor to execute a function to display human motion images representing the movements of a human and robot motion images representing the movements of the robot simulated based on the motion control program on the display device.

7. A generation system for generating robot motion control programs, A display control unit that displays multiple parameters related to human movement on a display device, A receiving unit that accepts the selection of an imitation parameter, which is a parameter to be imitated by the robot for each of the multiple parameters, from among the multiple parameters that have been displayed, The system includes a program generation unit that generates the motion control program by referring to the human actions corresponding to the received imitation parameters, The display control unit further generates a display device that displays human motion images representing the movements of the human and robot motion images representing the movements of the robot simulated based on the motion control program.

Images