Generation methods, computer programs, and generation systems

Abstract

This invention discloses a generation method, computer program, and generation system capable of generating motion control programs that achieve desired robot movements. The method for generating a robot motion control program includes: a step of displaying multiple parameters related to human movements; a step of selecting, from the displayed multiple parameters, a parameter that the robot will imitate, i.e., an imitation parameter; and a step of generating the motion control program by referring to the human movement corresponding to the selected imitation parameter.

Description

Technical Field

[0001] This disclosure relates to techniques for generating motion control programs for robots. Background Technology

[0002] Previously, there were known technologies for generating motion control programs that enable robots to mimic actual human tasks and control the robot's movements (Patent Document 1).

[0003] Patent Document 1: Japanese Patent Application Publication No. 2019-188477

[0004] In previous technologies, when generating motion control programs by making robots uniformly imitate all human actions, it is possible that motion control programs cannot be generated to achieve the desired robot movements. For example, when making robots uniformly imitate all human movements based on the location where they are used, the content of the work, and the shape of the workpiece being worked on, it is possible that the robot's movements are not suitable for the work scenario involving the workpiece. Summary of the Invention

[0005] According to a first aspect of this disclosure, a method for generating a motion control program for a robot is provided. The method includes: displaying a plurality of parameters related to human actions; selecting from the displayed plurality of parameters a parameter, i.e., an imitation parameter, that the robot will imitate; and generating the motion control program by referring to the human action corresponding to the selected imitation parameter.

[0006] According to a second aspect of this disclosure, a computer program is provided that causes a processor to execute the generation of a motion control program for a robot. This computer program causes the processor to perform: a function to display a display device showing multiple parameters related to human actions; a function to accept selection from the displayed multiple parameters of the parameters to be imitated by the robot, i.e., an imitation parameter; and a function to generate the motion control program by referring to the human actions corresponding to the accepted imitation parameter.

[0007] According to a third aspect of this disclosure, a system for generating motion control programs for robots is provided. The system comprises: a display control unit that displays multiple parameters related to human actions on a display device; a receiving unit that receives a selection of parameters, i.e., imitation parameters, from the displayed multiple parameters; and a program generation unit that generates the motion control program by referring to the human action corresponding to the received imitation parameters. Attached Figure Description

[0008] Figure 1 This is an overall structural diagram of the generation system in this embodiment.

[0009] Figure 2 It is a diagram representing the specific structure of the generating system.

[0010] Figure 3 It is a diagram used to illustrate the initial information of the parameters.

[0011] Figure 4 It is a diagram used to illustrate human movement information.

[0012] Figure 5 This is a flowchart representing the teaching process of a robot.

[0013] Figure 6 This is a flowchart illustrating the generation and processing of the robot's motion control program.

[0014] Figure 7 This is a diagram used to illustrate the generation and processing of motion control programs.

[0015] Figure 8 It is a diagram representing a dynamic image confirming an action.

[0016] Figure 9 It is the screen displayed on the display device by executing step S80.

[0017] Figure 10 It means Figure 6 The first flowchart showing the details of step S30 is shown.

[0018] Figure 11 It means Figure 6 The second flowchart shows the details of step S30.

[0019] Figure 12 It means Figure 6 The third flowchart shows the details of step S30.

[0020] Figure 13 This is the first example of a motion confirmation image and a received image.

[0021] Figure 14 This is the second example of a diagram representing an action confirmation animation and a received image.

[0022] Figure 15 This is the third example of a diagram representing an action confirmation animation and a receiving image.

[0023] Figure 16 This is the fourth example of a motion confirmation image and a processing image.

[0024] Figure 17 This is the fifth example of a motion confirmation image and a processing image.

[0025] Figure 18This is the sixth example of a motion confirmation image and a processing image.

[0026] Explanation of reference numerals in the attached figures

[0027] 10…Display device; 20…Robot control device; 22…Second processor; 24…Second storage device; 25…Motion control program; 30…Sensor type; 50…System control device; 52…First processor; 53…Display control unit; 54…Receiver unit; 55…Determination unit; 56…Program generation unit; 57…Imitation teaching decision unit; 58…Robot motion conversion unit; 59…Dynamic image generation unit; 61…Finger determination unit; 62…Object determination unit; 63…Force determination unit; 64…Motion determination unit; 71…First storage device; 72…Initial parameter information; 74…Human motion information; 75…Pressure sensor; 80…Human; 81…First imaging device; 82…Second imaging device; 100 …robot; 110…arm; 120…arm flange; 130…force detector; 140…end effector; 160…position sensor; 200…robot control unit; 1000…generation system; D1a…first point image; D1b…second point image; IM…acceptance image; IM0…input description image; IM1…parameter description image; IM2…parameter selection image; IM3…confirmation image; IM4…decision image; IWK1…first workpiece image; IWK2…second workpiece image; J1~J6…joints; PI…action confirmation dynamic image; PIA1…human dynamic image; PIA2…robot dynamic image; PIB1…human auxiliary image; PIB2…robot auxiliary image. Detailed Implementation

[0028] A. Implementation method:

[0029] Figure 1 This is an overall structural diagram of the generation system 1000 according to this embodiment. The generation system 1000 is a system that generates a motion control program for the robot 100 based on the actions of a human 80 teaching the robot 100 to move. In this embodiment, an example of generating a motion control program for the robot 100 based on the actions of a human 80 moving a workpiece WK on a workbench 90 from a first position to a second position will be described. The generation system 1000 includes a display device 10, a robot control device 20, a system control device 50, sensors 30, and a robot 100.

[0030] The robot 100 in this embodiment is a single-arm robot, and various end effectors 140 are installed on the arm flange 120 at the front end of the arm 110 for use.

[0031] Arm 110 has six joints J1 to J6. Joints J2, J3, and J5 are bending joints, while joints J1, J4, and J6 are torsional joints. Each joint is equipped with a servo motor and a position sensor. The servo motor generates a rotational output to drive each joint. The position sensor 160 detects the angular position of the servo motor's output shaft. Additionally, for ease of understanding, in... Figure 1 The servo motor and position sensor are not shown in the diagram.

[0032] Various end effectors 140 for holding, processing, and other operations on objects are mounted on the arm flange 120 at the front end of joint J6. In this specification, the objects processed by the robot 100 are also referred to as "workpieces".

[0033] The position near the tip of arm 110 can be set as the tool center point. Hereinafter, the tool center point will be referred to as "TCP". TCP is the position used as the position reference for end effector 140. For example, a predetermined position on the rotation axis of joint J6 can be set as TCP.

[0034] The robot 100 can configure the end effector 140 in any position and any posture within the range of motion of the arm 110. A force detector 130 and the end effector 140 are provided on the arm flange 120. In this embodiment, the end effector 140 is a gripper. The end effector 140 is capable of holding the workpiece WK.

[0035] Force detector 130 is disposed on robot 100 and is capable of measuring the external force applied to robot 100. Specifically, force detector 130 is a six-axis sensor. Force detector 130 is capable of detecting the magnitude of the force and the magnitude of the torque about these three axes that are parallel to the x-axis, y-axis and z-axis, which are orthogonal to each other in the inherent coordinate system, i.e., the sensor coordinate system.

[0036] The coordinate system used to define the space where robot 100 is located is called the "robot coordinate system". The robot coordinate system is a three-dimensional orthogonal coordinate system defined by mutually orthogonal x-axis and y-axis on a horizontal plane, and a z-axis with the vertically upward direction as positive. Figure 1 The coordinate system shown is the robot coordinate system. Arbitrary positions in three-dimensional space are represented by the x, y, and z axes, and arbitrary postures in three-dimensional space are represented by the rotation angles around the x, y, and z axes. In this specification, the term "position" can also refer to both position and posture. Furthermore, the term "force" can also refer to both force and torque.

[0037] The display device 10 displays various information such as text and images. The display device 10 is, for example, a liquid crystal monitor. In this embodiment, the display device 10 is a monitor for a personal computer.

[0038] Sensor 30 includes a first imaging device 81, a second imaging device 82, and a pressure sensor 75. Sensor 30 and system control device 50 are connected via wired or wireless means, and sensor 30 can transmit data to system control device 50. The first imaging device 81 includes an image sensor that captures a person 80's movements as a time-lapse image while working on workpiece WK, and a depth sensor that detects the distance to the person 80, workpiece WK, and other objects, as well as the three-dimensional shape of the objects. The second imaging device 82 includes an image sensor that captures a robot 100's movements as a time-lapse image while working on workpiece WK, and a depth sensor that detects the distance to the robot 100, workpiece WK, and other objects, as well as the three-dimensional shape of the objects. Data acquired and detected by the first and second imaging devices 81 and 82, respectively, are transmitted to system control device 50. The pressure sensor 75 is, for example, installed on the hand of the person 80 to detect the force with which the person 80 grips workpiece WK.

[0039] The robot control unit 20 sends control commands to the robot 100, controlling the arm 110 and the end effector 140. The system control unit 50 generates a motion control program for the robot 100 based on data from the sensor class 30 and received reference parameters (described later). The generated motion control program is sent to the robot control unit 20 and stored in its memory. Furthermore, the robot control unit 200 and the system control unit 50 can be either a single device or separate devices.

[0040] Figure 2 This diagram illustrates the specific structure of the generation system 1000. The system control unit 50 is a computer equipped with a first processor 52 and a first storage device 71. The system control unit 50 functions as a display control unit 53, a receiving unit 54, a determining unit 55, a program generation unit 56, and a motion image generation unit 59 by executing various programs stored in the first storage device 71. The display control unit 53 sends various information to the display device 10 to control the display operation of the display device 10. The receiving unit 54 receives input from the outside via input interfaces such as a keyboard and mouse.

[0041] The determination unit 55 determines the action of the person 80 based on data such as detection results from the sensor type 30. The determination unit 55 includes a finger determination unit 61, an object determination unit 62, a force determination unit 63, and an action determination unit 64.

[0042] The finger determination unit 61 uses the captured image and depth information obtained from the first imaging device 81 to determine the position and posture of the person 80's fingers at preset time intervals. Regarding the finger determination unit 61, for example, a mark is installed on the finger of the person 80 that needs to be determined, and the position and posture of the finger are determined by detecting the mark in the captured image. Alternatively, in other embodiments, the finger determination unit 61 may use other methods such as pattern matching to determine the position and posture of the fingers.

[0043] The object determination unit 62 uses the captured image and depth information acquired from the first imaging device 81 to determine the position and orientation of the workpiece WK at preset time intervals. For example, the object determination unit 62 performs image processing such as edge extraction on the captured image and determines the workpiece WK based on the extracted edges. Furthermore, the object determination unit 62 determines the position and orientation of the workpiece WK determined using the captured image and depth information.

[0044] The force determination unit 63 receives detection signals from the pressure sensor 75 at preset time intervals, thereby determining the magnitude of the force exerted by the person 80 on the workpiece WK.

[0045] The motion determination unit 64 arranges (i) the position of the finger of the person 80 determined by the finger determination unit 61, (ii) the position of the workpiece WK determined by the object determination unit 62, and (iii) the magnitude of the force in a time sequence according to a preset time interval, thereby determining the motion of the person 80 using a predetermined coordinate system and physical quantities. In this embodiment, the predetermined coordinate system refers, for example, to the camera coordinate system of the first imaging device 81. The determined motion of the person 80 is stored in the first storage device 71 as human motion information 74. The human motion information 74 includes parameters for causing the robot 100 to perform actions and the content of each parameter. Details regarding the human motion information 74 will be explained later.

[0046] The first storage device 71 is composed of ROM, RAM, etc., and in addition to the various programs executed by the first processor 52, it also stores parameter initial information 72 and human action information 74. The parameter initial information 72 contains the parameters and their contents required for the robot 100 to perform actions, and is used when the robot 100 performs actions without referring to the parameters determined by the determination unit 55. Details about the parameter initial information 72 will be explained later.

[0047] The program generation unit 56 generates a motion control program for the robot 100 based on the initial parameter information 72 and the actions of the person 80 determined by the determination unit 55. The program generation unit 56 includes an imitation teaching decision unit 57 and a robot motion conversion unit 58. The imitation teaching decision unit 57 determines whether to use the content of the parameters determined by the determination unit 55 or the content of the parameters in the initial parameter information 72 for each parameter used when generating the motion control program based on the imitation parameters received by the receiving unit 54. The robot motion conversion unit 58 uses the content specified by the parameters determined by the imitation teaching decision unit 57 and generates the motion control program using a conversion program such as a conversion matrix formula pre-set according to inverse kinematics.

[0048] The motion image generation unit 59 generates a motion confirmation motion image PI that represents the actions of the person 80 determined by the determination unit 55 and the actions of the robot 100 when executing the motion control program generated by the program generation unit 56. The motion confirmation motion image PI is displayed on the display device 10 under the control of the display control unit 53. The motion confirmation motion image PI includes a person motion image PIA1 obtained by capturing the actions of the person 80 through the first capturing device 81 and a robot motion image PIA2 that represents the actions of the robot 100 simulated based on the animation control program generated by the program generation unit 56. The robot motion image PIA2, as a simulated animation, is generated by the motion image generation unit 59. The person motion image PIA1 uses motion images captured by the first capturing device 81 when capturing the actions of the person 80 for imitation teaching. Alternatively, in other embodiments, the robot motion image PIA2 may also include motion images captured by the second capturing device 82 that cause the robot 100 to execute the animation control program generated by the program generation unit 56.

[0049] Figure 3 This is a diagram illustrating the initial parameter information 72. The initial parameter information 72 specifies multiple parameters used to cause the robot 100 to perform actions and the pre-set actions of the robot 100 for each parameter. The parameters are: "trajectory," "workpiece position," "workpiece gripping position," "force applied to the workpiece," "trajectory speed," and "preparatory action." "Trajectory" refers to the parameter related to the trajectory of the workpiece WK. "Workpiece position" refers to the parameter related to the position of the workpiece WK before and after its movement. "Force applied to the workpiece" refers to the parameter related to the magnitude of the force applied to the workpiece WK. "Trajectory speed" refers to the parameter related to the movement speed of the workpiece WK. "Preparatory action" refers to the parameter related to the actions of the robot 100 performed before causing the workpiece WK to move.

[0050] When the program generation unit 56 generates a motion control program for the robot using the initial parameter information 72, the robot 100 performs the following actions. Regarding the movement trajectory, the robot 100 moves in a manner that minimizes the distance from the starting point before movement to the ending point after movement. Alternatively, in other embodiments, the robot 100 may move in a manner that minimizes the movement of each joint of the robot 100, or a predetermined trajectory otherwise. Regarding the position of the workpiece WK before and after movement, the robot 100 moves in a manner that corresponds to the position of the movement trajectory specified in the parameters. Alternatively, the position of the workpiece WK before movement may be preset. Regarding the gripping position of the workpiece WK, the robot 100 moves in a manner that grips the workpiece WK at a predetermined position. The predetermined position, for example, refers to the coordinate position of the center of the gripping surface of the workpiece WK. The robot 100 moves in a manner that the magnitude of the force applied to the workpiece WK by the robot 100 gripping it reaches a preset initial value. The robot 100 moves in a manner that maximizes the moving speed of the workpiece WK, achieving the fastest possible moving speed within the robot 100's TCP (Coordinate Control Point). No preparatory actions for the robot 100 are set in the initial parameter information 72. The parameters in the initial parameter information 72 are determined using various coordinate systems, velocities, and other physical quantities. For example, the moving speed of the workpiece WK is determined using physical quantities. Relationships between various coordinate systems, such as the robot coordinate system and the sensor coordinate system, and formulas for converting physical quantities into the robot 100's control program are predefined. Therefore, regardless of which coordinate system or physical quantity is used, the content of each parameter can be converted into the robot 100's motion control program.

[0051] Figure 4 This is a diagram used to illustrate human motion information 74. Human motion information 74 specifies multiple parameters used to cause robot 100 to perform actions and the content of the actions of human 80 determined by determination unit 55. The parameters of human motion information 74 are related to the points in the action performed by human 80 during action teaching. Figure 3The parameters in the initial parameter information 72 shown are different. That is, each parameter in the initial parameter information 72 is a parameter related to the action of the person 80. In the initial parameter information 72, the "movement trajectory" parameter represents the movement trajectory of the workpiece WK. Furthermore, in the initial parameter information 72, the "workpiece position" parameter represents the position of the workpiece WK before and after movement when the person 80 moves it. Furthermore, in the initial parameter information 72, the "workpiece gripping position" parameter represents the gripping position of the workpiece WK by the person 80. Furthermore, in the initial parameter information 72, the "force applied to the workpiece" parameter represents the magnitude of the force applied to the workpiece WK by the person 80 gripping it. Furthermore, in the initial parameter information 72, the "moving speed" parameter represents the moving speed of the workpiece WK when the person 80 moves it. Furthermore, in the initial parameter information 72, the "preparatory action" parameter represents the action performed before the person 80 moves the workpiece WK. The content of each parameter is determined by the determination unit 55 using physical quantities such as camera coordinate system and velocity.

[0052] like Figure 2 As shown, the robot control device 20 is a computer equipped with a second processor 22 and a second storage device 24. The second processor 22 controls the movement of the robot 100 by executing the motion control program 25 stored in the second storage device 24. The second storage device 24 is composed of RAM and ROM. The motion control program 25 generated by the program generation unit 56 of the system control device 50 is stored in the second storage device 24.

[0053] Figure 5 This is a flowchart illustrating the teaching process of robot 100. First, in step S10, a person 80, equipped with a pressure sensor 75, performs a teaching action on robot 100 in front of the first imaging device 81, and the system control device 50 acquires detection information from the sensor class 30. Specifically, in step S10, the system control device 50 acquires captured images and depth information from the first imaging device 81 and acquires detection signals from the pressure sensor 75 at preset time intervals.

[0054] Then, in step S12, the determination unit 55 of the system control device 50 uses the acquired captured images, depth information and detection signals to determine the position and shape of the finger of the person 80, the position of the workpiece WK and the magnitude of the force applied to the workpiece WK by the person 80 at a preset time interval.

[0055] Then, in step S14, the determination unit 55 determines the action of the person 80, including a preparatory action, using the position and shape of the person 80's fingers, the position of the workpiece WK, and the magnitude of the force determined in step S12. The determined action of the person 80 is stored in the first storage device 71 as person action information 74. In addition, the magnitude of the force applied by the person 80 is defined as the parameter of the person action information 74 as "force applied to the workpiece". As a preparatory action, for example, the content is defined by the person 80 performing a pre-set gesture before grasping the workpiece WK. For example, if the person 80 performs a gesture of approaching the workpiece WK with their hands closed and opening their hands when they are approaching the workpiece WK at a predetermined distance, the content of the above-mentioned gesture action is defined as the "preparatory action" in the person action information 74. When generating a motion control program by referring to the preparatory action of the human motion information 74, the robot 100 generates a motion control program in a manner that mimics the preparatory action specified by the human motion information 74, that is, in a manner that the gripper, which is the end effector 140, maintains a closed state before approaching the workpiece WK and opens when it approaches the workpiece WK to a predetermined distance.

[0056] Figure 6 This is a flowchart showing the generation and processing of the motion control program for robot 100. Figure 7 This is a diagram used to illustrate the generation and processing of motion control programs. Figure 7 This refers to the screen displayed on the display device 10. Figure 6 The generation process shown can be performed in both Figure 5 The teaching process shown will start automatically after it is completed, or it can start by accepting a start command from the user.

[0057] like Figure 6 As shown, in step S20, the display control unit 53 of the system control device 50 displays a received image IM that receives instructions from the user in order to generate an action control program. Figure 7 As shown, the image IM is the one that is accepted in Figure 5 The image is the selection image of the action that the robot 100 will imitate from the action of the human 80 determined in step S14. The acceptance image IM has an input description image IM0, a parameter description image IM1, a parameter selection image IM2, a confirmation image IM3, and a decision image IM4. The input description image IM0 is an image that prompts the user to input through the acceptance image IM. The parameter description image IM1 is an image representing multiple parameters and is a text image explaining each parameter. Each parameter in the parameter description image IM1 corresponds to each parameter in the parameter initial information 72 and the human action information 74. Each parameter in the parameter description image IM1 is a parameter related to the action of the human 80 and is a candidate parameter for the robot 100 to refer to and imitate.

[0058] The parameter selection image IM2 is an image that determines whether to select each parameter from among multiple parameters as the parameter that the robot 100 will imitate, i.e., the imitation parameter. The parameter selection image IM2 is an image indicating "yes" or "no" next to each parameter in the parameter description image IM1. The user uses an input device such as a mouse to select "yes" for parameters that cause the robot 100 to imitate the actions of the human 80, and "no" for parameters that do not cause the robot 100 to imitate the actions of the human 80. Parameters selected as "yes" are chosen as imitation parameters.

[0059] The confirmation image IM3 is the image used by the receiving unit 54 to receive information selected by the parameter selection image IM2. The user selects the confirmation image IM3 using an input interface such as a mouse, thereby the selection information selected by the parameter selection image IM2 is received by the receiving unit 54. The received selection information is stored in the first storage device 71. That is, by confirming that the image IM3 is selected as in step S22, the receiving unit 54 receives the selection of the parameter that the robot 100 will imitate, i.e., the imitation parameter, from the multiple parameters displayed in the parameter description image IM1. Furthermore, by confirming that the image IM3 is selected, the motion image generation unit 59 generates the aforementioned motion confirmation motion image PI. The decision image IM4 is used to forward the motion control program generated by the program generation unit 56 to the robot control device 20. When the decision image IM4 is selected, the motion control program generated by the program generation unit 56 is forwarded to the robot control device 20.

[0060] Then, in Figure 6 In step S30, the program generation unit 56 generates a motion control program by referring to the actions of the person 80 corresponding to the imitation parameters and the parameter initial information 72. Specifically, the program generation unit 56 determines the content of the actions of the person 80 corresponding to the imitation parameters based on the person action information 74, and determines the content of the actions of the parameters other than the imitation parameters among the multiple parameters based on the parameter initial information 72. Furthermore, the program generation unit 56 generates a motion control program for the determined action content using a transformation program such as a transformation matrix formula pre-set according to inverse kinematics. As described above, in the process of generating the motion control program in step S30, the motion control program is generated for the parameters other than the imitation parameters among the multiple parameters using the parameter initial information 72.

[0061] Then, in step S70, the motion image generation unit 59 generates an action confirmation motion image PI. Figure 8 This is a graph representing the motion confirmation dynamic image (PI). The motion confirmation dynamic image (PI) includes the aforementioned human dynamic image (PIA1), robot dynamic image (PIA2), human auxiliary image (PIB1), and robot auxiliary image (PIB2). Regarding... Figure 8The human motion image PIA1 and robot motion image PIA2 shown are illustrated with dashed arrows YA1 and YA2 to represent the movement trajectory of workpiece WK, and the length of the dashed line represents the movement speed of workpiece WK. The longer the dashed line, the slower the movement speed. The human auxiliary image PIB1 is an image showing the gripping position of workpiece WK and the magnitude of the gripping force on workpiece WK in the action determined by the determination unit 55 during action teaching. In addition, the human motion image PIA1 and robot motion image PIA2 also include an image of the worktable 90.

[0062] The human-aided image PIB1 has a first workpiece image IWK1 that mimics the workpiece WK and a first point image D1a that coincides with the first workpiece image IWK1. The position and size of the first point image D1a in the first workpiece image IWK1 indicate the gripping position and the magnitude of the gripping force exerted by the human 80 when teaching the action. The larger the size of the first point image D1a, the greater the gripping force.

[0063] The robot-assisted image PIB2 has a second workpiece image IWK2 that mimics the workpiece WK and a second point image D1b that coincides with the second workpiece image IWK2. The position of the second point image D1b in the second workpiece image IWK2 indicates the gripping position of the robot 100 on the workpiece WK. The size of the second point image D1b indicates the magnitude of the force by which the robot 100 grips the workpiece WK. The larger the size of the second point image D1b, the greater the gripping force.

[0064] like Figure 6 As shown, in step S80 after step S70, the display control unit 53 causes the display device 10 to display the generated motion confirmation dynamic image PI. Figure 9 This indicates the screen displayed on the display device 10 after executing step S80. The display control unit 53 causes the display device 10 to display the action confirmation animation P on the same screen as the received image IM. Figure 9 The example shown represents the case where all parameters are selected as the imitation parameters for the actions of the reference person 80.

[0065] exist Figure 6 Following step S70, in step S80, the program generation unit 56 determines in step S90 whether to forward the motion control program generated in step S30 to the robot control device 20. That is, in step S90, it determines whether to determine the motion control program generated in step S30 as the final program for controlling the actions of the robot 100. Figure 9 If the decision image IM4 shown is selected, a "yes" judgment is made, and the motion control program is forwarded to the robot control device 20.

[0066] On the other hand, if the user wants to refer to the motion control program generated by modifying the motion image PI for confirmation, they can again select either "Yes" or "No" from the parameter selection image IM2, and then select the confirmation image IM3. Thus, the steps from step S22 onwards are executed again. Alternatively, if the user wants to refer to the motion control program generated by modifying the motion image PI for confirmation, the user 80 can again perform the teaching action and execute it. Figure 5 The teaching process is shown.

[0067] Figure 10 It means Figure 6 The first flowchart shows the details of step S30. In step S32, the imitation teaching decision unit 57 determines whether "movement trajectory" has been selected as the imitation parameter. If the robot motion conversion unit 58 determines "yes" in step S32, in step S36, it determines the timed movement trajectory of the workpiece WK, referring to the parameter "movement trajectory" in the human motion information 74, when generating a motion control program. Furthermore, in step S38, the robot motion conversion unit 58 generates a program related to the movement trajectory of the workpiece WK in the motion control program of the robot 100. Specifically, in step S38 after step S36, the robot motion conversion unit 58 generates a motion control program in such a way that the movement trajectory of the workpiece WK based on the action of the robot 100 becomes the movement trajectory specified by the human motion information 74.

[0068] On the other hand, if the robot motion conversion unit 58 determines "no" in step S32, in step S34, it decides to refer to the "movement trajectory" parameter in the parameter initial information 72 when generating a motion control program. Furthermore, in step S38 after step S34, the robot motion conversion unit 58 generates a motion control program in such a way that the movement trajectory of the workpiece WK based on the robot 100's motion becomes the movement trajectory specified in the parameter initial information 72.

[0069] In step S42, the imitation teaching decision unit 57 determines whether "workpiece position" has been selected as the imitation parameter. If the robot motion conversion unit 58 determines "yes" in step S42, in step S46, it determines the position of the workpiece WK before and after movement, referring to the content of the human motion information 74 where the parameter is "workpiece position," when generating a motion control program. Furthermore, in step S48, the robot motion conversion unit 58 generates a program related to the position of the workpiece WK before and after movement in the motion control program of the robot 100. Specifically, in step S48 after step S46, the robot motion conversion unit 58 generates a motion control program in a manner that makes the position of the workpiece WK before and after movement based on the movement of the robot 100 the position before and after movement specified in the human motion information 74.

[0070] On the other hand, if the robot motion conversion unit 58 determines "no" in step S42, in step S44, it decides to refer to the parameter "workpiece position" in the parameter initial information 72 when generating the motion control program. Furthermore, in step S48 after step S44, the robot motion conversion unit 58 generates the motion control program in a manner that makes the position of the workpiece WK before and after movement based on the robot 100's motion the position specified by the parameter initial information.

[0071] Figure 11 It means Figure 6 The second flowchart shows the details of step S30. In step S52, the imitation teaching decision unit 57 determines whether "workpiece gripping position" has been selected as the imitation parameter. If the determination in step S52 is "yes", the robot motion conversion unit 58, in step S56, determines that the parameter "grip position" in the human motion information 74 is referenced when generating a motion control program. Furthermore, in step S58, the robot motion conversion unit 58 generates a program in the motion control program of the robot 100 regarding the gripping position of the robot 100 gripping the workpiece WK. Specifically, in step S58 after step S56, the robot motion conversion unit 58 generates a motion control program in a manner that makes the gripping position of the workpiece WK based on the robot 100's motion the gripping position specified by the human motion information 74.

[0072] On the other hand, if the robot motion conversion unit 58 determines "no" in step S52, in step S54, it decides to refer to the parameter "workpiece gripping position" in the parameter initial information 72 when generating a motion control program. Furthermore, in step S58 after step S54, the robot motion conversion unit 58 generates a motion control program such that the gripping position of the workpiece WK based on the robot 100's motion, i.e., the gripping position of the robot 100 holding the workpiece WK, becomes the gripping position specified in the parameter initial information 72.

[0073] In step S62, the imitation teaching decision unit 57 determines whether "force applied to the workpiece" has been selected as the imitation parameter. If the robot motion conversion unit 58 determines "yes" in step S62, in step S66, it decides to refer to the parameter "force applied to the workpiece" in the human motion information 74 when generating a motion control program. Furthermore, in step S68, the robot motion conversion unit 58 generates a program in the motion control program of the robot 100 concerning the force applied to the workpiece WK. Specifically, in step S68 after step S66, the robot motion conversion unit 58 generates a motion program in such a way that the magnitude of the force used to hold the workpiece WK based on the robot 100's motion is the magnitude of the force specified in the human motion information 74.

[0074] On the other hand, if the robot motion conversion unit 58 determines "no" in step S62, in step S64, it decides to refer to the parameter "workpiece gripping position" in the parameter initial information 72 when generating the motion control program. Furthermore, in step S68 after step S64, the robot motion conversion unit 58 generates the motion control program in such a way that the magnitude of the force used to grip the workpiece WK based on the robot 100's motion is the magnitude of the force specified in the parameter initial information 72.

[0075] Figure 12 It means Figure 6 The third flowchart shows the details of step S30. In step S72, the imitation teaching decision unit 57 determines whether "movement speed" has been selected as the imitation parameter. If the robot motion conversion unit 58 determines "yes" in step S72, in step S76, it decides to refer to the parameter "movement speed" in the human motion information 74 when generating a motion control program. Furthermore, in step S78, the robot motion conversion unit 58 generates a program for the movement speed of the workpiece WK in the motion control program of the robot 100. Specifically, in step S78 after step S76, the robot motion conversion unit 58 generates a motion control program in a manner that makes the movement speed of the workpiece WK based on the movement of the robot 100 the movement speed specified by the human motion information 74.

[0076] On the other hand, if the robot motion conversion unit 58 determines "no" in step S72, in step S74, it decides to refer to the parameter "movement speed" in the parameter initial information 72 when generating the motion control program. Furthermore, in step S78 after step S74, the robot motion conversion unit 58 generates the motion control program in such a way that the movement speed of the workpiece WK based on the robot 100's motion is the movement speed specified in the parameter initial information 72.

[0077] In step S82, the imitation teaching decision unit 57 determines whether a "preparatory action" has been selected as the imitation parameter. If the robot motion conversion unit 58 determines "yes" in step S82, in step S86, it decides to refer to the parameter in the human motion information 74 as the content of the "preparatory action" when generating a motion control program. Furthermore, in step S88, the robot motion conversion unit 58 generates the program related to the preparatory action in the motion control program of the robot 100. Specifically, in step S88 after step S86, the robot motion conversion unit 58 generates a motion control program in a manner that makes the preparatory action in the robot 100's motion conform to the content specified by the human motion information 74.

[0078] On the other hand, if the robot motion conversion unit 58 determines "no" in step S82, in step S84, it decides to refer to the parameter initial information 72 as the content of the "preparatory action" when generating the motion control program. Furthermore, in step S88 after step S84, the robot motion conversion unit 58 generates the motion control program in a manner that makes the preparatory action in the robot 100's motion the content specified by the parameter initial information 72. For example... Figure 3 As shown, in the parameter initial information 72, it is specified that no "preparatory action" will be performed. Therefore, in this embodiment, the clamp, which is the end effector 140, moves to approach the workpiece WK in an open state.

[0079] Figure 13 This is the first example of the action confirmation dynamic image PI and the acceptance image IM. It shows that the image with a single shadow line is selected in the parameter selection image IM2. This is also true for the subsequent figures. Figure 13 In this process, only the "workpiece position" among multiple parameters is selected as the simulation parameter for the actions of the reference human 80. In this case, in the robot motion image PIA2 displayed on the display device 10, the position of the workpiece WK before and after movement becomes the position specified by the human action information 74. That is, the position of the workpiece WK before and after movement on the worktable 90 in the human motion image PIA1 is the same as the position of the workpiece WK before and after movement on the worktable 90 in the robot motion image PIA2.

[0080] Figure 14 This is a second example of a diagram representing the motion confirmation image (PI) and the reception image (IM). Figure 14 In this case, only the "movement trajectory" among multiple parameters is selected as the parameter to mimic the action of the reference human 80. In this situation, in the robot motion image PIA2 displayed on the display device 10, only the movement trajectory of the workpiece WK becomes the movement trajectory specified by the human action information 74. That is, the movement trajectory of the workpiece WK in the human motion image PIA1 is the same as the movement trajectory of the workpiece WK in the robot motion image PIA2.

[0081] Figure 15 This is a diagram representing the third example of an action confirmation dynamic image (PI) and a reception image (IM). Figure 15 In this process, only the "workpiece gripping position" is selected as the parameter to mimic the action of the reference human 80 among multiple parameters. In this case, in the robot motion image PIA2 displayed on the display device 10, only the gripping position of the workpiece WK becomes the gripping position specified according to the human action information 74. That is, the position of the first point image D1a relative to the first workpiece image IWK1 is the same as the position of the second point image D1b relative to the second workpiece image IWK2.

[0082] Figure 16 This is the fourth example of a diagram representing the motion confirmation image (PI) and the reception image (IM). Figure 16 In this process, only the "force applied to the workpiece" is selected as the parameter to mimic the action of the reference human 80 among multiple parameters. In this case, in the robot motion image PIA2 displayed on the display device 10, only the magnitude of the force applied to the workpiece WK becomes the magnitude of the force specified according to the human action information 74. That is, the size of the first point image D1a is the same as the size of the second point image D1b.

[0083] Figure 17 This is the fifth example of a diagram representing the motion confirmation image (PI) and the reception image (IM). Figure 17 In this case, only "movement speed" is selected as the parameter to mimic the actions of the reference human 80 among multiple parameters. In the robot motion image PIA2 displayed on the display device 10, the movement speed of the workpiece WK based solely on the actions of the robot 100 becomes the movement speed of the workpiece WK specified according to the human action information 74. That is, the movement speed of the workpiece WK in the human motion image PIA1 is the same as the movement speed of the workpiece WK in the robot motion image PIA2.

[0084] Figure 18 This is the sixth example of a diagram representing the motion confirmation image (PI) and the reception image (IM). Figure 18In this case, only the "preparatory action" is selected as the parameter to mimic the action of the reference human 80 among multiple parameters. In the robot motion image PIA2 displayed on the display device 10, only the preparatory action in the robot 100's action becomes the preparatory action specified according to the human action information 74. Figure 18 In the robot motion image PIA2 shown, the gripper 140, which is the end effector, remains closed before approaching the workpiece WK and opens when it approaches the workpiece WK to a predetermined distance.

[0085] The user confirms the above. Figures 13-18 Taking the motion confirmation dynamic image PI as an example, the system determines the parameters related to the motion of the human 80 that the robot 100 imitates, and the system control device 50 generates the motion control program.

[0086] According to the above embodiment, the imitation parameters that the robot 100 can mimic can be selected from multiple parameters related to the actions of the human 80, thus enabling the generation of a motion control program that accurately achieves the desired robot actions. However, when the motion control program is generated in such a way that the movement trajectory of the workpiece WK based on the actions of the human 80 becomes the movement trajectory of the workpiece WK based on the actions of the robot 100, it is possible that movements that are not easily performed by the robot 100 may occur. As a result, there is a potential risk that the desired movement speed of the workpiece WK cannot be achieved. Furthermore, this is not limited to situations where the actions of the human 80 and the robot 100 are exactly the same. There are also cases where it is not necessary to move the workpiece WK to the same position as the position after the workpiece WK has moved based on the actions of the human 80 through the actions of the robot 100. Therefore, as... Figure 7 As shown, by being able to select from multiple parameters related to the actions of the human 80, the system control device 50 can generate a motion control program that accurately achieves the desired actions of the robot 100.

[0087] Furthermore, according to the above-described embodiments, such as Figure 4 As shown, the parameters are set to those that are most likely to be imitated by the robot 100. That is, the parameters are those corresponding to the basic actions performed by the robot 100 when the workpiece WK is moved. Furthermore, according to the above embodiment, as... Figures 10-12 As shown, the program generation unit 56 can generate an action control program using the parameter initial information 72 for parameters other than the simulation parameters.

[0088] B. Other implementation methods:

[0089] B-1. Other implementation method one:

[0090] In the above embodiments, such as Figure 4 As shown, there are six parameters, but any two or more are acceptable, and other parameters can also be included.

[0091] C. Other methods:

[0092] This disclosure is not limited to the embodiments described above, and can be implemented in various ways without departing from its spirit. For example, this disclosure can also be implemented in the following aspects. The technical features in the above embodiments corresponding to the technical features in the various aspects described below can be appropriately replaced and combined in order to solve part or all of the problems of this disclosure, or to achieve part or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential features in this specification, they can be appropriately deleted.

[0093] (1) According to a first aspect of this disclosure, a method for generating a motion control program for a robot is provided. The method includes: displaying a plurality of parameters related to human actions; accepting a selection from the displayed plurality of parameters of a parameter that the robot will imitate, i.e., an imitation parameter; and generating the motion control program by referring to the human action corresponding to the accepted imitation parameter. According to this method, an imitation parameter that the robot will imitate can be selected from a plurality of parameters related to human actions, thus enabling the generation of a motion control program that accurately achieves the desired robot actions.

[0094] (2) In the above-described manner, the plurality of parameters may also include at least two of the following: the position of the workpiece before and after movement by the robot, the movement trajectory of the workpiece, the gripping position of the person holding the workpiece, the movement speed of the workpiece, the force applied by the person to the workpiece, and the preparatory actions performed by the person before moving the workpiece. According to this method, the plurality of parameters can include at least two parameters corresponding to the basic actions performed by the robot when moving the workpiece.

[0095] (3) In the above method, the generated process may also involve generating the motion control program in such a way that the robot performs an action that mimics the preparatory action, provided that the imitation parameters include the preparatory action. According to this method, the robot can mimic a human preparatory action as an action performed before moving the workpiece.

[0096] (4) Alternatively, in the above method, the generation process may use pre-set parameter initial information, which determines the robot's action, to generate the motion control program for parameters other than the imitation parameters. According to this method, motion control programs can be generated using parameter initial information for parameters other than the imitation parameters.

[0097] (5) In the above method, the displayed process may also display an acceptance image, which includes a parameter description image representing the plurality of parameters and a parameter selection image for deciding whether to select each of the plurality of parameters as the simulated parameter. According to this method, the user can easily use the parameter selection screen to decide whether to select each of the plurality of parameters as the simulated parameter.

[0098] This disclosure can also be implemented in various ways other than those described above. For example, it can be implemented by using a computer program that generates a robot motion control program, a non-transitory storage medium that records the computer program, or a system that generates a robot motion control program.

Claims

1. A method of generating, characterized by, generating a motion control program of a robot, the generating method includes: a step of displaying a plurality of parameters related to a motion of a person; a step of receiving selection of a parameter to be imitated by the robot, i.e., an imitated parameter, from the displayed plurality of parameters; and a step of generating the motion control program with reference to the motion of the person corresponding to the received imitated parameter, in the step of displaying, the receiving image including an input instruction image that prompts a user to make an input, a parameter instruction image that indicates the plurality of parameters, a parameter selection image that determines whether to select each of the plurality of parameters as the imitated parameter, a confirmation image that receives information selected through the parameter selection image, and a determination image that forwards the generated motion control program to a robot control device.

2. The generating method according to claim 1, wherein the plurality of parameters include at least two or more of a position before and after movement of a workpiece handled by the robot, a movement trajectory of the workpiece, a holding position of the workpiece held by the person, a movement speed of the workpiece, a force applied to the workpiece by the person, and a preparatory motion performed by the person before moving the workpiece.

3. The generating method according to claim 2, wherein the step of generating generates the motion control program in such a manner that the robot performs a motion that imitates the preparatory motion when the imitated parameter includes the preparatory motion.

4. The generating method according to any one of claims 1 to 3, wherein the step of generating generates the motion control program using parameter initial information that is set in advance for parameters other than the imitated parameter among the plurality of parameters and that is parameter initial information that determines a motion of the robot.

5. A storage medium, characterized by a computer program that causes a processor to execute generation of a motion control program of a robot, the computer program causes the processor to execute: a function of causing a display device to display a plurality of parameters related to a motion of a person; a function of receiving selection of a parameter to be imitated by the robot, i.e., an imitated parameter, from the displayed plurality of parameters; and a function of generating the motion control program with reference to the motion of the person corresponding to the received imitated parameter, the function of displaying further causes the display device to display a receiving image including an input instruction image that prompts a user to make an input, a parameter instruction image that indicates the plurality of parameters, a parameter selection image that determines whether to select each of the plurality of parameters as the imitated parameter, a confirmation image that receives information selected through the parameter selection image, and a determination image that forwards the generated motion control program to a robot control device.

6. A generation system, characterized by, generating a motion control program of a robot, the generating system includes: a display control section that causes a display device to display a plurality of parameters related to a motion of a person; a receiving section that receives selection of a parameter to be imitated by the robot, i.e., an imitated parameter, from the displayed plurality of parameters; and a generating section that generates the motion control program with reference to the motion of the person corresponding to the received imitated parameter, the display control section causes the display device to display a receiving image including an input instruction image that prompts a user to make an input, a parameter instruction image that indicates the plurality of parameters, a parameter selection image that determines whether to select each of the plurality of parameters as the imitated parameter, a confirmation image that receives information selected through the parameter selection image, and a determination image that forwards the generated motion control program to a robot control device. a program generation section that generates the motion control program with reference to the person's motion corresponding to the received mimic parameter, The display control section also causes the display device to display a received image including an image that prompts the user to input, an input instruction image, an image that indicates the plurality of parameters, a parameter instruction image, an image that determines whether to select each of the plurality of parameters as the mimic parameter, a parameter selection image, an image that receives confirmation of information selected by the parameter selection image, and an image that determines to forward the generated motion control program to a robot control device.

Images