CONTROL DEVICE, ROBOT SYSTEM, CONTROL METHOD, AND PROGRAM

The control device predicts robot trajectories to minimize delays, improving operational feeling and efficiency by using adaptive control parameters based on operator and environmental data.

JP7577017B2Active Publication Date: 2024-11-01HONDA MOTOR CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2021060904
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-03-31
Publication Date
2024-11-01
Estimated Expiration
2041-03-31

AI Technical Summary

Technical Problem

Existing robot systems experience delays in following operator movements, leading to discomfort, decreased work speed, and reduced efficiency due to communication and control delays in motion information.

Method used

A control device and method that predicts the trajectory of a robot's effector based on operator movements and environmental information, adjusting control commands to reduce delays and improve operational feeling by using predicted trajectories and adaptive control parameters.

Benefits of technology

Reduces or eliminates follow-up delays, enhancing operational feeling and work efficiency while reducing operator burden by accurately adjusting the robot's movements based on the operating situation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007577017000001
    Figure 0007577017000001
  • Figure 0007577017000002
    Figure 0007577017000002
  • Figure 0007577017000003
    Figure 0007577017000003
Patent Text Reader

Abstract

To provide a control device, a robot system, a control method and a program which can improve an operational feeling.SOLUTION: In all of a control device, a robot system, a control method and a program according to the embodiment, a track predicting part determines a predicted track on which an effector of a robot runs until a predicted time at which a predicted time elapses after a current time, on the basis of at least motion information showing motion of the robot and operation information showing an operation situation, and a control command generating part generates a control command on the basis of the predicted track.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a control device, a robot system, a control method, and a program. [Background technology]

[0002] Conventionally, robot systems have been proposed that accept the actions of an operator and control a robot according to the accepted actions. For example, a robot system described in Patent Document 1 acquires first user posture information indicating the posture of a first user who operates the robot, and acquires pre-change posture information indicating a pre-change posture before changing the posture of the robot based on the first user posture information. The robot system determines a target posture different from the posture of the first user as the posture of the robot based on the first user posture information at a time when the robot is in the pre-change posture indicated by the pre-change posture information and the pre-change posture information. In addition, the robot system described in Patent Document 1 acquires user information on which the first user posture information is based from a user device via a network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6476358 Summary of the Invention [Problem to be solved by the invention]

[0004] In a robot system, it is important for the robot to follow the operator's movements without delay in order to improve work efficiency and reduce the operator's burden. However, it is impossible to completely eliminate delays in the control of the robot based on the operator's movements and in the communication of operation information. On the other hand, delays in the robot's movements (tracking delays) give the operator an uncomfortable feeling, slow down the work speed, cause the operator to fail the work, and reduce work efficiency.

[0005] An aspect of the present invention has been made in consideration of the above points, and one of its objectives is to provide a control device, a robot system, a control method, and a program that can improve the operability. [Means for solving the problem]

[0006] (1) The present invention has been made to solve the above problems. One aspect of the present invention is a method for detecting a motion of a robot, comprising: At a minimum, indicate the operator's hand position a trajectory prediction unit that determines a predicted trajectory of an effector of the robot from a current time to a predicted time after a predetermined predicted time based on operation information; and a control command generation unit that generates a control command based on the predicted trajectory; and an operation status estimation unit that estimates an operation status of the robot based on at least environmental information indicating an operation environment of the robot and the operation information, and the trajectory prediction unit , The predicted time is determined based on the operating condition. It is a control device.

[0008] (2) Another aspect of the present invention is (1) a drive control unit that determines an amount of operation for the movement mechanism of the robot based on a target value of a displacement of the movement mechanism of the robot that provides a target position of the effector for each time of the predicted trajectory, and the movement status estimating unit The target gain may be determined based on the operating conditions.

[0009] (3) Another aspect of the present invention is (2) wherein the drive control unit determines the manipulated variable by combining a first component based on a deviation between the displacement output value providing a current position of the effector and the target value and a first gain, and a second component based on the target value and a second gain, and the operation status estimation unit may determine the first gain and the second gain based on the operation status.

[0010] (4) Another aspect of the present invention is the (3) In any one of the control devices described above, the operation status estimating unit may further estimate the operation status based on operator information indicating a status of an operator operating the robot.

[0011] (5)Another aspect of the present invention is to provide a method for controlling a computer to execute a program comprising: (3) The present invention may also be a program for causing the device to function as any one of the control devices described above.

[0012] (6) Another aspect of the present invention is (1) et al. (3) The present invention may be a robot system including any one of the control devices described above and the robot.

[0013] (7) Another aspect of the present invention is a control method in a control device, comprising: motion information indicating at least a motion of a robot; At a minimum, indicate the operator's hand position A first step of determining a predicted trajectory of an effector of the robot from a current time to a predicted time after a predetermined predicted time based on operation information, and a second step of generating a control command based on the predicted trajectory; and a third step of estimating an operating condition of the robot based on environmental information indicating at least an operating environment of the robot and the operation information, The first step is determining the predicted time based on the operating condition; It is a control method. Effect of the Invention

[0014] As mentioned above, (5)、(6)、(7) According to this configuration, the effector of the robot is driven according to a control command generated based on the predicted trajectory of the effector from the current time to a predicted time after the current time, thereby reducing or eliminating the delay until the operation is reflected in the operation of the robot 20. Since the feeling of operation is improved for the operator, it is possible to achieve both improved work efficiency and reduced burden.

[0015] As mentioned above (1)、(5)、(6)、(7) According to the configuration of (1), the predicted time is determined according to the robot's operating environment and the operating situation of the robot, and therefore the balance between the improvement of the operability and the accuracy of the position of the effector to be controlled is adjusted according to the operating situation.

[0016] As mentioned above (2) According to the configuration of (1), the contribution of the target value to the amount of operation of the operating mechanism is adjusted in accordance with the operating situation. Therefore, the sensitivity of the operation of the effector to the operation by the operator is adjusted in accordance with the operating situation.

[0017] As mentioned above(3) According to the configuration of (1), the balance between the feedback term and the feedforward term is adjusted according to the operating situation. Therefore, the balance between the sensitivity and accuracy of the operation of the effector organ in response to the operation of the operator is adjusted according to the operating situation.

[0018] As mentioned above (4) According to the configuration of (a), the operating status can be accurately estimated by further referring to the status of the operator, which further improves the work efficiency and reduces the work burden of the robot. [Brief description of the drawings]

[0019] [Figure 1] 1 is a schematic block diagram illustrating an example of the configuration of a robot system according to a first embodiment. [Diagram 2] 2 is a block diagram showing an example of a functional configuration of a portion of a control device according to the first embodiment. FIG. [Diagram 3] 2 is a schematic block diagram illustrating an example of a hardware configuration of a control device according to the first embodiment. FIG. [Figure 4] 5 is a flowchart showing an example of an operation control process according to the first embodiment. [Diagram 5] FIG. 11 is a schematic block diagram illustrating a configuration example of a robot system according to a second embodiment. [Figure 6] FIG. 11 is a block diagram showing an example of a functional configuration of a portion of a control device according to a second embodiment. [Figure 7] 10 is a flowchart showing an example of an operation control process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] <First embodiment> Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing an example of the configuration of a robot system S1 according to this embodiment. The robot system S1 is a control system that can control the operation of the robot 20 according to the operation of an operator who is a user. The robot system S1 is also a manipulation system that manipulates the operation of the robot 20 according to the operation. The robot 20 is equipped with an effector (also called an end-effector, robot hand, end effector, effector, etc.). The effector is a member that mechanically acts on and influences other objects. The effector has multiple finger parts and is equipped with a mechanism that can grasp or release other objects. Each finger part can operate according to the movement of the corresponding finger of the operator. This allows various tasks to be realized.

[0021] The robot system S1 according to this embodiment estimates the motion status of the robot 20 based on environmental information indicating the motion environment of the robot 20 and operation information indicating the operation status. As described later, when estimating the motion status, the robot system S1 may also use driving state information acquired from the driving sensor 26 and gaze information of the operator. The robot system S1 generates a control command for operating the effector of the robot 20 to a target position based on characteristic parameters indicating control characteristics corresponding to the motion status. The robot system S1 controls the motion of the robot 20 based on the generated control command. In the present application, the operation situation mainly includes the form of the operation performed by the robot 20, that is, the task or operation mode. The operation situation may include, for example, the type of work to be performed, the positional relationship with an object related to the work, the type or characteristics of the object, and the like as elements. The operation situation may be defined to include any element or a combination of any elements. Even if the operation information is common, the target position to which the effector of the robot 20 is moved may differ depending on the operation situation.

[0022] The robot system S1 includes a robot 20, a display device 50, an operating device 70, and an environmental information acquisition unit 80. The robot 20 includes one or more manipulators. The manipulator is also called a robot arm. Each manipulator includes a plurality of segments, which are connected to one another. The plurality of segments includes one effector. The effector is a member that is connected to one end of the manipulator and that comes into contact with an object to exert an action. The operation mechanism of the robot 20 includes a joint for each pair of segments including two of the plurality of segments, and includes an actuator for each joint. The position of the effector can be moved by changing the angle between each pair of segments using the actuator. In the following description, the case where the number of manipulators is one (single-arm type) is mainly described, but the number of manipulators may be two or more. The robot 20 may be a stationary type that is fixed at a predetermined position, or a movable type that can move its own position. In the following description, the robot 20 will be mainly a stationary type.

[0023] The control device 30 of the robot 20 is connected to the display device 50, the operation device 70, and the environmental information acquisition unit 80 wirelessly or via wires so as to be able to input and output various types of information. The display device 50 and the operation device 70 may be located in a space physically separated from the robot 20 and the environmental information acquisition unit 80. In that case, the display device 50 and the operation device 70 may be configured as a remote operation system connected to the control device 30 via a communication network.

[0024] The robot 20 includes a drive unit 24 , a drive sensor 26 , a power source 28 , and a control device 30 . The drive unit 24 includes an actuator for each joint, and operates according to a control command input from a drive control unit 40 of the control device 30. Each actuator corresponds to a so-called motor, and changes the angle between the two segments connected to the actuator according to a target value of the drive amount specified by the control command. The drive sensor 26 detects the drive state of the robot 20 by the drive unit 24, and outputs drive state information indicating the detected drive state to the control device 30. The drive sensor 26 includes, for example, a rotary encoder that detects the angle between two segments for each joint.

[0025] The power supply 28 supplies power to each component of the robot 20. The power supply 28 includes, for example, a power supply terminal, a secondary battery, and a voltage converter. The power supply terminal allows a power line to be connected, receives power from the outside, and supplies the supplied power to the secondary battery or the voltage converter. The secondary battery stores power supplied via a power supply terminal. The secondary battery is capable of supplying power to each component via a voltage converter. The voltage converter converts the voltage of the power supply terminal or the power supplied from the power supply terminal to a predetermined voltage required by each component, and supplies each component with the converted voltage.

[0026] Next, the functional configuration of the control device 30 according to this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of a portion of the control device 30 according to this embodiment. The control device 30 includes an information acquisition unit 32, an operation status estimation unit 34, a target position estimation unit 36, a control command generation unit 38, a drive control unit 40, a communication unit 42, and a storage unit 44. The information acquisition unit 32 acquires various information related to the status of the operator and the status of the robot 20. For example, the information acquisition unit 32 acquires gaze information and first operator movement information from the display device 50. The gaze information and the first operator movement information constitute operator information indicating the status of the operator. The gaze information is information indicating the gaze direction of at least one eye of the operator at a certain time. The first operator movement information is information indicating the movement of the operator's head. The head movement is represented by the direction and position of the head at each time. In the present application, the gaze information and the first operator movement information may be collectively referred to as operator information.

[0027] The information acquisition unit 32 acquires the second operator motion information from the operation device 70. The second operator motion information is information indicating a body motion related to the operation by the operator. The second operator motion information constitutes operation information indicating the operation status of the robot 20. The second operator motion information mainly includes information indicating the motion of the hand as the body part of the operator. The hand motion includes the motion of at least two fingers. The second operator motion information may include information indicating the motion of the wrist in addition to the motion of the hand. The wrist motion may be represented using a representative position of the wrist, and may further include information on the posture. The motion of the body part is represented using information on the position of the body part at each time. In the present application, the second operator motion information may be referred to as operation information.

[0028] The information acquiring unit 32 acquires environmental information from the environmental information acquiring unit 80. The environmental information indicates the operating environment of the robot 20. The environmental information includes image data indicating an image showing the operating environment of the robot 20, and distance data indicating distances to objects distributed in the operating environment of the robot 20. The information acquiring unit 32 acquires the driving state information from the driving sensor 26. The driving state information can also be considered as posture information indicating the posture of the robot 20. The time series of the driving state information at each time corresponds to robot operation information indicating the operation of the robot 20. The information acquiring unit 32 stores the acquired various information in the storage unit 44.

[0029] The motion situation estimation unit 34 reads the latest environmental information and operation information at that time from the storage unit 44, and estimates the motion situation of the robot 20 based on the read environmental information and operation information. The motion situation estimation unit 34 uses, for example, a predetermined machine learning model to estimate the motion situation of the robot 20. When the environmental information and operation information are input as input information to the motion situation estimation unit 34, a parameter set learned using training data is set in advance so that output information in which the reliability of a known motion situation corresponding to the input information is set to 1 and the reliability of other candidates of motion situations is set to 0 is obtained as output information. The training data is configured to include many pairs of input information and known output information. The parameter set is learned so that the difference between the estimated value of the output information estimated from the input information and the known output information is minimized as the entire training data. In this case, the reliability of each motion situation obtained from the machine learning model can be determined as a real value between 0 and 1. The motion situation estimation unit 34 can specify a motion situation candidate in which the reliability calculated for each candidate motion situation is higher than a predetermined reliability and is the maximum, as the motion situation of the robot 20. As the motion situation, for example, an approach to an object (target object) that is located in the direction closest to the motion direction of the hand indicated by the operation information at that time among a plurality of objects located in the motion environment of the robot 20, a form of action of an object in contact with the effector at that time (e.g., grasping, releasing, friction, etc.), etc. are determined.

[0030] The motion situation estimation unit 34 may read the latest operator information at that time from the storage unit 44, and estimate the motion situation of the robot 20 based on the operator information in addition to the environmental information and operation information. The operator information indicates the line of sight direction taking into consideration the movement of the operator's head. Therefore, when estimating the motion situation, the direction in which an event in which the operator is interested occurs is further taken into consideration based on the operator information. The motion situation estimation unit 34 pre-sets a parameter set learned as described above using training data so that, when the environmental information, operation information, and operator information are input as input information to the machine learning model, output information in which the reliability of a known motion situation corresponding to the input information is 1 and the reliability of other motion situation candidates is 0 is obtained as output. The motion status estimation unit 34 may read out the latest driving state information at that time from the storage unit 44, and further estimate the motion status of the robot 20 based on the driving state information. The driving state information indicates the posture of the robot 20 and its change over time. Therefore, when estimating the motion status, the posture of the robot 20 is further taken into consideration. In the motion status estimation unit 34, a parameter set learned as described above is set in advance using training data so that when driving state information is input as input information to the machine learning model in addition to environmental information and operation information, output information in which the reliability of a known motion status corresponding to the input information is 1 and the reliability of other motion status candidates is 0 is obtained as output.

[0031] The operation situation estimation unit 34 determines characteristic parameters corresponding to the estimated operation situation. The characteristic parameters are parameters related to the control characteristics for the operation of the effector. The characteristic parameters include, for example, a convergence determination parameter, a weighting coefficient for each factor of the objective function, or a combination of these. The storage unit 44 stores in advance a characteristic parameter table indicating characteristic parameters for each operation situation. The operation situation estimation unit 34 refers to the characteristic parameter table, identifies characteristic parameters corresponding to the estimated operation situation, and outputs the identified characteristic parameters to the control command generation unit 38.

[0032] The convergence judgment parameter is a parameter indicating a convergence judgment condition for judging whether the position of the effector has converged to the target position. The strength of the convergence judgment condition corresponds to the degree of constraint of the converged position (sometimes referred to as the "convergence position" in this application) with respect to the target position, in other words, the accuracy with respect to the target position. For example, a threshold value of the distance from the target position can be used as the convergence judgment parameter. The smaller the threshold value of the distance from the target position, the stronger the convergence judgment condition. The operation situation estimation unit 34 may set a convergence judgment parameter indicating a stronger convergence judgment condition for an operation situation requiring the tracking of the position of the effector to be controlled. Tracking means a characteristic that allows the position of the effector to be accurately controlled with respect to the target position, or a characteristic that requires accurate control, that is, accuracy. Such operation situations include, for example, a situation where a delicate object is installed on the effector and the operation instructed by the operation information is delicate, and the like.

[0033] Inverse kinematics calculation is an analytical method for determining the displacement of each joint from a target position, that is, either or both of the angle and angular acceleration, and is also called inverse kinematics analysis. Note that inverse dynamics calculation is an analytical method for determining the driving force of a joint for realizing the motion of each joint (that is, the time change of either or both of the angle and angular acceleration) in a link mechanism in which multiple segments are connected, and is also called inverse dynamics analysis. Therefore, inverse kinematics calculation is closely related to inverse dynamics calculation. In this embodiment, characteristic parameters related to inverse kinematics calculation may include characteristic parameters related to inverse dynamics calculation. On the other hand, the stronger the convergence judgment condition, the higher the possibility that no solution of the inverse kinematics calculation exists, or that even if a solution exists, the higher the possibility that the solution is unstable. Therefore, the motion situation estimation unit 34 may set a convergence judgment parameter indicating a weaker convergence judgment condition (e.g., distance from the target position) for a motion situation in which flexibility of the position of the effector to be controlled is permitted. Flexibility means a characteristic that allows deviation of the position of the effector from the target position. Flexibility means a characteristic that allows reliable control, that is, safety. An example of a motion situation in which flexibility is permitted is a case where multiple objects are distributed around the path to the target position.

[0034] The objective function is used to numerically obtain an optimal solution for the operation of the robot 20 using a known method in the inverse kinematics calculation (optimization problem). In other words, the objective function is a function that quantitatively indicates the magnitude of the load for operating the effector to the target position, and includes multiple types of factors, and is configured by combining these factors. The weighting coefficient indicates the contribution of each factor to the objective function. The load means the cost related to control. The multiple factors include the magnitude of the control error at each time as one factor. The magnitude of the control error is expressed, for example, by the magnitude (distance) of the error between the final target position and the convergence. The effector is controlled so that the tracking ability of the effector to the target position increases as the weighting coefficient for the control error increases. The objective function may include any one or a combination of factors such as the arrival time from the current position of the effector to the convergence position, the magnitude of the jerk from the current position to the convergence position, the amount of power from the current position to the convergence position, the momentum of the robot 20 until the convergence position is reached, the magnitude of the difference between the current posture of the robot 20 and the posture of the robot 20 at the time when the effector is located at the convergence position, and the like. In this application, optimization means searching for a solution that reduces the function value of the objective function, and is not limited to absolutely minimizing the function value. Therefore, the function value may temporarily increase.

[0035] Therefore, the motion situation estimation unit 34 may set the weighting coefficient for the control error to be larger in a motion situation requiring the tracking of the position of the effector to be controlled. The motion situation estimation unit 34 may set the weighting coefficient for the control error to be smaller in a motion situation allowing flexibility in the position of the effector to be controlled. The motion situation estimation unit 34 may set the weighting coefficient for the magnitude of the jerk to be larger in a motion situation requiring continuity in the position of the effector to be controlled.

[0036] The motion situation estimating unit 34 determines drive control parameters corresponding to the estimated motion situation. The drive control parameters are parameters related to the control characteristics for driving each segment constituting the robot 20. The drive control parameters are used when a controller constituting the drive control unit 40 determines the operation amount for the actuator that drives each joint. The storage unit 44 stores in advance a drive control parameter table indicating the drive control parameters for each motion situation. The motion situation estimating unit 34 refers to the drive control parameter table to identify the drive control parameters corresponding to the estimated motion situation, and outputs the identified drive control parameters to the drive control unit 40. For example, when the controller is a PID (Proportional-Integral-Differential) controller, the drive control parameters include a proportional gain, an integral gain, and a differential gain. The proportional gain is a gain for multiplying a proportional term, which is one component of the manipulated variable, by the deviation between the target value and the output value at that time (present). The integral gain is a gain for multiplying an integral term, which is another component of the manipulated variable, by the integral value of the deviation up to that time. The differential gain is a gain for multiplying a differential term, which is yet another component of the manipulated variable, by the differential value of the deviation at that time. For example, the operation situation estimation unit 34 may determine each gain such that the differential gain is relatively larger than other types of gains for an operation situation requiring more tracking ability. For example, the operation situation estimation unit 34 determines each gain such that the integral gain is relatively larger than other types of gains for an operation situation requiring more flexibility.

[0037] The target position estimation unit 36 ​​reads out the latest environmental information and operation information at that time, and estimates the target position of the effector of the robot 20 based on the read environmental information and operation information. The target position estimation unit 36 ​​outputs target position information indicating the estimated target position to the control command generation unit 38. The target position estimation unit 36 ​​uses, for example, a machine learning model to estimate the target position. To distinguish it from the machine learning model in the operation situation estimation unit 34, the machine learning model in the operation situation estimation unit 34 and the machine learning model in the target position estimation unit 36 ​​may be called a first machine learning model and a second machine learning model, respectively.

[0038] The target position estimation unit 36 ​​is preset with a parameter set that has been learned so that when environmental information and operation information are input as input information to the machine learning model, output information relating to the target position corresponding to the input information is obtained. As the target position, a position on the surface of the object that is empirically likely to be acted upon by the effector is set. The target position may depend on the shape of the object. In the case of a vertically elongated cylinder whose height is greater than its diameter, the position closest to the effector among the center of the bottom surface, the center of the surface, and the cross section of the midpoint in the height direction may be more likely to be the target position. The output information from the machine learning model may be, but is not limited to, information indicating the coordinates of the target position. The output information may include, for example, information indicating the object on which the target position is set, the type, shape, and direction of the object, and the position on the object where the target position is set. The target position estimation unit 36 ​​can determine the coordinates of the target position from the output information.

[0039] The target position estimation unit 36 ​​may also read the latest operator information at that time from the storage unit 44 and estimate the target position based on the operator information in addition to the environmental information and operation information. Therefore, the target position is estimated based on the operator information, taking into consideration the direction in which an event of interest to the operator occurred. However, a parameter set is set in advance in the target position estimation unit 36, which has been learned so that when the operator information is input as input information to the machine learning model in addition to the environmental information and operation information, the coordinate value of the target position corresponding to the input information can be obtained as output information.

[0040] However, the target position estimation unit 36 ​​does not necessarily need to estimate the target position using the second machine learning model. For example, when the motion situation notified from the motion situation estimation unit 34 is a motion situation that does not involve continuous translational motion of the effector, the target position estimation unit 36 ​​does not estimate the target position using the second machine learning model. Such motion situations include, for example, stillness, rotation, gripping and releasing of an object, and friction of the object surface. When the target position estimation unit 36 ​​does not estimate the target position using the second machine learning model, the target position estimation unit 36 ​​may adopt the position of the operator's hand indicated by the operation information as the target position. The target position estimation unit 36 ​​can use mathematical models such as a neural network and a random forest as the first machine learning model and the second machine learning model.

[0041] The control command generating unit 38 generates an operation command for moving the effector from the current position toward the target position indicated by the target position information input from the target position estimating unit 36. "Towards the target position" or "moving towards the target position" includes the meaning that the purpose is to move to the target position, but reaching the target position is not guaranteed. The control command generating unit 38 can read the drive state information stored in the storage unit 44 and calculate the current position of the effector based on the angle of each joint and the dimension of each segment indicated by the drive state information. The control command generating unit 38 can calculate the velocity of the effector based on the angle of each joint and the dimension of each segment indicated by the drive state information, as well as the angular velocity of each joint. The control command generating unit 38 performs a known inverse kinematic calculation based on the characteristic parameters and drive state information (robot operation information) input from the operation situation estimating unit 34, and determines the angle of each joint so as to move the effector to the convergence position. The control command generating unit 38 outputs an operation command (joint command) indicating the determined angle of each joint (joint angle) to the drive control unit 40.

[0042] When the characteristic parameters include a convergence judgment parameter, the control command generating unit 38 may determine, in the inverse kinematics calculation, any position within the region from the target position indicated by the convergence judgment parameter as a convergence position, and may stop the movement of the effector 38. When determining whether the current position of the effector has converged, the control command generating unit 38 may, for example, determine whether the current position of the effector is within a predetermined distance threshold indicated by the characteristic parameter from the target position. When determining that the position of the effector has converged, the control command generating unit 38 generates a control command indicating the stop of operation and outputs it to the drive control unit 40. When the drive control unit 40 receives a control command indicating the stop of operation from the control command generating unit 38, it stops updating the target values ​​for each joint and fixes them. At the convergence position, the control command generating unit 38 may impose a constraint condition that the velocity of the effector is zero, or the drive control unit 40 may impose a constraint condition that the angular velocity of each joint is zero.

[0043] When the convergence characteristic parameters include a weighting coefficient for each factor of the objective function, the control command generator 38 performs optimization calculations based on the objective function synthesized using the factors corresponding to the weighting coefficients in the inverse kinematic calculations. The objective function is calculated, for example, as a sum (weighted sum) between factors of the products of the factors and the weighting coefficients. In the optimization calculations, the control command generator 38 may recursively calculate the angles of each joint so that the objective function indicating the load generated until the effector reaches the target position or convergence position is minimized.

[0044] Note that, when target position information is not acquired from the target position estimation unit 36, the control command generation unit 38 may adopt the position of the hand indicated by the operation information as the target position. The control command generation unit 38 performs inverse kinematic calculations to generate a motion command having the angle of each joint as a target value for moving the effector from the current position toward the adopted target position.

[0045] The drive control unit 40 determines a drive command based on the control command input from the control command generation unit 38, the drive control parameters input from the operation status estimation unit 34, and the drive state information (robot operation information) read from the storage unit 44. More specifically, the drive control unit 40 includes a controller that calculates an operation amount for controlling the angle of each joint indicated by the control command as a target value, and determines a control command indicating the calculated operation amount. The drive control unit 40 outputs the determined control command to the drive unit 24. The controller sets the angle indicated by the drive state information as an output value, and calculates the operation amount by applying the drive control parameters to the deviation between the target value and the output value according to a control method set in the controller. The PID controller calculates the sum of multiplied values ​​obtained by multiplying the deviation, the time integral value of the deviation, and the time derivative value of the deviation by a proportional gain, an integral gain, and a derivative gain, respectively, as the operation amount.

[0046] The communication unit 42 transmits and receives various information wirelessly or via wires between each component of the robot 20 and other components (i.e., the display device 50, the operation device 70, and the environmental information acquisition unit 80). The communication unit 42 includes, for example, an input / output interface, a communication interface, and the like. The storage unit 44 temporarily or permanently stores data used for various processes in the control device 30, various data acquired by the control device 30, etc. The storage unit 44 includes a storage medium such as a RAM (Random Access Memory) or a ROM (Read Only Memory).

[0047] The display device 50 includes a display unit 52, a gaze detection unit 54, a motion detection unit 56, a control unit 58, and a communication unit 60. The display device 50 includes a support member that can be worn on the head of an operator who is a user, and may be configured as a head mounted display (HMD).

[0048] The display unit 52 receives image data from the environmental information acquisition unit 80 via the control device 30 using the communication unit 60. Based on the received image data, the display unit 52 displays an image showing the operating environment of the robot 20. When the display unit 52 is worn on the head of the operator, the display unit 52 may be disposed in a position where the screen faces directly in front of both eyes. The gaze detection unit 54 includes a gaze sensor that detects the gaze direction of one or both eyes of the operator. The gaze detection unit 54 transmits gaze information indicating the gaze direction detected at each time to the control device 30 using the communication unit 60. The gaze detection unit 54 may be disposed at a position exposed to at least one eye when worn on the operator's head. The motion detection unit 56 detects the motion of the head of the operator, and transmits first operator motion information indicating the detected motion to the control device 30 using the communication unit 60. The motion detection unit 56 is configured to include, for example, an acceleration sensor for detecting the motion of the operator.

[0049] The control unit 58 controls the functions of the display device 50. The control unit 58 includes a processor such as a CPU (Central Processing Unit). The communication unit 60 transmits and receives various information to and from the control device 30. The communication unit 60 includes a communication interface.

[0050] The operation device 70 includes a motion detection unit 72, a control unit 74, and a communication unit 76. The operation device 70 includes a support member that can be worn on the hand of an operator who is a user, and may be configured as a data glove. The motion detection unit 72 detects the motion of the operator's hand, and transmits second operator motion information indicating the detected motion to the control device 30 using the communication unit 76. The motion detection unit 72 includes, for example, an acceleration sensor for detecting the motion of the operator's hand. A wrist tracker may be further connected to the operation device 70. The wrist tracker includes a support member that can be worn on the operator's wrist, and an acceleration sensor for detecting the wrist motion. The acceleration sensor that detects the wrist motion constitutes a part of the motion detection unit 72. The second operator motion information is transmitted to the control device 30 including information indicating the wrist motion.

[0051] The control unit 74 controls the functions of the operation device 70. The control unit 74 includes a processor such as a CPU. The communication unit 76 transmits and receives various information to and from the control device 30. The communication unit 76 includes a communication interface. The motion detection unit 72 may be connectable to the communication unit 60 of the display device 50. The second operator motion information may be transmitted to the control device via the communication unit 60. In that case, the control unit 74 and the communication unit 76 may be omitted.

[0052] The environmental information acquisition unit 80 includes an image capture unit 82, a distance measurement unit 84, and a communication unit 86. The environmental information acquisition unit 80 may be installed in the housing of the robot 20, or may be installed in a location separate from the robot 20. The photographing unit 82 photographs an image of the operating environment within a predetermined range from the robot 20. The operating environment includes the range that the effector of the robot 20 can reach. The photographed image does not necessarily include the entire image of the robot 20. The photographing unit 82 is a digital video camera that photographs images at predetermined time intervals. The photographing unit 82 transmits image data showing the photographed image to the control device 30 via the communication unit 86.

[0053] The distance measuring unit 84 measures the distance from the robot 20 to the surface of an object in the operating environment within a predetermined range. The distance measuring unit 84 includes, for example, a wave transmitting unit, a wave receiving unit, and a distance detecting unit. The wave transmitting unit transmits waves such as infrared rays. The wave transmitting unit includes, for example, a light emitting diode. The wave receiving unit receives reflected waves generated by reflection on the surface of the object. The wave receiving unit includes, for example, a photodiode. The reflected waves are generated in response to the wave transmitted from the wave transmitting unit being incident on the surface of the object as an incident wave. The distance detecting unit detects the phase difference between the transmitted wave and the reflected wave, and can determine the distance to the surface of the object based on the detected phase difference. The distance detecting unit transmits distance data indicating the distance determined for each direction corresponding to each pixel of the image to the control device 30 via the communication unit 86. Note that the wave used to measure the distance is not limited to infrared rays, and may be millimeter waves, ultrasonic waves, or the like. The communication unit 86 transmits and receives various information to and from the control device 30. The communication unit 86 includes a communication interface. In addition, when the environmental information acquisition unit 80 is installed in the housing of the robot 20 and can transmit and receive various information to and from other functional units of the robot 20, the communication unit 86 may be omitted.

[0054] (Hardware configuration) Next, an example of the hardware configuration of the control device 30 according to the present embodiment will be described. 3 is a schematic block diagram showing an example of a hardware configuration of the control device 30 according to this embodiment. The control device 30 functions as a computer including a processor 102, a ROM 104, a RAM 106, an auxiliary storage unit 108, and an input / output unit 110. The processor 102, the ROM 104, the RAM 106, the auxiliary storage unit 108, and the input / output unit 110 are connected to each other so as to be able to input and output various data.

[0055] The processor 102, for example, reads out programs and various data stored in the ROM 104, and executes the programs to execute processes for realizing the functions of each part of the control device 30 and processes for controlling the functions. The processor 102 is, for example, a CPU. In this application, the execution of processes instructed by various commands written in a program may be referred to as "executing a program" or "running a program."

[0056] The ROM 104 stores, for example, a program for execution by the processor 102 . The RAM 106 functions as a working area for temporarily storing various data and programs used by the processor 102, for example. The auxiliary storage unit 108 permanently stores various data. The auxiliary storage unit 108 stores data acquired by the control device 30. The auxiliary storage unit 108 includes a storage medium such as a hard disk drive (HDD) or a solid state drive (SSD).

[0057] The input / output unit 110 is capable of wirelessly or wired input / output of various data with other devices. The input / output unit 110 may be connected to other devices via a network. The input / output unit includes, for example, one or both of an input / output interface and a communication interface. The display device 50, the operation device 70, and the environmental information acquisition section 80 may also have a hardware configuration similar to the hardware configuration exemplified in FIG. 3, and may be configured as a computer that realizes the functions of each device.

[0058] (Operation control processing) Next, an example of the operation control process according to the present embodiment will be described with reference to a flowchart shown in FIG 4. (Step S102) The motion status estimation unit 34 acquires the latest environmental information, operation information, and gaze information at that time, and estimates the motion status of the robot 20 based on these. (Step S104) The operation condition estimation unit 34 refers to the characteristic parameter table and the drive control parameter table, and determines the characteristic parameters and the drive control parameters corresponding to the estimated operation condition, respectively.

[0059] (Step S106) The target position estimation unit 36 ​​acquires the latest environmental information, operation information, and line of sight information at that time, and estimates the target position of the effector organ of the robot 20 based on these. (Step S108) The control command generator 38 generates a control command for moving the effector from the current position to the target position based on the determined characteristic parameters. (Step S110) The drive control unit 40 uses the drive control parameters to generate a drive command for controlling the angle of each joint to a target value indicated in the control command.

[0060] (Step S112) The control command generator 38 determines whether the position of the effector has converged. The control command generator 38 can determine whether the position of the effector has converged, for example, based on whether the current position of the effector is within a predetermined distance threshold indicated by a characteristic parameter from the target position. When it is determined that the position of the effector has converged (step S112 YES), a control command indicating the stop of operation is generated and output to the drive control unit 40. This stops the operation of the robot 20. Thereafter, the processing of FIG. 4 is terminated. When it is determined that the position of the effector has not converged (step S112 NO), the processing returns to step S102.

[0061] <Second embodiment> Next, a second embodiment of the present invention will be described. In the following description, differences from the first embodiment will be mainly described, and functions and configurations common to the first embodiment will be referred to in the description of the first embodiment unless otherwise specified. The functional configuration of the robot system according to this embodiment will be described with reference to FIG. 5 as well as FIG. 6. FIG. 5 is a schematic block diagram showing an example of the configuration of the robot system S1 according to this embodiment. FIG. 6 is a block diagram showing an example of the functional configuration of a part of the control device 30 according to this embodiment.

[0062] The robot system S1 according to this embodiment determines a predicted trajectory of the effector of the robot 20 from the current time to a predicted time after a predetermined predicted time based on position information indicating the position of the robot 20 and operation information indicating an operation status. The robot system S1 generates a control command based on the determined predicted trajectory. The robot system S1 according to this embodiment further includes a trajectory prediction unit 46 in the control device 30.

[0063] The trajectory prediction unit 46 predicts a predicted trajectory indicating the operation of the effector of the robot 20 from the present to a predicted time after a predetermined prediction time using robot operation information (driving state information) indicating at least the operation of the robot 20 at that time and operation information. The operation information is a factor in the combination with acceleration or deceleration of the effector, or a change in direction, for example. The trajectory prediction unit 46 performs linear prediction on the driving state information and operation information within a predetermined period up to the present, and calculates the position and speed of the effector up to the predicted time. The trajectory prediction unit 46 configures a predicted trajectory as a time series of the position of the effector at each time up to the predicted time. The predicted time indicates the elapsed time from the present to the predicted time, and is not the time spent for prediction.

[0064] The trajectory prediction unit 46 may sequentially predict the position of the effector using, for example, a Kalman filter. The Kalman filter is a method that has (a) a prediction step and (b) an update step, and sequentially repeats these steps. In the (a) prediction step, the trajectory prediction unit 46 adds the time evolution (weighted sum) of the state quantity of the effector up to the previous time and the external force to calculate an estimate of the current state quantity of the effector. However, the state quantity is a vector having a position and a velocity as elements. The external force is a vector having a velocity and an acceleration as elements. The velocity is calculated based on the angle and angular velocity of each joint indicated in the drive state information, and the dimension of the segment. The acceleration is calculated based on the angle, each velocity, and angular acceleration of each joint indicated in the drive state information, and the dimension of the segment. In addition, the trajectory prediction unit 46 adds the time evolution of the prediction error of the state quantity up to the previous time and the time evolution of the error to calculate the current prediction error. (b) In the update step, the trajectory prediction unit 46 calculates the difference between the current position (observation value) of the effector and the estimated value of the current position of the effector as the observation residual. The trajectory prediction unit 46 calculates the covariance of the current observation error from the current prediction error. The trajectory prediction unit 46 calculates the Kalman gain from the covariance of the current prediction error and the observation residual. The trajectory prediction unit 46 updates the current state quantity of the effector to a value obtained by adding the estimated value of the current state quantity of the effector to the product of the observation residual and the Kalman gain. The trajectory prediction unit 46 updates the current prediction error to a value obtained by multiplying the prediction error with the residual of the mapping from 1 to the observation space multiplied by the Kalman gain.

[0065] The predicted time may be fixed to a predetermined value (for example, 0.2 s to 2 s) or may be variable. The predicted time is set to a value equal to or less than the delay time until the operator's movement is expressed as the movement of the robot 20 (also called "tracking delay"). The longer the predicted time, the larger the prediction error of the effector position, which may impair operability. When the predicted time is made variable, a predicted time table showing the predicted time for each operation situation may be stored in advance in the storage unit 44. In particular, if the predicted time exceeds the delay time, the movement of the robot 20 precedes the movement of the operator, and there is a risk of a significant prediction error. Therefore, the delay time may be set as the upper limit of the predicted time. The operation situation estimation unit 34 can estimate the operation situation of the robot 20 using the above method, and specify the predicted time corresponding to the estimated operation situation by referring to the predicted time table.

[0066] The operation situation estimation unit 34 may set a smaller prediction time for an operation situation requiring accuracy of the position of the effector to be controlled. Such an operation situation is an operation situation in which a prediction error of the effector is not permitted, but a delay is permitted. For example, when the object to be operated is a minute object, or when the operation of the operator is delicate, it corresponds to such an operation situation. The operation situation estimation unit 34 may set a larger prediction time for an operation situation requiring quick response of the position of the effector. Such an operation situation is suitable for an operation situation in which a prediction error is permitted, but the responsiveness of the effector to the operation is expected. For example, it corresponds to a case where the effector is operated globally toward the target object, or a case where the effector is retreated to a space where no other objects are present.

[0067] The operation status estimation unit 34 outputs predicted time information indicating the specified predicted time to the trajectory prediction unit 46. The trajectory prediction unit 46 calculates the position and speed of the effector up to the predicted time determined using the predicted time indicated in the predicted time information input from the operation status estimation unit 34. The trajectory prediction unit 46 outputs predicted trajectory information indicating a predicted trajectory indicating at least the position and speed of the effector from the present to the predicted time to the control command generation unit 38.

[0068] When generating an operation command for moving the effector from the current position toward the target position, the control command generation unit 38 performs known inverse kinematic calculations based on the predicted trajectory input from the trajectory prediction unit 46 and the characteristic parameters input from the motion status estimation unit 34, instead of the drive state information, and determines the angles of each joint so as to move the effector to a convergence position. In contrast, the drive state information up to the present can also be regarded as an actual trajectory indicating the position of the effector at each time up to the present. In a supplementary embodiment, the delay in the operation of the effector can be compensated for by using the predicted trajectory instead of the actual trajectory.

[0069] Moreover, the controller provided in the drive control unit 40 according to this embodiment may calculate the manipulated variable by combining the first component calculated by the feedback term and the second component calculated by the feedforward term. The feedback term is a term that calculates the first component, which is a component of the manipulated variable, based on the deviation between the target value and the output value. The angle of each joint corresponding to the target position is used as the target value. The angle of each joint indicated in the drive state information is used as the output value. This output value corresponds to the current position of the effector at that time. The above PI control and PID control correspond to a method of determining the first component as the manipulated variable without including the second component. That is, the above proportional term, integral term, and differential term are each included in the first component. The feedforward term is a term that calculates the second component, which is the other component of the manipulated variable, based on the target value without considering the output value. The second component includes a second proportional term that is proportional to the target value. The second proportional term is obtained by multiplying the target value by a second proportional gain.

[0070] Therefore, in the drive control parameter table, in addition to the first gain (either the proportional gain, integral gain, or differential gain, or a combination of these) related to the first component, the second gain related to the second component is set as a drive control parameter corresponding to each operating situation. The second gain includes the second proportional gain. The second component may further include one or both of a second integral term proportional to a time integral value of a target value and a second differential term proportional to a time differential value of a target value. The second integral term is obtained by multiplying the target value by the second integral gain. The second differential term is obtained by multiplying the target value by the second differential gain. The second gain further includes one or both of a second integral gain and a second differential gain. The ratio of the first gain to the second gain may be different depending on the operating situation. For example, the first gain and the second gain may be determined so that the ratio of the first factor to the second factor is relatively large as the operating situation requires greater accuracy in the position of the effector to be controlled. In such an operating situation, the feedback term is relatively emphasized. Also, the first gain and the second gain should be determined so that the ratio of the second factor to the first factor is relatively large in an operating situation that requires quick response of the position of the effector. In such an operating situation, the feedforward term is relatively emphasized.

[0071] The motion status estimation unit 34 refers to the drive control parameter table to identify drive control parameters including a first gain and a second gain corresponding to the estimated motion status, and outputs the identified drive control parameters to the drive control unit 40. The drive control unit 40 calculates a first factor and a second factor for each joint based on the first gain and the second gain indicated in the drive control parameters input from the motion status estimation unit 34, and determines the amount of operation by combining the calculated first factor and second factor. The drive control unit 40 outputs a drive command indicating the amount of operation determined for each joint to the drive unit 24. In addition, the controller according to the present embodiment may determine the second component as the manipulated variable without including the first component. In that case, the setting of the first gain may be omitted.

[0072] (Operation control processing) Next, an example of the operation control process according to this embodiment will be described. Fig. 7 is a flowchart showing an example of the operation control process according to this embodiment. The process shown in Fig. 7 further includes steps S122, S124, and S126 in addition to the process shown in Fig. 4. After the target position estimation unit 36 ​​finishes the process of step S106, the process proceeds to step S122. (Step S122) The operation status estimation unit 34 refers to the predicted time table and determines a predicted time corresponding to the operation status determined by the unit itself. (Step S124) The trajectory prediction unit 46 uses the robot motion information and the operation information to estimate a predicted trajectory of the effector from the present to a predicted time that is a predetermined predicted time later. (Step S126) The control command generator 38 generates a control command indicating the angle of each joint that moves the effector toward the target position as a target value, using the predicted trajectory and the characteristic parameters. Then, the process proceeds to step S110.

[0073] In step S122, the operation condition estimating unit 34 may refer to a drive control parameter table to identify the drive control parameters corresponding to the operation condition. In step S110, the drive control unit 40 may use the identified drive control parameters to calculate a first factor and a second factor based on the target value and the deviation between the target value and the output value, and determine the manipulated variable by combining the calculated first factor and second factor. The drive control unit 40 generates a drive command indicating the determined manipulated variable, and outputs the generated drive command to the drive unit 24.

[0074] As described above, the control device 30 according to the above embodiment includes a trajectory prediction unit 46 that determines a predicted trajectory of the effector organ of the robot 20 from the current time to a predicted time a predetermined predicted time later, based on at least operation information indicating the operation of the robot 20 and operation information indicating an operation status, and a control command generation unit 38 that generates a control command based on the predicted trajectory. According to this configuration, the effector of the robot 20 is driven according to a control command generated based on the predicted trajectory of the effector from the current time to a predicted time after the current time, thereby reducing or eliminating the delay (follow-up delay) until the operation is reflected in the operation of the robot 20. Since the feeling of operation is improved for the operator, it is possible to achieve both improved work efficiency and reduced burden.

[0075] The control device 30 may further include an operation status estimation unit 34 that estimates an operation status of the robot 20 based on at least environmental information indicating an operation environment of the robot 20 and operation information. The trajectory prediction unit 46 may determine a prediction time based on the operation status. According to this configuration, the predicted time is determined according to the motion status of the robot 20 estimated from the motion environment and operation status of the robot 20. Therefore, the balance between the improvement of the operability and the accuracy of the position of the effector to be controlled is adjusted according to the motion status.

[0076] The control device 30 may also include a drive control unit 40 that determines an amount of operation for the movement mechanism based on a target value of the displacement of the movement mechanism of the robot 20 that gives a target position of the effector at each time of the predicted trajectory. The movement status estimator 34 may determine a gain of the target value based on the movement status. According to this configuration, the contribution of the target value to the amount of operation of the operating mechanism is adjusted in accordance with the operating situation, and therefore the sensitivity of the operation of the effector to the operation by the operator is adjusted in accordance with the operating situation.

[0077] The drive control unit 40 may determine the manipulated variable by combining a first component based on the deviation between the output value of the displacement that gives the current position of the effector and the target value and the first gain, and a second component based on the target value and the second gain. The operational status estimation unit 34 may determine the first gain and the second gain based on the operational status. According to this configuration, the balance between the feedback term and the feedforward term is adjusted according to the operating situation, and therefore the balance between the sensitivity and accuracy of the operation of the effector organ in response to the operation of the operator is adjusted according to the operating situation.

[0078] Furthermore, the motion status estimating unit 34 may further estimate the motion status based on operator information indicating the status of the operator operating the robot. According to this configuration, the operating status can be accurately estimated by further referring to the status of the operator, which further improves the work efficiency and reduces the workload of the robot.

[0079] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.

[0080] The above-described embodiments and their modified examples may be combined with each other as long as no contradiction occurs, or some of them may be omitted. For example, in each of the above embodiments, the target position estimation unit 36 ​​may be omitted, and further, the processing based on the target position estimated by the target position estimation unit 36 ​​may be omitted. In the second embodiment, when the predicted time is fixed, the operational status estimating unit 34 may be omitted, and further, the processing based on the operational status may be omitted.

[0081] In the environmental information acquisition unit 80, the distance measurement unit 84 may be integrated with the photographing unit 82 and configured as a three-dimensional image photographing unit. Moreover, the environmental information acquisition unit 80 may include a multi-viewpoint photographing unit instead of the photographing unit 82 and the distance measurement unit 84. The multi-viewpoint photographing unit is a photographing unit that has a plurality of viewpoints and can photograph an image for each viewpoint. The multi-viewpoint photographing unit includes a so-called stereo camera. The operating environment of the robot 20 is three-dimensionally expressed by the images (multi-viewpoint images) for each viewpoint photographed by the multi-viewpoint photographing unit. The target position estimation unit 36 ​​can identify the area occupied by an object from the operating environment of the robot 20 by analyzing the multi-viewpoint images.

[0082] Also, a part or the whole of the display device 50 may be omitted. Although the gaze information mainly indicates the gaze direction of one eye (for example, the left eye) of the operator, the present invention is not limited to this. The gaze information may indicate the gaze direction of each of the left eye and the right eye. When the gaze direction of one eye is not available at a certain time (for example, blinking), the target position estimation unit 36 ​​may use the gaze direction of the other eye when determining the attention point. When the gaze directions of both eyes are available, the target position estimation unit 36 ​​may determine the intersection of the line segments of the gaze directions of each of the two eyes, or the midpoint of the closest point of each of these line segments, as the gaze point. Then, the target position estimation unit 36 ​​may determine the direction from the head to the gaze point as the gaze direction representing the gaze directions of both eyes, and determine the intersection of the determined gaze direction and the surface of the object. The determined intersection is used to set the target position.

[0083] The robot 20 may include a plurality of manipulators. The plurality of manipulators may correspond to one forearm and one hand of each of a plurality of operators. Each manipulator may be operable in response to the movement of one forearm and one hand of each of the operators. The control device 30 may determine one target position for each manipulator using the above-mentioned method, and may operate the manipulator toward the determined target position. The robot 20 may be a dual-arm robot having two manipulators. For a dual-arm robot, for example, one and the other manipulator may correspond to the left and right hands of one operator, respectively. [Explanation of symbols]

[0084] S1...robot system, 20...robot, 24...drive unit, 26...drive sensor, 28...power supply, 30...control device, 32...information acquisition unit, 34...operation status estimation unit, 36...target position estimation unit, 38...control command generation unit, 40...drive control unit, 42...communication unit, 44...storage unit, 46...trajectory prediction unit, 50...display device, 52...display unit, 54...gaze detection unit, 56...motion detection unit, 58...control unit, 60...communication unit, 70...operation device, 72...motion detection unit, 74...control unit, 76...communication unit, 80...environmental information acquisition unit, 82...image capture unit, 84...distance measurement unit, 86...communication unit

Claims

1. a trajectory prediction unit that determines a predicted trajectory of the effector of the robot from a current time to a predicted time after a predetermined predicted time based on motion information indicating at least a motion of the robot and operation information indicating at least a hand position of an operator; a control command generating unit that generates a control command based on the predicted trajectory; a motion status estimation unit that estimates a motion status of the robot based on at least environmental information indicating an operational environment of the robot and the operation information; The trajectory prediction unit is The predicted time is determined based on the operating condition. Control device.

2. a drive control unit that determines an amount of operation for the movement mechanism of the robot based on a target value of a displacement of the movement mechanism of the robot that provides a target position of the effector for each time on the predicted trajectory, The operation status estimation unit is The target gain is determined based on the operating condition. The control device according to claim 1 .

3. The drive control unit is the operation status estimation unit determines the manipulated variable by combining a first component based on a deviation between the output value of the displacement that provides a current position of the effector and the target value and a first gain, and a second component based on the target value and a second gain; determining the first gain and the second gain based on the operating condition The control device according to claim 2.

4. The operation status estimation unit is Furthermore, the operating status is estimated based on operator information indicating the status of an operator operating the robot. The control device according to any one of claims 1 to 3.

5. A program for causing a computer to function as the control device according to any one of claims 1 to 3.

6. A robot comprising the control device according to any one of claims 1 to 3 and the robot. Robot system.

7. A control method in a control device, comprising: a first step of determining a predicted trajectory of an effector of the robot from a current time to a predicted time after a predetermined predicted time based on motion information indicating at least a motion of the robot and operation information indicating at least a hand position of an operator; a second step of generating control commands based on the predicted trajectory; and a third step of estimating an operating condition of the robot based on environmental information indicating at least an operating environment of the robot and the operation information, The first step comprises: determining the predicted time based on the operating condition; Control methods.

Citation Information

Patent Citations

  • Qualitative inference processing method

    JP1989076358A

  • Robot motion control device

    JP2017196678A

  • Remote control method and remote control system

    JP2019217557A

  • Remote control manipulator system and control device

    WO2019059364A1