Robot control system, robot control device, robot control method, and program

Abstract

This robot control system is a control system for a hybrid robot that uses a combination of two or more robots, each of which is controlled by an individual controller, the robot control system comprising: a controller that acquires a robot state and issues a drive command for a robot; and an integrated controller that performs control in a state in which the two or more robots are combined. The integrated controller comprises: an acquisition unit that acquires a robot command value and a robot state held by each of the controllers; a calculation unit that calculates, from the robot state, an appropriate correction amount for the hybrid robot, in which the two or more robots are combined; and a distribution unit that distributes the correction amount to each of the controllers.

Description

Robot control system, robot control device, robot control method, and program

[0001] The present invention relates to a robot control system, a robot control device, a robot control method, and a program. This application claims priority to Japanese Patent Application No. 2024-042680, filed March 18, 2024, the contents of which are incorporated herein by reference.

[0002] A system for integrating and controlling a robot and peripheral devices has been proposed (see, for example, Patent Document 1). In such a system, for example, when a robot arm, hand, and cart are combined and used as a single robot, each has its own individual controller.

[0003] Japanese Patent Application Laid-Open No. 2021-20259

[0004] However, conventional technologies have individual controllers, so the accuracy of the entire system, for example, fingertip position in three-dimensional space, cannot be guaranteed, which leads to issues with reduced work efficiency and task success rates.

[0005] The aspects of the present invention have been made in consideration of the above-mentioned problems, and aim to provide a robot control system, a robot control device, a robot control method, and a program that can improve work efficiency and task success rates.

[0006] In order to solve the above problems, the present invention employs the following aspects: (1) A robot control system according to one aspect of the present invention is a control system for a composite robot that uses a combination of two or more robots, each controlled by an individual controller, and includes: a controller that acquires the states of the robots and issues drive commands for the robots; and an integrated controller that controls the two or more robots in a combined state, wherein the integrated controller includes an acquisition unit that acquires robot command values ​​and the states of the robots held by each of the controllers, a calculation unit that calculates, from the states of the robots, an appropriate correction amount for the composite robot that combines two or more robots, and a distribution unit that distributes the correction amount to each of the controllers.

[0007] (2) In the above aspect (1), the calculation unit may calculate the correction amount from actual machine information on either one of the tracking performance and movable angle limit of each drive axis of the robot.

[0008] (3) In the above aspect (1) or (2), the calculation unit may calculate the correction amount by preferentially using an axis of the robot having high tracking performance.

[0009] (4) In any one of the above aspects (1) to (3), the integrated controller may include a part designation unit that designates a part on the robot to be operated among the plurality of robots; a degree of freedom determination unit that determines the degrees of freedom to be used among the plurality of robots; and a joint angle calculation unit that receives as input the state quantities of the robot, the part on the robot to be operated, a target position of the part on the robot to be operated, and the degrees of freedom to be used, and calculates a joint angle target for moving the part on the robot among the plurality of robots to a target value of the specific part.

[0010] (5) A robot control device according to one aspect of the present invention is a controller that acquires the state of a robot and issues drive commands for the robot, and is a control device for a composite robot that uses a combination of two or more robots, each controlled by an individual controller. The robot control device includes: an acquisition unit that controls the combined state of the two or more robots and acquires the robot command values ​​and the states of the robots held by each of the controllers; a calculation unit that calculates, from the state of the robots, an appropriate correction amount for the composite robot that combines two or more robots; and a distribution unit that distributes the correction amount to each of the controllers.

[0011] (6) A robot control method according to one aspect of the present invention is a control method for a control device of a composite robot that uses a combination of two or more robots, each controlled by an individual controller, and the controller acquires the state of the robot and issues drive commands for the robot, wherein an acquisition unit acquires the robot command values ​​and the state of the robot held by each of the controllers, a calculation unit calculates an appropriate correction amount for the composite robot that combines two or more of the robots from the state of the robot, and a distribution unit distributes the correction amount to each of the controllers.

[0012] (7) A program according to one aspect of the present invention is a controller that acquires the state of a robot and issues drive commands for the robot, and causes a computer of a control device for a composite robot that uses a combination of two or more robots, each controlled by an individual controller, to acquire the robot command values ​​and the state of the robot held by each of the controllers, calculate an appropriate correction amount for the composite robot that combines two or more robots from the state of the robot, and distribute the correction amount to each of the controllers.

[0013] According to the aspects of the present invention, it is possible to improve work efficiency and task success rates.

[0014] FIG. 1 is a diagram showing a schematic configuration example of a robot control system according to a first embodiment. FIG. 2 is a diagram showing a configuration example of a robot control system according to the first embodiment. FIG. 3 is a flowchart of processing performed by a robot control device according to the first embodiment. FIG. 4 is a diagram showing a configuration example of a robot control system according to a second embodiment. FIG. 5 is a diagram showing examples of parts in the second embodiment and examples of behavior when parts are changed. FIG. 6 is a diagram showing an example of changing the degree of freedom according to a contact point, as viewed from the front of the robot. FIG. 7 is a flowchart of processing performed by a robot control system according to the second embodiment.

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scale of each component has been appropriately changed so that each component can be recognized. In all drawings used to explain the embodiments, components having the same function are designated by the same reference numerals, and repeated explanations will be omitted. In addition, "based on XX" in this application means "based on at least XX" and includes cases where the component is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where the component is based on XX after calculation or processing. "XX" is any element (for example, any information).

[0016] First Embodiment In this embodiment, two or more robots, each having a controller, are combined and controlled, as shown in Fig. 1. Fig. 1 is a diagram showing an example of the schematic configuration of a robot control system according to this embodiment. As shown in Fig. 1, the robot control system 1 includes, for example, a first robot 2-1, a second robot 2-2, a third robot 2-3, a first controller 3-1, a second controller 3-2, a third controller 3-3, a motion generator 4 (robot control device), and an integrated controller 5 (robot control device).

[0017] The first robot 2-1 is, for example, a hand. The second robot 2-2 is, for example, an arm. The third robot 2-3 is a cart. Each robot 2 (2-1, 2-2, 2-3) is equipped with sensors such as encoders and force sensors at joints, etc. It is also desirable to combine multiple robots 2 in a way that allows them to interact with each other when working together.

[0018] The first controller 3-1 acquires actual measurement values ​​detected by the sensors of the first robot 2-1 and controls the first robot 2-1 by outputting command values ​​in accordance with the control of the motion generator 4 and the integrated controller 5. The second controller 3-2 acquires actual measurement values ​​detected by the sensors of the second robot 2-2 and controls the second robot 2-2 by outputting command values ​​in accordance with the control of the motion generator 4 and the integrated controller 5. The third controller 3-3 acquires actual measurement values ​​detected by the sensors of the first robot 2-1 and controls the third robot 2-3 by outputting command values ​​in accordance with the control of the motion generator 4 and the integrated controller 5.

[0019] The motion generator 4 outputs the generated command values ​​to the first controller 3-1, the second controller 3-2, the third controller 3-3 and the integrated controller 5.

[0020] The integrated controller 5 acquires command values ​​from the motion controller 4, acquires actual measured values ​​from each robot 2 (2-1, 2-2, 2-3), and generates a correction amount (also called a "compensation amount") to be fed back using the acquired information. The integrated controller 5 distributes the generated correction amount and outputs it to the first controller 3-1, the second controller 3-2, and the third controller 3-3.

[0021] That is, in this embodiment, the individual controllers 3 (3-1, 3-2, 3-3) remain independent, and the integrated controller 5 acquires the command values ​​and actual measured values ​​of the entire system and distributes the correction amounts to each controller 3 (3-1, 3-2, 3-3).

[0022] [Configuration of Robot Control System] Next, an example configuration of the robot control system 1 will be described. Fig. 2 is a diagram showing an example configuration of the robot control system according to this embodiment. As shown in Fig. 2, the robot control system 1 includes, for example, a first robot 2-1, a second robot 2-2, a first control unit 3-1, a second control unit 3-2, and a control device 6 (robot control device).

[0023] The robot 2-n (n is an integer equal to or greater than 1) includes, for example, an actuator 21-n, a sensor 22-n, and a communication unit 23-n. The number of robots 2 may be two or more. The robot 2 may include a drive circuit that drives the actuator 21-n. The controller 3-n includes, for example, a control unit 31-n and a communication unit 32-n. The control device 6 includes, for example, a motion generator 4, an integrated controller 5, an acquisition unit 61, an output unit 62, and a storage unit 63. The integrated controller 5 includes, for example, a calculation unit 51 and a distribution unit 52.

[0024] The robot 2-n transmits and receives various information to and from the controller 3-n via a wired or wireless network NW-1, and transmits and receives various information to and from the control device 6 via a wired or wireless network NW-2.

[0025] [Functions of each device in the robot control system] Next, the functions of each device in the robot control system 1 will be described using Figure 2. (Robot 2) Actuators 21-n are attached to each joint. Sensors 22-n are, for example, six-axis sensors attached to the joints, tactile sensors attached to the fingers, force sensors, etc. The six-axis sensors detect forces along three axes (x, y, z) and moments along three axes (α, β, γ).

[0026] The communication unit 23-n transmits the actual measurement values ​​detected by the sensor 22-n to the controller 3-n corresponding to the robot 2-n and to the control device 6. The communication unit 23-n acquires the drive command values ​​output by the corresponding controller 3-n. Note that the data output by the robot 2-n includes identification information that can identify the robot 2-n. Also, the data acquired by the robot 2-n includes identification information that can identify that the data is addressed to the robot 2-n.

[0027] (Controller 3) The control unit 31-n generates a drive command value using the actual measurement value acquired by the communication unit 32-n, the command value output by the action generator 4, and the correction amount output by the integrated control device 5, and controls the action of the corresponding robot 2-n according to the generated drive command value. That is, each control unit 31 acquires the state of the respective robot (at least the actual measurement value), and outputs a drive command for the robot based on the acquired state of the robot.

[0028] The communication unit 32-n acquires actual measured values ​​from the corresponding robot 2-n. The communication unit 32-n outputs drive command values ​​to the corresponding robot 2-n. The communication unit 32-n acquires command values ​​from the motion generator 4. The communication unit 32-n acquires correction amounts from the integrated controller 5.

[0029] (Control device 6) The motion generator 4 generates a command value for each robot 2 in response to, for example, an instruction from an operator. The motion generator 4 outputs the generated command value to the corresponding controller 3-n. The motion generator 4 also outputs a command value for the entire system to the integrated controller 5. Note that the command value generated by the motion generator 4 is a command value that does not take into account the tracking ability of each robot 2.

[0030] The central controller 5 acquires the command values ​​held by each controller 3 and the state (actual measured values) of the robot from the information acquired by the acquisition unit 61. The central controller 5 may have some of the functions of the acquisition unit 61 and acquire the command values ​​held by each controller 3 and the state (actual measured values) of the robot. The central controller 5 may also include the motion generator 4. The central controller 5 acquires and uses actual measured values ​​that are not input to the motion generator 4, and therefore generates correction amounts taking into account the responsiveness of each robot 2, etc.

[0031] The calculation unit 51 of the central controller 5 calculates the amount of correction using actual measurement values ​​obtained from each robot 2 and command values ​​for the entire system obtained from the motion generator 4. The calculation unit 51 calculates the amount of correction by, for example, preferentially using an axis of the robot 2 with high tracking performance. Note that, for example, if there is a non-negligible amount of delay between the central controller 5, the controller 3, and the robot 2, the calculation unit 51 may calculate the amount of correction taking the amount of delay into account.

[0032] The distribution unit 52 of the central controller 5 distributes the calculated correction amount to each robot 2 and outputs it to the controller 3 via the output unit 62. The distribution unit 52 distributes the correction amount to each robot 2 based on, for example, the tracking performance and movable angle limit of each drive axis stored in the memory unit 63. In this way, the central controller 5 controls a combination of multiple robots 2. Note that the central controller 5 may also distribute the correction amount using information on the task, work content, and environment.

[0033] The acquisition unit 61 acquires actual measured values ​​from each robot 2. The acquisition unit 61 acquires command values ​​for the entire system from the motion generator 4.

[0034] The output unit 62 outputs the correction amount distributed to each robot 2 output by the integrated control unit 5 to the controller 3 .

[0035] The storage unit 63 stores, for example, programs, mathematical expressions, thresholds, identification information of the robot 2, identification information of the controller 3, etc. used by each part of the control device 6. The storage unit 63 stores, for each robot 2, the tracking performance and movable angle limit of each drive axis.

[0036] [Example of Processing Procedure] Next, a description will be given of an example of processing procedure performed by the robot control system 1. Fig. 3 is a flowchart of processing performed by the robot control device according to this embodiment.

[0037] (Step S1) The motion generator 4 of the control device 6 acquires, for example, an instruction input by an operator. Note that if an environmental sensor equipped with, for example, an image capture device is installed in the robot workspace, the motion generator 4 may estimate the task content based on an image captured by the environmental sensor and generate an instruction based on the estimated task content.

[0038] (Step S2) Based on the acquired instructions, the motion generator 4 generates command values ​​for each robot 2. Subsequently, the motion generator 4 outputs the generated command values ​​to each controller 3.

[0039] (Step S3) The controller 3 acquires an actual measurement value from the corresponding robot 2. The hourly speed value includes the state of the robot 2. For example, if the robot 2 is a hand, the state of the robot 2 includes information such as the finger joint angles and positions. For example, if the robot 2 is an arm, the state of the robot 2 includes information such as the arm joint angles, positions, and shoulder joint angles. The controller 3 uses the acquired actual measurement value and command value to generate a drive command value for the corresponding robot 2.

[0040] (Step S4) The controller 3 drives the corresponding robot 2 using the generated drive command value.

[0041] (Step S5) The integrated controller 5 acquires command values ​​for the entire system from the motion controller 4 and acquires actual measured values ​​from each of the robots 2.

[0042] (Step S6) The calculation unit 51 of the integrated controller 5 calculates the amount of correction using the acquired command values ​​for the entire system and the actual measurement values.

[0043] (Step S7) The distribution unit 52 of the integrated controller 5 distributes the acquired correction amount to the multiple controllers 3 based on the tracking performance and movable angle limit of each drive axis.

[0044] (Step S8) The controller 3 outputs the drive command value to which the acquired correction amount has been added to the controller 3.

[0045] (Step S9) The controller 3 drives the corresponding robot 2 using the generated drive command value.

[0046] <Example of Distribution of Correction Amount> Assume that the plurality of robots 2 are configured with, for example, a hand, an arm, and a cart. The integrated controller 5 performs feedback compensation by, for example, preferentially using arm axes with high tracking performance, rather than using cart axes with low tracking performance.

[0047] Assume that the multiple robots 2 are composed of, for example, a first robot 2-1, a second robot 2-2, and a third robot 2-3. Assume that one task is performed using these three robots 2. Assume also that the three robots 2-1, 2-2, and 2-3 have varying tracking capabilities. In such a case, the integrated controller 5 may improve tracking capabilities by, for example, allocating a larger correction amount to the robot 2 with poor tracking capabilities than to the robot 2 with good tracking capabilities.

[0048] The distribution priority and the like may be stored in advance in the storage unit 63, or may be input or set by the worker. Alternatively, the central controller 5 may set the priority and the like according to the task, work content, environment (for example, the relationship between the robot 2 and surrounding walls, etc.) based on images captured by an environmental sensor (not shown) installed in the robot work space. The central controller 5 may also change the distribution ratio sequentially according to the work status.

[0049] As described above, in this embodiment, an integrated controller 5 is added in addition to the individual controllers 3. The integrated controller 5 acquires information on command values ​​and actual measurement values ​​individually held by each controller 3. The integrated controller 5 also calculates an appropriate feedback correction amount for the entire system from actual machine information such as the tracking performance and movable angle limit of each drive axis. Furthermore, the integrated controller 5 distributes the calculated correction amount to each controller.

[0050] As a result, according to this embodiment, the position tracking capability of the end effector is improved for the entire system, thereby improving work efficiency and the success rate of tasks. For example, in a situation where the fingertip position of a robot 2 composed of a robot arm, hand, and cart is important, this embodiment can improve the success rate of tasks in which the fingertip position is important by using arm axes with high tracking performance preferentially for feedback compensation, rather than using cart axes with low tracking performance.

[0051] [Modification] The robot 2 may be operated, for example, by an operator operating an operation unit (not shown) while watching the robot 2, or the robot 2 may perform work automatically. Alternatively, a worker may wear an HMD (head mounted display) on his head and an operation unit such as a data glove on his hand to remotely control multiple robots 2.

[0052] Second Embodiment In this embodiment, for example, in a robot with a hand equipped with multiple fingers, the tip of the hand is not fixed as an end effector as in conventional technology, but the end effector is switched depending on the operation content and the scene. Thus, in this embodiment, an "end effector" is a part that can be used in a task. Furthermore, the usable part is, for example, the fingertips, wrist, arm, etc. of the end effector (hand) equipped in the robot.

[0053] [Configuration of Robot Control System] Next, an example configuration of the robot control system 1A will be described. Fig. 4 is a diagram showing an example configuration of the robot control system according to this embodiment. As shown in Fig. 2, the robot control system 1A includes, for example, a first robot 2-1, a second robot 2-2, a first control unit 3-1, a second control unit 3-2, a control device 6A (robot control device), and an environmental sensor 7.

[0054] The controllers 3 (3-1, 3-2, ...) include, for example, control units 31 (31-1, 31-2, ...) and communication units 32 (32-1, 32-2, ...). The control device 6A includes, for example, a motion generator 4, an integrated controller 5A, an acquisition unit 61, an output unit 62A, and a storage unit 63. The integrated controller 5A includes, for example, a calculation unit 51, a distribution unit 52, a part designation unit 53, a degree of freedom determination unit 54, and a joint angle calculation unit 55. The environmental sensor 7 includes, for example, a sensor 71 and a communication unit 72.

[0055] (Environmental Sensor 7) The environmental sensor 7 is installed, for example, in the robot workspace. The sensor 71 is, for example, an RGB-D camera, and acquires RGB (red, green, blue) information and depth information. The information is acquired, for example, at predetermined time intervals. The communication unit 72 outputs the detection value detected by the sensor 71 to the controller 3A (3A-1, 3A-2, ...). The data output by the environmental sensor 7 includes identification information that allows the environmental sensor 7 to be identified. The environmental sensor 7 may also output the detection value to the control device 6.

[0056] (Control device 6A) The part designation unit 53 designates the part and number of parts on a specific robot 2 to be operated among the robots 2. The part designation unit 53 may designate a part by acquiring information indicating the part input by the operator. Alternatively, the part designation unit 53 may designate a part by estimating the relationship between the operator's operation intention (see, for example, Japanese Patent Application No. 2022-006498) and the operation target object based on information acquired from the environmental sensor 7. The part to be set is not limited to one, and may be multiple (for example, the thumb and index finger). The part designation unit 53 designates a part for at least one of the multiple robots 2.

[0057] The degree of freedom determination unit 54 determines the degrees of freedom to be used for each designated part, for example, depending on the task and the environment. The degree of freedom determination unit 54 may determine the degrees of freedom based on, for example, the detection results of the environmental sensor 7 or the estimated operation content, or may determine the degrees of freedom based on the degrees of freedom input by the operator. For example, when the arm of the robot 2 performs a task while in contact with a wall, the contact with the wall may be detected based on the detection results of the force sensor or the environmental sensor 7 provided in the robot 2, or may be input by the operator. The degree of freedom determination unit 54 determines the degrees of freedom for at least one of the multiple robots 2.

[0058] The joint angle calculation unit 55 calculates the target joint angle value of the robot 2 moving a specific part on the robot 2, using as input the robot state quantity, the part on the robot 2 to be operated, the target position of the part on the robot 2 to be operated, and the degrees of freedom to be used.

[0059] The output unit 62A outputs the joint angle target value calculated by the joint angle calculation unit 55 to the controller 3A corresponding to the robot 2 being used, or to each of the controllers 3A.

[0060] (Controller 3A) In addition to the processing performed by the control unit 31, the control unit 31A generates a drive command by correcting, for example, the joint angle target value calculated by the joint angle calculation unit 55 as a command value with a correction amount. Note that the correction target and correction amount are not limited to the joint angle, and may be at least one of the joint angle, velocity, acceleration, and hand position. Note that the correction amount is used as the correction amount for the command value received by each control unit 31A from the motion generation unit 4. For example, if the correction amount is x' and the original command value is x, then x + x' becomes the corrected command value.

[0061] The communication unit 32A acquires, in addition to the information acquired by the communication unit 32, the first sensor value detected by the sensor 213 from the robot 2, the first sensor value detected by the environmental sensor 7, and the operation result of the operator. The communication unit 32A acquires the joint angle target value from the control device 6A. The communication unit 32A outputs the drive command calculated by the control unit 31A to the robot 2.

[0062] The part designation unit 53, the degree of freedom determination unit 54, and the joint angle calculation unit 55 may be included in each controller 3A. Alternatively, the part designation unit 53, the degree of freedom determination unit 54, and the joint angle calculation unit 55 may be included in the motion generator 4. In this case, the integrated controller 5A may be provided with the motion generator 4.

[0063] [Joint Angle Target Value] Next, a description will be given of an example of a method for generating a joint angle target value performed by the joint angle calculation unit 55. The joint angle calculation unit 55 includes, for example, an inverse kinematics calculation unit 552. A determiner 551 corresponds to the part designation unit 53 and the degree of freedom determination unit 54.

[0064] The decision unit 551 is, for example, a trained model. The first decision unit 551-1 corresponds to the part designation unit 53, and the second decision unit 551-2 corresponds to the degree of freedom determination unit 54. Note that the decision unit 551 outputs the end effector command value as it is, since the end effector command value is a command value indicating where to move the designated end effector part.

[0065] During learning, the first determinator 551-1 receives "information necessary to determine the location of the end effector (location on the robot)" and teacher data that is the correct output, and outputs "the location of the end effector." During use, the first determinator 551-1 receives "information necessary to determine the location of the end effector (location on the robot)" and outputs "the location of the end effector." Note that the "information necessary to determine the location of the end effector" is at least one of the following: objects in the vicinity of the robot, the work content, objects in the vicinity of each of the candidate locations, and position information or posture information of each of the candidate locations.

[0066] During learning, the second determiner 551-2 receives "information necessary to determine usable degrees of freedom" and teacher data that is the correct output, and outputs "usable degrees of freedom." During use, the second determiner 551-2 receives "information necessary to determine usable degrees of freedom" and outputs "usable degrees of freedom." The "information necessary to determine usable degrees of freedom" is at least one of the degrees of freedom of each candidate part, objects around the robot, the work content, objects around each candidate part, and position information or posture information of each candidate part.

[0067] The information required to determine the available degrees of freedom may be setting information set by the operator, or may be information based on the detection values ​​of the sensor 22 and the environmental sensor 7. Here, the available degrees of freedom refer to the degrees of freedom of the joints of the specified part, excluding, for example, directions in which movement is not possible due to the environment. For example, if there is a wall or other object around the specified part, moving the joint in that direction may result in a collision with the wall or other object. Therefore, the degree of freedom determination unit 34 sets the degrees of freedom based on, for example, the detection results of the environmental sensor 7.

[0068] The inverse kinematics calculation unit 552 calculates the objective function C of the following equation (1) using the input "end effector part, available degrees of freedom, and end effector command value." ａｌｌ is solved to calculate the joint angle target value.

[0069]

[0070] In addition, in the formula (1), W １ , W ２ is the weight of the joint angular acceleration task, and is expressed as a vector of degrees of freedom, so the degree of freedom available is expressed by the magnitude of the weight. １ and W ２ is a parameter to be updated. As shown in equation (1), the objective function is, for example, C １ and C related to joint angles ２ The objective function is constructed from the following. Each cost is expressed by a weighted norm, for example, and the dimension of the weight vector corresponds to the dimension of the cost. For example, C ２ The dimension of the weight vector corresponds to the joint degrees of freedom, and by changing this for each joint, the degrees of freedom to be actively used are controlled. １ and C related to joint angles ２ The objective function is constructed from the following. Each cost is expressed by a weighted norm, for example, and the dimension of the weight vector corresponds to the dimension of the cost. For example, C ２ The dimension of the weight vector corresponds to the joint degrees of freedom, and by changing this for each joint, the degrees of freedom to be actively used are controlled.

[0071] As shown in Equation (1), the end effector part is the objective function of the quadratic programming problem, and the available degrees of freedom are reflected in the weights of the quadratic programming problem. In other words, in this embodiment, mathematical optimization is used to solve the inverse kinematics, an objective function related to the end effector target is added according to the set part, and the weights of the objective function related to the joint angle are changed according to the determined degrees of freedom. Minimizing the norm of the first term in Equation (1) means placing (moving) the specified part to a certain position in three-dimensional space. Then, by changing the weights W1 and W2, the weights for the first and second terms are changed, thereby actively controlling the degrees of freedom.

[0072] In the control, the coordinate system used is, for example, a robot coordinate system. Positions in different coordinate systems are converted into the robot coordinate system using a well-known method.

[0073] [Examples of parts and examples of behavior when the part is changed] Next, examples of parts and examples of behavior when the part is changed will be described. Fig. 5 is a diagram showing examples of parts in this embodiment and examples of behavior when the part is changed. In Fig. 5, reference symbol g11 is the tip of the hand, reference symbol g12 is the hand 211, reference symbol g13 is the wrist, and reference symbol g14 is the arm 212.

[0074] Reference symbol g20 indicates an example of behavior when the tip g11 of the index finger is set as the part. In this case, the control device 6A issues a joint angle command centered on the tip g11 of the index finger, as shown by a circle g21. In this case, since the specified part is the fingertip, control is performed using the degrees of freedom of each of the arm and finger, for example.

[0075] Reference symbol g30 indicates an example of behavior when the wrist g13 is set as the part. In this case, the control device 6A issues a joint angle command centered on the wrist g13, as shown by the circle g31. In this case, since the specified part is the wrist, control is performed using the respective degrees of freedom of the arm and hand, for example, to achieve this.

[0076] [Example of changing the degree of freedom according to the contact point] Next, an example of changing the degree of freedom according to the contact point will be explained using Fig. 6. Fig. 6 is a front view of the robot, showing an example of changing the degree of freedom according to the contact point. Note that Fig. 6 is an example of the results of a simulation.

[0077] Image g50 in Fig. 6 is a diagram showing the degrees of freedom etc. before the contact point is changed. Image g60 in Fig. 6 is a diagram showing the degrees of freedom etc. after the contact point is changed. Reference symbol g71 indicates the fingertip joint, reference symbol g72 indicates the wrist joint, reference symbol g73 indicates the arm joint, reference symbol g74 indicates the first shoulder joint, and reference symbol g75 indicates the second shoulder joint. Reference symbol g76 indicates the contact point. Reference symbol g77 indicates the direction of the degrees of freedom at each joint.

[0078] Before contact, as shown by the dashed-line rectangle g51 in the diagram indicated by reference symbol g50, the degrees of freedom of the arm, hand, and fingers are all controlled and available without any restrictions. In contrast, after contact, as shown by the dashed-line rectangle g61 indicated by reference symbol g60, the area below the contact point g71 is used without moving the area above the contact point g71, so that the available degrees of freedom are restricted (applied). Note that such switching of the available degrees of freedom is determined by the degree-of-freedom determination unit 54, which determines whether or not an object has come into contact with the contact point based on the detection results of, for example, a sensor provided in the robot 2 or the environmental sensor 7.

[0079] In the above example, an example of controlling one manipulator has been described, but when controlling two manipulators, each manipulator is controlled by the above-mentioned control method. Note that the number of manipulators may be three or more, and in that case, each manipulator may be controlled by the above-mentioned control method.

[0080] [Example of Processing Procedure] Next, an example of processing procedure performed by the robot control system 1A will be described. Fig. 7 is a flowchart of processing performed by the robot control system according to this embodiment.

[0081] (Step S101) The acquisition unit 61 acquires the operation result input by the operator.

[0082] (Step S102) The acquisition unit 61 acquires the first sensor value (robot state quantity) detected by the sensor 22 from the robot 2. The acquisition unit 61 acquires the first sensor value detected by the environment sensor 7.

[0083] (Step S103) The part designation unit 53 estimates the relationship between the operator's operation intention and the operation target object based on, for example, information acquired from the environment sensor 7 and the operation result of the operator.

[0084] (Step S104) The part designation unit 53 sets the number and specific parts of the robot 2 to be operated based on the estimated operation intention, the detection results of the environmental sensor 7, etc. The part designation unit 53 may also designate the parts by acquiring information indicating the parts designated by the operator.

[0085] (Step S105) The degree of freedom determination unit 54 determines the degrees of freedom to be used among the joints of the specified part based on, for example, the specified part and the detection results of the environmental sensor 7.

[0086] (Step S106) The inverse kinematics calculation unit 552 of the joint angle calculation unit 55 calculates the joint angle target value using the "end effector portion, available degrees of freedom, and end effector command value" and equation (1). Subsequently, the control device 6A outputs the joint angle target value to the controller 3.

[0087] (Step S107) The control unit 31A of the controller 3A generates a drive command by correcting the target joint angle value as a command value using the correction amount. As described above, the correction target and correction amount are not limited to the joint angle, and may be at least one of the joint angle, velocity, acceleration, and hand position. The control unit 31A transmits the generated drive command to the robot 2 via the communication unit 32A.

[0088] As described above, in this embodiment, the position and number of end effectors are determined based on the judgment of a person or the control device 6A. Also, in this embodiment, the degrees of freedom that can be used are actively determined based on the judgment of a person or the control device 6A. Furthermore, in this embodiment, mathematical optimization is used to solve the inverse kinematics, an objective function related to the end effector target is added depending on the determined position and number, and the weight of the objective function related to the joint angle is changed depending on the determined degrees of freedom.

[0089] As a result, according to this embodiment, by changing the end effector part depending on the situation, it becomes possible to perform operations suited to the task or scene. Furthermore, according to this embodiment, even when the degrees of freedom are limited due to contact with the environment (for example, when contacting or being close to a wall), it is possible to continue moving with the remaining degrees of freedom, thereby expanding the range of applications.

[0090] In this embodiment, the robot 2 may also be remotely controlled by an operator.

[0091] In addition, a program for implementing all or part of the functions of the control device 6 (or 6A) or the control unit 3 in the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform all or part of the processing performed by the control device 6 (or 6A) or the control unit 3. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage provision environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain programs for a certain period of time, such as volatile memory (RAM) within computer systems that function as servers or clients when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Alternatively, some or all of these components may be realized by LSIs (Large Scale Integration) such as ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), GPUs (Graphics Processing Units), and SOCs (System On Chips), or by hardware (including circuitry), or may be realized by a combination of software and hardware.

[0092] The program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program that realizes part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0093] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0094] 1, 1A... robot control system, 2, 2-1, 2-2... robot, 3, 3-1, 3-2... 3A, 3A-1, 3A-2... control unit, 6, 6A... control device, 7... environment sensor, 21, 21-1, 21-2... actuator, 22, 22-1, 22-2... sensor, 23, 23-1, 23-2... communication unit, 31, 31-1, 31-2... 31A , 31A-1, 31A-2, ....Control unit, 32, 32-1, 32-2, ..., 32A, 32A-1, 32A-2, ....Communication unit, 4...Movement generator, 5...Integrated controller, 51...Calculation unit, 52...Distribution unit, 53...Part designation unit, 54...Degree of freedom determination unit, 55...Joint angle calculation unit, 61...Acquisition unit, 62...Output unit, 63...Memory unit, 71...Sensor, 72...Communication unit, NW, NW-1, NW-2...Network

Claims

1. A control system for a composite robot that uses a combination of two or more robots, each controlled by an individual controller, comprising: a controller that acquires the state of the robot and issues drive commands for the robot; and an integrated controller that controls the two or more robots in a combined state, wherein the integrated controller comprises: an acquisition unit that acquires the robot command values ​​and the state of the robot held by each of the controllers; a calculation unit that calculates an appropriate amount of correction for the composite robot that combines two or more robots from the state of the robot; and a distribution unit that distributes the amount of correction to each of the controllers.

2. The robot control system according to claim 1, wherein the calculation unit calculates the amount of correction from actual machine information on either the tracking performance of each drive axis of the robot or the movable angle limit.

3. A robot control system according to claim 1 or claim 2, wherein the calculation unit calculates the correction amount by preferentially using an axis of the robot with high tracking performance.

4. A robot control system as described in claim 1 or claim 2, wherein the integrated controller comprises: a part designation unit that designates a part on one of the plurality of robots to be operated; a degree of freedom determination unit that determines the degrees of freedom to be used among the plurality of robots; and a joint angle calculation unit that receives as input the state quantity of the robot, the part on the robot to be operated, a target position of the part on the robot to be operated, and the degrees of freedom to be used, and calculates a joint angle target for moving the part on the robot of the plurality of robots to a target value for the specific part.

5. A control device for a composite robot that uses a combination of two or more robots, each controlled by an individual controller, and that acquires the state of the robot and issues drive commands to the robot, the control device comprising: an acquisition unit that controls the combined state of the two or more robots and acquires the robot command values ​​and the state of the robots held by each of the controllers; a calculation unit that calculates, from the state of the robots, an appropriate correction amount for the composite robot that combines two or more robots; and a distribution unit that distributes the correction amount to each of the controllers.

6. A control method for a control device of a composite robot that uses a combination of two or more robots, each controlled by an individual controller, that acquires the robot's state and issues drive commands for the robot, wherein an acquisition unit acquires the robot command values ​​and the robot's state held by each of the controllers, a calculation unit calculates an appropriate correction amount for the composite robot that combines two or more of the robots from the robot's state, and a distribution unit distributes the correction amount to each of the controllers.

7. A program for causing a computer of a control device for a composite robot that uses a combination of two or more robots, each controlled by an individual controller, to acquire the robot command values ​​and the robot status held by each controller, calculate appropriate correction amounts for the composite robot that combines two or more robots, from the robot status, and distribute the correction amounts to each controller.

Images