Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program which enable an object to be grasped stably. The information processing device according to the present technology includes a detecting unit that detects slippage occurring at an object grasped by a grasping unit, and a collaboration control unit that collaboratively controls operations of a whole body of a robot having the grasping portion, in accordance with the slippage of the object. The collaboration control unit controls operations of the configurations making up the whole body of the robot, including at least a manipulator unit to which the grasping portion is attached, and a movement mechanism of the robot. The present technology is applicable to a mobile manipulator that grasps objects.

Description

[Technical Field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and particularly to an information processing device, an information processing method, and a program that enable an object to be stably grasped. [Background technology]

[0002] In recent years, research and development has been conducted on slip detection functions, which are often used in systems that control the gripping force used to grasp an object. The slip detection function is a function that detects slippage that occurs in an object being grasped by a hand or the like attached to a manipulator.

[0003] For example, Patent Document 1 describes a slippage detection system that acquires the pressure distribution when an object comes into contact with the curved surface of a fingertip and derives the critical amount of gripping force that prevents the object from slipping. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-297542 [Patent Document 2] Japanese Patent Application Publication No. 2019-018253 [Patent Document 3] Japanese Patent Application Laid-Open No. 2007-111826 [Patent Document 4] International Publication No. 2014 / 129110 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, when a human being feels that an object they are holding is about to slip, they do not simply tighten their grip, but also unconsciously move their entire body in coordination, such as by changing the position of their arms to reduce the slippage of the object or by moving their feet in the direction the object is being pulled. Humans also adaptively adjust the degree of movement of each part of their body depending on their surrounding environment and their own posture.

[0006] In systems such as robots, it is believed that objects can be grasped stably by coordinating the movements of the entire body to counteract any slippage that occurs in the object.

[0007] The present technology has been developed in light of these circumstances, and makes it possible to stably grasp an object. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present technology includes a detection unit that detects slippage that occurs in an object being grasped by a gripping unit, and a cooperative control unit that coordinates and controls the movement of the entire body of a robot having the gripping unit in accordance with the slippage of the object.

[0009] In one aspect of the present technology, slippage that occurs in an object being grasped by a gripping unit is detected, and the movement of the entire body of a robot having the gripping unit is controlled in a coordinated manner in accordance with the slippage of the object. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of an appearance of a robot according to an embodiment of the present technology; [Figure 2] FIG. [Figure 3] FIG. 2 is an enlarged view of a part of the fingertip. [Figure 4] FIG. 10 is a diagram showing how the device is gripped with fingertips. [Figure 5] 10A and 10B are diagrams illustrating an example of a method for measuring the displacement of a contact portion in the shear direction. [Figure 6]10A and 10B are diagrams illustrating an example of operation using the whole-body cooperative control function. [Figure 7] FIG. 1 is a diagram illustrating an example of coordinated control of the whole body. [Figure 8] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a robot. [Figure 9] FIG. 2 is a block diagram showing an example of a functional configuration of a robot. [Figure 10] FIG. 10 is a block diagram showing another example of the functional configuration of the robot. [Figure 11] 10 is a flowchart illustrating a process executed by a robot. [Figure 12] FIG. 10 is a diagram illustrating an example of cooperative control of the whole body when multiple robots cooperate to transport an object. [Figure 13] FIG. 10 is a block diagram showing an example of the functional configuration of robots when a plurality of robots cooperate to transport one object. [Figure 14] FIG. 10 is a diagram illustrating an example of the behavior of a leader and a follower. [Figure 15] FIG. 1 is a diagram illustrating an example of the configuration of a control system. [Figure 16] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present technology will be described in the following order. 1. Robot gripping function 2. Whole body coordinated control function 3. Robot Configuration 4. Robot behavior 5. Application Examples 6. Variations

[0012] <<1. Robot gripping function>> FIG. 1 is a diagram showing an example of the appearance of a robot 1 according to an embodiment of the present technology.

[0013] As shown in Figure 1, the robot 1 has a humanoid upper body and a wheeled mobility mechanism. A flattened spherical head 12 is provided on top of a torso 11. Two cameras 12A are provided in front of the head 12 in a shape that mimics human eyes.

[0014] Manipulator units 13-1 and 13-2, which are manipulators with multiple degrees of freedom, are provided at the upper end of the body unit 11. Hand units 14-1 and 14-2 are provided at the tips of the manipulator units 13-1 and 13-2, respectively. The robot 1 has the function of grasping an object with the hands 14-1 and 14-2.

[0015] Hereinafter, when it is not necessary to distinguish between the manipulator units 13-1 and 13-2, they will be collectively referred to as the manipulator unit 13. When it is not necessary to distinguish between the hand units 14-1 and 14-2, they will be collectively referred to as the hand unit 14. Other configurations that are provided in pairs will also be collectively described when appropriate.

[0016] A carriage-like mobile unit 15 is provided at the bottom end of the body 11 as a movement mechanism for the robot 1. The robot 1 can move by rotating wheels provided on the left and right sides of the mobile unit 15 or by changing the direction of the wheels.

[0017] In this way, the robot 1 is a so-called mobile manipulator that can freely lift and transport an object while grasping the object with the hand unit 14.

[0018] The robot 1 may be configured as a single-arm robot (with one manipulator unit 13) instead of the dual-arm robot shown in Fig. 1. Also, legs may be provided as a movement mechanism instead of the carriage-like moving body unit 15. In this case, the body unit 11 is provided on the legs.

[0019] FIG. 2 is an enlarged view of the hand unit 14. As shown in FIG.

[0020] 2, the hand unit 14 is a two-fingered gripper-type holding unit. Finger units 32A and 32B, which constitute two fingers, are attached to a base unit 31. The base unit 31 functions as a support unit that supports the multiple finger units 32.

[0021] Finger portion 32A is formed by connecting member 41A and member 42A, which are plate-like members having a predetermined thickness. Member 42A is provided on the tip side of member 41A attached to base portion 31. The connecting portion between base portion 31 and member 41A, and the connecting portion between member 41A and member 42A, each have a predetermined range of motion. Contact portion 43A, which comes into contact with an object to be grasped, is provided on the inside of member 42A. Member 42A and contact portion 43A form fingertip portion 51A.

[0022] Finger portion 32B has the same configuration as finger portion 32A. Member 42B is provided at the tip side of member 41B attached to base portion 31. The connecting portion between base portion 31 and member 41B, and the connecting portion between member 41B and member 42B each have a predetermined range of motion. Contact portion 43B is provided on the inside of member 42B. Member 42B and contact portion 43B form fingertip portion 51B.

[0023] Although the hand portion 14 is a two-fingered gripping portion, a multi-fingered gripping portion having a different number of fingers, such as a three-fingered or five-fingered type, may be provided.

[0024] Fig. 3 is an enlarged view of a part of fingertip 51. Fig. 3A shows the side of fingertip 51, and Fig. 3B shows the front (inner surface) of fingertip 51.

[0025] As indicated by hatching, a pressure distribution sensor 44 capable of detecting pressure at each position on the contact portion 43 is provided below the contact portion 43 .

[0026] The contact portion 43 is made of an elastic material such as rubber, and forms a semi-spherical flexible deformation layer.

[0027] Fingertip 51A and fingertip 51B have a parallel link mechanism. Fingertip 51A and fingertip 51B are driven so that their inner surfaces remain parallel. As shown in FIG. 4, object O, which is an object to be grasped, is grasped by being sandwiched between contact portion 43A on the fingertip 51A side and contact portion 43B on the fingertip 51B side, which are arranged so that their inner surfaces are parallel.

[0028] Because the contact portion 43 is made of an elastic material, it deforms in response to gravity acting on the object O. In the robot 1, the gripping state of the object is observed based on the pressure distribution detected by the pressure distribution sensor 44. For example, the displacement of the contact portion 43 in the shear direction is measured based on the pressure distribution.

[0029] The pressure distribution sensor 44, on the surface of which the flexible deformation layer is formed, functions as a slip sensor that calculates displacement in the shear direction.

[0030] FIG. 5 is a diagram showing an example of a method for measuring the displacement of the contact portion 43 in the shear direction.

[0031] The flexible deformation layer 61 shown in FIG. 5 corresponds to the contact portion 43 of the hand unit 14.

[0032] The left side of the top row of FIG. 5 shows the state where an object O is in contact with the flexible deformation layer 61 in the horizontal direction. On the other hand, the right side shows the state where a normal force F N is applied, and a horizontal shear force F x This shows how it has been added.

[0033] Shear force F x By applying the shear force F x The position of contact between the object O and the flexible deformation layer 61 is determined by the shear force F x The displacement u from the position before x Move only.

[0034] Displacement in the shear direction u xis expressed by the following equation (1) according to Hertz's contact theory.

number

[0035] In formula (1), R is the radius of curvature of the flexibly deformable layer 61. * is the composite transverse elastic modulus of the flexible deformation layer 61 and the object O, and E * is the composite longitudinal elastic modulus of the flexibly deformable layer 61 and the object O.

[0036] When the flexibly deforming layer 61 deforms in the shear direction, the pressure distribution at the contact portion 43 also changes, as shown in the lower part of Figure 5. Therefore, by detecting this pressure distribution, it is possible to measure the amount of displacement in the shear direction. For example, the amount of displacement in the shear direction is calculated based on the amount of movement of the CoP (Center of Pressure). The amount of displacement in the shear direction represents the amount of slippage of the object O. Furthermore, the shear direction represents the direction of slippage of the object O.

[0037] <<2. Whole-body coordination control function>> The robot 1 has a whole-body coordination control function that coordinates the movements of the whole body according to the measurement results of the slip sensors.

[0038] FIG. 6 is a diagram showing an example of operation by the whole-body cooperative control function.

[0039] The whole-body cooperative control by the whole-body cooperative control function is performed when the robot 1 is gripping an object O as shown in FIG.

[0040] For example, if object O slips to the left as shown by arrow #1 on the left side of Figure 6, the robot 1 moves the moving body unit 15 to the left as shown by arrow #11, and moves each of the manipulator units 13-1 and 13-2 to the left as shown by arrows #12 and #13, respectively, as shown on the right side of Figure 6, in order to counteract the slip.

[0041] In this way, the robot 1 controls the whole body movements to be coordinated according to the state of slippage of the object O. Although the description will be given mainly assuming that the components of the robot 1 whose movements are controlled by the whole body cooperative control function are the manipulator unit 13 and the moving body unit 15, the movements of other operable components may also be controlled.

[0042] That is, the whole body of the robot 1 includes components other than the manipulator unit 13 and the mobile body unit 15. For example, the movement of the waist, which is the connecting part between the torso unit 11 and the mobile body unit 15, may be controlled, or the movement of the head 12 may be controlled. Instead of the movement of the entire manipulator unit 13, the movement of only a part of the manipulator unit 13, such as the elbow or shoulder, may be controlled.

[0043] FIG. 7 is a diagram showing an example of coordinated control of the whole body.

[0044] 7 shows a top view of the robot 1 gripping a rectangular object O with the hands 14-1 and 14-2. For ease of explanation, the hands 14-1 and 14-2 are shown as rectangular, but in reality, the hands 14-1 and 14-2 are configured as two-fingered gripper-type hands as described above.

[0045] When object O slips, the white arrow As shown by #21, the displacement amount u1 is measured by the slip sensor of the hand unit 14-1, and as shown by the outline arrow #22, the displacement amount u2 is measured by the slip sensor of the hand unit 14-2.

[0046] The displacement amount measured by the slip sensor of the hand unit 14 provided on the manipulator unit i is expressed as u i Then, the control target value Δx of the moving body 15 is b is calculated by the following formula (2). As shown by the dashed arrow, the displacement u i is expressed in the hand coordinate system, and the control target value Δx of the moving body 15 is b is expressed in the mobile coordinate system.

[0047]

number

[0048] In equation (2), n indicates the number of manipulator units 13 (n=2 in FIG. 7), and w indicates a weight. Here, the weight w indicates the proportion of the moving body unit 15 in the control amount for counteracting slippage of the object O. The weight w is determined, for example, according to a priority indicating the degree to which the moving body unit 15 is preferentially operated in the coordinated control of the movement of the entire body.

[0049] For example, the higher the priority of the moving body unit 15, the greater the control amount relative to the control amount of the manipulator unit 13 becomes. b It is calculated as:

[0050] Control target value Δx b The function f(u1, ,u n ) is, for example, expressed as the following equation (3): i This is a function that calculates the average value of

[0051]

number

[0052] Depending on how it is operated, a function that calculates a weighted average value or a function that performs nonlinear calculations can be used as the function f(u1, ,u n ) can also be used.

[0053] Control target value Δx of the moving body unit 15 b After the calculation, the control target value Δx b and the displacement amount u measured by the slip sensor of each hand unit 14. i Based on this, the control target value Δx of the manipulator unit i is calculated. i The control target value Δx of the manipulator unit i is calculated. i is expressed by the following equation (4).

[0054]

number

[0055] As shown by the white arrow #31, the robot 1 adjusts the control target value Δx b In conjunction with the operation of the moving body unit 15, the robot 1 also operates the manipulator unit 13-1 by the control target value Δx1 and the manipulator unit 13-2 by the control target value Δx2.

[0056] As described above, in the robot 1, the displacement amount u measured by the slip sensor i The movement of the whole body including the manipulator unit 13 and the moving body unit 15 is controlled according to the state of slippage expressed by . In addition, the degree of cooperative control for each part of the whole body is changed by the weight w.

[0057] Normally, when a human being grasps an object that is about to slip, they do not simply tighten their grip, but rather unconsciously move their entire body in coordination, such as by changing the position of their arms to reduce the slippage of the object or by moving their feet in the direction the object is being pulled. Humans also adaptively adjust the degree of movement of each part of their body depending on the surrounding environment and their own posture. The robot 1's whole-body coordinated control function will enable it to achieve the same movements as humans.

[0058] The robot 1 can grip the object O stably by controlling the movement of the entire body so as to counteract the slippage of the object O.

[0059] <<3. Robot Configuration>> <Hardware configuration> FIG. 8 is a block diagram showing an example of the hardware configuration of the robot 1.

[0060] As shown in FIG. 8, the robot 1 is configured by connecting a body section 11, a head section 12, a manipulator section 13, a hand section 14, and a moving body section 15 to a control device 101.

[0061] The control device 101 is configured by a computer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, etc. The control device 101 is housed, for example, in the body part 11. The control device 101 executes a predetermined program using the CPU and controls the overall operation of the robot 1.

[0062] The control device 101 recognizes the environment around the robot 1 based on the detection results from sensors and images captured by cameras, and performs an action plan based on the recognition results. Each of the body 11, head 12, manipulator 13, hand 14, and mobile unit 15 is provided with various sensors and cameras.

[0063] The control device 101 generates a task for realizing a predetermined action and performs an operation based on the generated task. For example, the control device 101 performs an operation such as moving an object by operating the manipulator unit 13 while holding the object, or transporting the object by operating the mobile unit 15 while holding the object.

[0064] The control device 101 also detects the displacement u measured by the slip sensor. x Depending on the situation, it performs coordinated control of the entire body.

[0065] The manipulator unit 13 is provided with an encoder 71 and a motor 72. A combination of the encoder 71 and the motor 72 is provided for each joint that constitutes the manipulator unit 13.

[0066] The encoder 71 detects the amount of rotation of the motor 72 and outputs a signal representing the amount of rotation to the control device 101. The motor 72 performs a rotational movement around the axis of the joint. The rotation speed, amount of rotation, etc. of the motor 72 are controlled by the control device 101.

[0067] The hand unit 14 is provided with an encoder 81, a motor 82, and a pressure distribution sensor 44. A combination of the encoder 81 and the motor 82 is provided for each joint that constitutes the hand unit 14.

[0068] The encoder 81 detects the amount of rotation of the motor 82 and outputs a signal representing the amount of rotation to the control device 101. The motor 82 performs a rotational movement around the axis of the joint. The rotation speed, amount of rotation, etc. of the motor 82 are controlled by the control device 101.

[0069] The moving body unit 15 is provided with an encoder 91 and a motor 92 .

[0070] The encoder 91 detects the amount of rotation of the motor 92 and outputs a signal representing the amount of rotation to the control device 101. The motor 92 rotates around the axis of the wheel. The rotation speed, amount of rotation, etc. of the motor 92 are controlled by the control device 101.

[0071] Encoders and motors are also provided in the body 11 and the head 12. The encoders provided in the body 11 and the head 12 output signals indicating the amount of rotation of the motors to the control device 101. The motors provided in the body 11 and the head 12 are driven according to the control of the control device 101.

[0072] <Functional configuration> Single-arm example Fig. 9 is a block diagram showing an example of the functional configuration of the robot 1. Fig. 9 shows an example of the functional configuration when the robot 1 is a single-arm robot (when only the manipulator unit 13-1 is provided).

[0073] At least some of the functional units shown in FIG. 9 are realized by the CPU of the control device 101 executing a predetermined program.

[0074] As shown in FIG. 9, the robot 1 includes a slippage detection unit 151, a whole-body coordination control unit 152, a moving body control unit 153, a manipulator control unit 154, and a hand control unit 155.

[0075] The slippage detection unit 151 acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided in the hand unit 14-1, and calculates the displacement amount u in the shear direction. x is measured based on the pressure distribution. x represents the amount and direction of slippage experienced by an object.

[0076] The displacement amount u measured by the slip detection unit 151 x are supplied as slip detection results to the moving body target value calculation unit 162 and the manipulator target value calculation unit 163 of the whole body cooperative control unit 152, and to the hand control unit 155, respectively.

[0077] The whole-body coordination control unit 152 is composed of a weight determination unit 161 , a moving body target value calculation unit 162 , and a manipulator target value calculation unit 163 .

[0078] The weight determination unit 161 recognizes the situation around the robot 1, the situation of each part of the robot 1, the task execution situation, etc. based on information acquired by sensors and cameras provided in each part. The weight determination unit 161 determines a weight w according to the recognized situation and outputs it to the moving object target value calculation unit 162. Details of how the weight w is determined will be described later.

[0079] The moving object target value calculation unit 162 calculates the displacement amount u measured by the slip detection unit 151. x and the weight w determined by the weight determination unit 161, the calculation expressed by the above formula (2) is performed to obtain the control target value Δx b The control target value Δx calculated by the moving object target value calculation unit 162 is calculated. b is supplied to the manipulator target value calculation unit 163 and the moving body control unit 153.

[0080] The manipulator target value calculation unit 163 calculates the displacement amount u measured by the slippage detection unit 151. x and the control target value Δx calculated by the moving object target value calculation unit 162. b The control target value Δx1 calculated by the manipulator target value calculation unit 163 is supplied to the manipulator control unit 154.

[0081] The moving object control unit 153 calculates the control target value Δx calculated by the moving object target value calculation unit 162. b The moving body unit 15 is controlled based on the above.

[0082] The manipulator control unit 154 controls the manipulator unit 13-1 based on the control target value Δx1 calculated by the manipulator target value calculation unit 163.

[0083] The hand control unit 155 controls the gripping force of the hand unit 14-1. The gripping force of the hand unit 14-1 is determined based on, for example, the displacement u measured by the slippage detection unit 151. x is controlled according to

[0084] Dual-arm example Fig. 10 is a block diagram showing another example of the functional configuration of the robot 1. Fig. 10 shows an example of the functional configuration when the robot 1 is a double-arm robot (when manipulator units 13-1 and 13-2 are provided).

[0085] 10, the robot 1 is configured with slip detection units 151-1 and 151-2, a whole-body coordination control unit 152, a moving body control unit 153, manipulator control units 154-1 and 154-2, and hand control units 155-1 and 155-2. Descriptions that overlap with those of FIG. 9 will be omitted as appropriate.

[0086] The slippage detection unit 151-1 acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided in the hand unit 14-1, and measures the displacement amount u1 in the shear direction based on the pressure distribution. The displacement amount u1 measured by the slippage detection unit 151-1 is supplied as the slippage detection result to the moving object target value calculation unit 162 and the manipulator target value calculation unit 163-1 of the whole-body coordination control unit 152, and the hand control unit 155-1, respectively.

[0087] The slippage detection unit 151-2 acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided in the hand unit 14-2, and measures the displacement amount u2 in the shear direction based on the pressure distribution. The displacement amount u2 measured by the slippage detection unit 151-2 is supplied as a slippage detection result to the moving object target value calculation unit 162 and the manipulator target value calculation unit 163-2 of the whole-body coordination control unit 152, and the hand control unit 155-2.

[0088] The whole-body cooperative control unit 152 is made up of a weight determination unit 161, a moving object target value calculation unit 162, and manipulator target value calculation units 163-1 and 163-2.

[0089] The moving body target value calculation unit 162 calculates the control target value Δx of the moving body unit 15 based on the displacement amount u1 measured by the slippage detection unit 151-1, the displacement amount u2 measured by the slippage detection unit 151-2, and the weight w determined by the weight determination unit 161. b The control target value Δx calculated by the moving object target value calculation unit 162 is calculated. b are supplied to the manipulator target value calculation units 163-1 and 163-2 and the moving body control unit 153.

[0090] The manipulator target value calculation unit 163-1 calculates the displacement u1 measured by the slippage detection unit 151-1 and the control target value Δx calculated by the moving body target value calculation unit 162. b The control target value Δx1 calculated by the manipulator target value calculation unit 163-1 is supplied to the manipulator control unit 154-1.

[0091] The manipulator target value calculation unit 163-2 calculates the displacement u2 measured by the slippage detection unit 151-2 and the control target value Δx calculated by the moving body target value calculation unit 162. b The control target value Δx2 calculated by the manipulator target value calculation unit 163-2 is supplied to the manipulator control unit 154-2.

[0092] The manipulator control unit 154-1 controls the manipulator unit 13-1 based on the control target value Δx1 calculated by the manipulator target value calculation unit 163-1.

[0093] The hand control unit 155-1 controls the gripping force of the hand unit 14-1.

[0094] The manipulator control unit 154-2 controls the manipulator unit 13-2 based on the control target value Δx2 calculated by the manipulator target value calculation unit 163-2.

[0095] The hand control unit 155-2 controls the gripping force of the hand unit 14-2.

[0096] <<4. Robot Movement>> Here, the operation of the robot 1 having the above configuration will be described.

[0097] The processing executed by the robot 1 will be described with reference to the flowchart of Fig. 11. The processing of Fig. 11 starts, for example, when the hand unit 14 grasps an object.

[0098] In step S1, the slippage detection unit 151 acquires the pressure distribution of the fingertips of the hand unit 14 and calculates the displacement amount u in the shear direction. i Calculate.

[0099] In step S2, the weight determination unit 161 determines the weight w according to the situation around the robot 1, the situation of each part of the robot 1, the execution situation of the task, and the like.

[0100] In step S3, the moving body target value calculation unit 162 calculates the displacement amount u x and the weight w, the control target value Δx of the moving body 15 is calculated. b Calculate.

[0101] In step S4, the manipulator target value calculation unit 163 calculates the displacement amount u x and the control target value Δx b Based on this, the control target value Δx of the manipulator unit 13 is i Calculate.

[0102] In step S5, the robot 1 performs cooperative control of the whole body. For example, the moving body control unit 153 controls the control target value Δx b The manipulator control unit 154 controls the moving body unit 15 based on the control target value Δx i The hand control unit 155 controls the manipulator unit 13 based on the displacement amount u in the shear direction. x The gripping force of the hand unit 14 is controlled in accordance with the detected pressure.

[0103] The above process enables the robot 1 to stably grasp an object.

[0104] <<5. Application Examples>> <Weight change> ·Determining the weight w according to the surrounding environment The weight determination unit 161 determines the weight w according to the surrounding environment of the robot 1.

[0105] For example, when it is recognized that the moving body unit 15 will collide with an obstacle, the weight determining unit 161 determines the weight w to be a lower value in accordance with the distance information to the obstacle.

[0106] By determining the weight w to be a low value, the operation of the manipulator unit 13 is prioritized in the cooperative control of the whole body. That is, the operations of the manipulator unit 13 and the mobile body unit 15 are controlled so that the operation of the manipulator unit 13 cancels out more slippage than the operation of the mobile body unit 15.

[0107] This allows the moving body unit 15 to take priority in avoiding the obstacle.

[0108] Different values ​​may be determined as the weight w that defines the movement in each direction of the x-axis and y-axis of the moving body coordinate system.

[0109] ·Determination of weight w according to the manipulability of the manipulator The weight determination unit 161 determines the weight w according to the manipulability of the manipulator unit 13. The manipulability is an index that indicates the degree of mobility of each part of the manipulator unit 13.

[0110] For example, if there is a possibility that the manipulator unit 13 will assume a unique posture such as being fully stretched, the weight determining unit 161 determines the weight w to be a higher value.

[0111] By determining the weight w to be a high value, the movement of the mobile body unit 15 is given priority in the cooperative control of the whole body. That is, the movements of the manipulator unit 13 and the mobile body unit 15 are controlled so that the movement of the mobile body unit 15 cancels out as much slippage as possible.

[0112] This makes it possible to control the movement of the entire body of the robot 1 so that the manipulator unit 13 does not assume an unusual posture.

[0113] ·Determine the weight w according to the actuator output The weight determination unit 161 determines the weight w according to the output of the actuators provided in the manipulator unit 13 and the moving body unit 15 .

[0114] For example, when it is difficult for the manipulator unit 13 to perform quick movements, such as when the output of the actuator mounted on the manipulator unit 13 is low, and the object being grasped is heavy, the weight determination unit 161 determines the weight w to be a higher value.

[0115] By determining the weight w to be a high value, it becomes possible to cause the moving body unit 15 with a high actuator output to perform the operation of counteracting the slippage of the object with priority.

[0116] <Multiple mobile manipulators> A case where a plurality of robots 1 cooperate to transport one object will be described.

[0117] 12 is a diagram showing an example of cooperative control of the whole body when a plurality of robots 1 cooperate to transport one object. Descriptions that overlap with the description of FIG. 7 will be omitted as appropriate.

[0118] 12 shows a situation in which robot 1A and robot 1B cooperate to transport object O. Both robot 1A and robot 1B have the same configuration as the above-described robot 1. The configurations of robot 1A and robot 1B that correspond to the configuration of robot 1 will be described with the letters "A" and "B" respectively.

[0119] In FIG. 12, the left end of the object O is grasped by the hand units 14A-1 and 14A-2 of the robot 1A, and the right end of the object O is grasped by the hand units 14B-1 and 14B-2 of the robot 1B.

[0120] When the object O slips, the slip sensor of the hand unit 14A-1 detects a displacement amount u as shown by the white arrow #41. 11 Also, as shown by the white arrow #42, the displacement amount u is measured by the slip sensor of the hand part 14A-2. 12 is measured.

[0121] Robot 1A moves with a displacement u 11 and displacement u 12Based on this, the control target value Δx of the moving body unit 15A is 1b , the control target value of the manipulator unit 13A-1 and the control target value of the manipulator unit 13A-2 are calculated.

[0122] As shown by the white arrow #51, the robot 1A adjusts the control target value Δx 1b In addition, in conjunction with the operation of the moving body unit 15A, the robot 1A operates each of the manipulator units 13A-1 and 13A-2 by the control target values.

[0123] On the other hand, as shown by the white arrow #61, the displacement amount u 21 Also, as shown by the white arrow #62, the displacement amount u is measured by the slip sensor of the hand part 14B-2. 22 is measured.

[0124] Robot 1B moves with a displacement u 21 and displacement u 22 Based on this, the control target value Δx of the moving body part 15B is 2b , the control target value of the manipulator unit 13B-1 and the control target value of the manipulator unit 13B-2 are calculated.

[0125] As shown by the white arrow #71, the robot 1B adjusts the control target value Δx 2b In addition, in conjunction with the operation of the moving body unit 15B, the robot 1B operates each of the manipulator units 13B-1 and 13B-2 by the control target values.

[0126] FIG. 13 is a block diagram showing an example of the functional configuration of the robot 1 when a plurality of robots 1 cooperate to transport one object.

[0127] The robot 1A and the robot 1B each have the same configuration as the robot 1 described with reference to Fig. 10. Descriptions that overlap with the description of Fig. 10 will be omitted as appropriate.

[0128] The slippage detection unit 151-1 of the robot 1A acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided in the hand unit 14A-1, and calculates the displacement amount u in the shear direction. 11 is measured based on the pressure distribution.

[0129] The slippage detection unit 151-2 of the robot 1A acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided in the hand unit 14A-2, and calculates the displacement amount u in the shear direction. 12 is measured based on the pressure distribution.

[0130] The weight determination unit 161 of the robot 1A recognizes the situation around the robot 1A, the situation of each part of the robot 1A, the task execution situation, etc. based on information obtained by sensors and cameras installed in each part, and determines the weight w_1 according to the recognized situation.

[0131] The moving object target value calculation unit 162 of the robot 1A calculates the displacement amount u measured by the slippage detection unit 151-1. 11 , the displacement amount u measured by the slip detection unit 151-2 12 , and the weight w_1 determined by the weight determination unit 161, the control target value Δx 1b Calculate.

[0132] The manipulator target value calculation unit 163-1 of the robot 1A calculates the displacement amount u measured by the slippage detection unit 151-1. 11 and the control target value Δx calculated by the moving object target value calculation unit 162. 1b Based on this, a control target value for the manipulator unit 13A-1 is calculated.

[0133] The manipulator target value calculation unit 163-2 of the robot 1A calculates the displacement amount u measured by the slippage detection unit 151-2. 12 and the control target value Δx calculated by the moving object target value calculation unit 162. 1b Based on this, a control target value for the manipulator unit 13A-2 is calculated.

[0134] The moving body control unit 153 of the robot 1A calculates the control target value Δx calculated by the moving body target value calculation unit 162. 1b The moving body unit 15A is controlled based on the above.

[0135] The manipulator control unit 154-1 of the robot 1A controls the manipulator unit 13A-1 based on the control target value calculated by the manipulator target value calculation unit 163-1.

[0136] The hand control unit 155-1 of the robot 1A detects the displacement amount u measured by the slippage detection unit 151-1. 11 The gripping force of the hand unit 14A-1 is controlled in accordance with the detected value.

[0137] The manipulator control unit 154-2 of the robot 1A controls the manipulator unit 13A-2 based on the control target value calculated by the manipulator target value calculation unit 163-2.

[0138] The hand control unit 155-2 of the robot 1A detects the displacement amount u measured by the slippage detection unit 151-2. 12 The gripping force of the hand unit 14A-2 is controlled in accordance with the detected value.

[0139] On the other hand, the slippage detection unit 151-1 of the robot 1B acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided on the hand unit 14B-1, and calculates the displacement amount u in the shear direction. 21 is measured based on the pressure distribution.

[0140] The slippage detection unit 151-2 of the robot 1B acquires the pressure distribution represented by the sensor data output from the pressure distribution sensor 44 provided on the hand unit 14B-2, and calculates the displacement amount u in the shear direction. 22 is measured based on the pressure distribution.

[0141] The weight determination unit 161 of the robot 1B recognizes the situation around the robot 1B, the situation of each part of the robot 1B, the task execution situation, etc. based on information acquired by sensors and cameras installed in each part, and determines the weight w_2 according to the recognized situation.

[0142] The moving object target value calculation unit 162 of the robot 1B calculates the displacement amount u measured by the slippage detection unit 151-1. 21 , the displacement amount u measured by the slip detection unit 151-2 22 , and the weight w_2 determined by the weight determination unit 161, the control target value Δx 2b Calculate.

[0143] The manipulator target value calculation unit 163-1 of the robot 1B calculates the displacement amount u measured by the slippage detection unit 151-1. 21 and the control target value Δx calculated by the moving object target value calculation unit 162. 2b Based on this, a control target value for the manipulator unit 13B-1 is calculated.

[0144] The manipulator target value calculation unit 163-2 of the robot 1B calculates the displacement amount u measured by the slippage detection unit 151-2. 22 and the control target value Δx calculated by the moving object target value calculation unit 162. 2b Based on this, the control target value of the manipulator unit 13B-2 is calculated.

[0145] The moving body control unit 153 of the robot 1B calculates the control target value Δx calculated by the moving body target value calculation unit 162. 2b The moving body unit 15B is controlled based on the above.

[0146] The manipulator control unit 154-1 of the robot 1B controls the manipulator unit 13B-1 based on the control target value calculated by the manipulator target value calculation unit 163-1.

[0147] The hand control unit 155-1 of the robot 1B detects the displacement amount u measured by the slippage detection unit 151-1. 21The gripping force of the hand unit 14B-1 is controlled in accordance with the detected value.

[0148] The manipulator control unit 154-2 of the robot 1B controls the manipulator unit 13B-2 based on the control target value calculated by the manipulator target value calculation unit 163-2.

[0149] The hand control unit 155-2 of the robot 1B detects the displacement amount u measured by the slippage detection unit 151-2. 22 The gripping force of the hand unit 14B-2 is controlled in accordance with the detected value.

[0150] Note that distributed cooperative control may be performed among multiple mobile manipulators. For example, robot 1A operates as a leader that takes the lead in a task, and robot 1B operates as a follower that assists in the task. Each robot 1 changes its operation mode depending on whether it operates as a leader or a follower.

[0151] FIG. 14 is a diagram illustrating an example of the operation of the leader and the follower.

[0152] As shown in the upper part of Fig. 14, the manipulator unit 13 of the leader maintains the posture, and the hand unit 14 of the leader controls the gripping force so that the grasped object O does not slip. The mobile unit 15 of the leader operates according to the motion plan of the task.

[0153] The manipulator unit 13 of the follower performs a following operation using the measurement results of the slip sensor in accordance with the cooperative control of the whole body as described above. The hand unit 14 of the follower maintains the gripping force. The moving body unit 15 of the follower performs a following operation using the measurement results of the slip sensor in accordance with the cooperative control of the whole body as described above.

[0154] As described above, different actions are performed by the multiple robots 1 according to the assigned roles. This makes it possible to realize distributed cooperative control, such as transporting an object using multiple mobile manipulators in cooperation with each other.

[0155] Because multiple mobile manipulators cooperate to grasp a single object, it is possible to transport larger or heavier objects than if a single mobile manipulator were to grasp it. By simply setting a mode for each mobile manipulator, multiple mobile manipulators can cooperate to transport an object (cooperative transport).

[0156] <Weight change for multiple mobile manipulators> Even when multiple mobile manipulators are used, various forms of cooperative transportation can be realized by changing the weight w.

[0157] Example of prioritizing the trajectory of a moving object During cooperative transport by multiple mobile manipulators, the weight determination unit 161 of each mobile manipulator (robot) determines, for example, a lower value for the weight w. By determining a lower value for the weight w, the mobile body unit 15 operates to follow a trajectory that was set in advance when planning the route. The measurement results from the slip sensor in the hand unit 14 are used to control the manipulator unit 13. This allows cooperative transport to be achieved while the manipulator unit 13 absorbs vibrations and object misalignment during transport.

[0158] ·Determining the weight w according to the surrounding environment Even during cooperative transportation by a plurality of mobile manipulators, it is possible to operate the manipulator unit 13 with priority by changing the weight w according to the surrounding environmental conditions.

[0159] As described above, when one object is grasped by a plurality of robots 1, a weight w that is different from that when the object is grasped by a single robot 1 is determined.

[0160] <<6. Modifications>> The operations of gripping an object with the hand unit 14 and transporting an object gripped by the hand unit 14 may be controlled based on an operation by the user.

[0161] When one object is grasped by a plurality of robots 1, one robot may control the operation of the other robots.

[0162] <System configuration> FIG. 15 is a diagram illustrating an example of the configuration of a control system.

[0163] 15 is configured by providing a control device 101 as a device external to the robot 1. In this way, the control device 101 may be provided outside the housing of the robot 1.

[0164] Between the robot 1 and the control device 101 in FIG. 15, wireless communication is performed using a wireless LAN or a mobile communication system.

[0165] The robot 1 transmits various types of information to the control device 101, such as information indicating the state of the robot 1 and information indicating the detection results of the sensors. The control device 101 transmits information to the robot 1 for controlling the operation of the robot 1.

[0166] The robot 1 and the control device 101 may be directly connected as shown in A of Fig. 15, or may be connected via a network such as the Internet as shown in B of Fig. 15. The operations of multiple robots 1 may be controlled by one control device 101.

[0167] <About computers> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0168] FIG. 16 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0169] A CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are interconnected by a bus 204.

[0170] An input / output interface 205 is also connected to the bus 204. An input unit 206 including a keyboard, a mouse, etc., and an output unit 207 including a display, a speaker, etc. are connected to the input / output interface 205. In addition, a storage unit 208 including a hard disk, a nonvolatile memory, etc., a communication unit 209 including a network interface, etc., and a drive 210 that drives removable media 211 are also connected to the input / output interface 205.

[0171] In the computer configured as above, the CPU 201 loads a program stored in the storage unit 208 into the RAM 203 via the input / output interface 205 and the bus 204 and executes the program, thereby performing the series of processes described above.

[0172] The program executed by the CPU 201 is installed in the storage unit 208 by being recorded on a removable medium 211, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0173] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0174] <Other> In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device with multiple modules housed in a single housing, are both systems.

[0175] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0176] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0177] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0178] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0179] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0180] <Configuration combination example> The present technology can also be configured as follows.

[0181] (1) a detection unit that detects slippage occurring in an object being gripped by the gripping unit; a cooperative control unit that coordinates and controls the movement of the entire body of the robot having the gripping unit in response to the slippage of the object; An information processing device comprising: (2) The cooperative control unit controls the operation of each component constituting the entire body of the robot, including at least a manipulator unit to which the gripping unit is attached and a movement mechanism of the robot. The information processing device according to (1) above. (3) The cooperative control unit controls the movement of each component constituting the entire body of the robot based on a control target value indicating a control amount of the movement of each component so as to cancel out the slippage of the object. The information processing device according to (1) or (2). (4) The cooperative control unit calculates the control target value based on weights indicating the proportions of the control target values ​​of the components constituting the entire body of the robot. The information processing device according to (3) above. (5) The cooperative control unit calculates the control target value based on the weight according to a priority indicating a degree to which each component constituting the entire body of the robot is to be preferentially operated. The information processing device according to (4) above. (6) The cooperative control unit determines the weights according to the surrounding environment of the robot. The information processing device according to (4) or (5). (7) The cooperative control unit determines the weights according to manipulability indicating the degree of movement of each component constituting the entire body of the robot. The information processing device according to (4) or (5). (8) The cooperative control unit determines the weights in accordance with the outputs of the actuators mounted on each component of the robot. The information processing device according to (4) or (5). (9) When one object is grasped by a plurality of the robots, the cooperative control unit calculates the control target value based on a weight that is different from a weight when the object is grasped by one of the robots. The information processing device according to (4) or (5). (10) The multiple robots each perform different actions according to their assigned roles. The information processing device according to (9) above. (11) The gripping portion is a flexible deformation layer that comes into contact with the object when the object is grasped; a pressure distribution sensor that detects a distribution of pressure applied to the flexible deformation layer; and The detection unit detects slippage of the object based on the detection result of the pressure distribution sensor. The information processing device according to any one of (3) to (10). (12) The cooperative control unit controls the operation of one of the manipulator units based on a control target value of the one of the manipulator units. The information processing device according to (2) above. (13) The cooperative control unit controls the operations of the plurality of manipulator units based on respective control target values ​​of the plurality of manipulator units. The information processing device according to (2) above. (14) The cooperative control unit controls the operation of each component constituting the entire body of the robot based on the amount and direction of slippage of the object so as to counteract the slippage of the object. The information processing device according to any one of (1) to (13). (15) The information processing device Detecting slippage occurring in the object being gripped by the gripping unit; The robot controls the entire body of the robot having the gripping unit in a coordinated manner in response to the slippage of the object. Information processing methods. (16) On the computer, Detecting slippage occurring in the object being gripped by the gripping unit; The robot controls the entire body of the robot having the gripping unit in a coordinated manner in response to the slippage of the object. A program for executing a process. [Explanation of symbols]

[0182] 1 robot, 11 body part, 12 head part, 13 manipulator part, 14 hand part, 15 mobile part, 31 base part, 32 finger part, 41 member, 42 member, 43 contact part, 44 pressure distribution sensor, 51 fingertip part, 101 control device, 151 slip detection part, 152 whole body coordination control part, 153 mobile part control part, 154 manipulator control part, 155 hand control part

Claims

1. a detection unit that detects slippage occurring in an object being gripped by the gripping unit; a cooperative control unit that controls, in accordance with the slippage of the object, the operation of at least a manipulator unit to which the gripping unit is attached and a movement mechanism of the robot in a coordinated manner, among the components that constitute the entire body of the robot having the gripping unit; An information processing device comprising:

2. The cooperative control unit controls the operation of each component constituting the entire body of the robot based on a control target value indicating a control amount of the operation of each component so as to cancel out the slippage of the object. The information processing device according to claim 1 .

3. The cooperative control unit calculates the control target value based on weights indicating the proportions of the control target values ​​of the components that make up the entire body of the robot. The information processing device according to claim 2 .

4. The cooperative control unit calculates the control target value based on the weight according to a priority indicating a degree to which each component constituting the entire body of the robot is to be preferentially operated. The information processing device according to claim 3 .

5. The cooperative control unit determines the weights according to the surrounding environment of the robot.

5. The information processing device according to claim 3.

6. The cooperative control unit determines the weights according to the manipulability indicating the degree of movement of each component that constitutes the entire body of the robot.

5. The information processing device according to claim 3.

7. The cooperative control unit determines the weights in accordance with the outputs of actuators mounted on each component that constitutes the entire body of the robot.

5. The information processing device according to claim 3.

8. When one object is grasped by a plurality of the robots, the cooperative control unit calculates the control target value based on a weight that is different from a weight when the object is grasped by one of the robots.

5. The information processing device according to claim 3.

9. The multiple robots each perform different actions according to their assigned roles. The information processing device according to claim 8 .

10. The gripping portion is a flexible deformation layer that comes into contact with the object when the object is grasped; a pressure distribution sensor that detects a distribution of pressure applied to the flexible deformation layer; and The detection unit detects slippage of the object based on the detection result of the pressure distribution sensor.

10. The information processing device according to claim 2.

11. The cooperative control unit controls the operation of one of the manipulator units based on a control target value of the one of the manipulator units. The information processing device according to claim 1 .

12. The cooperative control unit controls the operations of the plurality of manipulator units based on respective control target values ​​of the plurality of manipulator units. The information processing device according to claim 1 .

13. The cooperative control unit controls the operation of each component constituting the entire body of the robot based on the amount and direction of slippage of the object so as to counteract the slippage of the object. The information processing device according to claim 1 .

14. The information processing device Detecting slippage occurring in an object being gripped by a gripping unit; In response to the slippage of the object, at least the operations of a manipulator unit to which the gripping unit is attached and a movement mechanism of the robot are controlled in a coordinated manner among the components constituting the entire body of the robot having the gripping unit. An information processing method including:

15. On the computer, Detecting slippage occurring in an object being gripped by a gripping unit; In response to the slippage of the object, at least the operations of a manipulator unit to which the gripping unit is attached and a movement mechanism of the robot are controlled in a coordinated manner among the components constituting the entire body of the robot having the gripping unit. A program for executing a process including:

Images