Robot control system, robot hand, robot control program, information management program, and information processing program

Abstract

The three finger parts are arranged on the holding part, the multiple adsorption pads are installed on the palm side of the holding part and the three finger parts, the adsorption pads adsorb goods through air adsorption structures, and the goods can be grabbed by bending the three finger parts. By mounting a palm sensor including a high-resolution camera and a MoPU on the palm side and mounting the grip section of the aforementioned configuration on the arm section of the humanoid robot, articles can be reliably sorted by the suction surface, and goods can be conveyed from the grip section without falling even if the humanoid robot operates quickly.

Description

Technical Field

[0001] The present disclosure relates to a control system for a robot, a robot hand, a control program for a robot, an information management program, and an information processing program. Background Art

[0002] On a production line in a factory, humanoid robots are used for automatically performing operations. Japanese Patent Laid-Open No. 2019-093506 describes the posture control of a humanoid robot.

[0003] In addition, WO2011 / 001569 describes a robot arm driven by an elastomer actuator and having a plurality of joints. A control unit controls a tip support member that supports the robot arm due to contact with a support surface provided at the tip of the robot arm and the force of contact between the tip support member and the support surface, and at the same time controls the position and posture of the tip of the robot arm.

[0004] However, in the conventional sorting operation performed by a humanoid robot in a warehouse, for example, when sorting goods (items with different shapes, weights, hardnesses, and fragilities such as shampoo, conditioner, cosmetics, toothpaste, instant noodles, and packaged snacks) from a shelf on which the goods are displayed and packaging them in a prescribed package (such as a box), the current situation is to rely on human hands.

[0005] In addition, even if an attempt is made to make the structure of the holding part of the robot correspond to a finger shape, the productivity is low because the movements of the fingers and the arm are slow. Summary of the Invention

[0006] In view of the above facts, an object of the present disclosure is to obtain a control system for a robot, a robot hand, a control program for a robot, an information management program, and an information processing program that can perform a series of operations of quickly and reliably confirming the position of a good, grasping, and holding the good by a holding part.

[0007] The control system for a robot according to the present disclosure is a control system for a robot that can confront a holding part with an object to hold the object. The holding part includes a palm part that serves as a basis for holding the object and a plurality of finger parts that extend radially from the palm part. The robot control system has: a suction pad provided on the palm part and the finger parts for sucking the object; a palm sensor part provided on the palm part for detecting object information including the shape and position of the object; and a control part that, based on the detection result of the palm sensor part, selects any one of a first function of sucking by the suction pad, a second function of grasping by the palm part, and a third function of using both the first function and the second function as a function of holding the object, and controls the operation of holding the object.

[0008] According to the present disclosure, based on the detection result of the palm sensor, the control unit selects any one of the first function adsorbed by the adsorption pad, the second function grasped by the palm part, and the third function that combines the first function and the second function as the function for holding the object, and controls the action of holding the object.

[0009] Thereby, a series of operations of position confirmation, grasping, and holding of the object (hereinafter sometimes referred to as goods) can be quickly and reliably performed by the holding part.

[0010] In the present disclosure, it is characterized in that the palm sensor unit includes a camera that captures an image of the object to identify the type of the object and a motion processing unit that determines the position of the object.

[0011] The camera identifies the captured object based on the captured image information. That is, it has the function of obtaining information for determining the type (shape, size, hardness, etc.) of the object.

[0012] The motion processing unit (MoPU) outputs vector information of the motion of the points representing the position where the object exists along the specified coordinate axes as motion information. That is, the motion information output from the MoPU only includes information representing the motion (moving direction and moving speed) of the center point (or center of gravity point) of the object on the coordinate axes (x-axis, y-axis, z-axis). That is, the trajectory when the holding part approaches the object can be guided with high precision.

[0013] In the present disclosure, it is characterized in that the control unit allows the selection of the first function when there is a surface on which the object can be adsorbed, and allows the selection of the second function when the holding force of the object is prioritized.

[0014] The object can be held with an appropriate function (the first function and / or the second function).

[0015] In the present disclosure, it is characterized in that it further has an execution unit that, when multiple robots exist within a specified range of each other, determines whether there is a coincidence point in their moving paths, and when it is determined that there is a coincidence point in the moving paths, selects a robot to be avoided from contact among the multiple robots, and causes the selected robot to be avoided from contact to execute contact avoidance.

[0016] When multiple robots travel according to their respective travel programs, they may come into contact with each other. In the travel program, there are cases where contact avoidance is considered, but the holding state of the object by the holding part is not considered, and due to the avoidance behavior, the object may fall off.

[0017] Therefore, when multiple robots are within a specified range of each other, the execution unit determines whether there are points of coincidence in their respective movement paths. If it is determined that there are points of coincidence in the movement paths, the robot to be the contact avoidance target is selected from among the multiple robots, and the selected robot to be the contact avoidance target is made to perform contact avoidance.

[0018] Thereby, contact between robots can be avoided.

[0019] In the present disclosure, it is characterized in that the execution unit selects, based on information mutually transmitted and received between the multiple robots, a robot with a lighter load for holding the object, and when there is no difference in the holding state of the object, selects according to the priority order of predetermined identification information.

[0020] The robot to be the contact avoidance target can be appropriately selected.

[0021] In the present disclosure, it is characterized in that, as the contact avoidance behavior of the robot to be the contact avoidance target, the execution unit selects and executes a detour action of deviating from the normal travel route and making a detour, and a deceleration or pause action of decelerating or pausing before contact while maintaining the normal travel route.

[0022] An avoidance action can be selected in response to the situation.

[0023] The robot control program according to the present disclosure is a robot control program that causes a computer to function as the above control unit.

[0024] The control system of the robot according to the present disclosure is a control system of a robot capable of confronting an object with a holding unit to hold the object. The holding unit includes a palm part that serves as a basis for holding the object and a plurality of finger parts that extend radially from the palm part. The robot control system has: an adsorption pad provided on the palm part and the finger parts for adsorbing the object; a palm sensor part provided on the palm part for detecting object information of the object including the shape and position; a center-of-gravity position recognition part capable of recognizing at least the center-of-gravity position of the object; and a control part that, based on the detection result of the palm sensor part and the recognition result of the center-of-gravity position recognition part, selects any one of a first function of adsorbing by the adsorption pad, a second function of grasping by the palm part, and a third function of using both the first function and the second function as the function for holding the object, and controls the action of holding the object.

[0025] According to the present disclosure, the control part selects, based on the detection result of the palm sensor and the recognition result in the center-of-gravity position recognition part, any one of a first function of adsorbing by the adsorption pad, a second function of grasping by the palm part, and a third function of using both the first function and the second function as the function for holding the object, and controls the action of holding the object.

[0026] Thus, a series of operations such as quickly and reliably confirming the position of the object, grasping, and holding it can be performed by the holding part.

[0027] In the present disclosure, it is characterized in that the center-of-gravity position recognition part includes: a state sensor, which is provided together with the palm sensor part and at least detects the internal state of the object; and an analysis part, which analyzes the center-of-gravity position of the object based on the internal state detected by the state sensor.

[0028] In each robot, center-of-gravity position information can be obtained.

[0029] In the present disclosure, it is characterized in that the center-of-gravity position recognition part functions as an information acquisition part, and the information acquisition part acquires the center-of-gravity position information of the object bound and sent from an analysis device that analyzes the center-of-gravity position of the object.

[0030] The center-of-gravity position information of the object can be uniformly managed.

[0031] The control system of the robot according to the present disclosure is a control system of a robot that performs an operation of holding an object by a holding part, and has: an information processing device, which is mounted on the robot and controls the actions of the robot; and an information management server, which uniformly manages information related to the holding of the object by a plurality of the robots. The information management server has: an analysis part, which generates cubic data based on the object waiting to be held by the robot; and a sending part, which binds the cubic data generated by the analysis part to the ID information for identifying the object and stores it in a database, and sends out the cubic data in response to a request based on the ID information from the information processing device. The information processing device has: a cubic data acquisition part, which acquires the cubic data from the database based on the ID information acquired by the palm sensor part for the information management server; and a setting part, which sets the holding point of the object according to the cubic data acquired by the cubic data acquisition part.

[0032] According to the present disclosure, the information processing device mounted on the robot and controlling the actions of the robot and the information management server that uniformly manages information related to the holding of the object by a plurality of robots cooperate to control the actions of the robot (especially the holding action of the object).

[0033] That is, in the information management server, cubic data is generated based on the object waiting to be held by the robot, the generated cubic data is bound to the ID information for identifying the object and stored in the database, and the cubic data is sent out in response to a request based on the ID information from the information processing device.

[0034] In addition, in the information processing device, based on the detection result of the palm sensor unit, the ID information and the confrontation state information of the object are received. By requesting the cube data from the information management server based on the ID information, the cube data is obtained from the database, and the gripping point of the object is set based on the obtained cube data.

[0035] Thereby, a series of operations such as position confirmation, grasping, and holding of the object can be quickly and reliably performed by the holding unit.

[0036] The control system of the robot according to the present disclosure is a control system of a robot including a holding unit, an adsorption pad, and a palm sensor unit that performs an operation of holding an object by the holding unit, wherein: the holding unit holds the object; the adsorption pad is provided on the holding unit and adsorbs the object; the palm sensor unit is provided on the holding unit and detects object information including the shape and position of the object. The robot control system includes: an information processing device mounted on the robot and controlling the operation of the robot; and an information management server that uniformly manages information related to the holding of the object by a plurality of the robots. The information management server includes: an analysis unit that divides the object to be held by the robot into a plurality of blocks with coordinates determining their relative positions to each other, and generates cube data recording attribute information including the internal information of the object for each of the plurality of blocks; a storage unit that binds the cube data generated by the analysis unit to the ID information identifying the object and stores it in a database; and a sending unit that retrieves and sends the cube data corresponding to the request based on the ID information from the information processing device. The information processing device includes: a palm sensor information receiving unit that receives the ID information and the confrontation state information of the object based on the detection result of the palm sensor unit; a cube data obtaining unit that requests the cube data from the information management server based on the ID information and the confrontation state information received by the palm sensor information receiving unit, and obtains the cube data from the database; and a setting unit that sets the gripping point of the object according to the cube data obtained by the cube data obtaining unit.

[0037] According to the present disclosure, the information processing device mounted on the robot and controlling the operation of the robot and the information management server that uniformly manages information related to the holding of the object by a plurality of robots cooperate to control the operation of the robot (especially the object holding operation).

[0038] That is, in the information management server, an object to be grasped by the robot is subdivided into a plurality of blocks with coordinates determining their relative positions to each other. Cubic data recording attribute information including the internal information of the object is generated for each of the plurality of blocks. The generated cubic data is bound to the ID information for identifying the object, stored in a database, and retrieved and sent out in response to a request based on the ID information from the information processing device.

[0039] In addition, in the information processing device, based on the detection result of the palm sensor unit, the ID information and the confrontation state information of the object are received. The information management server is requested for cubic data based on the ID information and the confrontation state information, thereby obtaining the cubic data from the database, and the grasping point of the object is set based on the obtained cubic data.

[0040] Thus, a series of operations of position confirmation, grasping, and holding of the object can be quickly and reliably performed by the holding part.

[0041] In the present disclosure, the information processing device further has an action control unit. Based on the detection result of the palm sensor unit, any one of a first function adsorbed by the adsorption pad, a second function grasped by the holding part, and a third function combining the first function and the second function is selected as the function for holding the object, and the action of holding the object is controlled.

[0042] Based on the detection result of the palm sensor, the action control unit selects any one of a first function adsorbed by the adsorption pad, a second function grasped by the holding part, and a third function combining the first function and the second function as the function for holding the object, and controls the action of holding the object.

[0043] Thus, a series of operations of position confirmation, grasping, and holding of the object can be quickly and reliably performed by the holding part.

[0044] The information management program according to the present disclosure is characterized in that it causes a computer to operate as each part of the above information management server.

[0045] The information processing program according to the present disclosure is characterized in that it causes a computer to operate as each part of the above information processing device.

[0046] The robot hand according to the present disclosure includes: a holding part composed of a plurality of finger-like structures for grasping and holding an object; an adsorption pad provided on the holding part to confront the object to adsorb the object; and a film-like membrane body spanned between the plurality of finger-like structures of the holding part and configured to be deformable in response to the action of the holding part.

[0047] According to the present disclosure, the film is provided between the plurality of finger-shaped structures of the gripping part, and is formed to be freely deformable during the operation of the gripping part, so as to wrap the object in conjunction with the gripping operation of the gripping part.

[0048] That is, the film body has a function of assisting the gripping part in gripping the object, preventing the object from falling, etc. Therefore, the gripping part can quickly and reliably perform a series of operations such as position confirmation, grabbing and gripping of the goods.

[0049] The control system of the robot involved in the present disclosure is a robot control system that can confront an object to hold the object, and is characterized in that it comprises: a holding part, which is composed of a plurality of finger-like structures for grasping and holding the object; a thin film-like membrane, which is arranged in the holding part, is spanned between the plurality of finger-like structures in the holding part, and is formed to be deformable in response to the movement of the holding part; a palm sensor part, which is arranged in the holding part and detects object information of the object including shape and position; and a control part, which controls the movement of holding the object based on the detection result of the palm sensor part.

[0050] According to the present disclosure, the robot includes a gripping portion, an adsorption pad, a film, and a palm sensor portion, and the control portion controls the action of gripping the object based on a detection result of the palm sensor portion.

[0051] Here, the object is wrapped in conjunction with the gripping action of the control unit.

[0052] As a result, the gripping unit can quickly and reliably perform a series of operations including position confirmation, grabbing, and gripping of the cargo.

[0053] In addition, the above-disclosed summary does not list all the necessary features of the present disclosure. In addition, sub-combinations of these feature groups may also constitute inventions.

[0054] Effects of the Invention

[0055] As described above, according to the present disclosure, an effect is achieved in which a series of operations including position confirmation, grasping, and grasping of cargo can be performed quickly and reliably by the grasping unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a front view of the humanoid robot involved in the first embodiment.

[0057] Figure 2 It is a side view of the humanoid robot according to the first embodiment.

[0058] Figure 3A This is a front view of the palm side of the grip portion according to the first embodiment.

[0059] Figure 3B It is a perspective view of the adsorption pad installed on the holding part.

[0060] Figure 4A It is a perspective view of the holding part related to the first embodiment.

[0061] Figure 4B It is a perspective view of the holding part related to the modified example.

[0062] Figure 5 It is a diagram schematically showing an example of the functional structure of a humanoid robot.

[0063] Figure 6 It is a diagram schematically showing an example of a processing routine executed by an information processing device.

[0064] Figure 7 It shows the Figure 1 Flowchart of the holding control sequence when the holding part holds an object in linkage with the overall movement of the humanoid robot.

[0065] Figure 8 Diagram schematically showing an example of computer hardware functioning as an information processing device.

[0066] Figure 9 It is a flowchart showing the travel control routine between humanoid robots related to the second embodiment.

[0067] Figure 10A It is a state transition diagram showing the action states of the face-to-face contact avoidance program mode in the travel control between humanoid robots related to the second embodiment.

[0068] Figure 10B It is a state transition diagram showing the action states of the face-to-face contact avoidance program mode in the travel control between humanoid robots related to the second embodiment.

[0069] Figure 10C It is a state transition diagram showing the action states of the face-to-face contact avoidance program mode in the travel control between humanoid robots related to the second embodiment.

[0070] Figure 10D It is a state transition diagram showing the action states of the face-to-face contact avoidance program mode in the travel control between humanoid robots related to the second embodiment.

[0071] Figure 11A It is a state transition diagram showing the action states of the collision course phenomenon contact avoidance program mode in the travel control between humanoid robots related to the second embodiment.

[0072] Figure 11BIt is a state transition diagram showing the operation states of a collision course phenomenon contact avoidance program mode in the travel control between humanoid robots according to the second embodiment.

[0073] Figure 11C It is a state transition diagram showing the operation states of a collision course phenomenon contact avoidance program mode in the travel control between humanoid robots according to the second embodiment.

[0074] Figure 12 It is a front view of the palm side of the holding part according to the third embodiment.

[0075] Figure 13A It is a perspective view of the holding part according to the third embodiment.

[0076] Figure 13B It is a perspective view of the holding part according to the modification example.

[0077] Figure 14 It is a comparison diagram showing the correspondence between the visible image based on the cargo type and the detection image of the X-ray sensor.

[0078] Figure 15 It is a diagram schematically showing an example of the functional structure of a humanoid robot.

[0079] Figure 16 It shows the Figure 1 flowchart of the grasping control sequence when the cargo is grasped by the holding part in linkage with the overall movement of the humanoid robot.

[0080] Figure 17 It is a front view of a conveyor device provided with an X-ray analysis device for analyzing the center of gravity position of the cargo at the sorting station according to the modification example.

[0081] Figure 18A It is a schematic structure diagram visually showing the cube data concept applied in the fourth embodiment.

[0082] Figure 18B It is a chart showing the attribute information classified by each coordinate of each block.

[0083] Figure 19 It is a functional block diagram dedicated to the cargo grasping point setting function within the processing functions executed by the information processing device of the functional structure (control system) of the humanoid robot according to the fourth embodiment.

[0084] Figure 20 It is a schematic diagram of an analysis device provided at the cargo consolidation station according to the fourth embodiment and a functional block diagram of an information management server for managing cube data generated based on the information received by the analysis device.

[0085] Figure 21 is a flowchart showing the gripping control sequence when the gripping part holds the goods in linkage with the overall movement of the humanoid robot shown in Figure 1 .

[0086] Figure 22 is a control flowchart showing the details of the goods gripping point setting processing subroutine in step 268 of Figure 21 .

[0087] Figure 23 is a front view of the palm side of the gripping part according to the fifth embodiment.

[0088] Figure 24A is a perspective view of the gripping part according to the fifth embodiment.

[0089] Figure 24B is a perspective view of the gripping part according to the modification example.

[0090] Figure 25 is a perspective view showing the state of holding the goods in the gripping part (humanoid) according to the fifth embodiment.

[0091] Figure 26A is a front view of the gripping part having a structure in which the film body can be extended and retracted according to the modification example.

[0092] Figure 26B is a front view of the gripping part having a structure in which the film body can be extended and retracted according to the modification example.

[0093] Figure 26C is a flowchart showing a control routine for extending and retracting the film body. Detailed Embodiment

[0094] Hereinafter, the present disclosure will be described by way of embodiments of the invention. However, the following embodiments do not limit the invention described in the claims. In addition, not all of the feature combinations described in the embodiments are necessary for the solution means of the invention.

[0095] [First Embodiment]

[0096] Figure 1 is a front view of the humanoid robot according to the first embodiment. As Figure 1 shown, the humanoid robot 1 according to the present embodiment includes an upper body part 2, a leg part 3, and a connecting part 4 that rotatably connects the upper body part 2 to the leg part 3. For example, it is arranged on the production line of a factory and performs operations on objects such as those on the line or on the ground of a shelf on which objects to be sorted (goods to be sorted, etc.) are displayed. In addition, the operations include, in addition to gripping the goods 100 from the shelf (refer to Figure 2In addition to sorting as described above, it also includes packaging the held goods 100 in a specified frame (such as a cardboard box).

[0097] The upper body part 2 has two arm parts 5 and 6. The arm parts 5 and 6 are rotatably mounted on the left and right of the upper body part 2. In addition, a gripping part 20 (described in detail later) for gripping an object is mounted at the front ends of the arm parts 5 and 6. In addition, the number of arm parts is not limited to two, and may be one or three or more.

[0098] Two wheels 7 and 8 of the leg part 2 are mounted at its lower part and can move on the ground where the humanoid robot 1 is placed.

[0099] The connecting part 4 rotatably connects the upper body part 2 and the foot part 3. Therefore, the upper body part 2 can lean forward and backward relative to the foot part 3. Therefore, as Figure 2 shown, the humanoid robot 1 according to the present embodiment can lean the upper body part 2 forward relative to the foot part 3 to pick up the goods placed on the shelf, the goods 100 placed on the ground, and the goods 100 that fall to the ground during the operation.

[0100] In addition, the foot part 2 has a balancing function for preventing the humanoid robot 1 from falling when the upper body part 2 leans forward or backward relative to the foot part 3, or when the humanoid robot 1 moves.

[0101] In addition, as Figure 1 shown, the connecting part 4 has a function of being able to change the distance between the upper body part 2 and the foot part 3. Therefore, it is possible to adjust the position of the upper body part 2 relative to the foot part 3 in the vertical direction as shown by the arrow A in a manner suitable for the height of the workbench on the production line.

[0102] In addition, the humanoid robot 1 according to the present embodiment is controlled to drive by a control system 10 actually installed in the humanoid robot 1.

[0103] (Structure of the gripping part 20)

[0104] As Figure 3A shown, the gripping part 20 mounted at the front ends of the arm parts 5 and 6 is configured as a hand structure similar to that of a human, and the gripping part 20 is rotatably mounted on the arm parts 5 and 6 (Intelligent Hand System).

[0105] The gripping part 20 shown in the present embodiment has three finger parts 22A, 22B, and 22C each having a plurality of joints. In the present embodiment, the three-finger structure is as Figure 4A shown, and is the same for both the right hand and the left hand, but may also be a five-finger structure as Figure 4B shown.

[0106] A plurality (three in the present embodiment) of suction pads 24 are mounted on the palm side 20A of the grip portion 20.

[0107] In addition, on the three finger portions 22A, 22B, and 22C, a plurality (two in the present embodiment) of suction pads 24 are respectively mounted with the joint portions as boundaries.

[0108] Furthermore, a palm sensor 26 is mounted on the palm side 20A of the grip portion 20 of the present embodiment. The palm sensor 26 includes a high-resolution camera for identifying the type of the cargo 100 and a MoPU (Motion Processing Unit) for identifying the position of the cargo 100.

[0109] As Figure 3B shown, the suction pad 24 is composed of a rubber pad portion 24A and a suction nozzle 24B. The pad portion 24A faces the cargo when gripping the cargo 100, and the suction nozzle 24B forms an air flow path for sucking the air in the closed space formed by the close contact between the pad portion 24A and the cargo 100.

[0110] That is, the suction pad 24 of the present embodiment is an air suction structure and has an adsorption force by sucking the air in the closed space from the hole 24C provided in the suction nozzle 24B and evacuating (substantially including vacuum). In addition, the suction pad 24 is not limited to the air suction structure, and may also be a structure that adsorbs simply due to the change in the volume of the closed space caused by the deformation of the pad portion 24A.

[0111] (Details of the palm sensor 26)

[0112] The high-resolution camera constituting the palm sensor 26 of the present embodiment identifies what the captured cargo 100 is based on the captured image information, whether it is a care product such as shampoo, conditioner, cosmetics, toothpaste, or a food such as instant noodles or packaged snacks.

[0113] In other words, the high-resolution camera has the function of obtaining information for determining the type (shape, size, hardness, etc.) of the cargo 100.

[0114] On the other hand, the MoPU that constitutes the palm sensor 26 of the present embodiment together with the high-resolution camera outputs motion information indicating the motion of the captured cargo 100 (in this case, the relative motion between the arm portions 5 and 6) at a frame rate of 1000 frames / second or more based on the object image captured at a frame rate of 1000 frames / second or more. In addition, when detecting the moving cargo 100, the frame rate can be increased, and when detecting a stationary object (non-moving cargo 100), the frame rate can be decreased.

[0115] The MoPU outputs, as motion information, vector information indicating the movement of a point representing the position of the goods 100 along a specified coordinate axis. That is, the motion information output from the MoPU does not include the information required to identify what the captured goods 100 are (the above-mentioned nursing goods, food), but only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity point) of the goods 100 on the coordinate axes (x-axis, y-axis, z-axis).

[0116] That is, it is possible to accurately guide the trajectory when the holding part 20 approaches the goods 100 with high precision.

[0117] The information output from the palm sensor 26 including the high-resolution camera and the MoPU is supplied to the information processing device 14.

[0118] Based on the information from the palm sensor 26 including the high-resolution camera and the MoPU, the information processing device 14 accurately determines the position of the goods 100, calculates the degree of expansion of the finger parts 22A, 22B, 22C during grasping, the strength during grasping, the adsorption force generated by the adsorption pad 24, etc., and accurately controls the minute movements of the arm parts 5, 6 and the holding part 20, and can handle the sorting operations of various goods 100.

[0119] Figure 5 It is a schematic diagram of an example of a humanoid robot control system according to the first embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and a MoPU, and an information processing device 14.

[0120] The sensor 12 sequentially obtains information indicating at least the distance and angle between the object operating on the humanoid robot 1 around the humanoid robot 1 and the arm parts 5, 6. As the sensor 12, the highest-performance camera, solid-state LiDAR, multi-color laser coaxial displacement meter, or other various sensor groups can be used. In addition, as the sensor 12, a vibration meter, a thermal camera, a hardness meter, a radar, a LiDAR, a high-pixel telescopic ultra-wide-angle 360-degree high-performance camera, visual recognition, micro sound, ultrasonic waves, vibration, infrared rays, ultraviolet rays, electromagnetic waves, temperature, humidity, real-time AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail event AI data, etc. can also be cited.

[0121] In addition, in addition to the above information, the sensor 12 also detects images, distances, vibrations, heat, odors, colors, sounds, ultrasonic waves, ultraviolet rays, or infrared rays, etc. In addition, as the information detected by the sensor 12, the center-of-gravity movement of the humanoid robot 1, the detection of the material of the ground on which the humanoid robot 1 is set, the detection of the external air temperature, the detection of the external air humidity, the detection of the vertical, horizontal, and inclined angles of the ground, the detection of the moisture content, etc. can be cited.

[0122] The sensor 12 performs these detections, for example, every nanosecond.

[0123] The palm sensor 26 (high-resolution camera and MoPU) is a sensor provided in the holding part 20 of the arm parts 5 and 6. In addition to the sensor 12, it also has a camera function for photographing the goods 100 and a position determination function for determining the position of the goods 100.

[0124] In addition, when using one MoPU, it is possible to obtain vector information indicating the movement of the points representing the existence positions of the goods 100 along two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system. Using the principle of a stereo camera, two MoPUs can be used to output vector information indicating the movement of the points representing the existence positions of the goods 100 along three coordinate axes (x-axis, y-axis, and z-axis) in a three-dimensional orthogonal coordinate system. The z-axis is the axis along the depth direction (the driving of the vehicle).

[0125] The information processing device 14 includes an information acquisition unit 140, a control unit 142, and an information storage unit 144.

[0126] The information acquisition unit 140 acquires information on the goods 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0127] The control unit 142 uses the information acquired by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence) to control the rotational movement of the connecting part 4, the vertical movement, and the movements of the arm parts 5 and 6, etc.

[0128] In addition, the control unit 142 uses the information acquired by the information acquisition unit 140 from the palm sensor 26 (high-resolution camera and MoPU) to grasp in detail the type (shape, size, hardness, etc.) and position of the goods 100. Based on the shape or position, it makes the palm side 20A face the goods, adsorbs them through the adsorption pad 24, and controls the grasping (holding control) with the three finger parts 22A, 22B, and 22C. Additionally, based on the shape information, the type of the object can be grasped, and the holding control (such as only "adsorption", only "grasping", or the combination of "adsorption" and "grasping") can be selected.

[0129] For example, as an overall operation, the control unit 142 performs the following processes.

[0130] (1) Drive the connecting part 4 to tilt the upper body part 2 forward or backward so as to be able to pick up the goods 100 located on the shelf and the ground.

[0131] (2) Drive the arm parts 5 and 6 and the holding part so as to be able to grasp the goods 100.

[0132] (3) Drive the upper body part 2 up and down relative to the foot part 3 to suit the height of the workbench on the production line.

[0133] (4) To prevent the humanoid robot 1 from falling and obtain balance.

[0134] (5) Control the drive of the wheels 7 and 8 so that the humanoid robot 1 can push a trolley or the like.

[0135] When the information processing device 14 picks up the goods 100 located on the ground, for example, it repeatedly executes Figure 6 the flowchart shown.

[0136] In step S100, the information acquisition unit 140 acquires the information of the object detected by the sensor 12.

[0137] In step S102, the control unit 142 uses the information of the goods 100 acquired in step S100 and AI to pick up the goods 100 located on the ground by controlling the connecting part 4 and the arm parts 5 and 6.

[0138] In step S104, the control unit 142 moves the picked-up goods 100 to a specified position.

[0139] According to this embodiment, the humanoid robot 1 includes an upper body part 2, a foot part 3, and a connecting part 4 that rotatably connects the upper body part 2 and the foot part 3. In addition, the rotation of the connecting part 4 is controlled based on the information acquired by the sensor 12. Therefore, the distance and angle between the humanoid robot 1 and the goods 100 can be judged, and thus, the action of picking up the goods 100 located on the ground can be performed.

[0140] In addition, since the connecting part 4 can change the distance between the upper body part 2 and the foot part 3, the position of the upper body part 2 relative to the foot part 3 in the vertical direction can be adjusted to suit the height of the workbench on the production line.

[0141] In addition, the foot part 2 has a balance function for preventing the humanoid robot 1 from falling when the upper body part 2 leans forward or backward relative to the foot part 3. Therefore, it is possible to prevent the humanoid robot 1 from falling when performing operations such as pushing and pulling the goods 100 on the production line. Therefore, it is possible to prevent malfunctions of the humanoid robot 1 caused by falling, or injuries to people around the humanoid robot 1.

[0142] (Grasping control of goods 100)

[0143] Figure 7 is a flowchart showing the grasping control sequence when the goods 100 are grasped by the grasping part 20 in linkage with the overall movement of the humanoid robot 1 of Figure 1 .

[0144] In step 150, it is determined whether there is a gripping instruction for the goods 100. If the determination is affirmative, the process proceeds to step 152, where the humanoid robot 1 moves (for example, the arm parts 5 and 6 are actuated) so that the palm side 20A faces the target goods 100, and then the process proceeds to step 154.

[0145] In step 154, the palm side 20A is made to face each other, and the information of the goods 100 is detected.

[0146] In the next step 156, the detection information based on the palm sensor 26 (high-resolution camera and MoPU) is analyzed to grasp in detail the type (shape, size, hardness, etc.) and position of the goods 100, and then the process proceeds to step 158.

[0147] In step 158, an operation for gripping the goods 100 is selected. For example, it is selected from only "adsorption", only "grasping", using both "adsorption" and "grasping", etc. Then, the process proceeds to step 160, where the angles (opening degrees) of the finger parts 22A, 22B, and 22C are set based on the shape of the goods 100, and then the process proceeds to step 162.

[0148] In step 162, the goods 100 are gripped (only "adsorption", only "grasping", "adsorption" and "grasping").

[0149] In the next step 164, it is determined whether the gripping of the goods 100 is successful. If the determination is affirmative, the gripped goods 100 are transported to a specified place, and the process proceeds to step 150 to wait for the next gripping instruction for the goods 100.

[0150] In addition, if the determination in step 164 is negative, the process proceeds to step 166 to perform error handling (such as retry or cancellation, etc.), and then returns to step 150.

[0151] As described above, according to the present embodiment, by providing three finger parts 22A, 22B, and 22C in the gripping part 20, and installing a plurality of adsorption pads 24 on the palm side 20A and the finger parts 22A, 22B, and 22C of the gripping part 20, the adsorption pads 24 adsorb the goods 100 by, for example, an air adsorption structure, and the goods 100 can be grasped by bending the finger parts 22A, 22B, and 22C.

[0152] By installing a palm sensor 26 including a high-resolution camera and MoPU on the palm side 20A, and installing the gripping part 20 with the above structure on the arm parts 5 and 6 of the humanoid robot 1, items can be reliably picked up by the adsorption surface, and even if the humanoid robot 1 moves quickly, the goods 100 can be transported without falling from the gripping part 20.

[0153] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the goods 100 can be captured with high precision, and it is also possible to handle operations that require fine movements.

[0154] Moreover, for very soft and fragile items, it is possible to grasp them by the operations of the finger portions 22A, 22B, and 22C without using the suction pad 24. By adjusting the grasping force, it is possible to prevent damage to the soft goods 100 and the like.

[0155] Figure 8 An example of the hardware configuration of the computer 1200 that functions as the information processing device 14 is schematically shown. The program installed in the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the device according to the present embodiment, and / or cause the computer 1200 to execute the process or stages of the process according to the present embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0156] The computer 1200 in the present embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected through a main controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the main controller 1210 via an input / output controller 1220. The DVD drive can be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 can be a hard disk drive, a solid state drive, etc. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0157] The CPU 1212 operates according to the programs stored in the ROM 1230 and the RAM 1214, and thereby controls each unit. The graphics controller 1216 acquires the image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in itself, and causes the image data to be displayed on the display device 1218.

[0158] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card, and / or writes programs and data to the IC card.

[0159] The ROM 1230 stores a boot program and / or a program that depends on the hardware of the computer 1200, which is executed by the computer 1200 when activated. The input / output chip 1240 can also connect various input / output units to the input / output controller 1220 through a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0160] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214 or the ROM 1230, which are examples of computer-readable storage media, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200, resulting in the cooperation between the program and the above various types of hardware resources. The device or method can be constituted by operating or processing information based on the use of the computer 1200.

[0161] For example, when performing communication between the computer 1200 and an external device, the CPU 1212 can execute a communication program loaded into the RAM 1214, and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads the transmission data stored in the transmission buffer area provided in a recording medium such as the RAM 1214, the storage device 1224, the DVD-ROM or the IC card, transmits the read transmission data to the network, or writes the received data received from the network into the reception buffer area provided on the recording medium, etc.

[0162] In addition, the CPU 1212 can read all or a necessary part of a file or a database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. in the RAM 1214, and perform various types of processing on the data on the RAM 1214. Then, the CPU 1212 can write the processed data back to the external recording medium.

[0163] Various types of information (such as programs, data, tables, and databases) can be stored in a recording medium and receive information processing. The CPU 1212 can perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branch, unconditional branch, retrieval / replacement of information, etc., specified by the instruction sequence of the program described anywhere in the present disclosure, and write the result back to the RAM 1214. In addition, the CPU 1212 can retrieve information in files, databases, etc. within the recording medium. For example, if multiple entries are stored in the recording medium, where each entry has an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 can retrieve an entry that matches the condition specifying the attribute value of the first attribute from the multiple entries, and read the attribute value of the second attribute stored in that entry, so as to obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0164] The programs or software modules described above can be stored in a computer-readable storage medium near or in the computer 1200. In addition, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, so as to provide the program to the computer 1200 via the network.

[0165] In the flowcharts and block diagrams in the present embodiment, the blocks can represent stages of a process of performing operations or "parts" of a device having the function of performing operations. Specific stages and "parts" can be implemented by dedicated circuits, programmable circuits provided together with computer-readable instructions stored on a computer-readable storage medium, and / or processors provided together with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit can include digital and / or analog hardware circuits, and can include integrated circuits (ICs) and / or discrete circuits. The programmable circuit can include reconfigurable hardware circuits, which include logical products, logical sums, exclusive ORs, negative logical products, negative logical sums, and other logical operations, flip-flops, registers, and storage elements, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), etc.

[0166] A computer-readable storage medium can include any tangible device capable of storing instructions executable by an appropriate device. As a result, a computer-readable storage medium having instructions stored therein has a product including instructions that can be executed to create elements for performing the operations specified in a flowchart or block diagram. Examples of computer-readable storage media can include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media can include floppy disks, magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0167] Computer-readable instructions can include either source code or object code described by any combination of one or more programming languages, which include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status-setting data, and object-oriented programming languages such as Smalltalk (registered trademark), JAVA (registered trademark), C++, etc., as well as conventional procedural programming languages such as the "C" programming language or similar programming languages.

[0168] Computer-readable instructions can be provided to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device via a local area network (LAN) or a wide area network (WAN) such as the Internet, so that the processor or programmable circuit of the general-purpose computer, special-purpose computer, or other programmable data processing device executes the computer-readable instructions to generate elements for performing the operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0169] [Second Embodiment]

[0170] Hereinafter, a second embodiment of the present disclosure will be described.

[0171] In the second embodiment, the same reference numerals are assigned to the same components as those in the first embodiment, and the description of their structures is omitted.

[0172] The feature of the second embodiment is that, in the above first embodiment, in addition to the gripping control and conveyance control operations of the goods 100 using the gripping portion 20 of the humanoid robot 1, avoidance control when the conveyance paths of multiple humanoid robots 1 cross is also applied. That is, in the control unit 142 of the information processing device 14 (refer toFigure 5 ) stores a contact avoidance program, and based on a need (for example, when two or more humanoid robots 1 exist within a specified radius as an example of a specified range, etc.), this contact avoidance program is executed.

[0173] (Details of avoidance control)

[0174] When multiple humanoid robots 1 are holding goods 100 and transporting them to a specified position, or when returning after transporting the goods 100 to a specified position, there are cases where their travel paths cross each other.

[0175] The transfer timings of the goods 100 are respectively programmed into each humanoid robot 1. Although countermeasures to prevent contact are taken while monitoring the surroundings, the mutual transfer timings are not clearly managed.

[0176] For example, in a case where the travel paths of two humanoid robots 1 are in a straight line in the direction of approaching each other and may come into contact in the future, in general programming, sometimes information about the goods 100 being transported (the presence or absence of the goods 100, weight, etc.) is not considered, and due to unnecessary avoidance actions, an adverse situation such as the goods 100 falling off may occur.

[0177] Therefore, in the second embodiment, in multiple humanoid robots 1 (hereinafter, two humanoid robots 1 are exemplified, but it may also be three or more), when they are within a specified radius of each other, (1) mutual information is transmitted and received; (2) determination of the contact avoidance object is made (which one takes contact avoidance actions, etc.); (3) based on the contact mode, contact avoidance actions are executed.

[0178] (1) Transmitting and receiving information

[0179] As the execution information transmitted and received by the humanoid robot 1, the identifier (ID) of the humanoid robot 1, whether it is holding and transporting the goods 100, and goods information (at least the weight is necessary, and in addition, sometimes information such as the center of gravity position, external shape, traveling speed, etc.) can be cited.

[0180] (2) Determination of the contact avoidance object

[0181] Contact avoidance object determination is a determination used to select the party that needs to perform contact avoidance behavior. The selected humanoid robot 1 acts according to the execution unit (contact avoidance operation program), which is one of the functions of the control unit 142 of the information processing device 14. In other words, the unselected humanoid robot 1 continues to move based on the normal driving program. The premise of contact avoidance object determination is as follows: Based on the information mutually transmitted and received by multiple humanoid robots 1, the robot with a lighter load of the grasped object is selected. In the case where there is no difference in the grasping state of the object, selection is made according to the preset priority order of identification information. The following shows specific examples of contact avoidance behavior selection.

[0182] (Example 1) The party that is not grasping the goods 100 becomes the contact avoidance object.

[0183] (Example 2) In the case where both are grasping the goods 100, the party with the lighter weight of the goods 100 becomes the contact avoidance object.

[0184] (Example 3) In the situation where it cannot be determined in the above Examples 1 and 2 (the conditions are the same), the party with the smaller ID number becomes the contact avoidance object.

[0185] In addition, in the case where there is information on the center of gravity of the goods 100, the party with a heavier weight of the goods 100 but a stable center of gravity (the center of gravity position is close to the central part of the goods 100) can be used as the contact avoidance object (the risk of dropping for the party with a stable center of gravity is low).

[0186] In addition, in the case where there is information on the appearance of the goods 100, the party that grasps the goods with a smaller appearance even if the weight of the goods 100 is heavier and / or the center of gravity is unstable can be used as the contact avoidance object (the risk of dropping for the small goods 100 is low).

[0187] Furthermore, in the case where there is information on the conveyance speed, the party with a slower conveyance speed can also be used as the contact avoidance object.

[0188] In addition, when comprehensively considering the goods information, a specified arithmetic expression with weight information, center of gravity information, appearance information, speed information, etc. as parameters can also be calculated and numericalized to determine the contact avoidance object.

[0189] (3) Contact avoidance behavior based on the contact mode

[0190] As the contact mode of the two humanoid robots 1, there are sometimes driving paths of each other on a straight line. In this case, the two humanoid robots 1 facing each other gradually approach and come into face-to-face contact.

[0191] Therefore, by executing a detour action program on the humanoid robot 1 that becomes the contact avoidance object (refer to Figures 10A to 10D, which will be described in detail later), to prevent face-to-face contact.

[0192] On the other hand, if there is a point where the driving paths of each other cross in the future driving direction, and they drive while maintaining a certain angle at their respective positions, they will come into contact at the crossing point (collision course phenomenon).

[0193] Therefore, by making the humanoid robot 1 that is the object of contact avoidance execute a deceleration (pause) action program (refer to Figures 11A to 11C , which will be described in detail later), to prevent contact due to the collision course phenomenon.

[0194] In addition, in addition to face-to-face contact and contact due to the collision course phenomenon, for example, as an emergency avoidance process incorporated into the normal driving program, two humanoid robots 1 execute contact avoidance based on free driving.

[0195] Hereinafter, the operation of the second embodiment will be described according to the Figure 9 flowchart.

[0196] Figure 9 is a control flowchart showing a contact avoidance processing routine in the driving control of each humanoid robot 1 executed by the control unit 142 of the information processing device 14 (refer to Figure 5 ).

[0197] In step 200, the driving control based on the driving program is started, and then it proceeds to step 202 to determine whether there is another humanoid robot 1 within a specified radius.

[0198] If the determination in this step 202 is negative, it is determined that there is no possibility of contact between the humanoid robots 1 (less than the specified probability), and it proceeds to step 220. On the other hand, if the determination in step 202 is positive, it is determined that there is a possibility of contact between the humanoid robots 1 (equal to or more than the specified probability), and it proceeds to step 204.

[0199] In step 204, the humanoid robots 1 within the specified radius perform mutual information transmission and reception.

[0200] That is, as shown in the above-mentioned "(1) Transmitting and receiving information", they mutually transmit and receive the ID, whether there is a carried cargo, and the information of the cargo.

[0201] In the next step 206, a determination process for the contact avoidance object is executed based on the received information.

[0202] That is, as shown in the above-mentioned "(2) Determination of the contact avoidance object", based on the holding state of the cargo 100, the weight comparison of the cargo 100, and the priority setting state bound to the ID, the humanoid robot 1 that is the contact avoidance object is determined.

[0203] In the next step 208, as a result of the contact avoidance object determination process in step 206, it is determined whether the own humanoid robot 1 is a contact avoidance object. If the determination in this step 208 is negative, the own humanoid robot 1 continues to travel based on the normal travel program without avoidance and transfers to step 220.

[0204] On the other hand, if the determination in step 208 is positive, the own humanoid robot 1 determines that contact avoidance is required and transfers to step 210.

[0205] In step 210, the contact mode is determined. In the second embodiment, it is divided into face-to-face contact, collision course phenomenon, and other types.

[0206] In step 210, when it is determined that the contact mode is face-to-face contact, in order to execute the face-to-face contact avoidance process, it transfers from step 210 to step 212 and executes a detour action program (described in detail later).

[0207] In step 210, when it is determined that the contact mode is the collision course phenomenon, in order to execute the collision course phenomenon avoidance process, it transfers from step 210 to step 214 and executes a deceleration (pause) action program (described in detail later).

[0208] On the other hand, in step 210, when it is determined that the contact mode is not any one of face-to-face contact and the collision course phenomenon, in order to execute the emergency avoidance process, it transfers from step 210 to step 216.

[0209] That is, in addition to face-to-face contact and collision course phenomenon contact, in the second embodiment, as an emergency avoidance process incorporated into the normal travel program, the two humanoid robots 1 execute contact avoidance based on free travel. In this case, since it does not depend on the contact avoidance object determined in step 206 and each takes an avoidance behavior, contact is avoided, but there may be a time loss (including travel suspension, etc.).

[0210] Therefore, in the second embodiment, by selecting the humanoid robot 1 of the contact avoidance object and identifying the contact mode, the avoidance process can be performed quickly and reliably.

[0211] When the processing of step 212, 214, or 216 ends, it transfers to step 218, instructs to return to the normal travel route, and then transfers to step 220 to determine whether the travel program has ended.

[0212] If the determination in step 220 is negative, it returns to step 202 and repeats the above process. On the other hand, if the determination in step 220 is positive, this routine ends.

[0213] (Example of the action in step 212)

[0214] Figures 10A to 10D This is an execution example of the detour action program in step 212.

[0215] As Figure 10A shown, two humanoid robots 1 move closer to each other along the same straight line (refer to the humanoid robot 1 shown by the dotted line in Figure 10A ).

[0216] Figure 10A The humanoid robot 1A moving from left to right in Figure 10A is an object to be avoided from contact. When a certain distance is reached, the humanoid robot 1 changes its direction to the left side of the forward direction and deviates from the normal driving route (refer to the solid line in

[0217] After that, as Figure 10B shown, the humanoid robot 1B that is not an object to be avoided from contact and travels on the normal route travels straight as shown by the dotted arrow in Figure 10B , but the humanoid robot 1A that is an object to be avoided from contact bypasses the normal route as shown by the solid arrow in Figure 10B , thus avoiding mutual contact.

[0218] As Figure 10C shown, the humanoid robot 1A that has avoided mutual contact changes its direction to the right side of the forward direction in a way to return to the normal route and returns to the normal driving route (refer to Figure 10D ).

[0219] (Example of the action in step 214)

[0220] Figures 11A to 11C This is an execution example of the deceleration (pause) action program in step 214.

[0221] As Figure 11A shown, two humanoid robots 1 move along different straight driving paths respectively (refer to the solid arrow and the dotted arrow in Figure 11A ), and the two driving paths will become intersecting paths in the future.

[0222] The humanoid robot 1A moving along the Figure 11A solid arrow is an object to be avoided from contact. At time t1 and time t2, it moves at almost the same speed as the humanoid robot 1B that is not an object to be avoided from contact, but at time t3, it decelerates or pauses.

[0223] On the other hand, the humanoid robot 1B reaches the intersection of the two driving paths at time t3. However, due to the deceleration or pause of the humanoid robot 1A, contact at the intersection is avoided.

[0224] After that, as Figure 11B, Figure 11C As shown in Figure 11C , at time t4, the humanoid robot 1A reaches the intersection of the travel path. However, since the humanoid robot 1B has already passed through the intersection, at time t5, the humanoid robot 1A and the humanoid robot 1B continue to travel along the normal travel routes respectively.

[0225] As described above, according to the avoidance control in the second embodiment, the object and effect of the present disclosure can be more firmly achieved, that is, for each humanoid robot 1 holding the goods 100 and in the process of transportation, a series of operations of quickly and reliably confirming the position of the goods, grasping and holding by the holding part can be carried out. In particular, it is possible to reliably prevent the held goods 100 from falling off during transportation.

[0226] In addition, in the above second embodiment, it is assumed that there is an area where two humanoid robots 1 can pass by alternately. However, in the case where there is no space for alternate passing, for example, they can execute actions such as turning sideways to pass by alternately, or one humanoid robot 1 retreating to a space where alternate passing is possible.

[0227] This avoidance behavior is Figure 9 a corresponding mode of the emergency avoidance process in step 216 of

[0228] [Third Embodiment]

[0229] Hereinafter, a third embodiment of the present disclosure will be described.

[0230] In the third embodiment, the same reference numerals are assigned to the same components as in the first embodiment, and the description of their structures is omitted.

[0231] (Structure of the holding part 20)

[0232] As Figure 12 shown, the holding part 20 installed at the front ends of the arm parts 5 and 6 has a hand structure similar to that of a human. The holding part 20 is rotatably installed on the arm parts 5 and 6 (Intelligent Hand System).

[0233] The holding part 20 shown in this embodiment has three finger parts 22A, 22B, and 22C each having a plurality of joints. In this embodiment, the three-finger structure is Figure 13A as shown, and is the same for both the right hand and the left hand. However, it can also be Figure 13B a five-finger structure as shown.

[0234] In addition, on the palm side 20A of the holding part 20, an X-ray sensor 27 of a state sensor, which is part of the function of the center-of-gravity position recognition part, is installed adjacent to the palm sensor 26 (described in detail later).

[0235] Details of the palm sensor 26 are as described above, and thus repeated description thereof is omitted.

[0236] (Details of the X-ray sensor 27)

[0237] The X-ray sensor 27 synchronizes (almost simultaneously) when the palm sensor 26 determines the type (shape, size, hardness, etc.) of the goods 100, and detects the state inside the package of the goods 100.

[0238] In the X-ray sensor 27, a transmission image obtained by irradiating the object to be inspected (goods 100) with X-rays can confirm and evaluate the internal (inside the package) state that is invisible to the naked eye.

[0239] For example, when the material stored inside the package is a fluid, by detecting its filling amount or position (filling inspection), the center of gravity position of the goods 100 can be determined, and the holding point implemented by the holding part 20 can be appropriately adjusted.

[0240] In addition, the material can be identified. In the case of metal or non-metal, in the case of non-metal, classification such as plastic, glass, rubber, stone, bone, shell, etc. can be performed. Through this classification, the holding point of the holding part 20 can be adjusted.

[0241] Generally speaking, for the X-ray sensor 27, compared with the advantage of being easy to identify objects with a large specific gravity, there is also the disadvantage of being unsuitable for identifying objects with a small specific gravity or small weight (such as plastic film, paper, flying insects, hair, etc.). However, since light-weight objects do not require determination of the holding point of the holding part 20, in the function (adjustment of the holding point) in the present embodiment, the disadvantage can be ignored.

[0242] Figure 14 It is a comparison chart showing the correspondence between the visible image based on the type of the goods 100 and the detection image obtained by the X-ray sensor 27.

[0243] In Figure 14 Food trays, spouted bags, vacuum packages, cans, and PTP (blister packaging) are exemplified. However, in any of them, the part with a large mass (specific gravity) is shown as an X-ray image. Therefore, compared with the visible image, analysis of the center of gravity position and the like becomes easy. In addition, in addition to the goods 100 exemplified in Figure 14 (for example, the goods 100 packaged in a cardboard box, the position of the items inside the cardboard box, etc.), the image obtained by the X-ray sensor 27 is also effective in the analysis of the center of gravity position.

[0244] The information output from the X-ray sensor 27 is provided to the information processing device 14 which forms a part of the center of gravity position recognition part together with the X-ray sensor 27, and the center of gravity position is recognized in the information processing device 14.

[0245] The center-of-gravity position recognition unit of this embodiment has the function of an analysis unit that analyzes the center-of-gravity position of the object based on the weight distribution of the part that cannot be seen from the appearance of the object detected by the X-ray sensor.

[0246] Here, by using the above-mentioned palm sensor 26 and X-ray sensor 27 in combination, in the grasping control of the cargo 100, first, the shape (outer shape) and position of the cargo 100 are analyzed in the palm sensor 26, an adsorption and / or grasping action is selected, and the finger angles of the grasping part 20 are set, etc.

[0247] At this time, in the X-ray sensor 27, the center-of-gravity position based on the mass of the items stored in the package is calculated, and the selection of the adsorption and / or grasping action and the finger angle or grasping point are set.

[0248] As a result, it is possible to appropriately perform the selection of the adsorption and / or grasping action based on weight that is difficult to see from the appearance alone and the setting of the grasping point corresponding to the weight shift.

[0249] The information processing device 14 determines the position of the cargo 100 with high precision through the information from the palm sensor 26 including a high-resolution camera and MoPU, calculates the degree of expansion of the finger parts 22A, 22B, 22C during grasping, the strength during grasping, the adsorption force applied by the adsorption pad 24, etc., and controls the minute movements of the arm parts 5, 6 and the grasping part 20 with high precision, and can handle the sorting operations of various cargos 100.

[0250] Figure 15 It is a schematic diagram of an example of the humanoid robot control system according to the third embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and MoPU, an X-ray sensor 27, and an information processing device 14.

[0251] The sensor 12 main body sequentially obtains information indicating at least the distances and angles between the goods on which the humanoid robot 1 operates around the humanoid robot 1 and the arm parts 5, 6, and different from the above-mentioned palm sensor 26 and X-ray sensor 27, obtains information related to the actions of the humanoid robot 1 itself.

[0252] (Grasping control of the cargo 100)

[0253] Figure 16 It shows the Figure 1 flowchart of the grasping control sequence when the grasping part 20 grasps the cargo 100 in linkage with the overall movement of the humanoid robot 1.

[0254] In step 230, it is determined whether there is a gripping instruction for the goods 100. If the determination is affirmative, the process proceeds to step 232, where the humanoid robot 1 moves (for example, the arm parts 5 and 6 move), causing the palm side 20A to face the target goods 100, and then the process proceeds to step 234.

[0255] In step 234, the palm side 20A faces each other to detect information of the goods 100.

[0256] In the next step 236, the detection information based on the palm sensor 26 (high-resolution camera and MoPU) is analyzed to grasp in detail the type (shape, size, hardness, etc.) and position of the goods 100, and then the process proceeds to step 238.

[0257] In step 238, the internal information of the goods 100 is detected by the X-ray sensor 27. Then, the process proceeds to step 240, where it is determined by a weight sensor or the like whether the weight of the goods 100 is above a specified value. If the determination is affirmative, the process proceeds to step 242, where the center-of-gravity position of the goods 100 is analyzed based on the detection information obtained by the X-ray sensor 27, and then the process proceeds to step 244. Additionally, if the determination in step 240 is negative, the process proceeds to step 244. Also, the weight determination of the goods 100 is not necessary, and the center-of-gravity position can also be analyzed for all the goods 100.

[0258] In step 244, an operation for gripping the goods 100 is selected. For example, it is selected from only "adsorption", only "grasping", a combination of "adsorption" and "grasping", etc. Then, the process proceeds to step 246, where the angles (opening degrees) of the finger parts 22A, 22B, and 22C are set based on the shape of the goods 100, and then the process proceeds to step 248.

[0259] That is, based on the appearance obtained by the palm sensor 26, the angles of the finger parts 22A, 22B, and 22C of the gripping part 20 are adjusted, and based on the center-of-gravity position obtained by the X-ray sensor 27, the gripping points of the respective finger parts 22A, 22B, and 22C are determined, thereby enabling the goods 100 to be stably gripped.

[0260] In addition, the X-ray sensor 27 has the drawback that it cannot detect lighter objects. However, as shown in step 240, by taking advantage of the function of determining whether the weight of the goods 100 is above a specified value, it can be used to determine whether a "grasping" action is required. In other words, no additional weight sensor is needed.

[0261] In step 248, the goods 100 are gripped (only "adsorption", only "grasping", "adsorption" and "grasping").

[0262] In the next step 250, it is determined whether the gripping of the goods 100 is successful. In the case of a positive determination, the gripped goods 100 are transported to a specified place, and the process proceeds to step 230 to wait for the gripping instruction of the next goods 100.

[0263] In addition, in the case of a negative determination in step 250, the process proceeds to step 252 to perform error handling (such as retry or cancellation, etc.), and then returns to step 230.

[0264] As described above, according to the present embodiment, by providing three finger portions 22A, 22B, and 22C in the gripping portion 20, and installing a plurality of adsorption pads 24 on the palm side 20A of the gripping portion 20 and the finger portions 22A, 22B, and 22C, the adsorption pads 24 adsorb the goods 100 by, for example, an air adsorption structure, and by bending the finger portions 22A, 22B, and 22C, the goods 100 can be grasped.

[0265] By installing the palm sensor 26 including a high-resolution camera and a MoPU on the palm side 20A, and installing the gripping portion 20 having the above structure on the arm portions 5 and 6 of the humanoid robot 1, items can be reliably picked up from the adsorption surface, and even if the movement of the humanoid robot 1 is fast, the goods 100 can be transported without falling from the gripping portion 20.

[0266] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the goods 100 can be captured with high precision, and it is also possible to handle operations that require minute movements.

[0267] Moreover, for very soft and fragile items, they can be grasped by the actions of the finger portions 22A, 22B, and 22C without using the adsorption pads 24. By adjusting the grasping force, it is possible to prevent damage to the soft goods 100.

[0268] In addition, in the present embodiment, as the center-of-gravity position recognition unit, an X-ray sensor 27 and an analysis unit for analyzing the information from the X-ray sensor 27 to analyze the center-of-gravity position of the goods 100 are provided, but an infrared sensor can be used instead of the X-ray sensor 27. According to the infrared sensor, based on the internal temperature difference, the offset of the internal items can be recognized, and the center-of-gravity position can be determined. In addition, the center-of-gravity position can be recognized by a combination of multiple sensors including various sensors exemplified as the sensor 12.

[0269] (Variant example)

[0270] In addition, in the present embodiment, the X-ray sensor 27 is installed on the humanoid robot 1, and when each humanoid robot 1 grips the goods 100, based on the detection information of the X-ray sensor 27, the center-of-gravity position of the goods 100 is analyzed. However, as Figure 17As shown, an X-ray analysis device 52 shared by multiple humanoid robots 1 can also be provided in the conveyor device 50 at the sorting station of the goods 100, and the information processing device 14 receives information from the X-ray analysis device 52. In other words, the center-of-gravity position recognition unit of the information processing device 14 functions as an information acquisition unit that acquires information related to the center-of-gravity position.

[0271] That is, as Figure 17 shown, the conveyor belt 50A of the conveyor device 50 moves in the direction of arrow A by a driving force (not shown), and the goods 100 are sequentially conveyed from Figure 17 left to right. Figure 17 The humanoid robot 1 faces the conveyor belt 50A and waits until the goods 100 to be sorted reach a specified position (the confrontation position of the humanoid robot 1). When the goods 100 reach the specified position, the humanoid robot 1 sorts the goods 100.

[0272] Here, an X-ray analysis device 52 is provided on the upstream side in the conveyor direction of the conveyor belt 50A relative to the confrontation position of the humanoid robot 1.

[0273] The X-ray analysis device 52 arranges an X-ray generating device 54 at the upper part of the housing (above the conveyor belt 50A) and arranges an X-ray sensor 56 at the lower part of the housing (below the conveyor belt 50A).

[0274] The X-ray generating device 54 and the X-ray sensor 56 are respectively connected to a controller 58. Based on the instruction of the controller 58, X-rays are irradiated from the X-ray generating device 54, and the X-rays passing through the goods 100 are detected by the X-ray sensor 56.

[0275] In the controller 58, when the goods 100 reach the inspection position, based on the detection information obtained by the X-ray sensor 56, the center-of-gravity position of the goods 100 is analyzed and stored in association with the ID (goods ID) for identifying the goods 100.

[0276] The controller 58 has a communication function capable of communicating with the humanoid robot 1, and sends the stored ID information (goods ID) and center-of-gravity position information of the goods 100 to the humanoid robot 1 facing the conveyor belt 50A.

[0277] In the humanoid robot 1, the goods ID and the center-of-gravity position information (center-of-gravity position recognition unit) are received through the information processing device 14, and by matching with the goods ID to be grasped currently, the center-of-gravity position information of the goods 100 can be identified.

[0278] In the humanoid robot 1, the goods ID and the center-of-gravity position information (center-of-gravity position recognition unit) are received through the information processing device 14, and by matching with the goods ID to be grasped currently, the center-of-gravity position information of the goods 100 can be identified.

[0279] [Fourth Embodiment]

[0280] Hereinafter, a fourth embodiment of the present disclosure will be described.

[0281] In the fourth embodiment, the same reference numerals are given to the components identical to those in the first embodiment, and the description of their structures is omitted.

[0282] (Grasping control of the goods 100)

[0283] Here, in the information processing apparatus 14 according to the present embodiment, in the grasping control of the goods 100, the optimal grasping point is set based on the internal condition of the goods 100. For example, dropping, slipping, and shaking during travel after grasping are suppressed.

[0284] That is, before grasping the goods 100, cubic data indicating the internal condition of the goods 100 is received from the information management server 60 (see Figure 20 , which will be described in detail later) that manages information related to the goods 100. Based on this cubic data, the internal information of the goods 100 is estimated, and the grasping point is set.

[0285] As Figure 18A shown, the cubic data refers to the shape of the goods 100 formed as an aggregate by small blocks (cubes) CB on the three-dimensional grid G (see the diagonal line in Figure 18A ). In one aggregate (goods 100), coordinates for determining the relative positions of each other are provided. If configured based on these coordinates, a three-dimensional shape based on the outer shape of the goods 100 is assembled. In addition, the cubic data is not only the outer shape of three-dimensional model data (e.g., polygon data) respectively. As Figure 18B shown, the coordinates of each block CB are used as identification information, and various attribute information such as the internal structure, material used, bonding strength, and weight is recorded. Therefore, the cubic data is different from polygon data that simply represents the appearance (outline) and is information capable of identifying the internal state. In addition, the block CB represents a part of the goods 100.

[0286] Hereinafter, a case of setting the grasping point based on the cubic data of the goods 100 will be exemplified. In addition, it is not denied that the grasping point is set using the cubic data other than the following examples.

[0287] (Example 1) Accurately grasp the outer shape of the goods 100.

[0288] For example, the outer shape of the goods 100 can be recognized based on the image information obtained by the palm sensor 26 (high-resolution camera and MoPU), but sometimes the surface of the point adsorbed by the adsorption pad 24 cannot be recognized. For example, since there are fine irregularities, the adsorption force of the adsorption pad 24 is reduced, so a flat surface is preferred. At this time, the material is read from the cubic data located in the contour portion of the goods 100, and a flat area is retrieved as the grasping point.

[0289] (Example 2) Accurately grasp the center of gravity of the goods 100.

[0290] When the goods 100 are packaged, the contents are not necessarily stored in the center. If the gripping points are set only based on the outer shape, it becomes a so-called single-sided support state, and stable gripping may sometimes not be possible. Therefore, the center of gravity position is calculated based on the overall weight distribution of the cube data, and well-balanced gripping points are set. For example, the gripping points are set such that the center of gravity position lies on the line connecting multiple gripping points.

[0291] (Example 3) Accurately grasp the hardness of the goods 100 (especially the hardness of the appearance).

[0292] When gripping the goods 100, if it is relatively hard and flat, gripping using the adsorption pad 24 is effective. If it is relatively soft and uneven, gripping using the finger part 22 is effective.

[0293] On the other hand, in the case where the packaging material is thin, whether it is the adsorption of the adsorption pad 24 or the grasping of the finger part 22, the gripping force becomes important.

[0294] Therefore, based on the cube data, the gripping points are set in the basic order, and based on the internal attribute information of the goods 100, the gripping points that can be gripped with an appropriate gripping force are adjusted (changed).

[0295] In addition, each block of the cube data, including the recorded attribute information, is sometimes referred to as a voxel.

[0296] A voxel is a three-dimensional pixel value. Just as an image is formed by arranging pixels as two-dimensional pixel values in a plane, an object is formed by arranging voxels as three-dimensional pixel values in a three-dimensional space.

[0297] Figure 19 It is a functional block diagram dedicated to the goods gripping point setting function among the processing functions executed by the information processing device 14.

[0298] The information processing device 14 includes a palm sensor information receiving unit 62 as a goods gripping point setting function.

[0299] In the palm sensor information receiving unit 62, the appearance image information of the goods 100 to be gripped is received from the palm sensor 26 (refer to Figure 20 described later), and is sent to the goods ID reading unit 64 and the contour analysis unit 66.

[0300] In the goods ID reading unit 64, the goods ID, which is the ID information for identifying the goods 100, is read from the appearance image of the goods 100, the label pasted on the goods 100, etc. The goods ID is bound to the information related to the goods content (cube data, sender, destination, etc.) pre-stored in the information management server 60 (refer to Figure 20 described later).

[0301] The goods ID reading unit 64 is connected to the goods ID sending unit 68, and sends the read goods ID to the goods ID sending unit 68.

[0302] In the goods ID sending unit 68, the goods ID is sent to the information management server 60 (refer to Figure 20 described later).

[0303] The receiving unit 70 receives the cube data as a response to the goods ID sent by the goods ID sending unit 68, and sends it to the contour matching unit 72.

[0304] That is, the goods ID sending unit 68 and the receiving unit 70 function as a cube data acquisition unit for acquiring cube data based on the goods ID.

[0305] On the other hand, in the contour analysis unit 66, based on the appearance image of the goods 100 from the palm sensor 26, the contour of the goods 100 in the state of facing the humanoid robot 1 is analyzed, and sent to the contour matching unit 72.

[0306] In the contour matching unit 72, matching considering the appearance orientation of the goods 100, the orientation of the humanoid robot 1 in the confrontation state, and the orientation in the data is performed, and sent to the internal information estimation unit 74.

[0307] The internal information estimation unit 74 estimates the weight, material, center of gravity position, etc. of the goods 100 based on the attribute information of each voxel of the cube data.

[0308] The internal information estimation unit 74 is connected to the grip point setting unit 76, and sends the estimated information such as the weight, material, and center of gravity position of the goods 100 to the grip point setting unit 76.

[0309] In the grip point setting unit 76, the grip point is set based on the estimated information. For example, the grip point is preferably as flat and hard as possible, and preferably a position close to the center of gravity position.

[0310] The grip point setting unit 76 is connected to the grip execution unit 78. In the grip execution unit 78, based on the grip point set by the grip point setting unit 76, the grip of the goods 100 is executed.

[0311] (Operation execution site)

[0312] As Figure 20 shown, for the goods 100, operations are performed through the goods consolidation site and the sorting site.

[0313] A conveyor device 50 is provided at the goods consolidation site. The conveyor belt 50A of the conveyor device 50 moves along the Figure 20 arrow A direction with a driving force not shown, and the goods 100 are sequentially fromFigure 20 It performs the distribution operation of transporting the goods 100 from left to right and storing the goods 100 in a specified shelf or the like.

[0314] On the other hand, at the sorting site, the humanoid robot 1 sorts the goods 100 stored in the shelf or the like at the goods distribution site and performs operations such as transportation and packaging.

[0315] An X-ray analysis device 52 is provided in the conveyor device 50 at the goods distribution site.

[0316] The X-ray analysis device 52 arranges an X-ray generating device 54 at the upper part of the housing (above the conveyor belt 50A) and arranges an X-ray sensor 56 at the lower part of the housing (below the conveyor belt 50A).

[0317] The X-ray generating device 54 and the X-ray sensor 56 are respectively connected to the controller 58. When the goods 100 reach the inspection position, X-rays are irradiated from the X-ray generating device 54 based on the instruction of the controller 58, and the X-rays passing through the goods 100 are detected by the X-ray sensor 56.

[0318] In the controller 58, the detection information (X-ray image information) obtained by the X-ray sensor 56 is bound to the ID (goods ID) for identifying the goods 100 and temporarily stored.

[0319] In the above Figure 14 Food trays, spouted bags, vacuum packaging, cans, and PTP (blister packaging) are exemplified. However, in any of them, the parts with large mass (specific gravity) are shown as X-ray images. Therefore, compared with visible images, the analysis of the data recorded in each voxel of the cube data such as weight, material, and center of gravity position becomes easier.

[0320] In addition, in addition to the goods 100 exemplified in the above Figure 14 (for example, the goods 100 packed in a cardboard box, the position of the items inside the cardboard box, etc.), the images obtained by the X-ray sensor 27 are also effective in the analysis of the center of gravity position. Sometimes, there are goods 100 including cardboard boxes, with the original goods in a part inside and the other part being space. In this case, the image information obtained by the X-ray sensor can grasp the position of the internal goods.

[0321] In addition, in the present embodiment, the X-ray analysis device 52 is used to analyze the inside of the goods 100, but other internal inspection devices such as an infrared analysis device can also be used.

[0322] (Information management server 60)

[0323] The controller 58 of the X-ray analysis device 52 is connected to the information management server 60 and sends the goods ID and X-ray image information temporarily stored in the controller 58 to the information management server 60.

[0324] The information management server 60 includes an information collection control unit 80 and an information provision control unit 82.

[0325] (Information collection control unit 80)

[0326] The information collection control unit 80 includes an information receiving unit 84 that receives the goods ID and X-ray image information temporarily stored in the controller 58.

[0327] The information receiving unit 84 is connected to the analysis unit 86 and sends the received goods ID and X-ray image information to the analysis unit 86.

[0328] In the analysis unit 86, the X-ray image information is analyzed and cubic data is generated. The generated cubic data is stored in the database 88 as cubic data bound to the goods ID.

[0329] The database 88 is connected to the sending unit 89 and sends the cubic data bound to the goods ID based on a request from the information provision control unit 82 described later. Additionally, the database 88 does not have to be provided in the information management server 60 and an external memory can also be used.

[0330] (Information provision control unit 82)

[0331] The information provision control unit 82 includes a receiving unit 90 that receives a request signal sent from Figure 19 the goods ID sending unit 68. The receiving unit 90 is connected to the goods ID extraction unit 92, extracts the goods ID from the sent request signal, and sends it to the cubic data reading unit 94.

[0332] The cubic data reading unit 94 reads the cubic data from the database 88 using the goods ID as a retrieval condition via the sending unit 89 of the information collection control unit 80 and sends it to the sending unit 96.

[0333] The sending unit 96 sends the cubic data to Figure 19 the receiving unit 70.

[0334] In the information processing device 14 of the humanoid robot 1 that has received the cubic data, the gripping point of the goods 100 is set by matching with the ID of the goods 100 that is about to be gripped now.

[0335] Figure 21 Shows the one related to Figure 1The overall motion linkage of the humanoid robot 1, a flowchart of the grasping control sequence when the information processing device 14 executes and the grasping part 20 grasps the goods 100.

[0336] In step 260, it is judged whether there is an instruction to grasp the goods 100. If the judgment is affirmative, it transfers to step 262 to move the humanoid robot 1 (for example, move the arm parts 5 and 6), make the palm side 20A face the target goods 100, and transfer to step 264.

[0337] In step 264, make the palm side 20A face each other and detect the information of the goods 100.

[0338] In the next step 266, analyze the detection information obtained by the palm sensor 26 (high-resolution camera and MoPU), master in detail the type (shape, size, hardness, etc.) and position of the goods 100, and transfer to step 268.

[0339] In step 268, execute the goods grasping point setting process (refer to Figure 22 , which will be described in detail later), and transfer to step 270.

[0340] In step 270, select the operation for grasping the goods 100. For example, based on the result of the goods grasping point setting process in step 268, select from only "adsorption", only "grasping", using both "adsorption" and "grasping", etc. Then, transfer to step 272, and based on the set grasping point, set the angles (opening degrees) of the finger parts 22A, 22B, and 22C, and transfer to step 274.

[0341] That is, based on the appearance obtained by the palm sensor 26, adjust the angles of the finger parts 22A, 22B, and 22C of the grasping part 20, and based on the cube data, determine the grasping points of the respective finger parts 22A, 22B, and 22C, thereby being able to stably grasp the goods 100.

[0342] In addition, the X-ray sensor 27 has the disadvantage that it cannot detect light objects. However, by making use of the advantage of judging whether the weight of the goods 100 is above the threshold value, it can be used to determine whether a "grasping" action is required.

[0343] In step 274, execute the grasping of the goods 100 (only "adsorption", only "grasping", "adsorption" and "grasping").

[0344] In the next step 276, judge whether the grasping of the goods 100 is successful. If the judgment is affirmative, transport the grasped goods 100 to a specified place, transfer to step 260, and wait for the next instruction to grasp the goods 100.

[0345] In addition, in the case of a negative determination in step 276, the process proceeds to step 278, where error handling (such as retry or cancellation) is performed, and then returns to step 260.

[0346] Figure 22 is a control flowchart of the cargo gripping point setting processing subroutine in Figure 21 step 268.

[0347] In step 280, based on the detection information of the palm sensor 26, the cargo ID is read. Then, the process proceeds to step 282, where the contour of the cargo 100 is identified based on the detection information of the palm sensor 26.

[0348] In the next step 284, the cargo ID is sent to the information management server 60 (request for cube data).

[0349] In the next step 286, it is determined whether the cube data has been received. If the determination is affirmative, the process proceeds to step 288, where the orientation of the cargo 100 facing the humanoid robot 1 is matched with the orientation of the object on the cube data, and then the process proceeds to step 290.

[0350] In step 290, the cube data is analyzed (combining the information recorded in each voxel) to estimate the internal information. For example, the weight, material, center of gravity position, etc. of the cargo 100 are estimated.

[0351] In the next step 292, the gripping point of the cargo 100 is set. This setting is, for example, a flat place suitable for adsorption, a hard place with strong adsorption force, a position close to the center of gravity position, etc., and then returns to Figure 21 step 270.

[0352] As described above, according to the present embodiment, by providing three finger portions 22A, 22B, and 22C in the gripping portion 20, and installing a plurality of adsorption pads 24 on the palm side 20A of the gripping portion 20 and the finger portions 22A, 22B, and 22C, the adsorption pads 24 adsorb the cargo 100 by, for example, an air adsorption structure, and the cargo 100 can be grasped by bending the finger portions 22A, 22B, and 22C.

[0353] By installing the palm sensor 26 including a high-resolution camera and a MoPU on the palm side 20A and installing the gripping portion 20 having the above structure on the arm portions 5 and 6 of the humanoid robot 1, it is possible to reliably pick up an item through the adsorption surface, and even if the movement of the humanoid robot 1 is fast, the cargo 100 can be transported without falling from the gripping portion 20.

[0354] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the cargo 100 can be captured with high precision, and it is also possible to handle operations that perform minute movements.

[0355] Moreover, for very soft and fragile items, it is possible to grasp them by the actions of the finger parts 22A, 22B, and 22C without using the adsorption pad 24. By adjusting the grasping force, it is possible to prevent breakage and the like of the soft goods 100.

[0356] In addition, in the present embodiment, the information management server 60 generates cubic data based on information from the X-ray analysis device 52 at the goods distribution site, but it is also possible to pre-store a database of goods ID - cubic data in the database of the information management server 60.

[0357] [Fifth Embodiment]

[0358] Hereinafter, the fifth embodiment of the present disclosure will be described.

[0359] In the fifth embodiment, the same reference numerals are given to the same components as in the first embodiment, and the description of their structures is omitted.

[0360] (Structure of the Gripping Portion 20)

[0361] As Figure 23 shown, the gripping portion 20 installed at the front ends of the arm portions 5 and 6 has a hand structure similar to that of a human, and the gripping portion 20 is rotatably installed with respect to the arm portions 5 and 6 (refer to Figure 1 ).

[0362] The gripping portion 20 shown in the present embodiment has three finger parts 22A, 22B, and 22C each having a plurality of joints. In the present embodiment, the three-finger structure is as Figure 24A shown, and is the same for both the right hand and the left hand, but it may also be a five-finger structure (finger parts 22A, 22B, 22C, 22D, and 22E) as Figure 24B shown.

[0363] That is, the gripping portion 20 of the present embodiment has two gripping actions, namely, gripping by the finger parts 22A, 22B, and 22C and adsorption by the adsorption pad 24. Hereinafter, only gripping, only adsorption, combined gripping and adsorption are collectively referred to as "gripping", and when distinguishing each gripping, an annotation is added each time (for example, "gripping by grasping", "gripping by adsorption", etc.).

[0364] Here, membrane bodies 102 that can be deformed into various shapes are respectively provided between the fingers of the gripping portion 20 (between the finger part 22A and the finger part 22B, between the finger part 22B and the finger part 22C, and between the finger part 22C and the finger part 22A). The membrane body 102 is in a thin film shape (for example, webbed shape), and is deformed based on the actions of the gripping portion 20 (finger parts 22A, 22B, and 22C), and as a whole, forms a bag-shaped space in front of the palm.

[0365] In addition, since the film body 102 is mounted on the sides of the respective finger portions 22A, 22B, and 22C, the pad portion 24A of the suction pad 24, as Figure 24A and Figure 24B shown, becomes a structure that does not interfere with the film body 102 and can directly contact the goods 100.

[0366] In terms of physical properties, as an example, the film body 102 is represented by elastic materials (soft materials) such as silicon, rubber, and polyethylene films, but it can also be wire or rope materials that can be freely deformed by being woven into a net shape.

[0367] In the present embodiment, the film body 102 has a function of assisting the holding force when holding the goods 100, and the held goods 100 are stored in a bag-shaped space formed by the film body 102. That is, for example, in the case of soft goods 100 or the like where there are restrictions on the holding force (grasping force, adsorption force), by wrapping the goods 100 with the film body 102, stable holding can be performed.

[0368] In addition, by being stored in the space of the bag body, the goods 100 held by the holding portion 20 are prevented from falling off even if the grasping force or the adsorption force decreases, and also function as a so-called fail-safe.

[0369] The details of the palm sensor 26 are as described above, so the repeated description thereof is omitted.

[0370] The information processing device 14 mounted on the humanoid robot 1 (refer to the above Figure 5 ) determines the position of the goods 100 with high precision through the information from the palm sensor 26 including a high-resolution camera and a MoPU, calculates the degree of expansion of the finger portions 22A, 22B, 22C during holding, the strength during grasping, the adsorption force of the suction pad 24, etc., and controls the minute movements of the arm portions 5, 6 and the holding portion 20 with high precision, and can cope with the sorting operations of various goods 100.

[0371] (Grasping control of goods 100)

[0372] Refer to the above Figure 7 , and the grasping control sequence when grasping the goods 100 according to the fifth embodiment will be described.

[0373] As described above, in step 150, it is determined whether there is an instruction to grasp the goods 100. If it is a positive determination, it transfers to step 152, and the humanoid robot 1 moves (for example, the arm portions 5, 6 act), so that the palm side 20A faces the target goods 100, and then transfers to step 154.

[0374] In step 154, the palm side 20A faces each other, and the information of the goods 100 is detected.

[0375] In the next step 156, based on the detection information of the palm sensor 26 (high-resolution camera and MoPU), the type (shape, size, hardness, etc.) and position of the goods 100 are analyzed in detail, and the process proceeds to step 158.

[0376] In step 158, an operation for holding the goods 100 is selected. For example, it is selected from only "adsorption", only "grasping", combined use of "adsorption" and "grasping", etc. Then, the process proceeds to step 160, and the angles (opening degrees) of the finger parts 22A, 22B, and 22C are set based on the shape of the goods 100, and the process proceeds to step 162.

[0377] In step 162, the goods 100 are held (only "adsorption", only "grasping", "adsorption" and "grasping").

[0378] Here, in the present embodiment, since the film body 102 is provided between the finger parts 22A, 22B, and 22C of the holding part 20, when performing a grasping action with the finger parts 22A, 22B, and 22C and / or an adsorption action of the adsorption pad 24, the goods 100 are received in the space in front of the palm formed by the film body 102 and are closely attached to the goods 100 in a wrapped manner.

[0379] (Example of the holding action performed by the holding part 20)

[0380] Figure 25 An example of the action when holding the goods 100 in the holding part 20 according to the fifth embodiment is shown.

[0381] The holding part 20 has three finger parts 22A, 22B, and 22C each having a plurality of joints, and has a structure imitating the shape of a human hand.

[0382] In the holding part 20, since the film body 102 is provided between the finger parts 22A, 22B, and 22C, when the holding part 20 holds the goods 100, it can be received in the bag-shaped space formed by the film body 102 and functions as an aid for the holding state of the goods 100.

[0383] For example, for very soft and easily damaged items, the finger parts 22A, 22B, and 22C are used to perform grasping without using the adsorption pad 24, but the wrapping of the film body 102 effectively functions as an aid for this grasping action. In other words, by the stability provided by the wrapping of the film body 102, the grasping force is adjusted, and it is possible to prevent breakage of the soft goods 100, etc.

[0384] In addition, the goods 100 held by the holding part 20 are prevented from falling off even if the grasping force or the adsorption force decreases (fail-safe function).

[0385] In the next step 164 , it is determined whether the gripping of the cargo 100 is successful. If it is determined to be positive, the gripped cargo 100 is transported to a predetermined location, and the process moves to step 150 to wait for an instruction to grip the next cargo 100 .

[0386] If a negative determination is made in step 164 , the process proceeds to step 166 , where an error process (such as a retry or a cancellation) is performed, and the process returns to step 150 .

[0387] As described above, according to this embodiment, by installing the film 102 on the gripping portion 20 , the film 102 forms a bag-like space for storing the object 100 , and can be used as an auxiliary gripping force for the grasping action of the fingers 22A, 22B, and 22C and the suction action of the suction pad 24 .

[0388] In addition, since the film body 102 stabilizes the gripping state of the object 100 , it is possible to prevent the film body 102 from falling off due to vibration when the humanoid robot 1 moves.

[0389] Furthermore, when silicone rubber having adhesiveness is used as the material of the film 102 , even if the cargo 100 is stored in a narrow box and the suction pad 24 cannot be positioned as expected, the film 102 can adhere to and pull out the cargo 100 .

[0390] In addition, in the present embodiment, the membrane 102 is always arranged between the finger portions 22A, 22B, and 22C, but it may be arranged to be appropriately accommodated. Figure 26A and Figure 26B As shown, a telescopic rod 104 that performs joint motion in conjunction with the fingers 22A, 22B and 22C is installed at the front end of the arm 5 (6), and the film body 102 is installed on the telescopic rod 104.

[0391] The telescopic rod 104 is composed of a plurality of short rods, and the membrane 102 is installed in a dotted manner on each short rod, so that when the telescopic rod 104 is extended, the membrane 102 is installed between the finger parts 22A, 22B and 22C ( Figure 26A When the telescopic rod 104 is retracted, the membrane 102 is folded ( Figure 26B status).

[0392] Figure 26C is in the above Figure 7 In the grip control process, the routine interrupted at the start of gripping acquires information related to the cargo 100 detected by the palm sensor 26 in step 300, and then moves to step 302 to determine whether the film 102 is needed. Whether the film 102 is needed is determined by comprehensively judging the weight, size, center of gravity offset, etc. of the cargo 100.

[0393] In the next step 304, the determination result of step 302 is confirmed. When it is determined in step 304 that the film body 102 is required, the process proceeds to step 308, and the telescopic rod 104 is extended to take out the film body 102. Thereby, the film body 102 is placed between the finger portions 22A, 22B, and 22C ( Figure 26A state).

[0394] On the other hand, if it is determined in step 304 that the film body 102 is not required, the process proceeds to step 308, and the telescopic rod 104 is retracted to store the film body 102. Thereby, the film body 102 is removed from between the finger portions 22A, 22B, and 22C and stored ( Figure 26B state).

[0395] The movement between the use state and the storage state of the film body 102 is not limited to the telescopic rod, and other structures such as a winding type may also be used.

[0396] In addition, in the present embodiment, taking the humanoid robot 1 as an example, the general operation in which the humanoid robot 1 moves to the first specified position to hold the cargo 100 and transports it to the second specified position has been described. However, the holding portion 20 of the present embodiment can effectively exhibit its effects in the following specific operations 1 to 4, for example. In addition, the holding portion 20 according to the present embodiment is not limited to the following specific operations 1 to 4, and can be applied to all operations including the process of holding the cargo 100.

[0397] (Specific operation 1) An operation of holding the cargo 100 at the first specified position, transporting it to the second specified position, and packing the cargo 100 into a predetermined storage portion (such as a cardboard box) at the second specified value.

[0398] (Specific operation 2) An operation of holding and picking up the cargo 100 transported by a belt conveyor or the like at a fixed position and packing it into a storage portion (such as a cardboard box) on the spot.

[0399] (Specific operation 3) An operation of receiving the cargo 100 held and transported by another humanoid robot 1 and packing it into a storage portion (such as a cardboard box) on the spot.

[0400] (Specific operation 4) An operation of unpacking the delivered storage portion (such as a cardboard box), taking out the cargo 100, and transporting it to a specified (based on a pre-assigned identifier) management shelf for storage.

[0401] As described above, according to the present embodiment, by providing three finger portions 22A, 22B, and 22C on the holding portion 20, and installing a plurality of adsorption pads 24 on the palm side 20A of the holding portion 20 and the finger portions 22A, 22B, and 22C, the adsorption pads 24 adsorb the goods 100 by, for example, an air adsorption structure, and the goods 100 can be grasped by bending the finger portions 22A, 22B, and 22C. Moreover, the film body 102 serves as a holding aid and can maintain a stable holding state.

[0402] In addition, for the holding action (the grasping action of the finger portions 22A, 22B, and 22C and / or the adsorption action of the adsorption pads 24), a bag-shaped space is formed around the palm by the film body 102 to enclose a part or all of the goods 100. Therefore, the holding state of the above-mentioned holding action can be assisted, and the goods 100 can be prevented from falling off.

[0403] Furthermore, by installing a palm sensor 26 including a high-resolution camera and a MoPU on the palm side 20A and installing the holding portion 20 with the above structure on the arm portions 5 and 6 of the humanoid robot 1, items can be sorted reliably through the adsorption surface. Even if the action of the humanoid robot 1 is fast, the goods 100 can be transported without falling from the holding portion 20.

[0404] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the goods 100 can be captured with high precision, and it is also possible to handle operations that require minute movements.

[0405] Moreover, for very soft and fragile items, they can be grasped by the actions of the finger portions 22A, 22B, and 22C without using the adsorption pads 24. By adjusting the grasping force, damage to the soft goods 100 can be prevented.

[0406] The above has described the present disclosure using the embodiments, but the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It is also obvious that the ways of adding such changes or improvements can be included in the technical scope of the present disclosure based on the description of the claims.

[0407] It should be noted that the execution order of each process such as operations, sequences, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings can be implemented in any order as long as it is not specifically stated as "before", "prerequisite", etc. or the output of the previous process is not used in the subsequent process. For the operation processes in the claims, the specification, and the drawings, even if they are described using "first", "then", etc. for convenience, it does not mean that they must be implemented in that order.

[0408] The disclosures of Japanese Patent Application No. 2022-195944, filed on December 7, 2022, Japanese Patent Application No. 2023-010484, filed on January 26, 2023, Japanese Patent Application No. 2023-010485, filed on January 26, 2023, Japanese Patent Application No. 2023-014884, filed on February 2, 2023, and Japanese Patent Application No. 2023-015496, filed on February 3, 2023 are hereby incorporated by reference in their entirety into this specification. For all documents, patent applications, and technical standards cited in this specification, each document, patent application, and technical standard is incorporated by reference into this specification to the same extent as if it were specifically and individually recited herein.

Claims

1. A control system for a robot, which is a control system for a robot capable of causing a holding part to confront an object to hold the object, wherein, The holding part includes a palm part that serves as the basis for holding the object and a plurality of finger parts that extend radially from the palm part. The control system of the robot has: An adsorption pad is provided on the palm part and the finger parts to adsorb the object; A palm sensor part is provided on the palm part to detect object information of the object including the shape and position; And A control part, based on the detection result of the palm sensor part, selects any one of a first function adsorbed by the adsorption pad, a second function grasped by the palm part, and a third function that combines the first function and the second function as the function for holding the object, and controls the action of holding the object.

2. The control system of the robot according to claim 1, wherein The palm sensor part includes: A camera that captures an image of the object to identify the type of the object; and A motion processing unit that determines the position of the object.

3. The control system of the robot according to claim 1, wherein The control part allows the selection of the first function when there is a surface on which the object can be adsorbed, and allows the selection of the second function when the holding force of the object is prioritized.

4. The control system of the robot according to claim 1, wherein The control system of the robot further has an execution part. When multiple robots are within a specified range of each other, the execution part determines whether there is a coincidence point in their moving paths. When it is determined that there is a coincidence point in the moving paths, the execution part selects a robot to be the contact avoidance object from the multiple robots, and causes the selected robot to be the contact avoidance object to perform contact avoidance.

5. The control system of the robot according to claim 4, wherein The execution part selects a robot with a lighter load for holding the object based on the information mutually transmitted and received between the multiple robots. When there is no difference in the holding state of the object, the selection is made in the order of priority of the predetermined identification information.

6. The control system of the robot according to claim 4, wherein As the contact avoidance behavior of the robot to be the contact avoidance object, the execution part selects and executes a detour action of deviating from the normal travel route and making a detour, and a deceleration or pause action of decelerating or pausing before contact while maintaining the normal travel route.

7. A control system for a robot, which is a control system for a robot capable of causing a holding part to confront an object to hold the object, wherein, The holding part includes a palm part that serves as the basis for holding the object and a plurality of finger parts that extend radially from the palm part. The control system of the robot has: An adsorption pad is provided on the palm part and the finger parts to adsorb the object; A palm sensor part is provided on the palm part to detect object information of the object including the shape and position; A center-of-gravity position recognition part that can recognize at least the center-of-gravity position of the object; And The control unit selects any one of a first function of adsorbing by the adsorption pad, a second function of grasping by the palm part, and a third function of using both the first function and the second function as the function of holding the object, based on the detection result of the palm sensor part and the recognition result of the center-of-gravity position recognition part, and controls the action of holding the object.

8. The control system of the robot according to claim 7, wherein the center-of-gravity position recognition part includes: a state sensor, provided together with the palm sensor part, for detecting at least the internal state of the object; and an analysis part, for analyzing the center-of-gravity position of the object according to the internal state detected by the state sensor.

9. The control system of the robot according to claim 7, wherein the center-of-gravity position recognition part functions as an information acquisition part, and the information acquisition part acquires the center-of-gravity position information of the object bound and sent with the identification code of the object from an analysis device that analyzes the center-of-gravity position of the object.

10. The control system of the robot according to claim 7, wherein the palm sensor part includes: a camera, for taking an image of the object to identify the type of the object; and a motion processing unit, for determining the position of the object.

11. The control system of the robot according to claim 7, wherein the control unit allows the selection of the first function when there is a surface on which the object can be adsorbed, and allows the selection of the second function when the holding force of the object is prioritized.

12. A control system of a robot, which is a control system of a robot that performs an operation of holding an object by a holding part, and has: An information processing device, mounted on the robot, controls the actions of the robot; and an information management server, for uniformly managing information related to the holding of the object by a plurality of the robots, the information management server has: an analysis part, for generating cubic data according to the object to be held by the robot; and a sending part, for binding the cubic data generated by the analysis part with the ID information for identifying the object and storing the same in a database, and sending the cubic data in response to a request based on the ID information from the information processing device, the information processing device has: a cubic data acquisition part, for acquiring the cubic data from the database from the information management server based on the ID information acquired by the palm sensor part; and a setting part, for setting the holding point of the object according to the cubic data acquired by the cubic data acquisition part.

13. A control system for a robot, which is a control system for a robot having a gripping portion, a suction pad, and a palm sensor portion and performing an operation of gripping an object by the gripping portion, wherein: The holding part holds the object; the adsorption pad is provided on the holding part and adsorbs the object; the palm sensor part is provided on the holding part and detects object information of the object including the shape and the position. The control system of the robot has: an information processing device, mounted on the robot, for controlling the action of the robot; and an information management server, for uniformly managing information related to the holding of the object by a plurality of the robots, the information management server has: An analysis unit that subdivides the object to be held by the robot into a plurality of blocks with coordinates determining their relative positions to each other, and generates cube data for each of the plurality of blocks, recording attribute information including internal information of the object; A storage unit that binds the cube data generated by the analysis unit to ID information for identifying the object and stores it in a database; And A sending unit that retrieves and sends out the cube data corresponding to the request for a request based on the ID information from the information processing device; The information processing device has: A palm sensor information receiving unit that receives the ID information and confrontation state information of the object based on the detection result of the palm sensor unit; A cube data acquisition unit that requests the cube data from the database for the information management server by based on the ID information and the confrontation state information received by the palm sensor information receiving unit, and obtains the cube data from the database; And A setting unit that sets a holding point of the object according to the cube data obtained by the cube data acquisition unit.

14. The control system of the robot according to claim 13, wherein, The information processing device further has an action control unit that, based on the detection result of the palm sensor unit, selects any one of a first function adsorbed by the adsorption pad, a second function grasped by the holding part, and a third function using both the first function and the second function as the function for holding the object, and controls the action of holding the object.

15. The control system of the robot according to claim 13, wherein, The palm sensor unit includes: A camera that captures an image of the object to identify the type of the object; and A motion processing unit that determines the position of the object.

16. A robot hand having: A holding part composed of a plurality of finger-like structures for grasping and holding an object; An adsorption pad provided on the holding part to confront the object to adsorb the object; and A film-like membrane body spanned between the plurality of finger-like structures of the holding part and formed to be deformable in response to the action of the holding part.

17. The robot hand according to claim 16, wherein, The membrane body can take a first state spanned between the plurality of finger-like structures and a second state excluded from between the plurality of finger-like structures.

18. A control system of a robot that can confront an object to hold the object, having: A holding part composed of a plurality of finger-like structures for grasping and holding the object; An adsorption pad provided on the holding part to confront the object to adsorb the object; A film-like membrane body spanned between the plurality of finger-like structures of the holding part and formed to be deformable in response to the action of the holding part; A palm sensor unit provided on the holding part to detect object information including the shape and position of the object; and A control unit that controls the action of holding the object based on the detection result of the palm sensor unit.

19. The control system of the robot according to claim 18, wherein the palm sensor unit includes: a camera that captures an image of the object to identify the type of the object; and a motion processing unit that determines the position of the object.

20. The control system of the robot according to claim 18, wherein the control unit selects, based on the detection result of the palm sensor unit, any one of a first function adsorbed by the adsorption pad, a second function grasped by the palm unit, and a third function that combines the first function and the second function as the function for holding the object, and controls the action of holding the object.

21. A control program for a robot, the control program for the robot causing a computer to operate as the control unit according to any one of claims 1 to 6.

22. A control program for a robot, the control program for the robot causing a computer to operate as the control unit according to any one of claims 7 to 11.

23. An information management program, the information management program causing a computer to operate as each unit of the information management server according to claim 12 or 13.

24. An information processing program, the information processing program causing a computer to operate as each unit of the information processing device according to claim 12 or 13.

25. A control program for a robot, the control program for the robot causing a computer to operate as the control unit according to any one of claims 18 to 20.

Images