Machine learning device and robot system

Abstract

To materialize a robot system capable of properly gripping an object that has low reaction force and an indeterminate form.SOLUTION: A robot system is equipped with a robot 20 that has a hand portion provided with a set of moving claws, and grips work by pinching it with the moving claws, and a control device 70 that controls the robot 20. The control device 70 is provided with a host controller 80 that controls the robot 20 so as to execute a disposing action disposing the hand portion to a gripping position and a gripping action displacing the moving claws so as to come close to each other at the gripping position, and a machine learning device 90 that obtains stop reference data set when the work is gripped, distance data indicating a distance between each moving claw of the hand portion disposed at the gripping position and the work, and comparison data indicating the deformation amount of the work before / after the gripping action, and constructs a model used for setting an action mode of the gripping action by machine learning using these stop reference data, distance date, and comparison data.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a machine learning device and a robot system. [Background technology]

[0002] Some robot systems are capable of gripping an object (workpiece) by pinching the object with a set of movable claws (gripping parts) provided at the tip of the robot's arm. In this type of robot system, for example, a system has been proposed that detects changes in the current value of a motor for the movable claws or a reaction force from the object, and stops the movable claws when the detected value reaches a preset reference value (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-24134 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, when food such as cream puffs, daifuku, bread, etc. are assumed as the target, the shape of the target varies more (irregular shape) and the reaction force from the target when gripped is smaller (low reaction force) than when industrial products made of metal or synthetic resin are assumed as the target. When gripping an object by pinching it with a pair of movable claws as described above, one movable claw may hit the object before the other movable claws, causing the pressure applied to the object to be locally high, or the object pushed by one movable claw may slide on the surface on which the object is placed. In particular, since the shape of objects such as cream puffs varies greatly, it is assumed that when the gripping unit is positioned according to the position of the object identified from an image, etc., there will be a difference in the distance between each movable claw and the object. If such a difference occurs, the timing at which each movable claw hits the object will be shifted. Furthermore, since objects such as cream puffs are soft, there is a high possibility that they will be damaged (including excessive deformation) when they are pushed by the movable claws and slide along the mounting surface while being deformed. Damage to the object in this way can be a factor in reducing the yield in the manufacturing process, etc., and there is a concern that it will be an obstacle to improving manufacturing efficiency through the application of a robot system. Thus, there is still room for improvement in the configuration of the robot system in terms of appropriately gripping objects that have low reaction force and irregular shapes.

[0005] The present invention has been made in consideration of the above circumstances, and a main object of the present invention is to realize a robot system capable of appropriately gripping an object with low reaction force and an irregular shape. [Means for solving the problem]

[0006] Means for solving the above problems will be described below.

[0007] First means: A robot (robot 20) having a gripping unit (hand unit 38) with a pair of movable claws (movable claws 38a, 38b) and configured to grip an object (workpiece W) by pinching it between the movable claws; a control device (control device 70) that controls the robot so that a placement operation for placing the gripper at a predetermined position (gripping position) where the object is to be located between the pair of movable claws and a gripping operation for displacing the movable claws toward each other at the predetermined position are performed; Equipped with the control device is configured to stop the displacement of the set of movable claws when a reaction force from the object during the gripping operation reaches a reference value or when a relative distance between the set of movable claws reaches a reference value, Under a condition in which the gripping portion is disposed at the predetermined position, an adjustment operation can be performed to adjust the position of the set of movable claws with respect to the object by displacing the gripping portion, the reference value for stopping the gripping operation is a variable value, Stop reference data indicating the reference value set when gripping the object, distance data indicating the distance between each movable claw of the gripping part arranged at the predetermined position and the object, and a state of the object before the gripping operation is performed and a state of the object after the gripping operation is performed of A model construction unit (machine learning device 90) acquires comparison data (e.g., the deformation amount of the workpiece W) showing a difference from the state of the target object, and constructs a model (motion setting model) used for setting the motion mode of the predetermined motion including the adjustment motion and the gripping motion by machine learning using the stop reference data, distance data, and comparison data, The control device includes: an acquisition unit (data acquisition unit 85) that acquires the distance data indicating a distance between each movable claw of the gripping unit and the object when the gripping unit is placed at the predetermined position; A setting unit (control unit 84) for setting the motion mode of the predetermined motion of the robot; having A robot system in which the setting unit is capable of setting the operation mode of the specified operation based on the distance data acquired by the acquisition unit and the model constructed by the model construction unit.

[0008] In a configuration in which an object (such as cream puffs, Daifuku, or bread) is gripped by a set of movable claws, when the gripping unit is arranged so that the object is located between the movable claws, a difference in distance between each movable claw and the object may occur. Such a difference is noticeable for objects with large variations in shape (objects of irregular shape). When such a difference occurs, the timing at which each movable claw hits the object is shifted. Here, when the object is soft, that is, when the reaction force is small, there is a concern that the object may be damaged due to such a timing shift. Here, in this means, a model used to set the operation mode of a predetermined operation including an adjustment operation and a gripping operation is constructed by machine learning using the stop reference data, the distance data, and the comparison data. With such a configuration, not only can an optimal reference value be set, but also a configuration in which the position of the movable claws is appropriately adjusted according to the arrangement of the gripping unit can be realized. In other words, it is possible to suppress damage to the object caused by the timing shift and contribute to the realization of a robot system that can appropriately grip objects of irregular shape with low reaction force. In addition to the above-mentioned timing deviation, damage to the object may also occur when the reference value is not set properly, and it is assumed that it is difficult to identify the cause of the damage from the above-mentioned comparison data. Therefore, by using each of the stop reference data, distance data, and comparison data as input data for machine learning, it can contribute favorably to optimizing the predetermined operation (adjustment operation, gripping operation).

[0009] Incidentally, the "comparison data" may be any data that can identify the degree of damage to the object, and may be, for example, data showing a change in shape or data showing a change in weight.

[0010] The "operation mode of the predetermined operation" in this means includes settings for whether the adjustment operation is possible and the amount of displacement of the movable claw during adjustment.

[0011] Second means: The model construction unit is configured to calculate the shape of the object before the gripping operation is performed and the shape of the object after the gripping operation is performed. of Data indicating a difference from the shape of the target object is obtained as the comparison data.

[0012] When a low-reaction object is damaged, changes in weight and shape occur. In many cases where a change in weight occurs, the shape also changes, so by using the shape as a comparison target, the degree of damage to the object can be appropriately reflected in machine learning.

[0013] Third means: The model construction unit acquires shape data indicating the shape of the object before the grasping action is performed, and direction data indicating the relationship between the reference direction of the object (for example, the direction connecting the longest two points in the image of the object: the longitudinal direction) and the direction in which the pair of movable claws clamp the object, and performs the machine learning by associating the stopping reference data, the distance data, the comparison data, the shape data, and the direction data.

[0014] When gripping an object with a low reaction force and an irregular shape, the above-mentioned timing deviation and the influence of the load may differ depending on the direction from which it is pinched. In other words, the positional relationship between the object and each movable claw, which does not require adjustment, may differ depending on the pinching direction. Therefore, as shown in this means, if machine learning is performed by adding shape data and direction data to the various data shown in the first means, etc., the effect shown in the first means can be more suitably exhibited.

[0015] Fourth Means: The part of the set of movable claws that contacts the object is planar, The model construction unit acquires shape data indicating the shape of the object before the grasping operation is performed and contact area data indicating the contact area between the set of movable claws and the object while the object is being grasped, and performs the machine learning by associating the stopping criteria data, the distance data, the comparison data, the shape data, and the contact area data.

[0016] When gripping an object with a low reaction force and an indefinite shape, the contact area may affect the effect on the object. In other words, the contact area may affect the positional relationship between the object and each movable claw, which does not require adjustment. Therefore, as shown in this means, if machine learning is performed by adding shape data and contact area data to the various data shown in the first means, etc., the effect shown in the first means can be more effectively achieved.

[0017] Fifth means: The control device is configured to displace the set of movable claws at a set speed when performing the gripping operation, The speed is variable; The model construction unit acquires speed data indicating the displacement speed at which each of the movable claws displaces toward the target object, and performs the machine learning by associating the stopping criteria data, the distance data, the comparison data, and the speed data.

[0018] When the above-mentioned timing deviation occurs, there is a possibility that the influence on the target object will differ depending on the displacement speed of the movable claw. In other words, there is a possibility that the positional relationship between the target object and each movable claw, which does not require adjustment, will differ depending on the displacement speed. Therefore, as shown in this means, if a configuration is adopted in which speed data is added to the various data shown in the first means etc. and machine learning is performed, the effect shown in the first means can be more suitably exhibited.

[0019] Sixth Means: The model construction unit acquires position data indicating the position of the object before the grasping operation is executed, and performs the machine learning by associating the stopping criteria data, the distance data, the comparison data, and the position data.

[0020] For a robot system, the convenience of the system can be improved by providing a certain tolerance range for the position of the object to be supplied. However, when such a range is provided, there is a possibility that the way in which the movable claws hit the object when performing a gripping operation on the object will differ depending on the position within the range to which the object is supplied. Therefore, as shown in this means, if machine learning is performed by adding position data to the various data shown in the first means, etc., the effect shown in the first means can be more suitably exhibited.

[0021] Seventh means: The model construction unit acquires posture data indicating the posture of the robot when grasping the object, and performs the machine learning by associating the stopping reference data, the distance data, the comparison data, and the posture data.

[0022] For a robot, the way in which the movable claws hit the object may differ depending on the posture when gripping the object. Therefore, as shown in this means, if machine learning is performed by adding posture data to the various data shown in the first means, etc., the effect shown in the first means can be more effectively achieved.

[0023] Eighth Means: The model construction unit acquires environmental data indicating the environment around the robot, and performs the machine learning by associating the stopping criterion data, the distance data, the comparison data, and the environmental data.

[0024] For objects with low reaction force and irregular shape such as cream puffs, daifuku, and bread, the hardness may change depending on environmental conditions such as temperature and humidity. In other words, there is a possibility that the positional relationship between the object and each movable claw, which does not require adjustment, may differ depending on the environmental conditions. Therefore, as shown in this means, if machine learning is performed by adding environmental data to the various data shown in the first means, etc., the effect shown in the first means can be more suitably exhibited.

[0025] Ninth means. The setting unit is configured to set an operation mode so as to grasp the object without adjusting the relative position between the set of movable claws and the object when the distance data acquired by the acquisition unit indicates a distance within a range defined by the model, and to set an operation mode so as to grasp the object after adjusting the relative position between the set of movable claws and the object when the distance data acquired by the acquisition unit indicates a distance outside the range defined by the model.

[0026] Machine learning is expected to improve the accuracy of determining whether position adjustment is necessary, which is favorable for improving the efficiency of transporting objects by a robot system while minimizing damage to the objects.

[0027] Tenth Means: A robot (robot 20) having a gripping unit (hand unit 38) with a pair of movable claws (movable claws 38a, 38b) and gripping an object (workpiece W) by pinching it with the movable claws; a control device (control device 70) that controls the robot so that a placement operation for placing the gripper at a predetermined position (gripping position) where the object is to be located between the pair of movable claws and a gripping operation for displacing the movable claws toward each other at the predetermined position are performed; A robot system comprising: Under a condition in which the gripping portion is disposed at the predetermined position, the position of the set of movable claws with respect to the object can be adjusted by displacing the gripping portion, Distance data indicating a distance between each movable claw of the gripping part arranged at the predetermined position and the object, and a state of the object before the gripping operation is performed and a state of the object after the gripping operation is performed of A model construction unit (machine learning device 90) that acquires comparison data (e.g., the amount of deformation of the workpiece W) that indicates a difference from the state of the target object, and constructs a model used to set the manner of adjusting the position of the gripper at the predetermined position by machine learning using the distance data and the comparison data, The control device includes: an acquisition unit (data acquisition unit 85) that acquires the distance data indicating a distance between each movable claw of the gripping unit and the object when the gripping unit is placed at the predetermined position; a setting unit that sets a manner of the position adjustment based on the distance data acquired by the acquisition unit and the model constructed by the model construction unit; A robot system having the above features.

[0028] In a configuration in which an object (such as cream puffs, Daifuku, or bread) is grasped by a set of movable claws, when the grasping unit is arranged so that the object is located between the movable claws, a difference in distance between each movable claw and the object may occur. Such a difference is more prominent for objects with a large variation in shape (objects of irregular shape). When such a difference occurs, the timing at which each movable claw hits the object is shifted. Here, when the object is soft, that is, when the reaction force is small, there is a concern that the object may be damaged due to such a timing shift. Here, in this means, a model used to set the mode of position adjustment is constructed by machine learning using distance data and comparison data. With such a configuration, it is possible to realize a configuration in which the position of the movable claws is appropriately adjusted according to the arrangement of the grasping unit. In other words, it is possible to contribute to the realization of a robot system that can suppress damage to the object caused by the timing shift and appropriately grasp objects of irregular shape with low reaction force.

[0029] 11th Means: A machine learning device applied to a robot system that has a gripping unit (hand unit 38) provided with a set of movable claws (movable claws 38a, 38b) and grips an object (workpiece W) by pinching it with the movable claws, and a control device (control device 70) that controls the robot so that a placement operation is performed to place the gripping unit at a predetermined position (gripping position) where the object is located between the set of movable claws, and a gripping operation is performed to displace the movable claws toward each other at the predetermined position, the control device being configured to stop the displacement of the set of movable claws when a reaction force from the object reaches a reference value during the gripping operation or when a relative distance between the set of movable claws reaches a reference value, and that is capable of performing an adjustment operation to adjust the position of the set of movable claws relative to the object by displacing the gripping unit under a situation where the gripping unit is placed at the predetermined position, The reference value for stopping the gripping operation is a variable value, Stop reference data indicating the reference value set when gripping the object, distance data indicating the distance between each movable claw of the gripping part arranged at the predetermined position and the object, and a state of the object before the gripping operation is performed and a state of the object after the gripping operation is performed of The machine learning device is equipped with a model construction unit (machine learning device 90) that acquires comparison data (e.g., the deformation amount of the workpiece W) that indicates the difference with the state of the target object, and constructs a model used for setting the operation modes of predetermined operations including the adjustment operation and the gripping operation through machine learning using the stopping reference data, distance data, and comparison data.

[0030] According to the configuration shown in the present invention, a model used to set the operation mode of a predetermined operation (adjustment operation, gripping operation) is constructed by machine learning using the stop reference data, distance data, and comparison data. With such a configuration, not only can an optimal reference value be set, but also a configuration in which the position of the movable claw is appropriately adjusted according to the arrangement of the gripping part can be realized. In other words, it is possible to suppress damage to the object caused by a timing difference when each movable claw hits the object, and contribute to the realization of a configuration that can appropriately grip an object with low reaction force and irregular shape. Note that damage to the object may occur in cases other than the above-mentioned timing difference, such as when the reference value is not set appropriately, while it is assumed that it is difficult to identify the cause of the damage from the above-mentioned comparison data. Therefore, by using each data of the stop reference data, distance data, and comparison data as input data for machine learning, it can contribute favorably to optimizing the gripping operation.

[0031] Twelfth Means: A robot (robot 20) having a gripping unit (hand unit 38) with a pair of movable claws (movable claws 38a, 38b) and gripping an object (workpiece W) by pinching it with the movable claws; a control device (control device 70) that controls the robot so that a placement operation for placing the gripper at a predetermined position (gripping position) where the object is to be located between the pair of movable claws and a gripping operation for displacing the movable claws toward each other at the predetermined position are performed; Equipped with the control device is configured to stop the displacement of the set of movable claws when a reaction force from the object during the gripping operation reaches a reference value or when a relative distance between the set of movable claws reaches a reference value, The clamping direction, which is the direction in which the pair of movable claws are displaced by the gripping action, can be adjusted. The reference value for stopping the gripping operation is a variable value, Stop reference data indicating the reference value set when gripping the object, shape data indicating the shape of the object before the gripping operation is performed, direction data indicating the relationship between the reference direction of the object (for example, the direction connecting the two longest points in the image of the object: the longitudinal direction) and the pinching direction, and the state of the object before the gripping operation is performed and the state after the gripping operation is performed. of A model construction unit (machine learning device 90) acquires comparison data (e.g., the amount of deformation of the workpiece W) that indicates a difference from the state of the target object, and constructs a model used to set each of the operation modes of the placement operation and the gripping operation by machine learning using the stop reference data, shape data, direction data, and comparison data, The control device has an acquisition unit that acquires the shape data of the object before the grasping operation is performed, and is a robot system capable of setting each of the operation modes of the placement operation and the grasping operation based on the shape data acquired by the acquisition unit and the model constructed by the model construction unit.

[0032] In this means, a model used to set the pinching direction is constructed by machine learning using the stop criterion data, shape data, direction data, and comparison data. With such a configuration, not only can an optimal reference value be set, but also a configuration in which the pinching direction is appropriately adjusted according to the shape of the object can be realized. In other words, it is possible to contribute to the realization of a robot system that can suppress damage to the object and appropriately grasp an object with a low reaction force and an irregular shape. Damage to the object can occur not only when the pinching direction is not set properly but also when the reference value is not set properly, while it is assumed that it is difficult to identify the cause of the damage from the above comparison data. Therefore, by using each of the data of the stop criterion data, shape data, direction data, and comparison data as input data for machine learning, it can contribute favorably to the optimization of the placement operation and the gripping operation. [Brief description of the drawings]

[0033] [Figure 1] 1 is a schematic diagram showing a robot according to a first embodiment. [Diagram 2]FIG. 2 is a block diagram showing the electrical configuration of the robot system. [Diagram 3] Schematic diagram showing an operation mode of the robot. [Figure 4] 11A to 11C are schematic diagrams showing a procedure for placing the hand unit at a gripping position. [Diagram 5] FIG. 2 is a functional block diagram of a main control device. [Figure 6] Schematic diagram showing types of input data. [Figure 7] FIG. 1 is a schematic diagram showing a neural network for constructing a behavioral configuration model. [Figure 8] 13A is a flowchart showing an operation mode setting process executed by the CPU of the upper controller, and FIG. 13B is a flowchart showing a position adjustment process executed by the CPU of the upper controller. [Figure 9] 4 is a flowchart showing a learning process executed by a CPU of the machine learning device. [Figure 10] FIG. 13 is a schematic diagram showing input data in the fourth embodiment. [Figure 11] FIG. 13 is a schematic diagram showing input data in the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] <First embodiment> Hereinafter, a first embodiment embodied in a robot system for use in a food factory or the like will be described with reference to the drawings.

[0035] 1, the robot system 10 includes a robot 20, which is a vertical articulated industrial robot, and a motion controller 60 that controls the robot 20, and the robot 20 and the motion controller 60 are connected so as to be able to communicate with each other. The robot 20 is composed of a robot body 30 and a servo amplifier 50 attached to the robot body 30.

[0036] The robot body 30 has a base portion 31 fixed to a floor or the like, a shoulder portion 32 supported by the base portion 31, a lower arm portion 33 supported by the shoulder portion 32, a first upper arm portion 34 supported by the lower arm portion 33, a second upper arm portion 35 supported by the first upper arm portion 34, a wrist portion 36 supported by the second upper arm portion 35, and a flange portion 37 supported by the wrist portion 36.

[0037] The base portion 31 and the shoulder portion 32 are formed with a first joint portion J1 that connects the base portion 31 and the shoulder portion 32, and the shoulder portion 32 is rotatable in the horizontal direction around a connecting axis AX1 of the first joint portion J1. The shoulder portion 32 and the lower arm portion 33 are formed with a second joint portion J2 that connects the shoulder portion 32 and the lower arm portion 33, and the lower arm portion 33 is rotatable in the vertical direction around a connecting axis AX2 of the second joint portion J2. The lower arm portion 33 and the first upper arm portion 34 are formed with a third joint portion J3 that connects the lower arm portion 33 and the first upper arm portion 34, and the first upper arm portion 34 is rotatable in the vertical direction around a connecting axis AX3 of the third joint portion J3. The first upper arm portion 34 and the second upper arm portion 35 are formed with a fourth joint portion J4 that connects the first upper arm portion 34 and the second upper arm portion 35, and the second upper arm portion 35 is rotatable in a torsional direction around a connecting axis AX4 of the fourth joint portion J4. The second upper arm portion 35 and the wrist portion 36 are formed with a fifth joint portion J5 that connects the second upper arm portion 35 and the wrist portion 36, and the wrist portion 36 is rotatable in a vertical direction around a connecting axis AX5 of the fifth joint portion J5. The wrist portion 36 and the flange portion 37 are formed with a sixth joint portion J6 that connects the wrist portion 36 and the flange portion 37, and the flange portion 37 is rotatable in a torsional direction around a connecting axis AX6 of the sixth joint portion J6.

[0038] The shoulder section 32, lower arm section 33, first upper arm section 34, second upper arm section 35, wrist section 36, and flange section 37 are arranged in a series to form an arm in the robot body 30, and a hand section 38 serving as an end effector is attached to the flange section 37, which is the tip of the arm. The connecting axes AX1, AX4, and AX6 are parallel to the longitudinal direction of the arm, and the connecting axes AX2, AX3, and AX5 are perpendicular to the longitudinal direction.

[0039] A motor 41 (specifically, a servo motor) is provided at each of the joints J1 to J6 as a drive unit for rotating the joints J1 to J6. The motor 41 is connected to a servo amplifier 50, and the servo amplifier 50 controls the drive of the motor 41 based on a command received from the motion controller 60 described above.

[0040] Here, the electrical configuration of the robot system 10 will be further described with reference to FIG. 2. The robot system 10 includes a main control device 70 that constitutes a "control device" or "control means" together with the motion controller 60. The main control device 70 includes a host controller 80 that issues operation instructions to the motion controller 60, and a machine learning device 90 that learns (so-called machine learning) an appropriate gripping operation or the like according to various conditions such as the shape and arrangement of the workpiece W. The host controller 80 includes a CPU 81, a memory 82 having a ROM that stores various control programs and fixed value data, and a RAM that can temporarily store various data when the control program is executed. The machine learning device 90 also includes hardware such as a CPU 91 and a memory 92, similar to the host controller 80. The memory 92 includes a ROM that stores a learning algorithm, which is software for learning, and a RAM that stores various input data for learning.

[0041] The motion controller 60 receives an operation instruction from a host controller 80 provided in the main control device 70, reads an operation program corresponding to the operation instruction from a program storage unit, and specifies an operation target position (hereinafter referred to as a target position or a control point) from the read operation program. Thereafter, it generates a target trajectory that smoothly connects the specified target position and the current position of the arm (each movable part) of the robot 20, and sequentially transmits interpolated positions, which are positions obtained by subdividing the target trajectory, to the servo amplifier 50.

[0042] The servo amplifier 50 is provided with a position control section, a speed control section, a current control section, and a storage section for storing various information. The position control section is connected to an encoder 42 attached to the motor 41. The position control section detects the rotational position of the motor 41 (i.e., the posture of the arm) based on the encoder value. The position control section and the speed control section calculate a target torque and a target rotational speed of each motor 41 based on the deviation between the detected rotational position and an interpolated position included in a command received from the motion controller 60. The current control section determines the power (current, voltage, pulses) to be supplied to each motor 41 based on the calculated target torque and target rotational speed, and supplies power to each motor 41.

[0043] Next, the basic operation of the robot 20 will be described with reference to FIG. 3. The robot 20 and the conveyor S1 constitute a part of a production line in a food factory. Food products that have passed through a production process (processing process), specifically soft products with large variations in shape such as cream puffs and eclairs (hereinafter referred to as work W), flow on the conveyor S1, and the robot 20 is responsible for a boxing process of boxing these products. Specifically, a table S2 is provided in parallel with the conveyor S1, and a case C capable of accommodating the work W is provided on the table S2. The conveyor S1 moves the work W that has passed through the production process to a predetermined area in front of the robot 20, and when the work W is transported to the case C by the robot 20, the conveyor S1 sends the next work W to the predetermined area. In this embodiment, the work W corresponds to the "target object" or the "object to be grasped."

[0044] The robot system 10 includes a camera 65 (see FIG. 1) fixed to the ceiling of the building, and an image captured by the camera 65 is transmitted to a host controller 80. When the host controller 80 identifies a workpiece W placed in the above-mentioned predetermined area by image analysis, it changes the posture of the robot 20 to place the hand unit 38 at a gripping position (corresponding to a "predetermined position") where the workpiece W can be gripped (corresponding to a "placement operation"). By thus placing the hand unit 38 at the gripping position, the workpiece W is positioned between the movable claws 38a, 38b of the hand unit 38 (see FIG. 3(a)).

[0045] As shown in Fig. 3(a) to Fig. 3(b), after the workpiece W is placed at the gripping position, both movable claws 38a, 38b of the hand unit 38 are displaced toward each other. A force sensor 45 (e.g., a pressure sensor) is provided on the movable claws 38a, 38b to detect a reaction force from the workpiece W (see Fig. 1), and the reaction force detected by the force sensor 45 is transmitted to the upper controller 80. When the reaction force detected by the force sensor 45 reaches a reference (a reference reaction force for stopping, which will be described later), the displacement of the movable claws 38a, 38b stops, and the clamping of the workpiece W is completed. After the clamping of the workpiece W is completed, the posture of the robot 20 is changed while the workpiece W is being gripped, and the workpiece W is transported to the case C, as shown in Fig. 3(b) to Fig. 3(c). After the workpiece W is stored in the case C, as shown in Figure 3(c) → Figure 3(d), each movable claw 38a, 38b is displaced (returned) to its initial position to release the workpiece W, and the robot 20 returns to its standby position.

[0046] Here, referring to FIG. 4, a supplementary explanation will be given of the flow when the hand unit 38 is placed at the gripping position. When a new workpiece W is supplied to the predetermined area, the predetermined area is photographed. This image data is transmitted to the upper controller 80, and the upper controller 80 extracts the outer shape of the workpiece W based on the acquired image data, and sets the virtual center (temporary center) of the workpiece W from the extracted outer shape as a target position TP, which is one of the target positions. Then, the operation mode (various control points) of the robot 20 is set so that the reference point CP, which is the robot's reference for placing the hand unit 38, coincides with the target position TP. This set operation mode is transmitted to the motion controller 60, and the motion controller 60 determines the operation trajectory of the robot 20 based on this operation mode and the current posture of the robot 20.

[0047] In addition, the opposing surfaces of the movable claws 38a, 38b facing each other are both planar and perpendicular to the opening and closing direction of the movable claws 38a, 38b, and the reference point CP is defined to be a position that is equidistant from both movable claws 38a, 38b (opposing surfaces).

[0048] Here, the workpiece W assumed in this embodiment is a food such as cream puffs or eclairs, and has a large variation in shape compared to industrial products. The distance from the reference point CP to the outer circumference can also vary. Therefore, even if the posture of the robot 20 is changed to match the reference point CP with the target position TP and the hand unit 38 is positioned at a gripping position where the workpiece W is located between both movable claws 38a, 38b, the distance Xa from one movable claw 38a to the workpiece W and the distance Xb from the other movable claw 38b to the workpiece W are not necessarily the same.

[0049] In the example shown in Fig. 4(b), the hand unit 38 is biased so that the distance Xb from the movable claw 38b to the workpiece W is shorter than the distance Xa from the movable claw 38a to the workpiece W. When the movable claws 38a, 38b are displaced from this state toward each other, the timing at which the movable claw 38a hits the workpiece W differs from the timing at which the movable claw 38b hits the workpiece W. Specifically, as shown in Fig. 4(c), the movable claw 38b hits the workpiece W before the movable claw 38a hits the workpiece W.

[0050] Since the reaction force of the workpiece W is small, it is assumed that the movable claw 38b will be significantly deformed by hitting the workpiece W, or that the movable claw 38b will slide toward the movable claw 38a on the placement surface of the conveyor S1 by being pushed by the movable claw 38b. Such an event may cause damage to the workpiece W (including excessive deformation). This may lead to a decrease in the yield in the manufacturing process, and may be an obstacle to improving the manufacturing efficiency of the robot system 10. One of the features of this embodiment is that such concerns are eliminated by making it possible to properly grip the workpiece W using machine learning. Below, a characteristic configuration of this embodiment will be described with reference to FIGS. 5 and 6. FIG. 5 is a functional block diagram showing the functions of the main control device 70, and FIG. 6 is a schematic diagram showing input data input to the machine learning device 90.

[0051] The upper controller 80 includes a control unit 84 that executes a control program stored in the memory 82, and a data acquisition unit 85 that acquires various data from the robot 20 and the camera 65. The data acquisition unit 85 includes an image data acquisition unit 86 that acquires an image of the above-mentioned predetermined area from the camera 65, an attitude data acquisition unit 87 that acquires attitude data indicating the attitude of the robot 20 (for example, the attitude before the gripping operation and the attitude when gripping the workpiece W) from the encoder 42, a distance data acquisition unit 88 that acquires distance data indicating each distance between the workpiece W and each of the movable claws 38a, 38b in a state in which the hand unit 38 is placed at the gripping position (before adjustment, which will be described later) from a distance sensor 46 (see FIG. 1) provided on each of the movable claws 38a, 38b, and a reaction force data acquisition unit 89 that acquires reaction force data indicating a reaction force from the workpiece W when the workpiece W is gripped by the hand unit 38 from the force sensor 45.

[0052] The control unit 84 identifies the position (target position TP) where the workpiece W is placed in the above-mentioned predetermined area from the image data acquired by the image data acquisition unit 86, and stores the position as workpiece position data in the memory 82. In addition, the control unit 84 extracts the shape of the workpiece W before the gripping operation from the image data acquired before the gripping operation and stores the shape as shape data in the memory 82, and extracts the shape of the workpiece W after the gripping operation (after release) from the image data acquired after the gripping operation (after release) and stores the shape as shape data in the memory 82. The timing of acquiring the image data after the gripping operation is the timing when a predetermined time (for example, the time when the workpiece W is expected to restore itself) has elapsed since the release of the workpiece W, but this predetermined time may also be estimated by machine learning.

[0053] In addition, the type of the workpiece W (cream puff, eclair, etc.) is identified from the shape data before the gripping operation, and the identified type is stored as type data in memory 82. The upper controller 80 uses part of this data to control the robot 20, and provides part of this data as input data to the machine learning device 90. The input data provided to the machine learning device 90 is broadly divided into state data and label data.

[0054] The machine learning device 90 is provided with a state observation unit 97 that acquires various state data from the upper controller 80, including work position data, posture data when the work W is gripped (in a state where the work W is placed at the gripping position), shape data before and after gripping, and type data. These data are stored in a data storage unit 95 of the learning unit 94. The state observation unit 97 also compares the shape data before the gripping operation and the shape data after the gripping operation (after release) by template matching to calculate the amount of deformation (damage) of the work W triggered by the gripping operation. Data indicating the amount of deformation (comparison data) is also stored in the data storage unit 95 of the learning unit 94. In this embodiment, the machine learning device 90 is configured to calculate and store the comparison data by itself, but this is not limited to the above. The comparison data can also be configured to be provided (input) to the machine learning device 90 by a user. The machine learning device 90 is also provided with a label data acquisition unit 98 that acquires various label data from the upper controller 80. The label data acquisition unit 98 acquires the stop criterion data indicating the reference reaction force set as the stop criterion and the above-mentioned distance data, and stores them in the data storage unit 95.

[0055] The source of the various input data input to the machine learning device 90 is not limited to the host controller 80. For example, it is also possible to configure the input data to be obtained directly from the robot 20 or the camera 65 without going through the host controller 80.

[0056] The learning unit 94 of the machine learning device 90 learns by associating the above-mentioned various state data and label data, and constructs a model (operation setting model) that represents the correlation between the state data and the label data through this learning. This operation setting model is stored in a learning model storage unit 96 of the learning unit 94, and is updated based on newly acquired input data. By repeatedly updating the operation setting model, a stopping reference reaction force and a position adjustment reference distance that increase the probability of successful grasping (transportation) and reduce damage (deformation amount) of the workpiece W are learned according to the situation. Then, when the learning progresses and a large number of data are reflected in the operation setting model, the use of the operation setting model is permitted.

[0057] The result output unit 99 of the machine learning device 90 presents the estimation result of the appropriate stop reference reaction force to the upper controller 80 based on the operation setting model permitted for use and the data for the operation control of the robot 20 (workpiece position data, posture data before the gripping operation, shape data before the gripping operation, type data), or presents the appropriate position adjustment reference distance to the upper controller 80 based on the operation setting model permitted for use and the data for the operation control of the robot 20 (workpiece position data, posture data when gripping the workpiece W, shape data before the gripping operation, type data). Here, the appropriate stop reference reaction force shown in this embodiment is a minimum estimated value that can grip the workpiece W and minimize damage (deformation amount) of the workpiece W due to the gripping operation, and the appropriate position adjustment reference distance is an estimated value of the difference in distance from each of the movable claws 38a, 38b to the workpiece W, at which the effect (damage) of the deviation in the contact timing is 0 or almost 0.

[0058] In this embodiment, various input data when the workpiece W is successfully grasped is accumulated in the data storage unit 95, while various input data when the workpiece W is unsuccessfully grasped is erased without being stored in the data storage unit 95.

[0059] In the machine learning device 90 described above in detail, so-called supervised learning is applied as a learning algorithm executed by the learning unit 94. Supervised learning is a method of learning a model for estimating an execution result for a new execution condition by identifying a feature that suggests a correlation between the execution condition and the execution result from a known data set (so-called teacher data) of the execution condition and the corresponding execution result.

[0060] In this supervised learning, a neural network is used to construct an action setting model. Below, an overview of the neural network will be described with reference to Fig. 7. For convenience, Fig. 7 illustrates a three-layered neural network with four types of input data and three types of output data, but the number of input data, the number of output data, and the number of intermediate layers are not limited to this.

[0061] A neural network is a collection of many nodes N. Each node N is connected to multiple other nodes N, and a weight w is set between the connected nodes N. The collection of nodes is roughly divided into a collection (nodes N11 to N14) that functions as an input layer D1 that receives various input data, a collection (nodes N21 to N23) that functions as an intermediate layer D2 that executes calculations using a weight w2, and a collection (nodes N31 to N33) that functions as an output layer D3 that outputs output data. The number of nodes in the input layer D1 is set according to the type of input data x, and the number of nodes in the output layer D3 is set according to the type of output data y.

[0062] In the machine learning device 90 shown in this embodiment, the work position data, posture data before the gripping operation, posture data when gripping the work W, shape data before the gripping operation, type data, comparison data, stop reference data, and distance data are input data x, and the learning unit 94 performs a multi-layered calculation according to the above-mentioned neural network to estimate an appropriate stop reference reaction force and position adjustment reference distance as appropriate output data y. Note that the operation modes of the neural network include a learning mode in which the above-mentioned learning is performed, and a value prediction mode. For example, the weight w can be learned in the learning mode, and the learned weight w can be used to judge the value of an action in the value prediction mode.

[0063] Next, an operation mode setting process and a position adjustment process that are periodically executed by the CPU 81 of the host controller 80 will be described with reference to FIG.

[0064] In the operation mode setting process, first, image data is acquired from the camera 65 (step S11), and shape data of the workpiece W before the gripping operation is created (extracted) from the acquired image data (step S12). Next, position data of the workpiece W, i.e., the target position TP, is set from the shape data (step S13). After that, posture data of the robot 20 during standby (before the placement operation) is acquired (step S14), and a control point for generating an operation trajectory is set based on the target position TP and the posture data (step S15). After that, type data indicating the type of the workpiece W identified from the image data is acquired (step S16), and the above-mentioned stop reference reaction force is set based on the type data (step S17). In the process of step S17, if the use of the operation setting model is permitted, the stop reference reaction force is determined based on the operation setting model, and if the use of the operation setting model is not permitted, the stop reference reaction force is determined (selected) from a candidate range set in advance.

[0065] In the position adjustment process, first, it is determined whether or not it is the timing when the hand unit 38 is placed at the gripping position (step S21). If it is not the timing when it is placed at the gripping position, this position adjustment process is terminated as it is. If the hand unit 38 is placed at the gripping position, posture data indicating the posture of the robot 20 and distance data indicating each distance between the workpiece W and each of the movable claws 38a, 38b are acquired (step S22). After that, it is determined whether or not the use of the operation setting model is permitted (step S23). If the use of the operation setting model is not permitted, this position adjustment process is terminated as it is. In other words, if the use of the operation setting model is not permitted, position adjustment is avoided. If the use of the operation setting model is permitted, it is confirmed whether or not position adjustment is necessary (step S24). If the difference in distance this time does not exceed the position adjustment reference distance, it is determined that position adjustment is not performed (step S25: NO), and this position adjustment process is terminated as it is. If the difference in distance this time exceeds the position adjustment reference distance, it is determined that position adjustment is performed (step S25: YES), and the details of the position adjustment are determined (step S26). Specifically, the direction and amount of displacement of the hand unit 38 are determined in order to adjust the position of the hand unit 38 so that the difference in the distance between the workpiece W and each of the movable claws 38a, 38b becomes zero. Based on the determined direction and amount of displacement, a motion trajectory correction process is executed. Specifically, each control point is reset in order to correct the motion trajectory of the robot 20 to a motion trajectory that takes into account the position adjustment. The reset control points are transmitted to the motion controller 60.

[0066] Next, a learning process periodically executed by the CPU 91 of the machine learning device 90 will be described with reference to the flowchart of FIG.

[0067] In the learning process, first, it is determined whether or not it is the timing when all steps of the transport operation of the workpiece W to the case C are completed and the workpiece W has returned to the standby posture (step S31). If it is not this timing, the learning process ends as it is. If it is this timing, various status data are acquired from the host controller 80 (step S32). Specifically, workpiece position data, posture data before the gripping operation, posture data when gripping the workpiece W, shape data before the gripping operation, type data, and image data of a specified area after the gripping operation are acquired. Next, various label data are acquired from the host controller 80 (step S33). Specifically, stop criterion data and distance data are acquired.

[0068] Thereafter, the image of the predetermined area after the gripping operation is analyzed to determine whether or not the workpiece W remains in the predetermined area, i.e., whether or not the transportation (grasping) of the workpiece W was successful (step S34). If the transportation of the workpiece W has failed, the various acquired input data is erased without being stored in the data storage unit 95, i.e., without being used as input data for machine learning, and this learning process is terminated.

[0069] On the other hand, if the transportation (grasping) of the workpiece W is successful, the deformation amount of the workpiece W is calculated. Specifically, the shape data of the workpiece W before the gripping operation is compared with the shape data of the workpiece W after the gripping operation (release), the deformation amount (damage) of the workpiece W is calculated, and comparison data is created (step S35). Thereafter, the workpiece position data, posture data, shape data, type data, comparison data, stop reference data, and distance data are stored in the data storage unit 95, and the operation setting model is updated based on the newly acquired data. After that, data is accumulated at least until the above-mentioned use is permitted, and the update of the operation setting model is repeated. Note that it is not necessarily necessary to update the operation setting model every time new data is acquired, and the operation setting model may be updated (constructed) when the amount of accumulated data reaches a reference amount.

[0070] According to the first embodiment described above in detail, the following excellent effects can be expected.

[0071] In this embodiment, a motion setting model used to set the motion mode of the motion (position adjustment motion and gripping motion) of the robot 20 is constructed by machine learning using various input data including stop criterion data, distance data, and comparison data. With such a configuration, not only can an appropriate stop criterion for the movable claws 38a, 38b be set, but also a configuration can be realized in which the position of the hand unit 38 is appropriately adjusted according to the arrangement of the hand unit 38. In other words, it is possible to suppress damage to the workpiece W caused by a difference in timing at which the movable claws 38a, 38b hit the workpiece W, and contribute to realizing a robot system that can appropriately grip a workpiece W with low reaction force and irregular shape.

[0072] When a low-reaction workpiece W is damaged, changes in weight and shape occur. In many cases where a change in weight occurs, the shape also changes, so by comparing the shape before and after the gripping operation, the degree of damage to the workpiece W can be appropriately reflected in machine learning.

[0073] The convenience of the robot system 10 can be improved by providing a certain tolerance range (the above-mentioned predetermined area) for the position of the workpiece W to be supplied. However, when such a range is provided, the way in which the movable claws 38a, 38b hit the workpiece W when the gripping operation is performed may differ depending on the position within the range to which the workpiece W is supplied. In addition, the way in which the movable claws 38a, 38b hit the workpiece W may differ depending on the posture of the robot 20 when gripping the workpiece W. Therefore, if a configuration is adopted in which machine learning is performed by adding workpiece position data indicating the position of the workpiece W and posture data indicating the posture of the robot 20 to the input data, it can contribute to realizing a robot system that can appropriately grip a workpiece W with low reaction force and an irregular shape.

[0074] <Second embodiment> In the above first embodiment, a configuration in which an action setting model is constructed by supervised learning has been exemplified, but the learning method for constructing an action setting model may be reinforcement learning.

[0075] When constructing an operation setting model by reinforcement learning, it is preferable to increase the reward if the difference between the shape of the workpiece W before the gripping operation and the shape of the workpiece W after the gripping operation (release) is within a reference amount (5% in this embodiment), and decrease the reward if it exceeds the reference amount. In addition, in the above second embodiment, only the data when the gripping of the workpiece W is successful is used as input data for learning, but it is also possible to set a reward for the success or failure of gripping and to use the data when the gripping fails as input data. In this case, for example, it is preferable to increase the reward when the gripping of the workpiece W is successful and decrease the reward when the gripping of the workpiece W fails. A configuration in which the success of gripping is prioritized can be realized by setting the reward for the successful gripping of the workpiece W to be higher than the reward given when the difference in shape is within a reference amount.

[0076] When the position adjustment is performed, the transport efficiency is slightly lower than when the position adjustment is not performed. In addition, when the position adjustment is performed, the transport efficiency is lower as the displacement amount of the hand unit 38 increases. Therefore, by applying reinforcement learning to set the reward related to the transport efficiency, it is possible to protect the workpiece W and improve the transport efficiency at the same time. For example, by configuring the reward to be increased as the displacement amount of the hand unit 38 when the position adjustment is performed is reduced and reduced as the displacement amount is increased, it is possible to minimize the displacement amount of the hand unit 38 even when the position adjustment is required. Alternatively, it is possible to add data on the time required from the start of the operation of the robot 20 to the end of the operation as input data, and to increase the reward as the required time is reduced and decrease the reward as the required time is increased.

[0077] <Third embodiment> In the above-described first embodiment and the like, the movable claws 38a, 38b are stopped when the reaction force detected by the force sensor 45 reaches a stop criterion. The smaller the reaction force of the workpiece W, the more difficult it may be to properly detect the reaction force. Therefore, for a workpiece W having an extremely small reaction force, the criterion for stopping the movable claws 38a, 38b may be the relative distance (spacing or width) between the two movable claws 38a, 38b and the reaction force, that is, the amount of displacement of the two movable claws 38a, 38b in the clamping direction.

[0078] In such a configuration, when the use of the operation setting model is permitted, a reference relative distance serving as a stop reference is set using various data and the operation setting model, and when the use of the operation setting model is not permitted, a stop reference reaction force is determined (selected) from a preset candidate range. During learning, data indicating the relative distance when a gripping operation is performed is set as the stop reference data, and the relative distance is set as input data for learning.

[0079] <Fourth embodiment> The workpiece W (particularly cream puffs and daifuku) shown in the first embodiment and the like has an indefinite shape, and the restoring force of the workpiece W after the gripping is completed may differ depending on the direction from which the workpiece W is pinched. In other words, the above-mentioned timing deviation and the influence of the load may differ depending on the direction from which the workpiece W is pinched. In other words, the positional relationship between the object that does not require adjustment and each of the movable claws 38a, 38b may differ depending on the pinching direction. In addition, there is a possibility that the deformation amount (damage) is less when the workpiece is pinched from a different direction than when the workpiece is pinched from a certain direction. In this embodiment, taking such circumstances into consideration, one of the features is that pinching direction data indicating the pinching direction is added to the input data. Hereinafter, the input data will be described with reference to FIG. 10. The pinching direction is the direction in which the movable claws 38a, 38b are displaced in the gripping operation.

[0080] An image of the workpiece W placed in a predetermined area is input from the camera 65 to the upper controller 80. The upper controller 80 refers to shape data showing the outer shape of the workpiece W created (extracted) from the captured image, and sets the reference direction of the workpiece W. Specifically, the point on the outer shape line that has the greatest distance is determined, and the direction of the straight line connecting these two points is set as the reference direction. During learning, pinch direction data is obtained as data showing the relationship of the current pinch direction to this reference direction. More specifically, the pinch direction data is data showing the angle of the pinch direction with respect to the reference direction. In this way, by adding the pinch direction to the input data, the influence of the pinch direction is reflected in the constructed operation setting model. Therefore, when the operation mode is determined based on the direction data and the operation setting model, the set stop reference reaction force and position adjustment reference distance take the pinch direction into account.

[0081] In this embodiment, the two points on the outline that are the greatest distance apart are determined, and the direction of the straight line connecting those two points is set as the reference direction. Alternatively, it is also possible to determine the two points on the outline that are the smallest distance apart, and set the direction of the straight line connecting those two points as the reference direction.

[0082] In the fourth embodiment, the stop reference reaction force and the position adjustment reference distance are set taking into consideration the clamping direction, but instead of this, when the use of the operation setting model is permitted, the clamping direction may be set using various data (shape data of the workpiece W) and the operation setting model. In other words, the orientation of the hand unit 38 may be set so as to obtain a clamping direction in which the stop reference reaction force is small and the position adjustment reference distance is large.

[0083] <Fifth embodiment> The workpieces W shown in the first embodiment and the like have a large variation in shape and a small reaction force. When gripping such a workpiece W, the contact area between the workpiece W and the movable claws 38a, 38b may cause a difference in the effect on the workpiece W. In other words, the contact area may cause a difference in the positional relationship between the workpiece W and each of the movable claws 38a, 38b, which does not require adjustment. In addition, the amount of deformation (damage) of the workpiece W may be reduced depending on the size of the contact area. In this embodiment, taking such circumstances into consideration, one of the features is that contact area data indicating the contact area is added to the input data. Hereinafter, the input data will be described with reference to FIG. 11.

[0084] Sheet-like contact sensors 47a, 47b are disposed on the opposing surfaces of the movable claws 38a, 38b, respectively. Detection results by the contact sensors 47a, 47b are transmitted to a host controller 80, which determines the contact area between the workpiece W and the movable claws 38a, 38b based on the detection results. When the workpiece W is clamped with the same strength, the larger the contact area is, the more the local load on the workpiece W can be reduced. In other words, the larger the contact area is, the better the workpiece W can be protected.

[0085] The contact area data indicating the contact area is provided to the machine learning device 90, which stores this contact area data as input data in the data storage unit 95 and uses the contact area data to construct an operation setting model. By identifying the relationship between the shape of the workpiece W and the contact area by machine learning, it becomes possible to estimate the contact area from the operation setting model and the shape of the workpiece W, and to set the stop reference reaction force and position adjustment reference distance based on the estimated contact area.

[0086] In the above fifth embodiment, the stop reference reaction force and the position adjustment reference distance are set taking into account the estimated contact area, but instead of this, when the use of the operation setting model is permitted, a pinching direction (see the fourth embodiment) that provides an appropriate contact area may be set using various data (shape data of the workpiece W) and the operation setting model. In other words, a configuration may be used in which the orientation of the hand unit 38 is set so as to provide a pinching direction that provides a small stop reference reaction force and a large position adjustment reference distance.

[0087] <Other embodiments> The present invention is not limited to the contents of the above-described embodiments, and may be implemented, for example, as follows. Each of the following configurations may be applied individually to each of the above-described embodiments, or may be applied to each of the above-described embodiments in combination in part or in whole. It is also possible to arbitrarily combine all or part of the various configurations shown in each of the above-described embodiments. In this case, it is preferable that the technical significance (effects to be exerted) of each of the configurations to be combined is guaranteed.

[0088] The operation setting model shown in each of the above embodiments can be divided into a model for position adjustment (position adjustment model) and a model for setting a stop criterion (grasping model), and these can be constructed separately.

[0089] In each of the above embodiments, the gripping operation and position adjustment operation, which are part of the operation of the robot 20, are defined as "predetermined operations," and a model for setting the predetermined operations is constructed by machine learning. However, this can be modified so that a model for setting the entire operation of the robot 20, including the positioning operation of the hand unit 38, the position adjustment operation of the hand unit 38, and the gripping operation of the workpiece W, is constructed by machine learning.

[0090] In each of the above embodiments, the robot system 10 is configured to monitor whether the workpiece W has been grasped and the degree of damage, but this is not limited to the above. For example, it is also possible to have an operator monitor whether the workpiece W has been grasped and the degree of damage, and input the monitoring results to the machine learning device 90.

[0091] Environmental data (temperature data and humidity data) indicating the environment around the robot 20 may be added to the learning input data. For a gripping object with low reaction force and irregular shape, such as a cream puff, the hardness may change depending on environmental conditions such as temperature and humidity. In other words, there is a possibility that the positional relationship between the gripping object and each of the movable claws 38a, 38b, which does not require adjustment depending on the environmental conditions, may differ. Therefore, as shown in this modified example, if machine learning is performed by adding environmental data to the various input data shown in each embodiment, further optimization of the position adjustment operation and gripping operation can be expected.

[0092] In the above embodiments, the displacement speed of the movable claws 38a, 38b when performing the gripping operation is constant, but this is not limited to this. The displacement speed may be a variable parameter, and speed data indicating the displacement speed when gripping the workpiece W may be added to the learning input data. When the above-mentioned timing deviation occurs, there is a possibility that the influence on the gripping target may differ depending on the displacement speed of the movable claws 38a, 38b. In other words, there is a possibility that the positional relationship between the target object and each of the movable claws 38a, 38b, which does not require adjustment due to the displacement speed, may differ. Therefore, as shown in this modified example, if a configuration is adopted in which machine learning is performed by adding speed data to the various input data shown in each embodiment, further optimization of the gripping operation can be expected.

[0093] A configuration may be added in which a target point setting model used to set the target position TP is constructed by machine learning based on shape data indicating the shape of the workpiece W before the gripping operation, workpiece position data indicating the set target position TP, and distance data indicating the distance between each of the movable claws 38a, 38b and the workpiece W when the hand unit 38 is placed at the gripping position. If the setting accuracy of the target position TP is improved, the opportunities for the above-mentioned position adjustment can be reduced, which contributes to improving the transport efficiency.

[0094] In each of the above embodiments, the gripping portion is moved when adjusting the position, but the present invention is not limited to this. For example, it is possible to move the movable claw that is closer to the workpiece W to make the distance uniform, or to move the movable claw that is closer to the workpiece W to make the distance uniform. However, in such a configuration, the workpiece W may be gripped at a position biased toward one of the movable claws. This is not preferable in terms of stabilizing the gripping function or improving the efficiency of the arrangement work in the case C. In other words, there is technical significance in using a configuration in which the position is adjusted by moving the hand portion 38 instead of the movable claws 38a and 38b, as shown in each of the above embodiments.

[0095] In each of the above embodiments, a configuration is exemplified in which the amount of change (damage) in the shape of the workpiece W is identified by comparing the shape data of the workpiece W before the gripping operation with the shape data of the workpiece W after the gripping operation, but the present invention is not limited to this. Alternatively or in addition to this, a configuration is also possible in which the weight of the workpiece W is monitored using a weight sensor or the like, and the amount of change in the weight of the workpiece W is identified by comparing the weight of the workpiece W before the gripping operation with the weight of the workpiece W after the gripping operation. In other words, the comparison data that is input data to the machine learning device 90 can also be data indicating a weight difference.

[0096] In each of the above embodiments, the camera 65 is fixed to the ceiling of the factory, but the camera 65 may be attached to the robot body 30 (for example, an arm).

[0097] In each of the above embodiments, the workpiece W is gripped by the two movable claws 38a, 38b, but the number of movable claws is arbitrary. For example, it may be three, or four or more.

[0098] In each of the above embodiments, the main control device 70 is configured by the host controller 80 and the machine learning device 90, but the host controller 80 and the machine learning device 90 may be provided separately. Also, a configuration equivalent to the machine learning device 90 may be provided on the cloud.

[0099] In the above embodiments, cream puffs and eclairs are exemplified as objects to be grasped by the robot 20, but the present invention is not limited to these. It is also possible to use other foods that have a small reaction force and a large variation in shape, such as daifuku and bread, as objects to be grasped by the robot 20. Furthermore, the objects to be grasped are not limited to processed foods, and even if fruits and vegetables such as mandarin oranges and tomatoes are used as the objects to be grasped, the objects to be grasped can be protected in an appropriate manner. [Explanation of symbols]

[0100] 10...robot system, 20...robot, 38...hand portion, 38a, 38b...movable claws, 42...encoder, 45...force sensor, 46...distance sensor, 65...camera, 70...control device, 80...host controller, 81...CPU, 90...machine learning device, 91...CPU, 92...memory, CP...reference point, TP...target position, Xa, Xb...distance, W...workpiece.

Claims

1. a robot having a gripping unit provided with a pair of movable claws, and configured to grip an object by pinching the object with the movable claws; a control device that controls the robot so as to execute a positioning operation for positioning the gripping unit at a predetermined position where the object is to be located between the pair of movable claws, and a gripping operation for displacing the movable claws toward each other at the predetermined position; Equipped with the control device is configured to stop the displacement of the set of movable claws when a reaction force from the object during the gripping operation reaches a reference value or when a relative distance between the set of movable claws reaches a reference value, Under a condition in which the gripping portion is disposed at the predetermined position, an adjustment operation can be performed to adjust the position of the set of movable claws with respect to the object by displacing the gripping portion, The reference value for stopping the gripping operation is a variable value, a model construction unit that acquires stop reference data indicating the reference value set when gripping the object, distance data indicating the distance between each movable claw of the gripping unit arranged at the predetermined position and the object, and comparison data indicating a difference between a state of the object before the gripping operation is performed and a state of the object after the gripping operation is performed, and constructs a model used to set an operation mode of the predetermined operation including the adjustment operation and the gripping operation by machine learning using the stop reference data, the distance data, and the comparison data; The control device includes: an acquisition unit that acquires the distance data indicating a distance between each movable claw of the gripping unit and the target object when the gripping unit is placed at the predetermined position; A setting unit that sets a motion mode of the predetermined motion of the robot; having A robot system in which the setting unit sets the operation mode of the specified operation so as to suppress damage to the object based on the distance data acquired by the acquisition unit and the model constructed by the model construction unit.

2. The robot system according to claim 1 , wherein the model construction unit acquires, as the comparison data, data indicating a difference between a shape of the object before the grasping operation is performed and a shape of the object after the grasping operation is performed.

3. The robot system according to claim 1 or claim 2, wherein the model construction unit acquires shape data indicating the shape of the object before the grasping operation is performed and directional data indicating the relationship between a reference direction of the object and a direction in which the set of movable claws pinch the object, and performs the machine learning by associating the stopping reference data, the distance data, the comparison data, the shape data, and the directional data.

4. The portion of the set of movable claws that comes into contact with the object is planar, The robot system according to any one of claims 1 to 3, wherein the model construction unit acquires shape data indicating the shape of the object before the grasping operation is performed and contact area data indicating the contact area between the set of movable claws and the object while the object is being grasped, and performs the machine learning by associating the stopping criterion data, the distance data, the comparison data, the shape data, and the contact area data.

5. The control device is configured to displace the set of movable claws at a set speed when performing the gripping operation, The speed is variable; The robot system according to any one of claims 1 to 4, wherein the model construction unit acquires speed data indicating a displacement speed when each of the movable claws is displaced toward the target object, and performs the machine learning by associating the stopping criteria data, the distance data, the comparison data, and the speed data.

6. The robot system according to any one of claims 1 to 5, wherein the model construction unit acquires position data indicating the position of the object before the grasping operation is performed, and performs the machine learning by associating the stopping criteria data, the distance data, the comparison data, and the position data.

7. 7. The robot system according to claim 1, wherein the model construction unit acquires posture data indicating the posture of the robot when grasping the object, and performs the machine learning by associating the stopping reference data, the distance data, the comparison data, and the posture data.

8. The robot system according to any one of claims 1 to 7, wherein the model construction unit acquires environmental data indicating an environment surrounding the robot, and performs the machine learning by associating the stopping criterion data, the distance data, the comparison data, and the environmental data.

9. The robot system according to any one of claims 1 to 8, wherein the setting unit is configured to set an operation mode so as to grasp the object without adjusting a relative position between the set of movable claws and the object when the distance data acquired by the acquisition unit indicates a distance within a range defined by the model, and to set an operation mode so as to grasp the object after adjusting the relative position between the set of movable claws and the object when the distance data acquired by the acquisition unit indicates a distance outside the range defined by the model.

10. a robot having a gripping unit provided with a pair of movable claws, and configured to grip an object by pinching the object with the movable claws; a control device that controls the robot so that a positioning operation is performed to position the gripping unit at a predetermined position where the object is to be located between the pair of movable claws, and a gripping operation is performed to displace the movable claws toward each other at the predetermined position; A robot system comprising: Under a condition in which the gripping portion is disposed at the predetermined position, the position of the set of movable claws with respect to the object can be adjusted by displacing the gripping portion, a model construction unit that acquires distance data indicating a distance between each movable claw of the gripping unit arranged at the predetermined position and the object, and comparison data indicating a difference between a state of the object before the gripping operation is performed and a state of the object after the gripping operation is performed, and constructs a model used to set a manner of adjusting the position of the gripping unit at the predetermined position by machine learning using the distance data and the comparison data; The control device includes: an acquisition unit that acquires the distance data indicating a distance between each movable claw of the gripping unit and the target object when the gripping unit is placed at the predetermined position; a setting unit that sets a manner of the position adjustment so as to suppress damage to the object based on the distance data acquired by the acquisition unit and the model constructed by the model construction unit; A robot system having the above features.

11. A machine learning device applicable to a robot system including a gripping unit provided with a set of movable claws, which grips an object by pinching the movable claws, and a control device which controls the robot so as to execute a positioning operation of positioning the gripping unit at a predetermined position where the object will be located between the set of movable claws, and a gripping operation of displacing the movable claws toward each other at the predetermined position, the control device being configured to stop the displacement of the set of movable claws when a reaction force from the object reaches a reference value during the gripping operation or when a relative distance between the set of movable claws reaches a reference value, and capable of executing an adjustment operation of adjusting a position of the set of movable claws relative to the object by displacing the gripping unit under a condition where the gripping unit is positioned at the predetermined position, The reference value for stopping the gripping operation is a variable value, A machine learning device that acquires stop reference data indicating the reference value set when grasping the object, distance data indicating the distance between each movable claw of the grasping part arranged at the predetermined position and the object, and comparison data indicating the difference between the state of the object before the grasping action is performed and the state of the object after the grasping action is performed, and is equipped with a model construction unit that constructs a model used for setting the operation modes of predetermined actions including the adjustment action and the grasping action so as to suppress damage to the object through machine learning using the stop reference data, distance data, and comparison data.

Images