Machine learning device and robot system

Abstract

The present invention realizes a robot system capable of appropriately gripping an object of low reaction force and indefinite shape. The robot system is provided with a robot (20) having a hand provided with a set of movable claws, which grips an article by sandwiching the article with the movable claws, and a control device (70) that controls the robot. In the control device, a higher-level controller (80) controls the robot to perform a placement action that places the hand at a gripping position and a gripping action that displaces the movable claws toward each other on the gripping position side, and a machine learning device (90) acquires, in the case of gripping the article, set stop reference data, distance data that indicates distances of each of the movable claws of the hand placed at the gripping position from the article, and comparison data that indicates deformation amounts of the article before and after the gripping action, respectively, and constructs a model for setting a manner of action of the gripping action by machine learning using the stop reference data, the distance data, and the comparison data.

Description

Technical Field

[0001] This invention relates to machine learning devices and robotic systems. Background Technology

[0002] There are robot systems that can grasp an object (workpiece) by clamping it with a set of movable grippers (gripping parts) located at the tip of the robot's arm. In such robot systems, for example, a robot system is proposed that is configured to detect changes in the current value of the motor used for the movable grippers and the reaction force from the object, and stop the movable grippers when the detected values ​​reach a preset reference value (for example, see Patent Document 1).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2014-24134 Summary of the Invention

[0006] The technical problem to be solved by the present invention

[0007] Here, compared to industrial products made of metal and synthetic resin, when food items such as cream puffs, daifuku, and bread are considered as objects, the shape of the object deviates significantly (amorphous) and the reaction force from the object during gripping is small (low reaction force). When the object is gripped by a set of movable claws as described above, one movable claw may contact the object first, resulting in locally higher pressure on the object, or the object being pressed by one movable claw may slide on the mounting surface. Especially for objects like cream puffs, due to the large shape deviation, it is likely that differences will arise in the distance between each movable claw and the object when arranging the gripping parts according to the position of the object determined by images, etc. When such differences occur, the timing of each movable claw contacting the object deviates. Furthermore, because objects like cream puffs are soft, they may slide on the mounting surface while being pressed by the movable claws, potentially increasing the possibility of damage (including excessive deformation). Such object damage is a major cause of reduced yield in manufacturing processes and may hinder the improvement of manufacturing efficiency through the application of robotic systems. Therefore, there is still room for improvement in the structure of robotic systems in terms of properly grasping low-reaction and amorphous objects.

[0008] The present invention is made in view of the above circumstances, and its main objective is to realize a robotic system capable of properly grasping low-reaction and amorphous objects.

[0009] Technical solutions for solving technical problems

[0010] The following describes the technical solutions used to solve the above-mentioned technical problems.

[0011] In the first technical solution, a robot system is provided, which comprises:

[0012] The robot (robot 20) has a gripping part (hand 38) equipped with a set of movable claws (movable claws 38a, 38b), which grips an object (workpiece W) by holding it with these movable claws; and

[0013] The control device (control device 70) controls the robot to perform a configuration action of positioning the gripper to a predetermined position (gripping position) between the set of movable claws, and a gripping action of moving the movable claws toward each other at the predetermined position.

[0014] The control device is configured to stop the displacement of the set of movable claws when the reaction force from the object during the gripping action reaches a reference value or when the relative distance between the set of movable claws reaches a reference value.

[0015] When the gripping part is positioned at the predetermined location, it is possible to perform an adjustment action by shifting the gripping part to adjust the position of the set of movable claws relative to the object.

[0016] The reference value for stopping the grasping action is a variable value.

[0017] The device includes a model building unit (machine learning device 90). When gripping the object, this unit acquires stop reference data representing a set reference value, distance data representing the distance between each movable claw of the gripping unit positioned at the predetermined location and the object, and comparison data (e.g., the deformation of the workpiece W) representing the difference between the object's state before and after the gripping action. By using machine learning with this stop reference data, distance data, and comparison data, a model (action setting model) is constructed for setting a motion pattern that includes the adjustment action and the gripping action.

[0018] The control device has:

[0019] The data acquisition unit (data acquisition unit 85) acquires distance data indicating the distance between each movable claw of the gripping unit and the object when the gripping unit is positioned at the predetermined position; and

[0020] The setting unit (control unit 84) sets the action mode of the robot for the specified action.

[0021] The setting unit can set the action mode of the prescribed action based on the distance data obtained by the acquisition unit and the model constructed by the model construction unit.

[0022] In structures that grasp objects (such as cream puffs, daifuku, and bread) using a set of movable grippers, differences in the distance between each gripper and the object can occur when the grippers are positioned between them. These differences are more pronounced with objects exhibiting large shape deviations (amorphous objects). Furthermore, these differences lead to deviations in the timing of each gripper's contact with the object. In cases where the object is soft, i.e., when the reaction force is low, the likelihood of damage to the object increases due to these timing deviations. In this technical solution, a model for setting a prescribed action mode that includes adjustment and grasping actions is constructed using machine learning based on stopping reference data, distance data, and comparison data. This structure not only allows for setting optimal reference values ​​but also enables the appropriate adjustment of the gripper positions based on the gripper configuration. In other words, it contributes to the realization of a robot system that can suppress damage to objects caused by the aforementioned timing deviations and can appropriately grasp amorphous objects with low reaction forces. Furthermore, it is conceivable that damage to the object could occur not only due to timing deviations mentioned above, but also due to improperly set reference values. On the other hand, it is difficult to determine the main cause of the damage based on the aforementioned comparison data. Therefore, it is preferable to contribute to the optimization of the prescribed actions (adjustment actions, grasping actions) by using the stop reference data, distance data, and comparison data as input data for machine learning.

[0023] Furthermore, as for "comparative data," it can be arbitrary as long as the degree of damage to the object can be determined. For example, it can be set as data representing changes in shape and data representing changes in weight.

[0024] In addition, the "method of the prescribed action" shown in this technical solution includes setting whether the adjustment action is possible and setting the displacement amount of the movable claw during adjustment.

[0025] In the second technical solution, the model building unit obtains data representing the difference between the shape of the object before the grasping action and the shape of the object after the grasping action, as the comparison data.

[0026] When an object is damaged under low reaction force, changes occur in both weight and shape. In most cases where weight changes occur, shape also changes, thus allowing machine learning to appropriately reflect the degree of damage by using shape as a comparison object.

[0027] In the third technical solution, the model building unit obtains shape data representing the shape of the object before the grasping action is performed, and direction data representing the relationship between the reference direction of the object (e.g., the direction connecting the two longest points in the image of the object: the length direction) and the direction in which the set of movable claws grips the object, and performs machine learning in a corresponding association with the stop reference data, the distance data, the comparison data, the shape data, and the direction data.

[0028] When gripping objects with low reaction force and irregular shape, the aforementioned timing deviations and weight effects may vary depending on the gripping direction. In other words, the positional relationship between the object and each movable claw, which does not require adjustment, may differ depending on the gripping direction. Therefore, as shown in this technical solution, by adding shape data and orientation data to the various data shown in the first technical solution and performing machine learning, the effects shown in the first technical solution can be further appropriately utilized.

[0029] In the fourth technical solution, the portion of the set of movable claws that contacts the object is planar.

[0030] The model building unit obtains shape data representing the shape of the object before the grasping action is performed, and contact area data representing the contact area between the set of movable claws and the object in the state of grasping the object, and performs machine learning in a corresponding association with the stopping reference data, the distance data, the comparison data, the shape data, and the contact area data.

[0031] When grasping objects with low reaction force and irregular shapes, the impact on the object may vary depending on the contact area. In other words, the positional relationship between the object and each movable claw may differ depending on the contact area. Therefore, as shown in this technical solution, by adding shape data and contact area data to the various data shown in the first technical solution and performing machine learning, the effects shown in the first technical solution can be further and appropriately achieved.

[0032] In the fifth technical solution, the control device is configured to, when performing the gripping action, move the set of movable claws at a predetermined speed.

[0033] The speed is a variable value.

[0034] The model building unit obtains speed data representing the displacement speed of each movable claw as it moves toward the object, and performs machine learning in a corresponding association with the stopping reference data, the distance data, the comparison data, and the speed data.

[0035] In the event of the aforementioned timing deviation, the impact on the object may differ depending on the movement speed of the movable claw. In other words, the positional relationship between the object and each movable claw, which does not require adjustment, may vary depending on the movement speed. Therefore, as shown in this technical solution, by adding velocity data to the various data presented in the first technical solution and performing machine learning, the effects shown in the first technical solution can be further appropriately achieved.

[0036] In the sixth technical solution, the model building unit obtains position data representing the position of the object before the grasping action is performed, and performs machine learning on the corresponding correlation between the stopping reference data, the distance data, the comparison data, and the position data.

[0037] For robotic systems, the convenience of the system can be improved by setting a certain allowable range for the position of the supplied object. However, when such a range is set, the contact method of the movable gripper when performing the grasping action of the object may vary depending on where the object is supplied within that range. Therefore, as shown in this technical solution, as long as a structure is constructed by adding position data to the various data shown in the first technical solution and performing machine learning, the effects shown in the first technical solution can be further appropriately and appropriately achieved.

[0038] In the seventh technical solution, the model building unit obtains posture data representing the robot's posture when grasping the object, and performs machine learning in a corresponding association with the stopping reference data, the distance data, the comparison data, and the posture data.

[0039] For robots, the way the movable claw touches the object may vary depending on the posture when grasping it. Therefore, as shown in this technical solution, as long as a structure is constructed by adding posture data to the various data shown in the first technical solution and performing machine learning, the effects shown in the first technical solution can be further and appropriately achieved.

[0040] In the eighth technical solution, the model building unit obtains environmental data representing the surrounding environment of the robot, and performs machine learning by correspondingly associating the stopping reference data, the distance data, the comparison data, and the environmental data.

[0041] For low-reaction, amorphous objects such as cream puffs, daifuku, and bread, hardness can vary depending on environmental conditions such as temperature and humidity. In other words, the positional relationship between the object and each movable claw may differ depending on environmental conditions, even if no adjustment is needed. Therefore, as shown in this technical solution, by adding environmental data to the various data presented in the first technical solution and performing machine learning, the effects shown in the first technical solution can be further and appropriately enhanced.

[0042] In the ninth technical solution, the setting unit is configured such that, when the distance data obtained by the acquisition unit is data representing a distance within a range defined by the model, the operation mode is set to grasp the object without adjusting the relative position of the set of movable claws and the object; and when the distance data obtained by the acquisition unit is data representing a distance outside the range defined by the model, the operation mode is set to grasp the object after adjusting the relative position of the set of movable claws and the object.

[0043] Machine learning can be used to optimize the identification of whether or not positional adjustments are needed. These are preferred for mitigating damage to objects and improving the handling efficiency of robotic systems.

[0044] The tenth technical solution provides a robot system that comprises:

[0045] The robot (robot 20) has a gripping part (hand 38) equipped with a set of movable claws (movable claws 38a, 38b), which grips an object (workpiece W) by holding it with these movable claws; and

[0046] The control device (control device 70) controls the robot to perform a configuration action of positioning the gripper to a predetermined position (gripping position) between the set of movable claws, and a gripping action of moving the movable claws toward each other at the predetermined position.

[0047] When the gripping part is positioned at the predetermined location, the position of the set of movable claws relative to the object can be adjusted by shifting the gripping part.

[0048] The device includes a model building unit (machine learning device 90) that acquires distance data representing the distance between each movable claw of the gripping unit positioned at the predetermined position and the object, and comparison data representing the difference between the state of the object before the gripping action is performed and the state of the object after the gripping action is performed. By using machine learning with this distance data and comparison data, the device builds a model for setting the position adjustment method of the gripping unit at the predetermined position.

[0049] The control device has:

[0050] The data acquisition unit (data acquisition unit 85) acquires distance data indicating the distance between each movable claw of the gripping unit and the object when the gripping unit is positioned at the predetermined position; and

[0051] The setting unit sets the position adjustment method based on the distance data obtained by the acquisition unit and the model constructed by the model construction unit.

[0052] In structures that grasp objects (such as cream puffs, daifuku, and bread) using a set of movable grippers, differences in the distance between each gripper and the object can occur when the grippers are positioned between them. This difference is more pronounced with objects that have large shape deviations (amorphous objects). When this difference occurs, the timing at which each gripper contacts the object deviates. In cases where the object is soft, i.e., when the reaction force is small, the possibility of damage to the object due to this timing deviation increases. In this technical solution, a model for setting the position adjustment method is constructed using machine learning with distance data and comparison data. This structure enables the appropriate adjustment of the position of the movable grippers based on the gripper configuration. In other words, it contributes to the realization of a robot system that can suppress damage to objects caused by the aforementioned timing deviation and can appropriately grasp amorphous objects with low reaction force.

[0053] In the eleventh technical solution, a machine learning device is provided, applicable to a robot system. The robot system includes: a robot (robot 20) having a gripping part (hand 38) with a set of movable claws (movable claws 38a, 38b) for gripping an object (workpiece W); and a control device (control device 70) controlling the robot to perform a configuration action of positioning the gripping part at a predetermined position (gripping position) between the set of movable claws, and a gripping action of shifting the movable claws towards each other at the predetermined position. The control device is configured to stop the shifting of the set of movable claws when the reaction force from the object during the gripping action reaches a reference value or when the relative distance between the set of movable claws reaches a reference value. When the gripping part is positioned at the predetermined position, the control device is capable of performing an adjustment action by shifting the gripping part to adjust the position of the set of movable claws relative to the object.

[0054] The reference value for stopping the grasping action is a variable value.

[0055] The device includes a model building unit (machine learning device 90), which, when grasping the object, acquires stop reference data representing the set reference value, distance data representing the distance between each movable claw of the grasping unit positioned at the predetermined position and the object, and comparison data (e.g., the deformation amount of the workpiece W) representing the difference between the state of the object before the grasping action is performed and the state of the object after the grasping action is performed. By using machine learning with these stop reference data, distance data, and comparison data, a model is built for determining the action mode of the predetermined action, including the adjustment action and the grasping action.

[0056] According to the structure shown in this technical solution, a model for setting the action mode for a specified action (adjustment action, gripping action) is constructed by using machine learning with stop reference data, distance data, and comparison data. With this structure, not only can optimal reference values ​​be set, but the position of the movable claw can also be appropriately adjusted according to the configuration of the gripping part. In other words, it contributes to the realization of a structure that can suppress damage to the object caused by deviations in the timing of each movable claw contacting the object, and can appropriately grip objects with low reaction force and irregular shapes. Furthermore, it is conceivable that damage to the object can occur not only due to the aforementioned timing deviations, but also when reference values ​​are not properly set, and on the other hand, it is difficult to determine the main cause of damage based on the aforementioned comparison data. Therefore, it is preferable to use the stop reference data, distance data, and comparison data as input data for machine learning to contribute to the optimization of the gripping action.

[0057] The twelfth technical solution provides a robot system that includes:

[0058] The robot (robot 20) has a gripping part (hand 38) equipped with a set of movable claws (movable claws 38a, 38b), which grips an object (workpiece W) by holding it with these movable claws; and

[0059] The control device (control device 70) controls the robot to perform a configuration action of positioning the gripper to a predetermined position (gripping position) between the set of movable claws, and a gripping action of moving the movable claws toward each other at the predetermined position.

[0060] The control device is configured to stop the displacement of the set of movable claws when the reaction force from the object during the gripping action reaches a reference value or when the relative distance between the set of movable claws reaches a reference value.

[0061] Among them, the clamping direction, which serves as the direction for the displacement of the set of movable claws during the grasping action, can be adjusted.

[0062] The reference value for stopping the grasping action is a variable value.

[0063] The system includes a model building unit (machine learning device 90) that acquires stop reference data representing the reference value set when gripping the object, shape data representing the shape of the object before the gripping action, direction data representing the relationship between the reference direction of the object (e.g., the direction connecting the two longest points in an image of the object: the length direction) and the gripping direction, and comparison data representing the difference between the state of the object before the gripping action and the state of the object after the gripping action (e.g., the deformation of the part W). By using machine learning with this stop reference data, shape data, direction data, and comparison data, the system builds models for setting the configuration action and various action modes of the gripping action.

[0064] The control device has an acquisition unit that acquires the shape data of the object prior to the execution of the gripping action, and is able to set the configuration action and each action mode of the gripping action based on the shape data acquired by the acquisition unit and the model constructed by the model construction unit.

[0065] In this technical solution, a model for setting the gripping direction is constructed using machine learning with stop reference data, shape data, orientation data, and comparison data. This structure not only allows for setting optimal reference values ​​but also enables appropriate adjustment of the gripping direction based on the shape of the object. In other words, it contributes to the realization of a robot system that suppresses damage to the object and can appropriately grasp low-reaction, amorphous objects. Furthermore, damage to the object can occur not only when the gripping direction is not properly set but also when the reference value is not properly set; on the other hand, it is considered difficult to determine the main cause of damage based on the aforementioned comparison data. Therefore, it is preferable to use the stop reference data, shape data, orientation data, and comparison data as input data for machine learning to contribute to the optimization of the configuration and gripping actions. Attached Figure Description

[0066] Figure 1 This is a schematic diagram showing the robot in the first embodiment.

[0067] Figure 2 This is a block diagram showing the electrical structure of the robot system.

[0068] Figure 3 It is a schematic diagram showing how the robot moves.

[0069] Figure 4 This is a schematic diagram showing the process of positioning the hand in the gripping position.

[0070] Figure 5 This is a functional block diagram of the main control device.

[0071] Figure 6 It is a schematic diagram showing the types of input data.

[0072] Figure 7 This is a schematic diagram showing the neural network used to construct the action setting model.

[0073] Figure 8 (a) is a flowchart showing the action mode setting process executed by the CPU of the upper-level controller, and (b) is a flowchart showing the position adjustment process executed by the CPU of the upper-level controller.

[0074] Figure 9 This is a flowchart illustrating the learning process performed by the CPU of a machine learning device.

[0075] Figure 10 This is a schematic diagram showing the input data in the fourth embodiment.

[0076] Figure 11This is a schematic diagram showing the input data in the fifth embodiment.

[0077] Explanation of reference numerals in the attached figures

[0078] 10: Robotic Systems

[0079] 20: Robot

[0080] 38: Hands

[0081] 38a, 38b: Movable claws

[0082] 42: Encoder

[0083] 45: Force sensor

[0084] 46: Distance sensor

[0085] 65: Camera

[0086] 70: Control device

[0087] 80: Upper-level controller

[0088] 81: CPU

[0089] 90: Machine Learning Device

[0090] 91: CPU

[0091] 92: Memory

[0092] CP: Baseline

[0093] TP: Target Location

[0094] Xa, Xb: Distance

[0095] W: Parts Detailed Implementation

[0096] <First Implementation Method>

[0097] Hereinafter, a first embodiment of a robot system used in a food factory or the like will be described with reference to the accompanying drawings.

[0098] like Figure 1 As shown, the robot system 10 includes a robot 20 as a vertical multi-jointed industrial robot and a motion controller 60 for controlling the robot 20. The robot 20 and the motion controller 60 are communicatively connected to each other. The robot 20 includes a robot body 30 and a servo amplifier 50 attached to the robot body 30.

[0099] The robot body 30 has: a base 31 fixed to the floor, etc.; a shoulder 32 supported by the base 31; a lower arm 33 supported by the shoulder 32; a first upper arm 34 supported by the lower arm 33; a second upper arm 35 supported by the first upper arm 34; a wrist 36 supported by the second upper arm 35; and a flange 37 supported by the wrist 36.

[0100] A first joint J1 is formed connecting the base 31 and the shoulder 32, and the shoulder 32 is capable of rotating horizontally about the connecting axis AX1 of the first joint J1. A second joint J2 is formed connecting the shoulder 32 and the lower arm 33, and the lower arm 33 is capable of rotating vertically about the connecting axis AX2 of the second joint J2. A third joint J3 is formed connecting the lower arm 33 and the first upper arm 34, and the first upper arm 34 is capable of rotating vertically about the connecting axis AX3 of the third joint J3. A fourth joint J4 is formed connecting the first upper arm 34 and the second upper arm 35, and the second upper arm 35 is capable of rotating in a torsional direction about the connecting axis AX4 of the fourth joint J4. A fifth joint J5 is formed connecting the second upper arm portion 35 and the wrist portion 36. The wrist portion 36 is capable of rotating in the vertical direction about the connecting axis AX5 of the fifth joint portion J5. A sixth joint J6 is formed connecting the wrist portion 36 and the flange portion 37. The flange portion 37 is capable of rotating in the torsional direction about the connecting axis AX6 of the sixth joint portion J6.

[0101] The shoulder portion 32, lower arm portion 33, first upper arm portion 34, second upper arm portion 35, wrist portion 36, and flange portion 37 are arranged in a series to form the arm of the robot body 30. A hand portion 38, which serves as an end effector, is mounted on the flange portion 37, which is the top part of the arm. Furthermore, the connecting shafts AX1, AX4, and AX6 are parallel to the length direction of the arm, while the connecting shafts AX2, AX3, and AX5 are orthogonal to the length direction.

[0102] Each joint J1 to J6 is equipped with a motor 41 (specifically a servo motor) that serves as a drive unit for rotating these joints J1 to J6. The motor 41 is connected to a servo amplifier 50, which controls the drive of the motor 41 according to instructions received from the motion controller 60.

[0103] Here, refer to Figure 2The electrical structure of the robot system 10 will be further explained below. The robot system 10 includes a main control unit 70, which constitutes a "control device" or "control unit," along with the motion controller 60. The main control unit 70 includes: a higher-level controller 80, which provides motion instructions to the motion controller 60; and a machine learning device 90, which learns (so-called machine learning) appropriate grasping actions based on various conditions such as the shape and configuration of the workpiece W. The higher-level controller 80 includes a CPU 81 and a memory 82, which has a ROM storing various control programs and fixed-value data, and a RAM capable of temporarily storing various data when executing control programs. The machine learning device 90 also has hardware such as a CPU 91 and a memory 92, similar to the higher-level controller 80. The memory 92 includes a ROM (not shown, functioning as a non-temporary computer-readable recording medium for machine learning) storing learning algorithms for learning software, and RAM storing various input data for learning.

[0104] Therefore, through the processing of the CPU91, i.e., the computer (processor), or together with its processing, the various elements 94 (95, 96), 97, 98, 99 (see reference) described later are functionally constructed. Figure 5 ).

[0105] The motion controller 60 receives motion instructions from the upper-level controller 80 located in the main control unit 70 and reads the motion program corresponding to the motion instructions from the program storage unit, and determines the motion target position (hereinafter referred to as the target position or control point) based on the read motion program. Then, it generates a target trajectory that smoothly connects the determined target position with the current position of the arm (each movable part) of the robot 20, and sequentially sends interpolated positions as positions to refine the target trajectory to the servo amplifier 50.

[0106] The servo amplifier 50 includes a position control unit, a speed control unit, a current control unit, and a storage unit for storing various information. An encoder 42, attached to the motor 41, is connected to the position control unit. The position control unit detects the rotational position (i.e., the arm's posture) of the motor 41 based on the encoded value. The position control unit and the speed control unit calculate the target torque and target rotational speed of each motor 41 based on the deviation between the detected rotational position and the interpolated position contained in the command received from the motion controller 60. The current control unit determines the amount of power (current, voltage, pulse) supplied to each motor 41 based on the calculated target torque and target rotational speed, and supplies power to each motor 41 accordingly.

[0107] Next, refer to Figure 3The basic operations of robot 20 will be explained below. Robot 20, together with conveyor belt S1, constitutes part of the production line in a food factory. Food products that have undergone manufacturing processes (processing processes), specifically soft products with large shape deviations such as cream puffs and finger pastries (hereinafter referred to as parts W), flow on conveyor belt S1, and robot 20 is responsible for the packaging process of these products. Specifically, a platform S2 is arranged on conveyor belt S1, and a box C capable of holding parts W is provided on the platform S2. Conveyor belt S1 is configured to move parts W that have undergone manufacturing processes toward a designated area in front of robot 20, and when a part W is transported to box C by robot 20, the next part W is sent to the designated area. In this embodiment, part W is equivalent to "object" or "grasping object".

[0108] The robot system 10 has a camera 65 fixed to the ceiling of the building (see reference). Figure 1 The image captured by camera 65 is sent to the upper-level controller 80. In the upper-level controller 80, image analysis determines that the workpiece W is positioned within the designated area. To position the hand 38 at a gripping position (equivalent to the "designated position") capable of holding the workpiece W, the robot 20's posture (equivalent to the "positioning action") is changed. Thus, by positioning the hand 38 at the gripping position, the workpiece W is positioned between the movable claws 38a and 38b of the hand 38 (see reference). Figure 3 (a) in the middle.

[0109] like Figure 3 (a) in Figure 3 As shown in (b), after being positioned in the gripping position, the two movable claws 38a and 38b of the hand 38 move closer to each other. Force sensors 45 (e.g., pressure sensors) that detect the reaction force from the workpiece W are provided in the movable claws 38a and 38b (see Figure 1). Figure 1 The reaction force detected by the force sensor 45 is sent to the upper controller 80. When the reaction force detected by the force sensor 45 reaches the reference (the stop reference reaction force, which will be explained later), the movement of the movable jaws 38a and 38b stops, and the clamping of the workpiece W is completed. After the workpiece is clamped, as... Figure 3 (b) in the middle → Figure 3 As shown in (c), the robot 20 changes its posture while holding part W, and part W is transported to box C. After part W is housed in box C, as... Figure 3 (c) in Figure 3 As shown in (d), each movable claw 38a and 38b is moved (restored) to its initial position and the component W is released, and the robot 20 returns to the standby position.

[0110] Here, refer to Figure 4 The process of positioning the hand 38 in the gripping position is further explained below. A designated area is photographed when a new part W is supplied to that area. This image data is sent to the upper-level controller 80, which extracts the shape of the part W based on the acquired image data and sets the virtual center (temporary center) of the part W as one of the aforementioned target positions, TP. Furthermore, the motion mode (various control points) of the robot 20 is set so that the reference point CP, which serves as the reference for positioning the hand 38 on the robot side, coincides with the target position TP. This set motion mode is sent to the motion controller 60, where the motion controller 60 determines the robot 20's motion trajectory based on the motion mode and the robot 20's current posture.

[0111] In addition, the opposing surfaces of the movable claws 38a and 38b are all planar and orthogonal to the opening and closing directions of the movable claws 38a and 38b. The reference point CP is defined as a position that is equidistant from the two movable claws 38a and 38b (opposing surfaces).

[0112] In this embodiment, the envisioned part W is a food product such as a cream puff or finger pastry, which has a large shape deviation compared to industrial products. Moreover, the distance from the reference point CP to the outer periphery can vary. Therefore, even if the robot 20's posture is changed to align the reference point CP with the target position TP and the hand 38 is positioned in the gripping position between the two movable claws 38a and 38b for the part W, the distance Xa from one movable claw 38a to the part W and the distance Xb from the other movable claw 38b to the part W may not be the same.

[0113] exist Figure 4 In the example shown in (b), the hand 38 is biased such that the distance Xb from the movable pawl 38b to the workpiece W is shorter than the distance Xa from the movable pawl 38a to the workpiece W. When the movable pawls 38a and 38b are moved closer to each other from this state, the timing of the movable pawl 38 touching the workpiece W is different from the timing of the movable pawl 38b touching the workpiece W. Specifically, as... Figure 4 As shown in (c), the movable claw 38b touches the workpiece W before the movable claw 38a touches the workpiece W.

[0114] Since the reaction force on the workpiece W is small, it is conceivable that it may deform significantly upon contact with the movable claw 38b, or slide towards the movable claw 38a side on the mounting surface of the conveyor belt S1 due to being pressed by the movable claw 38b. This situation could be a major cause of damage (including excessive deformation) to the workpiece W. This leads to a decrease in the yield rate in the manufacturing process, potentially hindering the improvement of manufacturing efficiency through the robot system 10. In this embodiment, as one of the features, this concern is eliminated by using machine learning to properly grasp the workpiece W. Hereinafter, refer to Figure 5 as well as Figure 6 The characteristic structure of this embodiment will be described. Figure 5 This is a functional block diagram illustrating the functions of the main control device 70. Figure 6 This is a schematic diagram showing the input data fed into the machine learning device 90.

[0115] The upper-level controller 80 includes a control unit 84 that executes control programs stored in the memory 82, and a data acquisition unit 85 that acquires various data from the robot 20 and the camera 65. The data acquisition unit 85 is functionally configured to cooperate with the robot 20's equipment through the operation of the control unit 84. Therefore, the data acquisition unit 85 functionally includes: an image data acquisition unit 86 that acquires images of the aforementioned defined area from the camera 65; a posture data acquisition unit 87 that acquires posture data representing the robot 20's posture (e.g., posture before the gripping action and posture when gripping the workpiece W) from the encoder 42; and a distance data acquisition unit 88 that acquires distance data from distance sensors 46 (see reference 46) installed on each of the movable grippers 38a and 38b. Figure 1 The unit acquires distance data between the workpiece W and each movable claw 38a, 38b when the hand 38 is positioned in the gripping position (before the adjustment will be explained later); and the reaction force data acquisition unit 89 acquires reaction force data from the force sensor 45, which represents the reaction force from the workpiece W when the workpiece W is gripped by the hand 38.

[0116] The control unit 84 determines the position (target position TP) of the workpiece W within the specified area based on the image data acquired by the image data acquisition unit 86, and stores this position as workpiece position data in the memory 82. Furthermore, it extracts the shape of the workpiece W before the grasping action based on the image data acquired before the grasping action and stores this shape as shape data in the memory 82. It also extracts the shape of the workpiece W after the grasping action (after release) based on the image data acquired after the grasping action (after release) and stores this shape as shape data in the storage unit 82. The timing of acquiring the image data after the grasping action is when a predetermined time has elapsed after the release of the workpiece W (e.g., the time during which the workpiece W can be expected to recover on its own), but this predetermined time can also be estimated using machine learning.

[0117] Furthermore, the type of the part W (cream puff, finger pastry, etc.) is determined based on the shape data prior to the grasping action, and the determined type is stored as type data in the storage unit 82. In the upper-level controller 80, a portion of this data is used to control the robot 20, and a portion of this data is provided as input data to the machine learning device 90. The input data provided to the machine learning device 90 is roughly divided into status data and label data.

[0118] The machine learning device 90 includes a state observation unit 97 that acquires workpiece position data, posture data when gripping the workpiece W (in the gripping position), shape data before and after gripping, and type data based on various state data from the upper-level controller 80. This data is stored in the data storage unit 95 of the learning unit 94. Furthermore, the state observation unit 97 compares the shape data before the gripping action and the shape data after the gripping action (after release) using template matching and calculates the deformation (damage) of the workpiece W triggered by the gripping action. Data representing this deformation (comparison data) is also stored in the data storage unit 95 of the learning unit 94. In this embodiment, the machine learning device 90 calculates and stores the comparison data itself, but it is not limited to this. The comparison data can also be provided (input) to the machine learning device 90 by a user. Additionally, the machine learning device 90 includes a tag data acquisition unit 98 that acquires various tag data from the upper-level controller 80. The tag data acquisition unit 98 acquires stop reference data representing the stop reference set as a stop reference, as well as the aforementioned distance data, and stores them in the data storage unit 95.

[0119] Furthermore, the sources of various input data fed into the machine learning device 90 are not limited to the upper-level controller 80. For example, it could also be a structure that directly acquires input data from the robot 20 or the camera 65 without going through the upper-level controller 80.

[0120] The learning unit 94 of the machine learning device 90 learns the correlation between the various state data and label data, and constructs a model (motion setting model) representing the correlation between the state data and label data through this learning. This motion setting model is stored in the learning model storage unit 96 of the learning unit 94 and is updated based on newly acquired input data. By repeatedly updating the motion setting model, the stopping reference reaction force and position adjustment reference distance are learned based on the situation to increase the success rate of grasping (transfer) and reduce damage (deformation) to the workpiece W. Furthermore, the motion setting model is configured such that the amount of data summarizing the learning progress is reflected in the motion setting model, allowing its use.

[0121] In the result output unit 99 of the machine learning device 90, based on the allowed motion setting model and the motion control data of the robot 20 (workpiece position data, posture data before the gripping action, shape data before the gripping action, and type data), the estimated result of the appropriate stopping reference reaction force is prompted to the upper-level controller 80, or based on the allowed motion setting model and the motion control data of the robot 20 (workpiece position data, posture data when gripping the workpiece W, shape data before the gripping action, and type data), the appropriate position adjustment reference distance is prompted to the upper-level controller 80. Here, the appropriate stopping reference reaction force shown in this embodiment refers to the minimum estimated value that can grip the workpiece W and can minimize the damage (deformation) of the workpiece W based on the gripping action, and the appropriate position adjustment reference distance refers to the estimated value of the difference in distances from each movable claw 38a, 38b to the workpiece W, where the influence (damage) of the contact timing deviation is 0 or approximately 0.

[0122] Furthermore, in this embodiment, various input data in the case of successfully gripping the workpiece W are stored in the data storage unit 95, while various input data in the case of failure to grip the workpiece W are not stored in the data storage unit 95 and are eliminated.

[0123] The machine learning device 90 described in detail above applies a learning algorithm executed in the learning unit 94, which is called teacher-led learning. Teacher-led learning is a method of learning to estimate the model of the execution result relative to new execution conditions by identifying features that suggest the correlation between the execution conditions and the execution result based on a known dataset (so-called teacher data) of the execution conditions and the corresponding execution results.

[0124] In this teacher-led learning process, neural networks are used in the construction of the action setting model. The following refers to... Figure 7 A general overview of neural networks is provided. Furthermore, for ease of explanation, [the following is omitted]. Figure 7 The example illustrates a three-layer neural network with four types of input data and three types of output data, but the number of input data, output data, and intermediate layers are not limited to these.

[0125] A neural network is a collection of multiple nodes N. Each node N is connected to multiple other nodes N, and weights w are assigned between the connected nodes N. The collection of nodes can be roughly divided into three groups: an input layer D1 (nodes N11-N14) that functions as the input layer receiving various input data; an intermediate layer D2 (nodes N21-N23) that performs calculations using weights w2; and an output layer D3 (nodes N31-N33) that outputs the output data. The number of nodes in the input layer D1 is determined by the type of input data x, and the number of nodes in the output layer D3 is determined by the type of output data y.

[0126] In the machine learning device 90 shown in this embodiment, the following data are taken as input data x: workpiece position data, posture data before the grasping action, posture data when grasping workpiece W, shape data, type data, comparison data, stopping reference data, and distance data. The learning unit 94 estimates the appropriate stopping reference reaction force and position adjustment reference distance as appropriate output data y by performing calculations according to the multi-layer structure of the neural network described above. Furthermore, the neural network's action modes include a learning mode for performing the above-described learning and a value prediction mode. For example, weights w can be learned through the learning mode, and the value of the action can be determined using the value prediction mode using the learned weights w.

[0127] Next, refer to Figure 8 The process of setting the action mode and adjusting the position, which are periodically executed by the CPU 81 of the upper controller 80, is explained.

[0128] In the motion setting process, firstly, image data is acquired from camera 65 (step S11), and shape data of the workpiece W before the grasping action is created (extracted) based on the acquired image data (step S12). Next, position data of the workpiece W, i.e., target position TP, is set based on the shape data (step S13). Then, posture data of the robot 20 in standby mode (before configuring the action) is acquired (step S14), and control points for motion trajectory generation are set based on the target position TP and posture data (step S15). Then, type data indicating the type of workpiece W identified from the image data is acquired (step S16), and the aforementioned stop reference reaction force is set based on the type data (step S17). In the processing of step S17, if the use of the motion setting model is allowed, the stop reference reaction force is determined based on the motion setting model; if the use of the motion setting model is not allowed, the stop reference reaction force is determined (selected) based on a pre-set candidate range.

[0129] In the position adjustment process, firstly, it is determined whether it is the right time for the hand 38 to be positioned in the gripping position (step S21). If it is not the right time for the hand to be positioned in the gripping position, the position adjustment process ends directly. If the hand 38 is positioned in the gripping position, posture data representing the posture of the robot 20 and distance data representing the distances between the workpiece W and each movable claw 38a, 38b are obtained (step S22). Next, it is determined whether the use of the motion setting model is allowed (step S23). If the use of the motion setting model is not allowed, the position adjustment process ends directly. That is, position adjustment is avoided if the use of the motion setting model is not allowed. If the use of the motion setting model is allowed, it is determined whether position adjustment is needed (step S24). If the difference in distance does not exceed the position adjustment reference distance, it is determined that no position adjustment will be performed (step S25: No), and the position adjustment process ends directly. If the difference in distance exceeds the position adjustment reference distance, it is determined that position adjustment will be performed (step S25: Yes), and the details of the position adjustment are determined (step S26). Specifically, in order to adjust the position of the hand 38 so that the difference in distance between the workpiece W and each movable claw 38a, 38b is zero, the direction and amount of displacement of the hand 38 are determined. Based on the determined direction and amount of displacement, motion trajectory correction processing is performed. Specifically, in order to modify the motion trajectory of the robot 20 to a position-adjusted motion trajectory, each control point is re-set. The re-set control points are sent to the motion controller 60.

[0130] Next, refer to Figure 9 The flowchart illustrates the learning processes that are periodically executed by the CPU 91 of the machine learning device 90.

[0131] In the learning process, firstly, it is determined whether the entire process of transferring workpiece W to box C is complete and the machine returns to a standby position (step S31). If not, the learning process ends directly. If it is the correct time, various status data are obtained from the upper-level controller 80 (step S32). Specifically, workpiece position data, posture data before the gripping action, posture data when gripping workpiece W, shape data before the gripping action, type data, and image data of the specified area after the gripping action are obtained. Next, various tag data are obtained from the upper-level controller 80 (step S33). Specifically, stop reference data and distance data are obtained.

[0132] Next, the image of the designated area after the grasping action is analyzed to determine whether the workpiece W remains in the designated area, i.e., whether the handling (grasping) of workpiece W was successful (step S34). If the handling of workpiece W fails, the various input data obtained are not stored in the data storage unit 95, i.e., they are not used as input data for machine learning, and the learning process ends.

[0133] In contrast, if the workpiece W is successfully handled (grabbed), the deformation of the workpiece W is calculated. Specifically, the shape data of the workpiece W before the grasping action is compared with the shape data of the workpiece W after the grasping action (after release), the deformation (damage) of the workpiece W is calculated, and comparison data is generated (step S35). Then, the workpiece position data, posture data, shape data, type data, comparison data, stop reference data, and distance data are saved in the data storage unit 95, and the motion setting model is updated based on the newly acquired data. Afterward, the data is accumulated until at least the above-mentioned use is allowed, and the update of the motion setting model is repeated. Alternatively, the motion setting model can be updated (built) when the accumulated data reaches a reference amount, without having to perform an update every time new data is acquired.

[0134] Based on the first embodiment described in detail above, the following excellent effects can be expected.

[0135] In this embodiment, a motion setting model for setting the motion modes (position adjustment motion and grasping motion) of the robot 20 is constructed by using machine learning with various input data including stopping reference data, distance data, and comparison data. Based on this structure, it is possible not only to set appropriate stopping references for the movable claws 38a and 38b, but also to appropriately adjust the position of the hand 38 according to its configuration. In other words, this contributes to the realization of a robot system that can suppress damage to the workpiece W caused by deviations in the timing of the movable claws 38a and 38b contacting the workpiece W, and can appropriately grasp low-reaction and amorphous workpieces W.

[0136] When a workpiece W is damaged under low reaction force, changes in weight and shape occur. In most cases where weight changes occur, shape also changes. Therefore, by using shape as the object of comparison before and after the gripping action, the degree of damage to workpiece W can be appropriately reflected in machine learning.

[0137] For the robot system 10, the system's convenience can be improved by setting a certain allowable range (the aforementioned defined area) for the position of the supplied workpiece W. However, when such a range is set, the contact methods of the movable claws 38a and 38b when performing the grasping action of the workpiece W may differ depending on where the workpiece W is supplied within that range. Furthermore, the contact methods of the movable claws 38a and 38b with the workpiece W may differ depending on the robot 20's posture when grasping the workpiece W. Therefore, any structure that adds workpiece position data representing the position of the workpiece W and posture data representing the posture of the robot 20 to the input data for machine learning can contribute to the realization of a robot system that appropriately grasps low-reaction and amorphous workpieces W.

[0138] <Second Implementation Method>

[0139] The first embodiment described above illustrates a structure for constructing an action setting model through teacher-led learning, but the learning method used to construct the action setting model can also be reinforcement learning.

[0140] When constructing an action setting model using reinforcement learning, the reward can be increased when the difference between the shape of the workpiece W before the grasping action and the shape of the workpiece W after the grasping action (after release) is within a reference amount (5% in this embodiment), and decreased when it exceeds the reference amount. Furthermore, in the second embodiment described above, only data from successful grasping of the workpiece W is used as input data for learning; however, it is also possible to set a reward for both successful and unsuccessful grasping, using data from failed grasping as input data. In this case, for example, the reward could be increased for successful grasping of the workpiece W, and decreased for failed grasping. A structure prioritizing successful grasping can be achieved by allocating a higher reward for successful grasping of the workpiece W than for grasping shapes within a reference amount.

[0141] Compared to the case without position adjustment, the handling efficiency decreases slightly but significantly when position adjustment is performed. Furthermore, the greater the displacement of the hand 38 during position adjustment, the lower the handling efficiency. Therefore, by using reinforcement learning to set the reward related to handling efficiency, it is possible to balance the protection of the workpiece W with improved handling efficiency. For example, by using a structure where a smaller displacement of the hand 38 during position adjustment increases the reward and a larger displacement decreases the reward, the displacement of the hand 38 can be suppressed to the necessary minimum even when position adjustment is required. Alternatively, the system can be configured to add the time data required from the start to the end of the robot 20's action as input data, where a shorter time increases the reward and a longer time decreases the reward.

[0142] <Third Implementation Method>

[0143] In the first embodiment described above, the movable claws 38a and 38b are configured to stop when the reaction force detected by the force sensor 45 reaches a stopping reference. The smaller the reaction force of the workpiece W, the more difficult it is to detect the reaction force smoothly. Therefore, for the workpiece W with extremely small reaction force, the reference for stopping the movable claws 38a and 38b can be changed from the reaction force to the relative distance (space or width) between the two movable claws 38a and 38b, that is, the amount of displacement of the two movable claws 38a and 38b in the clamping direction.

[0144] In this structure, when the motion setting model is allowed, various data can be used to set a reference relative distance as the stopping reference. When the motion setting model is not allowed, the stopping reference reaction force can be determined (selected) based on a pre-set candidate range. Furthermore, during learning, data representing the relative distance when performing a grasping action can be used as the aforementioned stopping reference data, and this relative distance can be used as input data for learning.

[0145] <Fourth Implementation Method>

[0146] For the parts W shown in the first embodiment above (especially cream puffs and daifuku), the shape is irregular, and the restoring force of the parts W after clamping may vary depending on the direction from which the parts W are clamped. In other words, the timing deviation and load effect may differ depending on the clamping direction. Conversely, the positional relationship between the object that does not require adjustment and each movable claw 38a, 38b may differ depending on the clamping direction. Furthermore, the amount of deformation (damage) may be reduced when clamping from other directions compared to clamping in a certain direction. In this embodiment, considering this situation, as one feature, clamping direction data indicating the clamping direction is added to the input data. Hereinafter, refer to... Figure 10 The input data will be explained. Furthermore, the gripping direction refers to the direction in which the movable claws 38a and 38b move during the gripping action.

[0147] An image of the workpiece W positioned within a designated area is captured and input from camera 65 to the upper-level controller 80. In the upper-level controller 80, a reference direction is set in the workpiece W by referring to shape data representing the outline of the workpiece W created (extracted) based on the captured image. Specifically, the two points with the largest distance between two points on the outline are determined, and the direction of the straight line connecting these two points is set as the reference direction. During learning, the clamping direction is obtained as data indicating the relationship between the clamping direction and the reference direction. More specifically, the clamping direction data is data representing the angle of the clamping direction relative to the reference direction. Thus, by adding the clamping direction to the input data, the influence of the clamping direction is reflected in the constructed motion setting model. Therefore, when determining the motion method based on the direction data and the motion setting model, the set stop reference reaction force and position adjustment reference distance are data that take the clamping direction into account.

[0148] In addition, in this embodiment, the two points with the largest distance between two points on the outline are determined, and the direction of the straight line connecting these two points is set as the reference direction. However, alternatively, the two points with the smallest distance between two points on the outline can also be determined, and the direction of the straight line connecting these two points can be set as the reference direction.

[0149] Furthermore, in the fourth embodiment described above, the stopping reference reaction force and the position adjustment reference distance are set considering the clamping direction. However, it is also possible to configure the clamping direction using various data (shape data of the part W) and the motion setting model, provided that the motion setting model is allowed to be used. In other words, the orientation of the hand 38 can also be set to a clamping direction that results in a small stopping reference reaction force and a large position adjustment reference distance.

[0150] <Fifth Implementation Method>

[0151] For the workpiece W shown in the first embodiment described above, the shape deviation is large and the reaction force is small. When gripping such a workpiece W, the influence on the workpiece W may vary depending on the contact area between the workpiece W and the movable claws 38a and 38b. In other words, the positional relationship between the workpiece W and each movable claw 38a and 38b, which does not require adjustment, may differ depending on the contact area. Furthermore, the amount of deformation (damage) of the workpiece W may be reduced depending on the size of the contact area. In this embodiment, considering this situation, as one of the features, contact area data representing the contact area is added to the input data. Hereinafter, refer to Figure 11 Please provide an explanation of the input data.

[0152] Plate-shaped contact sensors 47a and 47b are respectively provided on the opposing surfaces of the movable claws 38a and 38b. The detection results from the contact sensors 47a and 47b are sent to the upper-level controller 80, where the contact area between the workpiece W and the movable claws 38a and 38b is determined based on these results. When clamping the workpiece W with the same strength, a larger contact area reduces the localized load on the workpiece W. In other words, a larger contact area provides better protection for the workpiece W.

[0153] Contact area data, representing the contact area, is provided to the machine learning device 90. The machine learning device 90 stores the contact area data as input data in the data storage unit 95 and uses the contact area data to construct a motion setting model. By determining the relationship between the shape of the workpiece W and the contact area through machine learning, and estimating the contact area based on the motion setting model and the shape of the workpiece W, the stop reference reaction force and position adjustment reference distance can be set based on the estimated contact area.

[0154] Furthermore, in the fifth embodiment described above, the estimated contact area is considered when setting the stop reference reaction force and the position adjustment reference distance. However, instead, it can be configured to use various data (shape data of the part W) and the motion setting model to set an appropriate clamping direction for the contact area, provided that the motion setting model is allowed to be used (see the fourth embodiment). In other words, it can also be configured to set the orientation of the hand 38 to a clamping direction that minimizes the stop reference reaction force and maximizes the position adjustment reference distance.

[0155] <Other Implementation Methods>

[0156] Furthermore, the embodiments described above are not limited to the specific details described. For example, they can be implemented as follows. Additionally, the following structures can be applied individually to each of the above embodiments, or they can be combined in combination with some or all of them. Furthermore, it is possible to arbitrarily combine all or some of the various structures shown in the above embodiments. In such cases, it is preferable to ensure the technical significance (effectiveness) of each structure that is the object of the combination.

[0157] The motion setting models shown in the above embodiments can also be independently constructed as a model for position adjustment (position adjustment model) and a model for stopping reference setting (gripping model).

[0158] Furthermore, in the above embodiments, the gripping action and the position adjustment action, which are part of the robot 20's actions, are configured as "prescribed actions," and a model for setting these prescribed actions is constructed using machine learning. However, these can also be changed, and a model for setting the entire action of the robot 20, including the configuration action of the hand 38, the position adjustment action of the hand 3, and the gripping action of the part W, can be constructed using machine learning.

[0159] In the above embodiments, the robot system 10 is configured to monitor the success and degree of damage of the gripping of the workpiece W, but it is not limited to this. For example, it may also be configured so that a worker monitors the success and degree of damage of the gripping and inputs the monitoring results into the machine learning device 90.

[0160] Environmental data (temperature and humidity data) representing the surrounding environment of the robot 20 can also be added to the input data used for learning. For grasping objects with low reaction force and irregular shape, such as cream puffs, the hardness may vary depending on environmental conditions such as temperature and humidity. In other words, differences may arise in the positional relationship between the grasping object and each movable claw 38a, 38b, even if adjustments are not required based on environmental conditions. Therefore, as shown in this modified example, as long as the structure adds environmental data to the various input data shown in each embodiment and performs machine learning, further optimization of the position adjustment action and grasping action can be expected.

[0161] In the above embodiments, the displacement speed of the movable claws 38a and 38b during the gripping action is kept constant, but this is not a limitation. The displacement speed can also be used as a variable parameter, and speed data representing the displacement speed when gripping the workpiece W can be added to the input data for learning. In the event of the aforementioned timing deviation, the influence on the gripped object may differ depending on the displacement speed of the movable claws 38a and 38b. In other words, the positional relationship between the object that does not require adjustment and each of the movable claws 38a and 38b may differ depending on the displacement speed. Therefore, as shown in this modified example, as long as a structure is used to add speed data to the various input data shown in each embodiment and perform machine learning, further optimization of the gripping action can be expected.

[0162] Alternatively, a target point setting model for setting the target position TP can be constructed using machine learning based on shape data representing the shape of the workpiece W before the gripping action, workpiece position data representing the set target position TP, and distance data representing the distances between each movable claw 38a, 38b and the workpiece W when the hand 38 is positioned in the gripping position. By improving the accuracy of setting the target position TP, the need for the aforementioned position adjustments can be reduced, contributing to improved conveying efficiency.

[0163] In the above embodiments, the gripping part is configured to move during position adjustment, but it is not limited to this. For example, it may be a structure in which the movable claw on the side with a larger distance from the workpiece W moves closer to the workpiece W to align the distance, or it may be a structure in which the movable claw on the side with a smaller distance from the workpiece W moves away from the workpiece W to align the distance. However, in such a structure, the workpiece W may be gripped in a position biased towards one side of the movable claw. This is not preferred in terms of stabilizing the gripping function and improving the efficiency of the placement operation to the box C. In other words, as shown in the above embodiments, there is technical significance in a structure in which the hand 38 is moved instead of the movable claws 38a and 38b for position adjustment.

[0164] In the above embodiments, a structure is illustrated for determining the amount of change (damage) in the shape of the workpiece W by comparing its shape data before and after the grasping action, but this is not a limitation. Alternatively, or additionally, the structure may be configured to monitor the weight of the workpiece W using a weight sensor or the like, and determine the amount of change in the weight of the workpiece W by comparing its weight before and after the grasping action. In other words, the comparison data used as input data to the machine learning device 90 may also be data representing the weight difference.

[0165] In the above embodiments, the camera 65 is fixed in the ceiling of the factory, but it is also possible to mount the camera 65 on the robot body 30 (e.g., the arm).

[0166] In the above embodiments, the component W is gripped by two movable claws 38a and 38b, but the number of movable claws is arbitrary. For example, it can be three or more.

[0167] In the above embodiments, the main control device 70 is constituted by the upper-level controller 80 and the machine learning device 90, but these upper-level controllers 80 and machine learning devices 90 can also be set up independently. Alternatively, the structure equivalent to the machine learning device 90 can be set up in the cloud.

[0168] In the embodiments described above, cream puffs and finger pastries are examples of objects to be grasped by the robot 20, but it is not limited to these. Other foods with small reaction forces and large shape deviations, such as daifuku and bread, can also be grasped by the robot 20. In addition, the objects to be grasped are not limited to processed foods, and fruits and vegetables such as oranges and tomatoes can also be grasped, while these objects can be appropriately protected.

Claims

1. A robot system comprising: A robot has a gripping part equipped with a set of movable claws, which grips an object by holding it. When the object is gripped by the movable claws, the object deforms and its shape changes. as well as The control device controls the robot to perform a configuration action that positions the gripper at a predetermined position between the set of movable claws, and a gripping action that moves the movable claws toward each other at that predetermined position. The control device is configured to stop the displacement of the set of movable claws when the reaction force from the object during the gripping action reaches a reference value or when the relative distance between the set of movable claws reaches a reference value. in, With the gripping part positioned at the specified location, an adjustment action can be performed to adjust the position of the set of movable claws relative to the object by shifting the gripping part. The reference value for stopping the grasping action is a variable value. The system includes a model building unit that, while grasping the object, acquires stop reference data representing a set reference value, distance data representing the distance between each movable claw of the grasping unit positioned at the predetermined position and the object, and comparison data representing the difference between the shape of the object before and after the grasping action. By using machine learning with this stop reference data, distance data, and comparison data, the system builds a model for setting a predetermined action including the adjustment action and the grasping action. The control device has: The acquisition unit, when the gripping unit is positioned at the predetermined position, acquires distance data indicating the distance between each movable claw of the gripping unit and the object; as well as The setting unit sets the action mode of the robot for the specified action. The setting unit can set the action mode of the prescribed action based on the distance data obtained by the acquisition unit and the model constructed by the model construction unit.

2. The robot system according to claim 1, wherein, The model building unit obtains shape data representing the shape of the object before the grasping action is performed, and direction data representing the relationship between the reference direction of the object and the direction in which the set of movable claws grasp the object, and performs machine learning in a corresponding association with the stop reference data, the distance data, the comparison data, the shape data, and the direction data.

3. The robot system according to claim 1, wherein, The portion of the movable claw that contacts the object is planar. The model building unit obtains shape data representing the shape of the object before the grasping action is performed, and contact area data representing the contact area between the set of movable claws and the object in the state of grasping the object, and performs machine learning in a corresponding association with the stopping reference data, the distance data, the comparison data, the shape data, and the contact area data.

4. The robot system according to claim 1, wherein, The control device is configured to move the set of movable claws at a predetermined speed when the gripping action is performed. The speed is a variable value. The model building unit obtains speed data representing the displacement speed of each movable claw as it moves toward the object, and performs machine learning in a corresponding association with the stopping reference data, the distance data, the comparison data, and the speed data.

5. The robot system according to claim 1, wherein, The model building unit obtains position data representing the position of the object before the grasping action is performed, and performs machine learning on the corresponding correlation between the stopping reference data, the distance data, the comparison data, and the position data.

6. The robot system according to claim 1, wherein, The model building unit obtains posture data representing the robot's posture when grasping the object, and performs machine learning by correspondingly associating the stopping reference data, the distance data, the comparison data, and the posture data.

7. The robot system according to claim 1, wherein, The model building unit obtains environmental data representing the environmental conditions around the robot, including temperature or humidity, and performs machine learning by correspondingly associating the stopping reference data, the distance data, the comparison data, and the environmental data.

8. The robot system according to claim 1, wherein, The setting unit is configured such that, when the distance data obtained by the acquisition unit is data representing a distance within the range specified by the model, the operation mode is set to grasp the object without adjusting the relative position of the set of movable claws and the object; and when the distance data obtained by the acquisition unit is data representing a distance outside the range specified by the model, the operation mode is set to grasp the object after adjusting the relative position of the set of movable claws and the object.

9. A robot system, comprising: A robot has a gripping part equipped with a set of movable claws, which grips an object by holding it. When the object is gripped by the movable claws, the object deforms and its shape changes. as well as The control device controls the robot to perform a configuration action that positions the gripper at a predetermined position between the set of movable claws, and a gripping action that moves the movable claws toward each other at that predetermined position. in, With the gripping part positioned at the specified location, the position of the set of movable claws relative to the object can be adjusted by shifting the gripping part. The system includes a model building unit that acquires distance data representing the distance between each movable claw of the gripping unit positioned at the predetermined location and the object, as well as comparison data representing the difference between the shape of the object before and after the gripping action. By using machine learning with this distance data and comparison data, the system builds a model for setting the position adjustment method of the gripping unit at the predetermined location. The control device has: The acquisition unit, when the gripping unit is positioned at the predetermined position, acquires distance data indicating the distance between each movable claw of the gripping unit and the object; as well as The setting unit sets the position adjustment method based on the distance data obtained by the acquisition unit and the model constructed by the model construction unit.

10. A machine learning device suitable for a robot system, the robot system comprising: a robot having a gripping part having a set of movable claws, for gripping an object by gripping the object with the movable claws, wherein the object deforms and changes shape when gripped by the movable claws; and a control device for controlling the robot to perform a configuration action of positioning the gripping part at a predetermined position such that the object is located between the set of movable claws, and a gripping action of moving the movable claws toward each other at the predetermined position, the control device being configured to stop the movement of the set of movable claws when the reaction force from the object during the gripping action is a reference value or when the relative distance between the set of movable claws is a reference value, and when the gripping part is positioned at the predetermined position, being capable of performing an adjustment action of adjusting the position of the set of movable claws relative to the object by moving the gripping part. in, The reference value for stopping the grasping action is a variable value. The system includes a model building unit that, when grasping the object, acquires stop reference data representing the set reference value, distance data representing the distance between each movable claw of the grasping unit positioned at the predetermined position and the object, and comparison data representing the difference between the shape of the object before the grasping action is performed and the shape of the object after the grasping action is performed. The system uses machine learning based on these stop reference data, distance data, and comparison data to build a model for setting a prescribed action including the adjustment action and the grasping action.

Images