Robot system and picking method

Abstract

The robot system comprises: a robot mechanism having fingers for gripping workpieces; a detection unit for detecting a gripping state of the fingers gripping the workpieces; an identification unit for identifying the number of workpieces gripped by the fingers based on a detection result of the detection unit; and a control unit for controlling the movement of the robot mechanism so that the number of gripped workpieces reaches the specified number when the number of workpieces identified by the identification unit is different from a pre-specified number after causing the robot mechanism to grip the workpieces from a loading location where a plurality of workpieces are loaded, wherein at least one of the fingers is flexible, and when the workpieces are gripped by the fingers, the control unit controls the movement of the robot mechanism so that the position of at least one finger changes.

Description

Technical Field

[0001] The present disclosure relates to a robotic system and a picking method. Background Art

[0002] Patent Documents 1 to 3 disclose technologies for identifying pickable workpieces based on captured images of bulk workpieces and picking up the workpieces.

[0003] Patent Document 1: Japanese Patent No. 5767464

[0004] Patent Document 2: Japanese Patent Application Laid-Open No. 2017-42859

[0005] Patent Document 3: Japanese Patent No. 5196156 Summary of the Invention

[0006] Technical problem to be solved by the invention

[0007] However, in the above-mentioned prior art, it is difficult to accurately detect the position and posture of workpieces using image sensors and depth sensors, especially for small, metal, and shiny workpieces such as screws. As a result, it is difficult to accurately pick up a single workpiece. Furthermore, picking can fail due to inaccurate estimates of clearances from surrounding obstructions and inability to calculate grip probability. Furthermore, because picking up a single workpiece often changes the configuration of surrounding workpieces, time is required to detect the position and posture of the workpiece before each picking action.

[0008] The present disclosure has been made in view of the above-mentioned points, and an object of the present disclosure is to provide a robot system and a picking method that can pick up a workpiece with high accuracy even without detecting the position and posture of the workpiece in advance every time.

[0009] Solutions for solving technical problems

[0010] The first aspect disclosed is a robot system comprising: a robot mechanism having a plurality of fingers for gripping a workpiece; a detection unit for detecting a gripping state of the workpiece gripped by the plurality of fingers; an identification unit for identifying the number of workpieces gripped by the plurality of fingers based on a detection result of the detection unit; and a control unit for controlling the movement of the robot mechanism so that the number of the gripped workpieces reaches the specified number when the number of the workpieces identified by the identification unit is different from a predetermined number after causing the robot mechanism to grip the workpiece from a loading place where the plurality of workpieces are loaded, wherein at least one of the plurality of fingers is flexible, and in a state where the workpiece is gripped by the plurality of fingers, the control unit controls the movement of the robot mechanism so that the position of at least one finger changes.

[0011] In the above-described first aspect, at least one of the plurality of fingers may be formed of an elastic member.

[0012] The second aspect disclosed comprises: a robot mechanism having a plurality of fingers for gripping a workpiece; a detection unit for detecting a gripping state of the workpiece gripped by the plurality of fingers; an identification unit for identifying the number of workpieces gripped by the plurality of fingers based on the detection result of the detection unit; and a control unit for causing the robot mechanism to perform an action of gripping the workpiece from a loading place where the plurality of workpieces are loaded, and when the number of workpieces identified by the identification unit is different from a predetermined specified number, the control unit controls the movement of the robot mechanism so that the number of the gripped workpieces reaches the specified number, the detection unit comprising a pressure distribution sensor, which is provided on the gripping surface of at least one finger and detects The recognition unit recognizes the number of workpieces by using a grasping number recognition model, the grasping number recognition model is a model obtained by learning with the pressure distribution detected by the detection unit as input and the number of workpieces grasped by the multiple fingers as output, the control unit controls the movement of the robot mechanism based on the driving method output from the driving method model, the driving method model is a model obtained by learning with the pressure distribution and the driving state of the driving unit that drives the multiple fingers as input and the number of workpieces grasped by the multiple fingers and the driving method of the driving unit for setting the number of workpieces to the specified number as output.

[0013] In the second aspect, the pressure distribution sensor may be provided with an elastic member.

[0014] In the above-mentioned first aspect, it can also be set that until the number of the workpieces identified by the identification unit reaches the specified number, the control unit controls the movement of the robot mechanism in a manner that repeatedly causes at least one of the multiple fingers to move in at least one of a first direction as a movement direction for grasping, a second direction intersecting the first direction and the long side direction of the fingers, a third direction as the long side direction of the fingers, and a rotation direction rotating with the third direction as the rotation axis.

[0015] In the first aspect, the detection unit may detect the gripping state a plurality of times while the relative positional relationships of the plurality of fingers are different from each other, and the recognition unit may recognize the number of the workpieces based on results of the plurality of gripping state detections.

[0016] The third aspect disclosed is a picking method, which detects the gripping state of a workpiece by the multiple fingers of a robot mechanism having multiple fingers for gripping the workpiece, identifies the number of workpieces gripped by the multiple fingers based on the detection result of the gripping state of the workpiece, and after causing the robot mechanism to grip the workpiece from a loading place where the multiple workpieces are loaded, when the number of the identified workpieces is different from a pre-specified number, controls the movement of the robot mechanism so that the number of the gripped workpieces reaches the specified number, at least one of the multiple fingers is flexible, and in the state of gripping the workpiece by the multiple fingers, controls the movement of the robot mechanism so that the position of at least one finger changes.

[0017] The fourth aspect disclosed is a picking method, which detects the gripping state of a robot mechanism having multiple fingers for gripping the workpiece, identifies the number of workpieces gripped by the multiple fingers based on the detection result of the gripping state of the workpiece, and after causing the robot mechanism to grip the workpiece from a loading place where the multiple workpieces are loaded, when the number of the identified workpieces is different from a pre-specified number, controls the movement of the robot mechanism so that the number of the gripped workpieces reaches the specified number, including a pressure distribution sensor, which is provided on the gripping surface of at least one finger and detects The pressure distribution of the gripping surface contacted by the workpiece is measured, and the number of the workpieces is identified by a gripping number recognition model, wherein the gripping number recognition model is a model obtained by learning with the detected pressure distribution as input and the number of the workpieces gripped by the multiple fingers as output, and the movement of the robot mechanism is controlled based on the driving method output from the driving method model, wherein the driving method model is a model obtained by learning with the pressure distribution as input and the number of the workpieces gripped by the multiple fingers and the driving method of at least one of the multiple fingers for setting the number of the workpieces to the specified number as output.

[0018] Effects of the Invention

[0019] According to the present disclosure, a workpiece can be picked up with high accuracy even without detecting the position and posture of the workpiece in advance every time. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a diagram of the robot system.

[0021] Figure 2 This is a functional block diagram of the control device.

[0022] Figure 3 It is a block diagram showing the hardware configuration of the control device.

[0023] Figure 4 Flowchart showing the flow of the picking process.

[0024] Figure 5 A three-dimensional diagram of a finger.

[0025] Figure 6 This image shows the fingers gripping a workpiece, viewed from below along the Z-axis.

[0026] Figure 7 This image shows the fingers gripping a workpiece as viewed along the Y-axis.

[0027] Figure 8 This image shows the fingers gripping a workpiece as viewed along the X-axis.

[0028] Figure 9 A diagram showing a pressure distribution sensor and an elastic member.

[0029] Figure 10 A diagram for explaining an elastic member. DETAILED DESCRIPTION

[0030] An example embodiment of the present disclosure is described below with reference to the accompanying drawings. It should be noted that identical or equivalent components and parts are denoted by the same reference numerals in the various drawings. Furthermore, the dimensional ratios in the drawings may be exaggerated for ease of description and may differ from actual ratios.

[0031] <First embodiment>

[0032] Figure 1 FIG is a structural diagram of the robot system 10 according to the first embodiment. Figure 1 As shown, the robot system 10 includes a robot mechanism 20, a control device 30, and an imaging unit 40. The robot system 10 functions as a pickup device that picks up a workpiece W in this embodiment.

[0033] The robot mechanism 20 includes a robot arm AR, which is the subject of motion control during the picking operation, i.e., the mechanical portion, and a robot hand H mounted at the distal end of the robot arm AR. The robot hand H grasps the workpieces W from a box 50, which is an example of a loading area for multiple workpieces W. The robot hand H is an example of a grasping unit. It should be noted that in this embodiment, the workpieces W are described as relatively small parts such as screws, as an example, such that the robot hand H can grasp multiple workpieces W. However, the workpieces W are not limited to this embodiment.

[0034] The robot H has a plurality of fingers, and in this embodiment, two fingers F1 and F2 are provided as an example, but the number of fingers is not limited to 2. In addition, the fingers F1 and F2 are formed of plate-shaped members as an example in this embodiment, but the shapes of the fingers F1 and F2 are not limited to this.

[0035] Furthermore, the robot H includes a drive unit 22-1 and a drive unit 22-2. The drive unit 22-1 drives the finger F1 to change its position while the fingers F1 and F2 grip the workpiece W, and the drive unit 22-2 drives the finger F2 to change its position while the fingers F1 and F2 grip the workpiece W. It should be noted that in this embodiment, the drive units are provided on both fingers F1 and F2, but a drive unit may also be provided on either finger F1 or F2.

[0036] Furthermore, a pressure distribution sensor 24-1 is provided on the gripping surface of finger F1 to detect the pressure distribution of the gripping surface in contact with the workpiece W. Similarly, a pressure distribution sensor 24-2 is provided on the gripping surface of finger F2 to detect the pressure distribution of the gripping surface in contact with the workpiece W. The pressure distribution sensors 24-1 and 24-2 are examples of detection units that detect the gripping state of the workpiece W by fingers F1 and F2.

[0037] It should be noted that, in this embodiment, a pressure distribution sensor is provided on each of the gripping surfaces of fingers F1 and F2. However, a pressure distribution sensor may be provided on the gripping surface of any one of fingers F1 and F2.

[0038] As an example, the robot mechanism 20 may be a vertical articulated robot or a horizontal articulated robot having six degrees of freedom. However, the degrees of freedom and types of the robot are not limited to these.

[0039] The control device 30 controls the robot mechanism 20. Figure 2 As shown, the control device 30 functionally includes an identification unit 32 and a control unit 34 .

[0040] The recognition unit 32 identifies the number of workpieces W gripped by the fingers F1 and F2 based on the detection results, i.e., the pressure distribution, of the pressure distribution sensors 24-1 and 24-2. In this embodiment, the recognition unit 32 uses, as an example, a learned model, such as a learned model using a neural network, to identify the number of workpieces W gripped by the fingers F1 and F2. The learned model is a model learned using each of the pressure distributions detected by the pressure distribution sensors 24-1 and 24-2 as input and the number of workpieces W as output.

[0041] After the robot mechanism 20 has grasped the workpieces W from the box 50, if the number of workpieces identified by the recognition unit 32 differs from a pre-specified number, the control unit 34 controls the movement of the robot mechanism 20 so that the number of grasped workpieces W reaches the specified number. Note that in this embodiment, the case where the specified number is 1 is described. That is, if the recognition unit 32 has recognized multiple workpieces, the movement of the robot mechanism 20 is controlled so that the workpieces W are dropped until the number of workpieces identified by the recognition unit 32 reaches one.

[0042] Specifically, for example, to change the gripping state of the workpiece W held by fingers F1 and F2, the control unit 34 controls at least one of the drive units 22-1 and 22-2 to move at least one of the fingers F1 and F2. In other words, at least one of the fingers F1 and F2 is moved to change the relative position of the fingers F1 and F2. This changes the gripping state of the workpiece W held by fingers F1 and F2, allowing the workpiece W to be dropped.

[0043] The imaging unit 40 is provided at a position capable of imaging the workpieces W in the box 50 from above the box 50 , and outputs an image obtained by imaging the workpieces W in the box 50 to the control device 30 in response to an instruction from the control device 30 .

[0044] Next is a block diagram showing the hardware configuration of the control device 30 .

[0045] like Figure 3 As shown, the control device 30 includes a CPU (Central Processing Unit) 30A, a ROM (Read Only Memory) 30B, a RAM (Random Access Memory) 30C, a memory 30D, an input unit 30E, a monitor 30F, an optical disk drive 30G, and a communication interface 30H. These components are interconnected via a bus 30I so as to be able to communicate with each other.

[0046] In this embodiment, a pickup program is stored in memory 30D. CPU 30A is a central processing unit that executes various programs and controls various components. Specifically, CPU 30A reads programs from memory 30D and executes them using RAM 30C as a work area. CPU 30A controls the aforementioned components and performs various calculations based on the programs stored in memory 30D.

[0047] ROM 30B stores various programs and data. RAM 30C temporarily stores programs and data as a work area. Memory 30D is composed of a HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0048] The input unit 30E includes pointing devices such as a keyboard 30E1 and a mouse 30E2, which are used for various inputs. The monitor 30F, for example, is a liquid crystal display and displays various information such as the gripping status of the workpiece W. The monitor 30F can also be a touch panel and function as the input unit 30E. The optical disk drive 30G reads data stored on various recording media (such as CD-ROMs and Blu-ray discs) and writes data to the recording media.

[0049] The communication interface 30H is an interface for communicating with other devices, and for example, a standard such as Ethernet (registered trademark), FDDI, or Wi-Fi (registered trademark) can be used.

[0050] Figure 2 Each functional configuration of the control device 30 shown is realized by the CPU 30A reading a pickup program stored in the memory 30D, developing the pickup program in the RAM 30C, and executing the program.

[0051] Next, the operation of the robot system 10 will be described.

[0052] Figure 4 Flowchart 1 is a flowchart showing the flow of the picking process implemented by the robot system 10. When the user operates the input unit 30E and instructs execution of the picking process, the CPU 30A reads the picking program from the memory 30D, expands the picking program into the RAM 30C, and executes the program, thereby executing the picking process.

[0053] In step S100, the CPU 30A, acting as the control unit 34, controls the robot mechanism 20 to grasp the workpiece W within the box 50 using the manipulator H. Specifically, for example, the CPU 30A instructs the imaging unit 40 to capture images of the workpiece W within the box 50. The captured images are then analyzed to determine the location of the workpiece W. In this case, it is not necessary to determine the position or posture of the workpiece W to be grasped; it is sufficient to simply determine the location of the workpiece W. The robot arm AR is then controlled to move the manipulator H to the location of the workpiece W, and the drive units 22-1 and 22-2 are then controlled to grasp the workpiece W using the fingers F1 and F2. Alternatively, the imaging unit 40 may be omitted, and instead the fingers F1 and F2 may be closed at any location within the box 50, resulting in a certain probability of grasping the workpiece W. In particular, when a large number of workpieces W remain within the box 50, the workpiece W can be grasped with a high probability even without determining the location of the workpiece W in advance.

[0054] In step S102, CPU 30A, acting as recognition unit 32, obtains the pressure distributions of the gripping surfaces of fingers F1 and F2 from pressure distribution sensors 24-1 and 24-2, respectively. Based on the obtained pressure distributions of the gripping surfaces of fingers F1 and F2, CPU 30A identifies the number of workpieces W gripped by fingers F1 and F2.

[0055] In step S104, the CPU 30A, acting as the control unit 34, determines whether the number of workpieces W identified in step S102 is zero. If the number of workpieces W identified is not zero, that is, if at least one workpiece W is being gripped, the process proceeds to step S106. On the other hand, if the number of workpieces identified is zero, the process returns to step S100 to re-grip the workpiece W.

[0056] In step S106, the CPU 30A, acting as the control unit 34, determines whether the number of workpieces W identified in step S102 is the specified number, that is, whether the number of workpieces W identified is one. If the number of workpieces W identified is one, the process proceeds to step S108. On the other hand, if the number of workpieces W identified is not one, that is, if there are multiple workpieces, the process proceeds to step S110.

[0057] In step S108 , the CPU 30A, serving as the control unit 34 , controls the robot mechanism 20 to move the workpiece W gripped by the robot hand H to a predetermined location and place the workpiece W thereon.

[0058] In step S110, CPU 30A, acting as control unit 34, controls at least one of drive units 22-1 and 22-2 to move at least one of fingers F1 and F2. For example, the finger to be moved, the direction, and the amount of movement can be predetermined, or the finger to be moved, the direction, and the amount of movement can be determined based on the pressure distribution on the gripping surfaces of fingers F1 and F2 acquired in step S102. This changes the gripping state of the workpiece W held by fingers F1 and F2, making it easier for the workpiece W to drop.

[0059] In step S112, CPU 30A, acting as control unit 34, determines whether all workpieces W in box 50 have been removed. In other words, it determines whether box 50 is empty. Specifically, for example, image analysis is performed on the image captured by imaging unit 40 to determine whether any workpieces W remain in box 50. If no workpieces W remain in box 50, the routine ends. On the other hand, if workpieces W remain in box 50, the routine returns to step S100, repeating the same process until all workpieces W have been removed.

[0060] Thus, in this embodiment, rather than detecting the position and posture of the workpiece W to be grasped before grasping the workpiece W, the workpiece W is first grasped, and then at least one of the fingers F1 and F2 is moved until the specified number of grasped workpieces W is reached. This allows workpieces W to be picked up with high accuracy, even without always detecting the position and posture of the workpiece W beforehand.

[0061] (Variation 1)

[0062] Next, Modification 1 of the first embodiment will be described.

[0063] When identifying the number of grasped workpieces W, for example, the number of grasped workpieces W can be identified based on the contact area of ​​the workpieces W on the respective gripping surfaces of fingers F1 and F2. For example, the contact area of ​​the workpieces W on the respective gripping surfaces of fingers F1 and F2 can be calculated based on the pressure distributions detected by pressure distribution sensors 24-1 and 24-2. Alternatively, the number of grasped workpieces W can be identified using, for example, table data or a mathematical expression that indicates the correspondence between each contact area and the number of grasped workpieces. For example, if the workpieces W are spherical, the contact area formed by pressing a single workpiece W against the elastic gripping surface is relatively stable, making this approach suitable for identifying the number of grasped workpieces.

[0064] (Variation 2)

[0065] Next, a second modification of the first embodiment will be described.

[0066] When identifying the number of gripped workpieces W, for example, an imaging unit 40, serving as an example of a detection unit, may capture images of the workpieces W gripped by the fingers F1 and F2, and the number of gripped workpieces W may be identified based on the captured images captured by the imaging unit 40. In this case, the control unit 34 may move the robot H so that the imaging unit 40 captures images of the workpieces W gripped by the fingers F1 and F2 at a position where the image can be captured, i.e., a position where the workpieces W are not blocked by the fingers F1 and F2. Alternatively, a configuration may further include a mechanism for moving the imaging unit 40, and the imaging unit 40 may be moved to the position of the robot H. Furthermore, both the robot H and the imaging unit 40 may be moved.

[0067] (Variation 3)

[0068] Next, Modification 3 of the first embodiment will be described.

[0069] When identifying the number of gripped workpieces W, for example, a configuration may be provided that includes a six-axis force sensor as an example of a detection unit. The six-axis force sensor detects the force acting on the fingers F1 and F2 when the fingers F1 and F2 grip the workpieces W. The identification unit 32 identifies the number of workpieces W based on the increase in the vertical component of the force detected by the force sensor after gripping, relative to the increase before gripping, that is, the total weight of the workpieces W. In this case, for example, the number of workpieces W may be identified based on the total weight of the workpieces W and the weight of each workpiece W. In other words, the number of workpieces W may be calculated by dividing the total weight of the workpieces W calculated based on the force detected by the force sensor by the weight of each workpiece W.

[0070] (Variation 4)

[0071] Next, Modification 4 of the first embodiment will be described.

[0072] Alternatively, when the number of workpieces W identified by the recognition unit 32 is different from a specified number, for example, when the number is less than the specified number, the control unit 34 may control the movement of the robot mechanism 20 so as to re-grasp the workpieces W from the box 50. In other words, the robot mechanism 20 may be controlled so as to first open the fingers F1 and F2 on the box 50 to return the temporarily grasped workpieces W to the box 50, and then re-grasp the workpieces W in the box 50.

[0073] (Variant 5)

[0074] Next, Modification 5 of the first embodiment will be described.

[0075] Alternatively, when the number of workpieces W identified by the recognition unit 32 exceeds a predetermined number, the control unit 34 may control the robot mechanism 20 to apply an external force to at least a portion of the workpieces gripped by the fingers F1 and F2, thereby causing the workpieces W to drop. For example, the robot mechanism 20 may be controlled to cause the workpieces W to collide with a rod-shaped fixed fixture, thereby causing the workpieces W to drop. Alternatively, for example, an external force mechanism may be further provided that applies an external force to the workpieces W using a rod-shaped member, and the external force mechanism may be controlled to cause the rod-shaped member to collide with the workpieces W, thereby causing the workpieces W to drop.

[0076] <Second embodiment>

[0077] Next, a second embodiment will be described. Components identical to those in the first embodiment are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0078] In the second embodiment, at least one of the fingers F1 and F2 is described as having flexibility. In the second embodiment, both the fingers F1 and F2 are described as having flexibility, but only one of the fingers F1 and F2 may be flexible.

[0079] like Figure 5 As shown, the finger F1 according to the second embodiment has a corrugation F1A which is flexible in the X-axis direction when the longitudinal direction of the finger F1 is along the Z-axis direction, that is, the direction of gravity. Figure 5 The structure is flexible in the X-axis direction.

[0080] The finger F2 has a corrugation F2A that is flexible in the X-axis direction. Figure 5 The finger F2 has flexibility in the X-axis direction. In addition, the finger F2 has a corrugation F2B that is flexible in the Y-axis direction. Figure 5 It is also flexible in the Y-axis direction.

[0081] The driving unit 22-1 has a function of driving the finger F1 so that the finger F1 Figure 5 The rotating motor rotates in the direction of arrow A with the Z axis as the rotation axis. By driving the rotating motor, Figure 6 When the fingers F1 and F2 are viewed from below as shown in FIG. 1 (A), the finger F1 can be rotated from the state of grasping the workpiece W to the state of Figure 6 Move in the direction of arrow A as shown in (B).

[0082] In addition, the finger F2 is connected to the driving unit 22-2 via a first tendon (not shown) in a knot shape. In addition, the driving unit 22-2 includes a first stretching motor that is driven to stretch the first tendon. By driving the first stretching motor, the finger F2 can be moved from Figure 7 (A) shows the state as Figure 7 It should be noted that since the finger F1 has the bellows F1A having flexibility in the X-axis direction, it moves in the X-axis direction following the movement of the finger F2.

[0083] In addition, the fingertip of the finger F2 is connected to the driving unit 22-2 via a second tendon (not shown) in a knot shape. In addition, the driving unit 22-2 has a second stretching motor that is driven to stretch the second tendon. By driving the second stretching motor, the finger F2 can be moved from Figure 8 (A) shows the state as Figure 8 It should be noted that, since the finger F1 has no flexibility in the Y-axis direction, it does not follow the movement of the finger F2 and move in the Y-axis direction.

[0084] In addition, although Figure 1 Although not shown in the figure, the workpiece W is grasped by the fingertip of the finger F1. Figure 9 As shown, a pressure distribution sensor 24-1 is attached, and an elastic member 26-1 is attached thereon. Similarly, finger F2 has a pressure distribution sensor 24-2 attached to the gripping surface of the workpiece W at the fingertip, and an elastic member similar to the elastic member 26-1 is attached thereon.

[0085] The pressure distribution sensors 24-1 and 24-2 are composed of m×n (m and n are integers) pressure detection elements. For example, m=6 and n=4 can be used, but the values ​​of m and n are not limited to these.

[0086] For example, Figure 10 As shown in (A), if the elastic member 26-1 is not provided on the pressure distribution sensor 24-1, when the workpiece W is grasped, the workpiece W can only be detected at the two contact points S1 and S2, so there is a possibility that the workpiece W cannot be identified with high accuracy. Figure 10 As shown in (B), in the case of a configuration in which the elastic member 26-1 is provided on the pressure distribution sensor 24-1, as in Figure 10 As indicated by the arrows in (B), the pressure of the gripped workpiece W is transmitted to the pressure distribution sensor 24 - 1 , and the workpiece W can be identified with high accuracy.

[0087] It should be noted that the fingers F1 and F2 may also be formed of an elastic member. In other words, instead of providing a separate elastic member, the fingertips of the fingers F1 and F2 themselves may be formed of an elastic member.

[0088] In the second embodiment, as an example, the recognition unit 32 recognizes the number of workpieces W through a grasping number recognition model, wherein the grasping number recognition model is a model obtained by learning with the pressure distribution detected by the pressure distribution sensors 24-1 and 24-2 as input and the number of workpieces W grasped by the fingers F1 and F2 as output.

[0089] As the grasping number recognition model, for example, a learned model that has learned a learning model using a convolutional neural network (CNN) can be used, but the present invention is not limited thereto.

[0090] Pressure distribution detection for identifying the number of workpieces W can also be performed multiple times by varying the relative position of fingers F1 and F2. In this case, the control unit 34 controls movement of at least one of fingers F1 and F2 in a predetermined direction, allowing the pressure distribution sensors 24-1 and 24-2 to detect the gripping state of the workpieces W multiple times. For example, while finger F2 is moved in the X-axis direction to press against finger F1, the pressure distribution sensors 24-1 and 24-2 detect the pressure distribution multiple times. In other words, the pressure distribution on the gripping surfaces of fingers F1 and F2 is detected in a time series manner.

[0091] The recognition unit 32 inputs the results of multiple detections by each of the pressure distribution sensors 24-1 and 24-2, namely, the pressure distributions of fingers F1 and F2 detected multiple times, into the grasping number recognition model. By inputting multiple pressure distributions into the grasping number recognition model, the number of workpieces W can be recognized with high accuracy.

[0092] Note that, for example, a stretch sensor that detects the extent of the warping of the finger F1 in the X-axis direction may be provided on the finger F1 , and the output value of the stretch sensor may also be input into the grip count recognition model.

[0093] In a state where the workpiece W is gripped by the fingers F1 and F2 , the control unit 34 controls the movement of the robot mechanism 20 so as to change the position of at least one of the fingers F1 and F2 .

[0094] Specifically, in a state where the workpiece W is gripped by the fingers F1 and F2, the movement of the robot mechanism 20 is controlled in such a manner that at least one of the fingers F1 and F2 moves in at least one of a predetermined first direction, a second direction intersecting the first direction, a third direction intersecting the first and second directions, and a rotation direction rotating about the third direction as a rotation axis. Figure 5 When the X-axis direction is the first direction, the Y-axis direction is the second direction, the Z-axis direction is the third direction, and the direction of arrow A is the rotation direction.

[0095] The moved finger, direction, and amount of movement may be determined based on, for example, the pressure distribution of the gripping surfaces of the fingers F1 and F2 acquired in step S102 , as described in the first embodiment.

[0096] Alternatively, the control unit 34 may control the motion of the robot mechanism 20 based on a drive method output from a drive method model. The drive method model is a model learned using the pressure distribution detected by the pressure distribution sensors 24-1 and 24-2 and the drive states of the drive units 22-1 and 22-2 as inputs, and using the number of workpieces W grasped by the fingers F and F2 and the drive method of the drive units 22-1 and 22-2 used to set the number of workpieces W to a specified number as outputs. The "drive method" referred to herein is, for example, drive-related information such as drive commands or information that generates drive commands, and refers to information equivalent to "actions" in machine learning and "control inputs" in control theory. In the second embodiment, a case of controlling the motion of the robot mechanism 20 based on the drive method output from the drive method model will be described.

[0097] Here, the drive state of the drive unit 22-1 is the rotation angle of the rotation motor of the drive unit 22-1, which corresponds to the amount of movement of the finger F1 in the direction of arrow A from the initial state. Furthermore, the drive state of the drive unit 22-2 is the rotation angle of the first stretching motor and the rotation angle of the second stretching motor of the drive unit 22-2, which correspond to the amount of movement of the finger F2 in the X-axis direction and the amount of movement of the finger F2 in the Y-axis direction from the initial state, respectively.

[0098] In the second embodiment, as an example, the driving method model is configured using a learning model including a learning model based on LSTM (Long Short-Term Memory) which is a type of recurrent neural network (RNN) and / or a learning completion model based on a reinforcement learning model.

[0099] In this driving method model, the pressure distribution detected by the pressure distribution sensors 24-1 and 24-2 and the driving states of the drive units 22-1 and 22-2 are input to the LSTM-based learning model to determine the number of gripped workpieces W. The determined number of workpieces W, the pressure distribution detected by the pressure distribution sensors 24-1 and 24-2, and the driving states of the drive units 22-1 and 22-2 are then input as state information into the reinforcement learning model. The reinforcement learning model outputs the driving method for the drive units 22-1 and 22-2 as an action corresponding to the input state information. Specifically, it outputs the rotation angle of the rotation motor of the drive unit 22-1 and the rotation angles of the first and second stretching motors of the drive unit 22-2. The control unit 34 drives the drive units 22-1 and 22-2 according to the driving method output from the driving method model, causing at least one of the fingers F1 and F2 to move. It should be noted that the output value of the stretch sensor may be included in the state information and further input into the driving method model.

[0100] Next, the operation of the robot system 10 will be described.

[0101] The basic flow of the picking process performed by the robot system 10 according to the second embodiment is the same as that described in the first embodiment. Figure 4 The processing shown in the flowchart is the same. Next, the characteristic processing in the second embodiment will be described.

[0102] exist Figure 4 In step S102, while fingers F1 and F2 are gripping workpieces W, CPU 30A, acting as control unit 34, moves finger F2 in the X-axis direction while simultaneously acquiring, as recognition unit 32, the pressure distributions of the gripping surfaces of fingers F1 and F2 from pressure distribution sensors 24-1 and 24-2 in a time series. The pressure distributions of the gripping surfaces of fingers F1 and F2, acquired in a time series, are then input into a grip count recognition model to identify the number of workpieces W gripped by fingers F1 and F2.

[0103] In step S110, the CPU 30A, acting as the control unit 34, inputs the pressure distribution detected by the pressure distribution sensors 24-1 and 24-2 and the driving states of the drive units 22-1 and 22-2 into the driving method model. Based on the driving method output from the driving method model, the CPU 30A drives the drive units 22-1 and 22-2 to move at least one of the fingers F1 and F2, thereby changing the grip state of the workpiece W. This facilitates the workpiece W from being dropped.

[0104] Thus, in the second embodiment, the grip count recognition model is used to recognize the grip count of the workpiece W, and the driving method model is used to control the movements of the fingers F1 and F2. This allows the workpiece W to be picked up with high accuracy even without always detecting the position and posture of the workpiece W in advance.

[0105] It should be noted that the pick-up process implemented by the CPU reading in and executing the software (program) in each of the above embodiments can also be performed by various processors other than the CPU. As processors in this case, PLD (Programmable Logic Device) and ASIC (Application Specific Integrated Circuit) whose circuit configuration can be changed after manufacturing of FPGA (Field-Programmable Gate Array) etc. can be exemplified, which are processors with a circuit configuration designed for specific purposes in order to perform specific processing, i.e., dedicated circuits. In addition, the pick-up process can be performed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and a CPU and an FPGA, etc.). In addition, the hardware structure of these various processors is, more specifically, a circuit obtained by combining circuit elements such as semiconductor elements.

[0106] In addition, in the above-described embodiments, the pickup program is pre-stored (installed) in the memory 30D or ROM 30B. However, the present invention is not limited to this. The program may also be provided in the form of a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. Furthermore, the program may be downloaded from an external device via a network.

[0107] It should be noted that the disclosure of Japanese Patent Application No. 2020-151539 is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical specifications described in this specification are incorporated herein by reference to the same extent as if each document, patent application, or technical specification were specifically and individually described as incorporated by reference.

[0108] Description of Reference Numerals

[0109] 10 Robotic System

[0110] 20 Robot Mechanism

[0111] 22-1, 22-2 drive unit

[0112] 24-1, 24-2 pressure distribution sensors

[0113] 26-1 Elastic components

[0114] 30 Control device

[0115] 32 Identification Department

[0116] 34 Control Department

[0117] 40 Filming Department

[0118] 50 boxes

[0119] F1, F2 fingers

[0120] F1A, F2A, F2B corrugation

[0121] AR robotic arm

[0122] H Robot

[0123] W workpiece.

Claims

1. A robotic system comprising: A robotic mechanism having multiple fingers for grasping a workpiece; a detection unit configured to detect a gripping state of the workpiece by the plurality of fingers; an identification section that identifies the number of workpieces gripped by the plurality of fingers based on the detection result of the detection section; and The control unit controls the movement of the robot mechanism so that the number of the grasped workpieces reaches the specified number when the number of the workpieces identified by the identification unit is different from a predetermined number after the robot mechanism grasps the workpieces from a loading location where a plurality of the workpieces are loaded. At least one of the plurality of fingers is flexible, In a state where the workpiece is gripped by the plurality of fingers, the control unit controls the movement of the robot mechanism so as to change the position of at least one finger. Until the number of the workpieces recognized by the recognition unit reaches the specified number, the control unit controls the movement of the robot mechanism in a manner that repeatedly causes at least one of the plurality of fingers to move in at least one of a first direction as a movement direction for grasping, a second direction intersecting the first direction and the longitudinal direction of the fingers, a third direction as the longitudinal direction of the fingers, and a rotation direction with the third direction as a rotation axis, in a state in which the workpieces are grasped by the plurality of fingers. When at least one of the multiple fingers is moved along the first direction, the other fingers of the multiple fingers that are not at least one of the multiple fingers follow the movement of at least one of the multiple fingers and move in the first direction, and when at least one of the multiple fingers is moved along the second direction, the other fingers will not follow the movement of at least one of the multiple fingers and move in the second direction.

2. The robot system according to claim 1, wherein: At least one of the plurality of fingers is formed of an elastic member.

3. A robotic system comprising: A robotic mechanism having multiple fingers for grasping a workpiece; a detection unit configured to detect a gripping state of the workpiece by the plurality of fingers; an identification section that identifies the number of workpieces gripped by the plurality of fingers based on the detection result of the detection section; and The control unit controls the movement of the robot mechanism so that the number of the grasped workpieces reaches the specified number when the number of the workpieces identified by the identification unit is different from a predetermined number after the robot mechanism grasps the workpieces from a loading location where a plurality of the workpieces are loaded. The detection unit includes a pressure distribution sensor, which is provided on a gripping surface of at least one finger and detects a pressure distribution of the gripping surface contacted by the workpiece. The recognition unit recognizes the number of the workpieces using a grasping number recognition model, wherein the grasping number recognition model is a model obtained by learning using the pressure distribution detected by the detection unit as input and the number of the workpieces grasped by the plurality of fingers as output. The control unit controls the motion of the robot mechanism based on the driving method output from a driving method model, the driving method model being a model learned by taking the pressure distribution and the driving state of the driving unit that drives the plurality of fingers as input and taking the number of workpieces grasped by the plurality of fingers and the driving method of the driving unit for setting the number of workpieces to the specified number as output, Until the number of the workpieces recognized by the recognition unit reaches the specified number, the control unit controls the movement of the robot mechanism in a manner that repeatedly causes at least one of the plurality of fingers to move in at least one of a first direction as a movement direction for grasping, a second direction intersecting the first direction and the longitudinal direction of the fingers, a third direction as the longitudinal direction of the fingers, and a rotation direction with the third direction as a rotation axis, in a state in which the workpieces are grasped by the plurality of fingers. When at least one of the multiple fingers is moved along the first direction, the other fingers of the multiple fingers that are not at least one of the multiple fingers follow the movement of at least one of the multiple fingers and move in the first direction, and when at least one of the multiple fingers is moved along the second direction, the other fingers will not follow the movement of at least one of the multiple fingers and move in the second direction.

4. The robot system according to claim 3, wherein: An elastic component is provided on the pressure distribution sensor.

5. The robot system according to any one of claims 1 to 4, wherein: The detection unit detects the gripping state a plurality of times while the relative positional relationships of the plurality of fingers are different from each other. The recognition unit recognizes the number of the workpieces based on a plurality of results of detecting the gripping state.

6. A picking method, wherein: detecting a gripping state of a robot mechanism having a plurality of fingers for gripping a workpiece, wherein the plurality of fingers grip the workpiece; identifying the number of workpieces gripped by the plurality of fingers based on a result of detecting the gripping state of the workpiece, After causing the robot mechanism to perform an action of grasping the workpieces from a loading location where a plurality of the workpieces are loaded, if the number of the identified workpieces is different from a predetermined number, controlling the movement of the robot mechanism so that the number of the grasped workpieces reaches the predetermined number, At least one of the plurality of fingers is flexible, In a state where the workpiece is gripped by the plurality of fingers, the movement of the robot mechanism is controlled so that the position of at least one finger is changed. until the number of the identified workpieces reaches the specified number, the movement of the robot mechanism is controlled in a manner such that at least one of the plurality of fingers is repeatedly moved in at least one of a first direction as a movement direction for grasping, a second direction intersecting the first direction and the longitudinal direction of the fingers, a third direction as the longitudinal direction of the fingers, and a rotation direction with the third direction as a rotation axis, while the workpiece is grasped by the plurality of fingers. When at least one of the multiple fingers is moved along the first direction, the other fingers of the multiple fingers that are not at least one of the multiple fingers follow the movement of at least one of the multiple fingers and move in the first direction, and when at least one of the multiple fingers is moved along the second direction, the other fingers will not follow the movement of at least one of the multiple fingers and move in the second direction.

7. A picking method, wherein: detecting a gripping state of a robot mechanism having a plurality of fingers for gripping a workpiece, wherein the plurality of fingers grip the workpiece; identifying the number of workpieces gripped by the plurality of fingers based on a result of detecting the gripping state of the workpiece, After causing the robot mechanism to perform an action of grasping the workpieces from a loading location where a plurality of the workpieces are loaded, if the number of the identified workpieces is different from a predetermined number, controlling the movement of the robot mechanism so that the number of the grasped workpieces reaches the predetermined number, A pressure distribution sensor is provided on the gripping surface of at least one finger and detects the pressure distribution of the gripping surface contacted by the workpiece. The number of the workpieces is identified by a grasping number recognition model, wherein the grasping number recognition model is a model obtained by learning using the detected pressure distribution as input and the number of the workpieces grasped by the plurality of fingers as output, controlling the motion of the robot mechanism based on the driving method output from a driving method model, the driving method model being a model learned by taking the pressure distribution as input and taking the number of workpieces grasped by the plurality of fingers and the driving method of at least one of the plurality of fingers for setting the number of workpieces to the specified number as output, until the number of the identified workpieces reaches the specified number, the movement of the robot mechanism is controlled in a manner such that at least one of the plurality of fingers is repeatedly moved in at least one of a first direction as a movement direction for grasping, a second direction intersecting the first direction and the longitudinal direction of the fingers, a third direction as the longitudinal direction of the fingers, and a rotation direction with the third direction as a rotation axis, while the workpiece is grasped by the plurality of fingers. When at least one of the multiple fingers is moved along the first direction, the other fingers of the multiple fingers that are not at least one of the multiple fingers follow the movement of at least one of the multiple fingers and move in the first direction, and when at least one of the multiple fingers is moved along the second direction, the other fingers will not follow the movement of at least one of the multiple fingers and move in the second direction.

Images