Method for estimating the number of work-pieces, component supply system, and robot system
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- SEIKO EPSON CORP
- Filing Date
- 2023-08-04
- Publication Date
- 2026-06-24
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a method for estimating a quantity of workpieces, a part supply system, and a robot system. [Background technology]
[0002] A parts supply system that supplies the workpieces is used for the workpiece picking operation by the robot. Patent Document 1 discloses a parts supply system that estimates the remaining amount of parts. In this conventional technology, the number of works stored in the recessed portion of the tray is detected, and the remaining amount of works in the tray is estimated based on the detection result. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2013-249140 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, the above-mentioned conventional technology has a problem in that the remaining amount of work cannot be counted unless the number of works is reduced to the same as the number of recesses. [Means for solving the problem]
[0005] According to a first aspect of the present disclosure, there is provided a method for estimating the quantity of a plurality of workpieces contained in a source tray, the method including: (a) determining a component remaining quantity indicator indicating at least one of a volume and a height of the plurality of workpieces using a result of depth detection in the source tray by a depth detector; and (b) estimating the quantity of the plurality of workpieces using the component remaining quantity indicator.
[0006] According to a second aspect of the present disclosure, there is provided a component supply system. The component supply system includes a source tray that accommodates a plurality of workpieces, a depth detector that photographs the source tray to detect the depth, and a quantity estimation unit that estimates the quantity of the plurality of workpieces. The quantity estimation unit is configured to execute (a) a process of obtaining a component remaining quantity index that indicates at least one of the volume and height of the plurality of workpieces by using a result of the depth detection in the source tray by the depth detector, and (b) a process of estimating the quantity of the plurality of workpieces by using the component remaining quantity index.
[0007] According to a third aspect of the present disclosure, there is provided a robot system including the component supply system and a robot that performs a picking operation of the workpiece contained in the supply source tray. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is an explanatory diagram showing a configuration of a robot system according to an embodiment. [Diagram 2] FIG. 2 is a functional block diagram of an information processing apparatus according to an embodiment. [Diagram 3] FIG. 4 is an explanatory diagram showing functions of an object detection unit, a part remaining amount indicator estimation unit, and a part quantity estimation unit. [Figure 4] 11 is a flowchart showing a procedure for advance preparation of a picking operation. [Diagram 5] 5 is a flowchart showing a processing procedure for a picking operation in the first embodiment. [Figure 6] FIG. 13 is an explanatory diagram showing an example of a window that notifies a remaining quantity of parts and a remaining operation time. [Figure 7] FIG. 11 is an explanatory diagram showing an example of a plurality of types of workpieces used in the second embodiment. [Figure 8] 10 is a flowchart showing a processing procedure for a picking operation in a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] A. First embodiment: 1 is an explanatory diagram showing an example of a robot system according to an embodiment. The robot system includes a robot 100, a control device 200 that controls the robot 100, an information processing device 300, a depth detector 400, a supply source tray 510, a supply destination tray 520, and a hopper 530. The information processing device 300 is, for example, a personal computer. The information processing device 300, the depth detector 400, the supply source tray 510, and the hopper 530 function as a part supply system.
[0010] The robot 100 includes a base 110 and a robot arm 120. A robot hand 150 serving as an end effector is attached to an arm end 122, which is the tip of the robot arm 120. The robot hand 150 can be realized as a gripper or a suction pad capable of gripping a workpiece WK. A TCP (Tool Center Point) serving as a control point of the robot 100 is set at the tip of the robot hand 150. The control point TCP can be set at any position.
[0011] The robot arm 120 is connected in sequence by six joints J1 to J6. Of these joints J1 to J6, three joints J2, J3, and J5 are bending joints, and the other three joints J1, J4, and J6 are torsion joints. In this embodiment, a six-axis robot is exemplified, but a robot having any robot arm mechanism having a plurality of joints can be used. In addition, the robot 100 in this embodiment is a vertical multi-joint robot, but a horizontal multi-joint robot may also be used.
[0012] The robot 100 performs a picking operation of taking out one workpiece WK from the supply source tray 510 and placing it on the supply destination tray 520. The supply source tray 510 accommodates a plurality of workpieces WK in a bulk state. The supply destination tray 520 is used as a place to place the workpiece WK taken out from the supply source tray 510. The workpiece WK is placed in a predetermined position in the supply destination tray 520 in a predetermined posture. In order to perform this picking operation accurately, the position and posture of the workpiece WK are recognized. The workpiece WK is also called a "part" or an "object". In the first embodiment, the robot 100 performs a picking operation for a plurality of workpieces WK of the same specifications. The hopper 530 is a replenishing device that replenishing the workpiece WK in the supply source tray 510. However, the replenishing device such as the hopper 530 may be omitted and the user may replenishing the workpiece WK.
[0013] The depth detector 400 is installed above the supply source tray 510 and captures the workpiece WK in the supply source tray 510. In this embodiment, the depth detector 400 includes a still image camera 410 that captures a still image, and a depth sensor 420 that captures a depth image. As the depth detector 400, for example, an RGBD camera including an RGB camera and a D camera (depth camera) can be used. However, the still image camera 410 may capture a monochrome image instead of an RGB image. Also, the still image camera 410 may be omitted. It is preferable that the depth detector 400 has at least a function of detecting depth.
[0014] FIG. 1 shows a world coordinate system Σw, a robot coordinate system Σr, a camera coordinate system Σc, and a sensor coordinate system Σs. The X-axis and Y-axis of the world coordinate system Σw are horizontal axes, and the Z-axis is a vertical axis. For the other coordinate systems, the symbols distinguishing the three coordinate axes are omitted. The robot coordinate system Σr is an orthogonal coordinate system with a coordinate origin set to a predetermined position of the robot 100. The camera coordinate system Σc is an orthogonal coordinate system with a coordinate origin set to a predetermined position of the still image camera 410. The sensor coordinate system Σs is an orthogonal coordinate system with a coordinate origin set to a predetermined position of the depth sensor 420. The relative relationship between the sensor coordinate system Σs and the camera coordinate system Σc is known, and the depth within the field of view of the depth sensor 420 is associated with a pixel position within an image captured by the still image camera 410. Furthermore, the still image camera 410 is calibrated in advance, and the position and orientation in each of the coordinate systems Σw, Σr, and Σc can be converted between each other by coordinate transformation.
[0015] 2 is a block diagram showing functions of the information processing device 300. The information processing device 300 has a processor 310, a memory 320, and an interface circuit 330. An input device 340 and a display device 350 are connected to the interface circuit 330, and the control device 200 and the depth detector 400 are also connected to the interface circuit 330.
[0016] The processor 310 has functions as a quantity estimation unit 610 and a robot control execution unit 620. The quantity estimation unit 610 estimates the quantity of workpieces WK accommodated in the supply source tray 510, and also estimates the position of at least one workpiece WK, using the depth detection result of the depth detector 400. The quantity estimation unit 610 includes functions of an object detection unit 612, a part remaining amount index estimation unit 614, a part remaining amount estimation unit 616, and a position and orientation estimation unit 618.
[0017] The object detection unit 612 detects the workpieces WK accommodated in the supply source tray 510 using the detection result of the depth detector 400. The component remaining amount index estimation unit 614 obtains a component remaining amount index indicating at least one of the volume and height of the multiple workpieces WK in the supply source tray 510 using the depth detection result of the depth detector 400. The component remaining amount estimation unit 616 estimates the quantity of the workpieces WK in the supply source tray 510 using the component remaining amount index. The functions of the object detection unit 612, the component remaining amount index estimation unit 614, and the component remaining amount estimation unit 616 will be described further below.
[0018] The position and orientation estimation unit 618 estimates the position and orientation (rotation angle) of the workpiece WK that is the target of the picking operation by the robot 100. The position and orientation are also called "position and orientation." In this embodiment, the position and orientation estimation unit 618 estimates the position and orientation of the workpiece WK using depth information measured by the depth sensor 420. The position and orientation estimation unit 618 can be configured using, for example, a position and orientation estimation algorithm using deep learning. Note that, depending on the shape of the workpiece WK and the picking method, there are cases where it is not necessary to estimate the orientation of the workpiece WK. In that case, the position and orientation estimation unit 618 estimates only the position of the workpiece WK.
[0019] In the position and orientation estimation process, it is desirable to use an algorithm that uses image information obtained by the still image camera 410 in addition to depth information, as this increases the estimation accuracy. As such a position and orientation estimation algorithm, for example, PVN3D (A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation, https: / / doi.org / 10.48550 / arXiv.1911.04231) can be used. This position and orientation estimation algorithm estimates the positions of keypoints, which are important feature points of an object, and can obtain the position and orientation of the object from the positions.
[0020] The robot control execution unit 620 executes the picking operation of the workpiece WK by using the estimated position and orientation of the workpiece WK. The functions of the quantity estimation unit 610 and the robot control execution unit 620 are respectively realized by the processor 310 executing a computer program stored in the memory 320. However, some or all of the functions of the quantity estimation unit 610 and the robot control execution unit 620 may be realized by a hardware circuit.
[0021] The memory 320 stores CAD data CD representing the outer shape of the workpiece WK, a template TP used for object detection and estimation of the position and orientation of the workpiece WK, and a robot control program RP. The robot control program RP is composed of a number of commands for operating the robot 100.
[0022] 3 is an explanatory diagram showing the functions of the object detection unit 612, the part remaining amount index estimation unit 614, and the part remaining amount estimation unit 616. The depth detector 400 creates a detection result image IM including a first image M1 and a second image M2. The first image M1 obtained by the still image camera 410 is an image representing the RGB values of each pixel having pixel coordinates (x, y). The second image M2 obtained by the depth sensor 420 is an image representing the depth information of each pixel having pixel coordinates (x, y). The second image M2 is also called a "depth image."
[0023] The object detection unit 612 uses the first image M1 obtained by the still image camera 410 to generate a bounding box BB surrounding a detection target object existing in the field of view, and identifies the type of each object. The detection target object is a workpiece WK. The object detection unit 612 can be configured as an object detection algorithm using deep learning, such as R-CNN (Region Based Convolutional Neural Networks), SSD (Single Shot multibox Detector), YOLO (You Look Only Ones), etc. The learning of the object detection unit 612 may be performed within the information processing device 300, or may be performed in advance in another environment. Note that, as the object detection process, other processes such as instance segmentation process may be used instead of the process using the bounding box BB. The object detection unit 612 may perform object detection using the second image M2 obtained by the depth sensor 420 instead of the first image M1 obtained by the still image camera 410.
[0024] The part remaining quantity index estimating section 614 executes the following process to obtain the part remaining quantity index Vwk. (a1) A two-dimensional area obtained by taking the logical sum of all of the multiple bounding boxes BB detected by the object detection unit 612 is obtained as the part group area Swk. At this time, a process of expanding the area may be performed so that there are no gaps or holes. (a2) From the depth information of the second image M2 obtained by the depth sensor 420, the distance Dh(x, y) from the depth sensor 420 is calculated for each pixel. (a3) The distance D0(x, y) of each pixel when the supply source tray 510 is photographed without any parts such as the workpiece WK being present is obtained in advance, and the actual height H(x, y) = Dh(x, y) - D0(x, y) of each pixel is obtained by taking the difference between the two distances Dh(x, y) and D0(x, y). (a4) The sum of the effective heights H(x, y) of the pixels included in the component group area Swk is obtained as the component remaining amount index Vwk. In this example, the part remaining amount index Vwk is an index that indicates an estimated amount of the total volume of a plurality of workpieces WK.
[0025] The above process (a1) may be omitted. In this case, an area of the screen of the depth sensor 420 that corresponds to the supply source tray 510 is set in advance as a "supply source area", and in the process (a4), the sum of the effective heights H(x, y) of the pixels included in the supply source area can be obtained as the component remaining amount index Vwk.
[0026] The component remaining quantity estimation unit 616 estimates the quantity Nres of the multiple workpieces WK accommodated in the supply source tray 510 using the component remaining quantity index Vwk. This quantity Nres can be calculated by dividing the component remaining quantity index Vwk by the workpiece volume ΔV per one workpiece WK. The workpiece volume ΔV can be calculated, for example, from the amount of decrease in the component remaining quantity index Vwk due to one picking operation. Alternatively, the workpiece volume ΔV may be calculated from the average value of the amount of decrease in the component remaining quantity index Vwk in multiple picking operations. The workpiece volume ΔV may also be calculated using CAD data of the workpieces WK.
[0027] As the part remaining amount index Vwk, any of the following indexes may be used instead of the index indicating the estimated amount of the total volume of the multiple workpieces WK. (a) An indicator showing the distance from the surface of the source tray 510 to the highest point of multiple workpieces WK contained in the source tray 510. (b) An indicator showing the distance from the depth detector 400 to the highest point of the multiple workpieces WK contained in the supply source tray 510. In these examples, the component remaining amount index Vwk is an index indicating the height of the multiple workpieces WK. When using such a component remaining amount index Vwk, the component remaining amount estimation unit 616 can estimate the quantity Nres by using a table indicating the relationship between the component remaining amount index Vwk and the quantity Nres of the multiple workpieces WK.
[0028] The component remaining amount index Vwk may include an index indicating the volume of the multiple workpieces WK and an index indicating the height of the multiple workpieces WK. In other words, the component remaining amount index Vwk may be an index indicating at least one of the volume and the height of the multiple workpieces WK.
[0029] 4 is a flowchart showing the procedure for advance preparation of a picking operation. In step S11, detection is performed by the depth detector 400 when there are no parts such as workpieces WK in the supply source tray 510. In step S12, the part remaining amount index estimation unit 614 uses the depth information obtained by the depth detector 400 to measure the distance D0(x, y) from the depth detector 400 for each pixel. Data indicating the distance D0(x, y) is stored in the memory 320.
[0030] FIG. 5 is a flowchart showing the processing procedure of the picking operation in the first embodiment. In step S21, the depth detector 400 performs detection in a state where a plurality of workpieces WK are randomly piled up in the supply source tray 510. In step S22, the object detector 612 performs part detection processing using the detection result of the depth detector 400. As described in FIG. 3, in this embodiment, the object detector 612 uses the first image M1 obtained by the still image camera 410 to generate a bounding box BB surrounding each workpiece WK present in its field of view, and identifies the type of each workpiece WK. However, in the first embodiment, since only one type of workpiece WK is stored in the supply source tray 510, the identification of the type can be omitted.
[0031] In step S23, the part remaining quantity estimation unit 614 executes a part remaining quantity estimation process using the detection result of the object detection unit 612. As described in FIG. 3, in this embodiment, the part remaining quantity estimation unit 614 obtains the logical sum of a plurality of bounding boxes BB as the part group area Swk, and calculates the sum of the real heights H(x, y) of the pixels included in the part group area Swk as the part remaining quantity index Vwk. In step S24, the part remaining quantity estimation unit 616 executes a part remaining quantity estimation process. As described in FIG. 3, in this embodiment, the part remaining quantity estimation unit 616 calculates the quantity Nres of the workpieces WK by dividing the part remaining quantity index Vwk by the workpiece volume ΔV. When the routine of steps S21 to S29 in FIG. 5 is executed for the first time, it is preferable to use a preset initial value as the workpiece volume ΔV.
[0032] In step S25, quantity estimation unit 610 displays the remaining part quantity and remaining operation time on display device 350 to notify the user.
[0033] FIG. 6 is an explanatory diagram showing an example of a window W1 that notifies the remaining amount of parts and the remaining operation time. The remaining amount of parts is the quantity Nres of workpieces WK calculated in step S24. The remaining operation time is the time required to complete the pick-up operation of the workpieces WK contained in the supply source tray 510. The remaining operation time is calculated by multiplying the quantity Nres of workpieces WK by the operation time required for one pick-up operation. When the routine of steps S21 to S29 in FIG. 5 is executed for the first time, it is preferable to use a preset initial value as the operation time required for one pick-up operation. Note that step S25 may be omitted.
[0034] When the remaining amount of parts is equal to or less than a preset notification threshold, it is preferable to notify the user that the remaining amount of workpieces is low. In this way, the user can easily recognize that the remaining amount of workpieces is low. In addition, it is preferable that the part supply system is configured so that a replenishing device such as a hopper 530 automatically replenishes the workpieces WK to the supply source tray 510 when the remaining amount of parts is equal to or less than a preset replenishment threshold. In this way, a sufficient amount of workpieces WK can be maintained in the supply source tray 510.
[0035] In step S26, the position and orientation estimation unit 618 executes a position and orientation estimation process for the workpiece WK using the detection result of the depth detector 400. In this orientation estimation process, the position and orientation of at least one workpiece WK among the multiple workpieces WK is estimated. As described above, only the position may be estimated without estimating the orientation. In step S27, the quantity estimation unit 610 selects one workpiece WK to be picked. This selection is performed, for example, by selecting the workpiece WK whose position and orientation estimated in step S26 is most suitable for the picking operation from among the multiple workpieces WK stored in the supply source tray 510.
[0036] In step S28, the robot control execution unit 620 executes the picking operation. At this time, the operation time required for one picking operation is recorded. This operation time is used to calculate the remaining operation time in step S25 when the routine of steps S21 to S29 in FIG. 5 is executed for the second or subsequent times.
[0037] In step S28, the quantity estimation unit 610 determines whether the picking work is completed. If the picking work is completed, the process ends as shown in FIG. 5. On the other hand, if the picking work is not completed, the process returns to step S21, and the process from step S21 onward is executed again.
[0038] When the routine of steps S21 to S29 in Fig. 5 is executed for the second time, a decrease ΔVwk in the component remaining amount index Vwk between the previous time and the current time is calculated in step S23. This decrease ΔVwk can be used as the work volume ΔV per work WK in the component remaining amount estimation process in step S24. Furthermore, when the routine of steps S21 to S29 in Fig. 5 is executed for the third time or later, an average value of the decrease ΔVwk in the component remaining amount index Vwk over multiple picking operations may be used as the work volume ΔV.
[0039] As described above, in the first embodiment, the result of depth detection in the supply source tray by the depth detector 400 is used to determine a component remaining amount index Vwk indicating at least one of the volume and height of the multiple workpieces WK, and this component remaining amount index Vwk is used to estimate the quantity Nres of the multiple workpieces WK. Therefore, the quantity Nres of the multiple workpieces WK in the supply source tray 510 can be easily estimated. In addition, the result of depth detection by the depth detector 400 can be used to both estimate the quantity of the workpieces WK and the position of the workpieces WK.
[0040] In the first embodiment, a remaining quantity indicator of the part is obtained and a remaining quantity estimation process is performed in S23 and S24, and then the position and orientation of the workpiece is estimated, a picking target is selected, and the picking work is performed in S26 to S28. However, a remaining quantity indicator of the part may be obtained and a remaining quantity estimation process may be performed in S23 and S24 after the position and orientation of the workpiece is estimated, a picking target is selected, and the picking work is performed in S26 to S28.
[0041] B. Second embodiment: 7 is an explanatory diagram showing an example of multiple types of workpieces used in the second embodiment. In the second embodiment, multiple types of workpieces WK1, WK2, and WK3 having different sizes are used. These workpieces WK1, WK2, and WK3 are stored in separate supply source trays 510_1, 510_2, and 510_3, respectively, and are arranged on one supply destination tray 520 by a picking operation. The device configuration of the second embodiment is the same as that of the first embodiment.
[0042] Fig. 8 is a flowchart showing the processing procedure of the picking work in the second embodiment. The second embodiment differs from the first embodiment in that step S100 is inserted between step S21 and step S22 of the processing procedure shown in Fig. 5, and step S110 is inserted between step S28 and step S29, but the other steps are almost the same as those of the first embodiment.
[0043] Steps S100 and S110 mean that steps S23 to S28 sandwiched between them are executed for each type of workpiece. The loop of steps S23 to S28 is called a "part type loop."
[0044] The processing of steps S21 and S22 is the same as that of the first embodiment. In the second embodiment, steps S23 to S28 are executed for each type of workpiece. That is, the part remaining amount index Vwk is calculated for each type of workpiece, and the remaining amount of the workpiece and the remaining operation time are also calculated for each type of workpiece. After that, the position and orientation of the type of workpiece to be processed are estimated, and the picking operation is executed. In the example of FIG. 7, in the part type loop between steps S100 and S110, steps S23 to S28 are executed once for each of the first workpiece WK1 and the second workpiece WK2, and steps S23 to S28 are executed three times for the third workpiece WK3. In this case, if a process of erasing the bounding box BB of one workpiece WK3 is executed when picking of the workpiece WK3 is completed, it is possible to execute multiple picking operations in succession without performing detection by the depth detector 400 every time picking is executed.
[0045] It is preferable to notify the user of the remaining amount of parts for each type of workpiece. It is also preferable to notify the user of the remaining operation time taking into account all of the multiple types of workpieces. Furthermore, it is preferable to notify the user of the replenishment timing separately for each type of workpiece.
[0046] As described above, in the second embodiment, even in a task in which a variety of workpieces are the subject of picking work, it is possible to estimate the quantity of workpieces and the position of the workpieces using the depth detection results of the depth detector 400.
[0047] Other forms: The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various forms without departing from the spirit of the present disclosure. For example, the present disclosure can be realized in the following aspects. The technical features in the above-mentioned embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined in order to solve some or all of the problems of the present disclosure, or to achieve some or all of the effects of the present disclosure. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted.
[0048] (1) According to a first aspect of the present disclosure, there is provided a method for estimating the quantity of a plurality of workpieces contained in a source tray, the method including: (a) determining a component remaining quantity indicator indicating at least one of a volume and a height of the plurality of workpieces using a result of depth detection in the source tray by a depth detector; and (b) estimating the quantity of the plurality of workpieces using the component remaining quantity indicator. According to this method, the quantity of multiple workpieces in the source tray can be easily estimated.
[0049] (2) The above method may further include a step of estimating a position of at least one of the plurality of workpieces using the depth detection result, and a step of performing a picking operation using the estimated position of the workpiece. According to this method, the depth detection result by the depth detector can be used to both estimate the number of workpieces and the position of the workpieces.
[0050] (3) The above method may further include a step of notifying that the remaining amount of workpieces is low when the quantity is equal to or less than a notification threshold. This method allows the user to easily recognize when the remaining amount of work is low.
[0051] (4) The above method may further include a step of a replenishment device replenishing the workpieces to the supply source tray when the quantity is equal to or less than a replenishment threshold. This method allows a sufficient number of workpieces to be kept stored in the source tray.
[0052] (5) In the above method, the component remaining amount indicator may be an estimate of a total volume of the plurality of workpieces. According to this method, the quantity of multiple workpieces can be easily estimated from the estimated total volume of the multiple workpieces.
[0053] (6) In the above method, the step (b) may include a step of estimating the quantity of the plurality of workpieces by dividing the total volume of the plurality of workpieces by a workpiece volume per workpiece. This method makes it easy to estimate the quantity of multiple workpieces.
[0054] (7) In the above method, the step (b) may include a step of calculating the work volume from an amount of reduction in the total volume of the plurality of workpieces due to one picking operation. This method allows the workpiece volume per workpiece to be accurately estimated.
[0055] (8) In the above method, the step (b) may include a step of calculating the work volume from an average value of a reduction amount of the total volume of the plurality of workpieces in a plurality of picking operations. This method allows the workpiece volume per workpiece to be accurately estimated.
[0056] (9) The above method may further include a step of calculating a remaining time by multiplying the quantity of the plurality of workpieces by a picking operation time per workpiece, and notifying the remaining time. According to this method, the time when the picking operation of the workpieces stored in the supply source tray will be completed can be notified to the user.
[0057] (10) In the above method, the component remaining amount indicator may be a distance from a surface of the supply source tray to a highest point of the plurality of workpieces. According to this method, the quantity of multiple workpieces can be easily estimated using the distance from the surface of the source tray to the highest point of the workpiece.
[0058] (11) In the above method, the part remaining amount indicator may be a distance from the depth detector to a highest point of the plurality of workpieces. According to this method, the quantity of multiple workpieces can be easily estimated using the distance from the depth detector to the highest point of the workpiece.
[0059] (12) According to a second aspect of the present disclosure, there is provided a component supply system. The component supply system includes a source tray that accommodates a plurality of workpieces, a depth detector that photographs the source tray to detect the depth, and a quantity estimation unit that estimates the quantity of the plurality of workpieces. The quantity estimation unit is configured to execute: (a) a process of determining a component remaining quantity index that indicates at least one of the volume and height of the plurality of frames, using a result of the depth detection in the source tray by the depth detector; and (b) a process of estimating the quantity of the plurality of workpieces, using the component remaining quantity index.
[0060] (13) According to a third aspect of the present disclosure, there is provided a robot system including the component supply system and a robot that performs a picking operation of the workpiece contained in the supply source tray.
[0061] The present disclosure may be realized in various forms other than those described above, such as a robot system including a robot and a robot control device, a computer program for implementing the functions of the robot control device, a non-transitory storage medium on which the computer program is recorded, etc. [Explanation of symbols]
[0062] 100...robot, 110...base, 120...robot arm, 122...arm end, 150...robot hand, 200...control device, 300...information processing device, 310...processor, 320...memory, 330...interface circuit, 340...input device, 350...display device, 400...depth detector, 410...still image camera, 420...depth sensor, 500...work table, 510...supply source tray, 520...supply destination tray, 530...hopper, 610...quantity estimation unit, 612...object detection unit, 614...part remaining amount index estimation unit, 616...part remaining amount estimation unit, 618...position and orientation estimation unit, 620...robot control execution unit
Claims
1. A method for estimating a quantity of a plurality of workpieces contained in a supply source tray, comprising the steps of: (a) determining a component remaining quantity indicator indicating at least one of a volume and a height of the plurality of workpieces using a result of a depth detection in the supply source tray by a depth detector; (b) estimating a quantity of the plurality of workpieces using the part remaining quantity indicator; A method comprising:
2. 10. The method of claim 1 further comprising: estimating a position of at least one workpiece among the plurality of workpieces using the depth detection result; The method includes a step of causing a robot to perform a picking operation using information on the estimated position of the workpiece.
3. 10. The method of claim 1 further comprising: The method includes a step of notifying that a remaining amount of work is low when the quantity is equal to or less than a notification threshold.
4. 10. The method of claim 1 further comprising: a replenishment device replenishing the source tray with the workpiece if the quantity is less than or equal to a replenishment threshold.
5. 2. The method of claim 1 , The method of claim 1, wherein the part remaining indicator is an estimate of a total volume of the plurality of workpieces.
6. 6. The method of claim 5, The method, wherein step (b) includes estimating the quantity of the plurality of workpieces by dividing the total volume of the plurality of workpieces by a workpiece volume per workpiece.
7. 7. The method of claim 6, The method, wherein the step (b) includes a step of calculating the work volume from an amount of reduction in the total volume of the plurality of workpieces due to one picking operation.
8. 7. The method of claim 6, The method, wherein the step (b) includes a step of calculating the work volume from an average value of a reduction amount of the total volume of the plurality of workpieces in a plurality of picking operations.
9. 10. The method of claim 1 further comprising: a step of calculating a remaining time by multiplying the quantity of the plurality of workpieces by a picking operation time per workpiece, and notifying the remaining time.
10. 2. The method of claim 1 , The method of claim 1, wherein the component remaining indicator is a distance from a surface of the source tray to a highest point of the plurality of workpieces.
11. 2. The method of claim 1 , The method of claim 1, wherein the part remaining indicator is a distance from the depth detector to a highest point of the plurality of workpieces.
12. A parts supply system, comprising: A supply tray for accommodating a plurality of workpieces; a depth detector that photographs the source tray and detects its depth; A quantity estimation unit that estimates a quantity of the plurality of workpieces; Equipped with The quantity estimation unit is (a) determining a component remaining amount indicator indicating at least one of a volume and a height of the plurality of workpieces using a result of the depth detection in the supply source tray by the depth detector; (b) estimating the quantity of the plurality of workpieces using the part remaining quantity indicator; and The part supply system is configured to:
13. 1. A robot system comprising: A component supply system according to claim 12; A robot that performs a picking operation of the work contained in the supply source tray; A robot system comprising: