Picking system

Abstract

To provide a picking system which can move a robot arm in a horizontal direction and a vertical direction at high speed to cause the robot arm to approach a workpiece.SOLUTION: A picking system 1 of the invention serially picks up a plurality of workpieces W loaded in an irregular manner and includes: a two-dimensional camera 2 which captures images of the workpieces W from positions spaced apart from each other in a vertical direction; a robot arm 3 which may move in a multiaxis direction; a proximity sense sensor 5 which is equipped in the robot arm 3 to detect a distance from a predetermined reference surface to the workpiece W and an angle formed between the predetermined reference surface and the workpiece W; and a controller 7 which controls movement of the robot arm 3 in a horizontal direction based on the captured images of the two-dimensional camera 2 and controls movement of the robot arm 3 in the vertical direction based on a detection result of the proximity sense sensor 5.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a picking system that sequentially picks up a plurality of workpieces that are irregularly stacked. [Background technology]

[0002] In recent years, picking systems have become widely used in which a robot arm moves in multiple axial directions to sequentially pick up multiple irregularly loaded workpieces. In these picking systems, the horizontal and vertical movements of the robot arm are controlled based on the results of images of the multiple workpieces taken from above by a three-dimensional camera (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-130924 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional picking systems have a problem in that the vertical movement speed of the robot arm is slower than the horizontal movement speed. More specifically, the 3D camera used to control the movement of the robot arm has a characteristic in which errors in the depth direction are larger than errors in the vertical and horizontal directions. Therefore, when capturing images of a workpiece from above, careful control of the robot arm is required near the workpiece, taking into account the susceptibility to errors in the depth direction (thickness direction of the workpiece). In particular, when the workpiece is made of a fragile material, such as food, the robot arm must move slowly vertically near the workpiece to prevent damage to the workpiece at the tip of the robot arm.

[0005] The present invention was devised in light of the above circumstances, and its purpose is to provide a picking system that can move a robot arm at high speed both horizontally and vertically to approach a workpiece. [Means for solving the problem]

[0006] A picking system according to one aspect of the present invention is a picking system that sequentially picks up multiple workpieces that are irregularly loaded, and includes a two-dimensional camera that images the multiple workpieces from positions spaced apart in the vertical direction, a robot arm that can move in multiple axial directions, a proximity sensor that is attached to the robot arm and detects the distance to and angle between the multiple workpieces, and a control device that controls the horizontal movement of the robot arm based on the image captured by the two-dimensional camera, and controls the vertical movement of the robot arm based on the detection results of the proximity sensor.

[0007] In addition, in a picking system according to one aspect of the present invention, the control device extracts the contours of the plurality of workpieces from the image captured by the two-dimensional camera, detects the workpieces by detecting from the extracted contours those contours that match at least a portion of the planar shape of the workpieces that has been stored in advance, calculates a score for each of the detected workpieces based on the degree to which the contours of other workpieces are present inside the contour, and compares the calculated scores of the plurality of workpieces to select the topmost workpiece, detects the distance to and angle between the selected topmost workpiece and the topmost workpiece using the proximity sensor, and moves the robot arm to a horizontal position and a vertical position where the topmost workpiece can be picked up.

[0008] In addition, in a picking system according to one aspect of the present invention, the control device may further have a noise removal unit that removes noise other than the contour of the work from the image captured by the two-dimensional camera prior to the contour extraction unit extracting the contour.

[0009] In addition, the picking system according to one aspect of the present invention may further include a light irradiation device that irradiates light onto the plurality of workpieces when the two-dimensional camera images the plurality of workpieces.

[0010] In addition, in a picking system according to one aspect of the present invention, the light irradiation device may have a one-side light irradiation unit that irradiates light onto the plurality of works from one horizontal side across the two-dimensional camera, and a other-side light irradiation unit that irradiates light onto the plurality of works from the other horizontal side opposite the two-dimensional camera.

[0011] In addition, the picking system according to one aspect of the present invention may further include a suction device that is mounted on the robot arm and picks up the topmost workpiece by suction. [Effects of the Invention]

[0012] According to a picking system according to one aspect of the present invention, the robot arm can be moved at high speed both horizontally and vertically to approach the workpiece. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a side view schematically showing the configuration of a picking system 1 according to an embodiment of the present invention. [Figure 2] 1 is a plan view schematically showing the configuration of a picking system 1 according to an embodiment of the present invention. [Figure 3] 1 is a block diagram showing a functional configuration of a picking system 1 according to an embodiment of the present invention. [Figure 4] 10 is a flowchart showing the flow of a picking process performed by the control device 7. [Figure 5] 10 is a flowchart showing the process flow of the uppermost position workpiece selection process. DETAILED DESCRIPTION OF THE INVENTION

[0014] A picking system according to an embodiment of the present invention will be described below with reference to the drawings.

[0015] (Picking system configuration) First, the configuration of a picking system according to an embodiment of the present invention will be described. FIGS. 1 and 2 are diagrams schematically illustrating the configuration of a picking system 1 according to an embodiment of the present invention, with FIG. 1 being a side view and FIG. 2 being a plan view. FIG. 3 is a block diagram illustrating the functional configuration of the picking system 1. The picking system 1 sequentially picks up, i.e., picks up, a plurality of workpieces W that are irregularly stacked. This picking system 1 includes a two-dimensional camera 2, a robot arm 3, a light irradiation device 4, a proximity sensor 5, a suction device 6, and a control device 7. Note that in this embodiment, a case will be described in which rice crackers that are approximately circular in plan view are used as the workpieces W according to the present invention.

[0016] The two-dimensional camera 2 serves to capture images of multiple workpieces W from positions spaced apart in the vertical direction (Z-axis direction). This two-dimensional camera 2 is a camera that captures two-dimensional information in the vertical and horizontal directions. In this embodiment, a so-called WEB camera (a camera that can transfer captured images to a computer in real time and process them) is used as the two-dimensional camera 2. As shown in FIGS. 1 and 2, this two-dimensional camera 2 is installed above the multiple workpieces W at a height position that allows it to capture an image of the entire workpieces W, with its lens facing downward. This allows the two-dimensional camera 2 to capture XY plane images of the multiple workpieces W. As shown in FIG. 3, the capture timing of the two-dimensional camera 2 is controlled by a control device 7. Note that, in addition to a WEB camera, a conventionally known two-dimensional camera can also be used as appropriate.

[0017] The robot arm 3 moves and deforms in multiple axial directions to bring a suction device 6, which will be described later, closer to or further away from the plurality of workpieces W. As shown in FIGS. 1 and 2, the robot arm 3 is installed at a predetermined position spaced apart from the plurality of workpieces W in a plan view, and out of the imaging range of the two-dimensional camera 2. As shown in FIG. 3, the horizontal and vertical movements of the robot arm 3 are controlled by a control device 7.

[0018] The light irradiation device 4 illuminates the plurality of workpieces W with light to clarify the image captured by the two-dimensional camera 2. As shown in FIGS. 1 and 2 , the light irradiation device 4 includes front-side LED lighting 8 (corresponding to the “one-side light irradiation unit” according to the present invention) arranged in a V-shape in a plan view above the plurality of workpieces W, and rear-side LED lighting 9 (corresponding to the “other-side light irradiation unit” according to the present invention) arranged in a V-shape in a plan view. The front-side LED lighting 8 is arranged on the horizontal front side, opposite the robot arm 3, across the two-dimensional camera 2, and the rear-side LED lighting 9 is arranged on the horizontal rear side, on the same side as the robot arm 3, across the two-dimensional camera 2. In the light irradiation device 4 configured in this manner, the on / off of the front-side LED lighting 8 and the rear-side LED lighting 9 is individually controlled by the control device 7, as shown in FIG. 3 . The number and arrangement of the front-side LED lighting 8 and the rear-side LED lighting 9 are not limited to those in this embodiment and can be modified as appropriate. Furthermore, the light irradiation device 4 is not limited to LED lighting, and any conventionally known lighting fixture that emits visible light can be used as the light irradiation device 4.

[0019] The proximity sensor 5 serves to detect the distance from a predetermined reference plane (not shown) to the workpiece W and the angle formed by the workpiece W with respect to the reference plane in a non-contact manner in order to control the vertical movement of the robot arm 3. Although not shown in detail in the figure, this proximity sensor 5 is configured by arranging multiple infrared sensors on a substrate. The proximity sensor 5 configured in this manner is mounted at the tip of the robot arm 3 with the infrared sensors 10 facing downward. As shown in FIG. 3, the operation of the multiple infrared sensors 10 is controlled by a control device 7. Note that any conventionally known sensor (which broadly includes so-called proximity sensors) can be used as the proximity sensor 5.

[0020] The suction device 6 serves to pick up the desired workpiece W by sucking it. As shown in Fig. 1, the suction device 6 is provided at the tip of the robot arm 3 and is connected to a vacuum pump (not shown) and the like. As shown in Fig. 3, the operation or stop of the suction device 6 is controlled by a control device 7.

[0021] The control device 7 plays a role in controlling the operation of each part constituting the picking system 1. As shown in FIG. 3, the control device 7 is electrically connected to each part constituting the picking system 1 via a communication bus 11.

[0022] (Picking process procedure by control device and its effects) Next, the procedure for the picking process performed by the control device 7 according to an embodiment of the present invention and its effects will be described. FIG. 4 is a flowchart showing the procedure for the picking process performed by the control device 7. When the picking process for sequentially picking up a plurality of irregularly stacked workpieces W is started, the control device 7 first determines whether or not there are any workpieces W to be picked up, for example, by capturing an image using the two-dimensional camera 2 (step S1). As a result, if it is determined that there are no workpieces W to be picked up (step S1: No), the control device 7 ends the picking process. On the other hand, if it is determined that there are any workpieces W to be picked up (step S1: Yes), the control device 7 moves the robot arm 3 to a predetermined initial position (step S2).

[0023] Next, the control device 7 executes a top-position workpiece sorting process to select the workpiece W located at the top from among the multiple workpieces W (step S3). FIG. 5 is a flowchart showing the process flow of the top-position workpiece sorting process. In the top-position workpiece sorting process, the control device 7 first turns on the front-side LED lighting 8 constituting the light irradiation device 4 (step S4). As a result, LED light is irradiated from the front side of each workpiece W, casting a shadow that extends rearward. In this state, the control device 7 controls the two-dimensional camera 2 to capture images of the XY plane of the multiple workpieces W, thereby acquiring a first planar image (step S5). Here, because each workpiece W casts a shadow on its rear side as described above, the contours of the rear side of each workpiece W are more clearly defined than the contours of the front side in the first planar image. Thereafter, the control device 7 turns off the front-side LED lighting 8 (step S6).

[0024] Next, the control device 7 turns on the rear LED lighting 9 that constitutes the light irradiation device 4 (step S7). As a result, LED light is irradiated from the rear side of each workpiece W, forming a shadow that extends forward. In this state, the control device 7 controls the two-dimensional camera 2 to capture images of the XY plane of the multiple workpieces W, thereby acquiring a second planar image (step S8). Here, because a shadow is formed on the front side of each workpiece W as described above, the outline of the front side of each workpiece W is clearer than the outline of the rear side in the second planar image. Thereafter, the control device 7 turns off the rear LED lighting 9 (step S9).

[0025] Next, the control device 7 extracts only information related to the contours of each workpiece W by removing noise information caused by surface wrinkles and the like from the first and second planar images (step S10). The control device 7 then generates a composite planar image by combining the first and second planar images from which only the contour information has been extracted (step S11). In this composite planar image, both the contours of the front and rear sides of each workpiece W are clearly visible.

[0026] Next, the control device 7 compares the contour information in the composite planar image with the pre-stored planar view shapes of the workpieces W, which in this embodiment are circular shapes in plan view. As a result, the control device 7 detects contour information of a shape that matches at least a part of the circular shape in plan view, thereby detecting the location of each workpiece W (step S12).

[0027] Next, the control device 7 calculates, for each detected work W, the degree to which the contours of other work W are present inside the contour of that work W as a score based on the location of each detected work W (step S13). More specifically, for each detected work W, the control device 7 calculates, as a score, the proportion of the number of pixels in the contours of other work W to the total number of pixels present inside the contour of that work W. In this embodiment, the shape of the work W in a plan view is a circle of a fixed size, and the total number of pixels is also fixed, so the calculated score is proportional to the number of pixels in the contours of other work W.

[0028] Next, the control device 7 compares the scores of each workpiece W to select the topmost workpiece Wt (see FIG. 1) that is located at the top in the vertical direction among the multiple workpieces W (step S14). More specifically, the control device 7 selects the workpiece W with the smallest score, i.e., the workpiece W with the fewest outlines of other workpieces W inside its outline, as the topmost workpiece Wt. This completes the topmost workpiece selection process.

[0029] Next, the control device 7 moves the robot arm 3 to a horizontal position where the selected uppermost workpiece Wt can be picked up (step S15), as shown in Fig. 4. More specifically, the control device 7 moves the robot arm 3 horizontally to a position spaced a predetermined distance substantially directly above the uppermost workpiece Wt.

[0030] Next, the control device 7 controls the proximity sensor 5 to irradiate the uppermost workpiece Wt with infrared light from the infrared sensor 10, thereby detecting the distance from a predetermined reference plane to the uppermost workpiece Wt and the angle formed by the uppermost workpiece Wt with respect to the reference plane (step S16). Then, based on the detection results, the control device 7 moves the robot arm 3 to a vertical position where the uppermost workpiece Wt can be picked up (step S17).

[0031] Here, the control for lowering the robot arm 3 to the closest position to the workpiece W is performed as follows. That is, when the robot arm 3 is made to approach the workpiece W moving at a constant velocity, the optimal approach trajectory without waste is calculated by the acceleration a calculated by the following formula 1. m It is known that the value of "t" in Equation 1 can be calculated by numerical integration. f " denotes the target convergence time, and "t" denotes the time elapsed since the start of control. Also, "χ" denotes the target position deviation calculated by the following formula 2. Here, "χ" in formula 2 p " is the height position of the surface on which the workpiece W is placed, and "χ m" and " refer to the hand position of the robot arm 3. However, in this conventional method, the target position deviation χ needs to be a continuous value, so it is assumed that a sensor capable of continuously measuring a wide range of within a few meters, such as a 3D camera or a camera capable of stereo vision, is used. Therefore, there was an issue that this conventional method cannot be applied when using a sensor that measures short distances of within a few centimeters, such as a proximity sensor.

[0032]

number

number

[0033] The inventors of the present application have determined that the gain G d This gain G d It has been found that by calculating the target position deviation using the following formula 4 with the addition of the formula 1, the optimal approach trajectory can also be obtained for a robot arm 3 equipped with a proximity sensor 5 by applying the formula 1 above. The control device 7 controls the robot arm 3 to descend based on this optimal approach trajectory. Note that "d" in formula 3 represents the distance from the predetermined reference plane detected in step S16 to the uppermost workpiece Wt, and "d max ” is the maximum distance, and “d min " and "θ2" in Equation 3 represent the angle formed by the uppermost workpiece Wt with respect to the reference plane detected in step S16, respectively. max " is the maximum angle, "θ min " means the minimum angle value.

[0034]

number

number

[0035] Next, as shown in Fig. 4, the control device 7 operates the suction device 6 to suck the uppermost workpiece Wt, thereby picking up the uppermost workpiece Wt (step S18). Then, the control device 7 moves the robot arm 3 horizontally and vertically to move the robot arm 3 above a predetermined collection position (not shown) (step S19). Then, the control device 7 stops the suction device 6 and releases the suction of the uppermost workpiece Wt, thereby dropping off the uppermost workpiece Wt, i.e., releasing it and causing it to fall (step S20). As a result, the uppermost workpiece Wt is collected to the collection position.

[0036] Next, the control device 7 determines whether or not there is a workpiece W to be picked up, for example, by capturing an image with the two-dimensional camera 2 (step S21). As a result, if it is determined that there is no workpiece W to be picked up (step S21: No), the control device 7 moves the robot arm 3 to the initial position (step S22) and ends the picking process. On the other hand, if it is determined that there is still a workpiece W to be picked up (step S21: Yes), the control device 7 returns to step S2 and repeats the subsequent processes.

[0037] As described above, according to the picking process of this embodiment, the vertical movement of the robot arm 3 is controlled based on the detection results of the proximity sensor 5, which has small errors. Therefore, compared to when the control is based on images captured by a three-dimensional camera, which has large errors in the vertical direction (depth direction), the robot arm 3 can be moved vertically quickly and accurately even near the workpiece W. This effect is particularly beneficial when the workpiece W is made of a brittle material and is easily damaged by contact with the robot arm 3. Furthermore, the horizontal movement of the robot arm 3 is controlled based on the results of the topmost workpiece selection process for the images captured by the two-dimensional camera 2, so the robot arm 3 can be moved horizontally quickly and accurately at low cost.

[0038] (Variation) The technical scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention as defined by the claims. For example, the following modifications can be considered as embodiments of the present invention.

[0039] In this embodiment, as shown in Fig. 4, the robot arm 3 is controlled to first move horizontally and then move vertically. However, the horizontal and vertical movements of the robot arm 3 may be controlled in the reverse order to this embodiment, i.e., to move vertically first and then horizontally. Furthermore, the robot arm 3 may be controlled so that the horizontal and vertical movements are performed simultaneously and in a composite manner, rather than being performed separately and independently.

[0040] 5, in this embodiment, first, the first planar image is acquired with only the front LED lighting 8 turned on, and then the second planar image is acquired with only the rear LED lighting 9 turned on. However, the order may be reversed from this embodiment, where first, the first planar image is acquired with only the rear LED lighting 9 turned on, and then the second planar image is acquired with only the front LED lighting 8 turned on.

[0041] In this embodiment, a light irradiation device 4 is provided to irradiate light onto the workpiece W in order to clarify the contours of the workpiece W in the captured image. However, the light irradiation device 4 is not an essential component of the present invention, and the picking system 1 can be configured without providing the light irradiation device 4. Furthermore, in this embodiment, a composite planar image is generated by combining the first planar image and the second planar image in order to further clarify the contours of the workpiece W in the captured image. However, generating a composite planar image is not an essential step of the present invention. For example, it is also possible to detect the location of each workpiece W using a single captured image captured with the front LED lighting 8 and the rear LED lighting 9 simultaneously turned on. However, if light is irradiated from both the front and rear sides of the workpiece W simultaneously, the shadows of each workpiece W may disappear, making it difficult to clarify their contours. Therefore, generating a composite planar image as in this embodiment is preferable.

[0042] In this embodiment, a suction device 6 is provided at the tip of the robot arm 3 as a means for picking up the workpiece W. However, the means for picking up the workpiece W is not limited to this, and the robot arm 3 can be equipped with any conventionally known means, such as a suction pad capable of suctioning the workpiece W or a hand capable of gripping the workpiece W. [Industrial Applicability]

[0043] The picking system 1 according to the present invention can also store in advance a trained model that receives a planar image of multiple irregularly stacked workpieces W as input and selects and outputs the topmost workpiece Wt. [Explanation of symbols]

[0044] 1. Picking system 2. 2D camera 3. Robotic Arm 4 Light irradiation device 5 Proximity sensor 6 Suction device 7 Control Device 8 Front LED lighting (one side light irradiation part) 9 Rear LED lighting (other side light irradiation part) double work Wt Top position work

Claims

1. A picking system that sequentially picks up a plurality of irregularly loaded workpieces, A two-dimensional camera that captures images of the plurality of workpieces from positions spaced apart in the vertical direction; a robot arm that is movable in multiple axial directions; a proximity sensor provided on the robot arm for detecting the distance to and the angle between the robot arm and the plurality of workpieces; a control device that controls horizontal movement of the robot arm based on the image captured by the two-dimensional camera, and controls vertical movement of the robot arm based on the detection result of the proximity sensor; A picking system comprising:

2. The control device Extracting contours of the plurality of workpieces from the images captured by the two-dimensional camera; Detecting the workpiece by detecting, from the extracted contour, a contour that matches at least a part of a pre-stored planar view shape of the workpiece; For each of the detected workpieces, the degree to which the contour of another workpiece exists inside the contour of the detected workpiece is calculated as a score; By comparing the calculated scores of the plurality of workpieces, the topmost workpiece is selected; The distance to and angle between the selected topmost workpiece and the workpiece are detected by the proximity sensor. The robot arm is moved to a horizontal position and a vertical position where the uppermost workpiece can be picked up. The picking system according to claim 1 .

3. 3. The picking system according to claim 2, wherein the control device removes noise other than the contours of the workpieces from the image captured by the two-dimensional camera before extracting the contours of the plurality of workpieces.

4. The picking system according to claim 1 , further comprising a light irradiation device that irradiates light onto the plurality of workpieces when the two-dimensional camera images the plurality of workpieces.

5. The picking system described in claim 4, characterized in that the light irradiation device has a one-side light irradiation unit that irradiates light onto the multiple works from one horizontal side across the two-dimensional camera, and a other-side light irradiation unit that irradiates light onto the multiple works from the other horizontal side opposite the two-dimensional camera.

6. 2. The picking system according to claim 1, further comprising a suction device mounted on the robot arm for picking up the uppermost workpiece by suction.

Images