Method for detecting the 3D position of a workpiece, and robot

Abstract

This invention provides a method and robot for detecting the three-dimensional position of a workpiece using simple means. [Solution] The method for detecting the three-dimensional position of a workpiece includes the steps of: roughly detecting the horizontal position of a first vertex by scanning a displacement sensor located above the workpiece in the horizontal direction; detecting the horizontal position of a first workpiece located on the uppermost layer stacked in the region of the first corner on the upper surface of a pallet including the first vertex using a two-dimensional camera located above the workpiece; calculating the horizontal positions of one or more workpieces located on the uppermost layer stacked in multiple regions other than the region of the first corner on the upper surface of the pallet, based on the horizontal position of the first workpiece and the size of the workpiece; and scanning the displacement sensor to the horizontal position of each uppermost workpiece to measure the vertical distance from the displacement sensor to each uppermost workpiece.

Description

[Technical Field]

[0001] This disclosure relates to a method for detecting the three-dimensional position of a workpiece and to a robot. [Background technology]

[0002] Devices for detecting the three-dimensional position of an object are known. For example, the three-dimensional position recognition device for an object described in Patent Document 1 comprises a computer that generates multiple slit-shaped black and white patterns, a projector that projects the light of the black and white patterns generated by the computer onto the luggage to be recognized, and an imaging device that captures a two-dimensional image of the luggage illuminated with the black and white pattern light. This three-dimensional position recognition device stores the two-dimensional image captured by the imaging device in the computer for each projection of light with a changed black and white pattern, calculates the height information of the top surface of the luggage based on the multiple stored two-dimensional images, stores the grayscale image of the top surface of the luggage captured by the imaging device when the light of the all-white pattern is projected in the computer, calculates the number of luggage items loaded on the top surface and their two-dimensional arrangement information from the grayscale image, and calculates the three-dimensional position of the luggage on the top surface from this number, the two-dimensional arrangement information, and the height information. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2006-300929 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] The 3D position recognition device described in Patent Document 1 requires the projection of a black and white pattern of light and the generation of a 2D image for all target workpieces, resulting in a complex processing method.

[0005] The purpose of this disclosure is to provide a method for detecting the three-dimensional position of a workpiece and a robot capable of detecting the three-dimensional position of the workpiece by simple means. [Means for solving the problem]

[0006] This disclosure provides a method for detecting the three-dimensional positions of multiple workpieces arranged horizontally and stacked vertically on the upper surface of a pallet, wherein the upper surface of the pallet is a rectangle having a first side, a second side perpendicular to the first side, and a first vertex which is the intersection of the first side and the second side, and the method comprises the steps of: roughly detecting the horizontal position of the first vertex by scanning a displacement sensor located above the workpieces horizontally; detecting the horizontal position of a first workpiece on the uppermost layer stacked in a first corner region of the upper surface of the pallet including the first vertex using a two-dimensional camera located above the workpieces; calculating the horizontal positions of one or more workpieces on the uppermost layer stacked in multiple regions other than the first corner region of the upper surface of the pallet based on the horizontal position of the first workpieces and the size of the workpieces; and scanning the displacement sensor to the horizontal position of each uppermost workpiece to measure the vertical distance from the displacement sensor to each uppermost workpiece. [Effects of the Invention]

[0007] According to this disclosure, the three-dimensional position of a workpiece can be detected by simple means. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram shows an overview of the structure of the robot hand portion according to the embodiment. [Figure 2] This diagram shows the configuration related to the control of the robot according to the embodiment. [Figure 3] This is a view of the pallet PL and workpiece WK from the horizontal direction. [Figure 4] This is a view of the pallet PL and workpiece WK from the vertical direction. [Figure 5] This is a flowchart illustrating the operation procedure of the method for detecting the three-dimensional position of the workpiece WK in Embodiment 1. [Figure 6] This is a flowchart showing the detailed steps for Step S101. [Figure 7] This diagram shows the positional relationships of each point in the process of step S101. [Figure 8] This diagram shows the positional relationships of each point in the process of step S101. [Figure 9] This is a flowchart showing the detailed steps for step S102. [Figure 10] This diagram shows the positional relationships of each point in the process of step S102. [Figure 11] This is a diagram illustrating the process in step S104. [Figure 12] This is a flowchart illustrating the operation procedure for the method of detecting the three-dimensional position of the workpiece WK in Embodiment 2. [Figure 13] This is a flowchart illustrating the detailed steps of step S204. [Figure 14] This diagram shows the relative number of layers An of the workpiece WKn stacked on the top layer in region SPn. [Figure 15] This is a flowchart illustrating the operation procedure of the method for detecting the three-dimensional position of the workpiece WK according to Embodiment 3. [Figure 16] This diagram shows an example of the position of workpiece WK1 detected by displacement sensor 2. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the drawings.

[0010] Embodiment 1.

[0011] Figure 1 is a diagram illustrating the structure of the hand portion of the robot according to this embodiment. The robot is equipped with a hand 7. A displacement sensor 2 and a two-dimensional camera 3 are attached to the hand 7. A magnet is attached to the tip of the hand 7. The magnet allows the robot to grasp a workpiece. The displacement sensor 2 and the two-dimensional camera 3 are scannable in the horizontal (XY direction) and vertical (Z-axis direction) directions. The displacement sensor 2 and the two-dimensional camera 3 are separated by ΔX in the X-axis direction and ΔY in the Y-axis direction.

[0012] Figure 2 is a diagram showing the configuration related to the control of the robot according to the embodiment. The robot includes a displacement sensor 2, a two-dimensional camera 3, a control device 4, and a notification device 5.

[0013] Displacement sensor 2 measures the distance from displacement sensor 2 to an object (pallet, workpiece, floor, etc.). Displacement sensor 2 measures the distance from displacement sensor 3 to the object by irradiating the object with laser light and detecting the reflected light from the object.

[0014] The 2D camera 3 generates a 2D image.

[0015] The control device 4 can control the operation of the displacement sensor 2, the 2D camera 3, and the hand 7. The control device 4 can control the loading and unloading of workpieces onto the pallet. The control device 4 may also include a memory in which a program is stored and a processor that executes the program.

[0016] The notification device 5 consists of a display, a speaker, or a communication device. The notification device 5 can notify the administrator of processing errors during robot operation, etc.

[0017] Figure 3 shows the pallet PL and workpiece WK viewed from the horizontal direction. Figure 4 shows the pallet PL and workpiece WK viewed from the vertical direction.

[0018] The pallet PL is placed on the floor. The top surface of the pallet PL is roughly rectangular. Displacement sensor 2 and 2D sensor 3 are located above the workpiece WK.

[0019] Workpiece WK is a rectangular parallelepiped connecting plate that connects elevator rails. Workpiece WK has multiple holes for passing bolts and nuts through for connection to the elevator rails. Workpiece WK may have a rib in the center.

[0020] Multiple workpieces WL are laid out horizontally on the top surface of the pallet PL and stacked vertically. The top surface of the pallet PL has a first side SD1, a second side SD2 perpendicular to the first side SD1, a third side SD3 parallel to the first side SD1 and perpendicular to the second side SD2, and a fourth side SD4 parallel to the second side SD2 and perpendicular to the first side SD1. The top surface of the pallet PL has a first vertex E, which is the intersection of the first side SD1 and the second side SD2.

[0021] Workpieces WK are arranged in multiple regions SP1 to SP10 on the upper surface of pallet PL. Region SP1 is the region of the first corner with the first vertex E. The uppermost layers of workpieces stacked in regions SP1 to SP10 are denoted as WK1 to WK10. Due to shifting during the stacking of workpieces WK, when viewed from above, the edges of the lower layers of workpieces may be on the outermost side.

[0022] In this embodiment, the three-dimensional position of the workpiece WK can be detected even when the workpiece WK is stacked in a misaligned manner or when packaging materials or the like are attached to the workpiece WK.

[0023] Figure 5 is a flowchart showing the operation procedure of the method for detecting the three-dimensional position of the workpiece WK in Embodiment 1.

[0024] In step S101, the control device 4 scans the displacement sensor 2 horizontally to determine the horizontal position E(X) of the first vertex E on the upper surface of the pallet PL. E ,Y E ) is roughly detected.

[0025] FIG. 6 is a flowchart showing the detailed procedure of step S101. FIGS. 7 and 8 are diagrams showing the positional relationship of each point in the process of step S101. Although the installation position of the pallet PL on the floor is not strictly determined, the first vertex E is at the horizontal position (X OA , Y OA ) of the first point OA shown in FIG. 7 and the horizontal position (X OC , Y OC ) of the third point OC. It is assumed that the pallet PL is installed within a quadrilateral area having these as diagonal vertices. Also, it is assumed that the installed pallet PL does not overlap with any of the horizontal positions (X OA , Y OA ) of the first point OA, the horizontal position (X OB , Y OB ) of the second point OB, the horizontal position (X OC , Y OC ) of the third point OC, and the horizontal position (X OD , Y OD ) of the fourth point OD. Further, it is assumed that the long side of the workpiece WK1 straddles one side ((X OA , Y OA ), (X OC , Y OC )) of a quadrilateral having the horizontal positions (X OA , Y OA ) of the first point OA and the horizontal positions (X OA , Y OC ) of the third point OC as diagonal vertices.

[0026] In step S201, the control device 4 scans from the first point OA (X OA , Y OA , Z OA ) on the floor outside the first side SD1 of the upper surface of the pallet PL in the direction in which the first side SD1 exists to the inside of the pallet PL, thereby determining the horizontal position A (X A , Y AThe first edge point A is roughly detected. For example, the horizontal laser beam irradiation position of the displacement sensor 2 when the distance from the displacement sensor 2 changes, i.e., when something other than the floor is detected, can be taken as the horizontal position of the first edge point A. When the workpiece WK is stacked without shifting, the roughly detected first edge point A will precisely coincide with a point on the first side SD1. When the workpiece WK is stacked with a shift, or when packaging material is attached to the edge of the workpiece WK, the roughly detected first edge point A may be an approximation of a point on the first side SD1. Furthermore, the control device 4 controls the first point OA(X) of the displacement sensor 2. OA ,Y OA ,Z OA During scanning from the position (X), the distance to the workpiece WK1 is measured by the displacement sensor 2, and the minimum value of the measured distance is recorded. The recorded minimum value of the distance is used to determine the focal length of the 2D camera 3 when performing step S301 in Figure 9, which will be described later. Note that the horizontal position (X) of the first point OA of the displacement sensor 2 is OA ,Y OA If the horizontal scanning from the ) extends beyond the loading position of workpiece WK1, the distance to other workpieces will be measured, causing the 2D camera 3 to lose focus in step S301 described later. To avoid this, the control device 4 controls the displacement sensor 4 so as not to scan beyond the loading position of workpiece WK1.

[0027] In step S202, the control device 4 detects the displacement sensor 2 at a second point OB(X) on the floor outside the first side SD1 of the upper surface of the pallet PL. OB ,Y OB ,Z OB By scanning from ) in the direction where the first edge SD1 exists to the inside of the palette PL, the horizontal position B(X) of the second edge point B on the first edge SD1 can be determined. B ,Y BThe second point OB and the first point OA can be assumed to be separated by a distance L. For example, the horizontal position of the laser beam emitted by the displacement sensor 2 when the distance from the displacement sensor 2 changes, i.e., when something other than the floor is detected, can be taken as the horizontal position of the second edge point B. The roughly detected second edge point B precisely coincides with a point on the first side SD1 when the workpiece WK is stacked without shifting. The roughly detected second edge point B may be an approximation of a point on the first side SD1 when the workpiece WK is stacked with a shift, or when packaging material is attached to the edge of the workpiece WK.

[0028] In step S203, the control device 4 detects the displacement sensor 2 at a third point OC(X) on the floor outside the second side SD2 of the upper surface of the pallet PL. OC ,Y OC ,Z OC By scanning from ) in the direction where the second edge SD2 exists to the inside of the palette PL, the horizontal position C(X) of the third edge point C on the second edge SD2 can be determined. C ,Y C ) is detected. For example, the horizontal laser beam irradiation position of the displacement sensor 2 when the distance from the displacement sensor 2 changes, i.e., when something other than the floor is detected, can be taken as the horizontal position of the third edge point C. The roughly detected third edge point C precisely coincides with a point on the second side SD2 when the workpiece WK is stacked without shifting. The roughly detected third edge point C may be an approximation of a point on the second side SD2 when the workpiece WK is stacked with a shift, or when packaging material is attached to the edge of the workpiece WK.

[0029] In step S204, the control device 4 controls the displacement sensor 2 to a fourth point OD(X) on the floor outside the second side SD2 of the upper surface of the pallet PL. OD ,Y OD ,Z OD By scanning from ) in the direction where the second edge SD2 exists to the inside of the palette PL, the horizontal position D(X) of the fourth edge point D on the second edge SD2 can be determined. D ,Y D) is detected. The fourth point OD and the third point OC can be assumed to be separated by a distance L. For example, the horizontal position of the laser beam irradiation position of the displacement sensor 2 when the distance from the displacement sensor 2 changes, i.e., when something other than the floor is detected, can be taken as the horizontal position of the fourth edge point D. When the workpiece WK is stacked without shifting, the roughly detected fourth edge point D precisely coincides with a point on the second side SD2. When the workpiece WK is stacked with a shift, or when packaging material is attached to the edge of the workpiece WK, the roughly detected fourth edge point D may be an approximation of a point on the second side SD2.

[0030] In step S205, the control device determines the first edge point A(X) according to equations (1) to (5). A ,Y A ) and the second edge point B(X B ,Y B A first virtual line L1 passes through ) and the third edge point C(X C ,Y C ) and the fourth edge point D(X D ,Y D Calculate the angle φ that the second imaginary line L2 passing through () makes.

number

[0031] In step S206, if φ is within a predetermined range (greater than or equal to 90°-α and less than or equal to 90°+α), the process proceeds to step S207. If φ is outside the predetermined range (less than 90°-α or greater than 90°+α), the process proceeds to step S208. For example, α can be set to any value less than or equal to 10°.

[0032] In step S207, the control device 4 determines the intersection of the first virtual line L1 and the second virtual line L2 to the horizontal position E(X) of the first vertex E according to equations (3) and (4). E ,Y E It is calculated as follows:

[0033] In step S208, the control device 4 causes the notification device 5 to notify it that an error has been detected.

[0034] Let's continue the explanation by referring to Figure 5 again.

[0035] In step S102, the control device 4 uses the two-dimensional camera 3 to detect the horizontal position of the uppermost first workpiece WK1, which is stacked in the region SP1 of the first corner including the first vertex E on the upper surface of the pallet PL.

[0036] Figure 9 is a flowchart showing the detailed steps of step S102. Figure 10 is a diagram showing the positional relationships of each point in the process of step S102.

[0037] In step S301, the control device 4 controls the 2D camera 3 to the first vertex E(X E ,Y E The control device 4 moves the camera to a position where it can be photographed. The control device 4 keeps the focal length of the 2D camera 3 constant by using the minimum distance stored in step S201 and the Z coordinate of the first point OA shown in Figure 7. OA The focal length is set from there. The control device 4 causes the 2D camera 3 to perform imaging processing and capture the region including the first vertex E, thereby generating a 2D image IMG.

[0038] In steps S302 and S303, the control device 4 detects the horizontal position of vertex V of the first workpiece WK1 in the two-dimensional image IMG.

[0039] In step S302, the control device 4 determines the horizontal position P(X) of the first hole P located closest to the first vertex E1 in the 2D image IMG. p ,Y p ) is detected. For example, if the control device 4 is captured such that the first vertex E is located in the upper right corner of the two-dimensional image IMG, the control device 4 can detect the hole that is represented by the pixel closest to the pixel at the upper right corner of the two-dimensional image IMG as the first hole P.

[0040] In step S303, the control device 4 detects the horizontal position of the vertex V of the first workpiece WK1 based on the horizontal position of the first hole P in the two-dimensional image IMG. For example, the control device 4 detects the horizontal position V(Xv,Yv) of the vertex V of the first workpiece WK1 from a region to the upper right of the first hole P1 in the two-dimensional image IMG using a well-known image processing method (corner detection method or vertex detection method). Alternatively, the control device 4 may detect the horizontal position of the vertex V of the first workpiece WK1 by detecting the intersection of a horizontal edge and a vertical edge.

[0041] In steps S304 and S305, the control device 4 detects the horizontal rotation angle θ of the first workpiece WK1 based on the position of the edges present in the two-dimensional image IMG.

[0042] In step S304, the control device 4 detects the first horizontal edge EG, one of two edges connecting to vertex V of the first workpiece WK1 in the 2D image IMG. Alternatively, the vertical edge may be detected.

[0043] In step S305, the control device 4 calculates the direction of the first edge EG in the 2D image IMG, for example, the angle between the first edge EG and the X-axis, as the horizontal rotation angle θ of the first workpiece WK1.

[0044] In step S306, the control device 4 determines the horizontal position of the first workpiece WK1 based on the horizontal position (Xv, Yv) of the vertex V of the first workpiece WK, the length of the first side XL and the length of the second side YL of the first workpiece WK1, and the horizontal rotation angle θ of the first workpiece WK1, and the horizontal position of the weight WG1 of the first workpiece WK1 WG1(X WG1 ,Y WG1 Calculate ).

[0045] Let's continue the explanation by referring to Figure 5 again.

[0046] In step S103, the control device 4 determines the horizontal position of the center of gravity of the uppermost workpiece WKn, which is stacked in multiple regions SP2 to SP10 other than the first corner region SP1 on the upper surface of the pallet PL, based on the horizontal position of the first workpiece WK1 and the size of the workpiece WKn, WGn(X WGn ,Y WGn Calculate (n=2 to 10).

[0047] In step S104, as shown in Figure 11, the control device 4 scans the displacement sensor 2 to the horizontal center of gravity of each uppermost workpiece WK1 to WK10, and the vertical distance L1 to L from the displacement sensor 2 to each uppermost workpiece WK1 to WK10 10 In other words, it measures the position in the vertical direction.

[0048] In step S105, the control device 4 operates the robot's hand 7 based on the three-dimensional positions (horizontal and vertical positions) of the workpieces WK1 to WK10 to perform the loading and unloading process (depalletizing) of workpiece WK10. Before loading and unloading workpieces WK1 to WK10, the positions of the workpieces WK1 to WK10 may be reconfirmed by photographing them with a two-dimensional camera.

[0049] According to this embodiment, the three-dimensional position of workpieces loaded on a pallet can be detected without using a 3D camera. This reduces costs and simplifies equipment adjustments.

[0050] Embodiment 2.

[0051] Figure 12 is a flowchart illustrating the operation procedure of the method for detecting the three-dimensional position of the workpiece WK according to Embodiment 2.

[0052] The difference between the processing procedure of Embodiment 2 and the processing procedure of Embodiment 1 is that steps S204 and S205 are included instead of steps S104 and S105.

[0053] In step S204, the control device 4 scans the displacement sensor 2 to the horizontal position of each uppermost workpiece WK1 to WK10, and the vertical distance L1 to L from the displacement sensor 2 to each uppermost workpiece WK1 to WK10 10 This is calculated, and further, the relative number of stages for workpieces WK1 to WK10 is calculated.

[0054] Figure 13 is a flowchart showing the detailed procedure of step S204. Figure 14 shows area SP. n Work WK, which is stacked on the top level. n Relative number of stages A n This is a diagram representing [something].

[0055] In step S401, the control device 4 sets the number n of the area on the upper surface of the pallet PL to 1.

[0056] In step S402, the control device 4 determines the number of stages A of the uppermost workpiece WK loaded in area SPn. n Set it to 0.

[0057] In step S403, the control device 4 controls the displacement sensor 2 to the uppermost workpiece W loaded in area SP. n Move to the center of gravity position and measure the displacement sensor 3 to the workpiece W n Distance L n To measure it.

[0058] In step S404, if n is greater than 1, the process proceeds to step S405.

[0059] In step S405, the control device 4 controls L n and L n-1 The quotient a when the difference is divided by the thickness t of the workpiece WK is n And, the remainder b n Calculate the result.

[0060] In step S406, the remainder b n If the remaining b is greater than or equal to half (t / 2) of the thickness t of the workpiece WK, the process proceeds to step S407, and the remainder bn If the thickness t of the workpiece WK is less than half (t / 2), the process proceeds to step S408.

[0061] In step S407, the control device 4, n-1 ni (a n The value obtained by adding (+1) is A n Let's assume that.

[0062] In step S408, the control device 4, n-1 to a n The value obtained by adding these values ​​is A n In the example in Figure 14, A2=A1+a2, A3=A2+a3, A4=A3+a4, and A5=A4+a5 are calculated.

[0063] In step S409, the control device 4 increases n by 1.

[0064] In step S410, if n is greater than 10, the process terminates; if n is 10 or less, the process returns to step S403.

[0065] Let's continue the explanation by referring to Figure 12 again.

[0066] In step S205, the control device 4 operates the robot's hand 7 based on the three-dimensional positions (horizontal and vertical positions) of the workpieces WK1 to WK10 to perform the loading and unloading process (depalletizing) of the workpieces WK10. Before loading and unloading the workpieces WK1 to WK10, the positions of the workpieces WK1 to WK10 may be reconfirmed by photographing them with a two-dimensional camera. The movement order of the hand 7 can be set to prioritize the removal of workpieces from areas with a higher number of layers. For example, in the example in Figure 14, five workpieces WK are loaded and unloaded from area SP3, then five workpieces WK are loaded and unloaded from areas SP1, SP3, and SP5, and then five areas are loaded and unloaded from areas SP1, SP2, SP3, and SP5. This ensures that the number of layers of workpieces WK in areas SP1 to SP5 is the same. After that, one workpiece WK may be loaded and unloaded from each area.

[0067] As described above, according to this embodiment, similar to Embodiment 1, it is possible to detect the three-dimensional position of workpieces stacked on a pallet without using a three-dimensional camera, and to detect the relative number of workpieces on the top layer.

[0068] Embodiment 3.

[0069] In this embodiment, when the displacement sensor 2 measures the distance to the uppermost workpiece by irradiating the center of gravity of the uppermost workpiece with a laser beam, the problem of incorrect detection of the distance to the uppermost workpiece occurs when the calculated center of gravity of the workpiece coincides with the position of the hole.

[0070] Figure 15 is a flowchart showing the operation procedure of the method for detecting the three-dimensional position of workpiece WK in Embodiment 3. The difference between the processing procedure of Embodiment 3 and the processing procedure of Embodiment 1 is that step S304 is included instead of step S104.

[0071] In step S304, the control device 4 scans the displacement sensor 2 to the horizontal center-of-gravity positions of the uppermost workpieces WK1 to WK10 and two positions other than the center-of-gravity positions, and based on the vertical distances from the displacement sensor 2 to each point of the uppermost workpieces WK1 to WK10, the distances L1 to L from the displacement sensor 2 to the uppermost workpieces WK1 to WK10 10 are calculated.

[0072] FIG. 16 is a diagram showing an example of the position of the workpiece WK1 detected by the displacement sensor 2.

[0073] The distance L from the displacement sensor 2 to the center-of-gravity position WG1 of the workpiece Wk1 1a and the distance L from the displacement sensor 2 to the position P1x that is dx away from the center-of-gravity position WG1 in the X direction 1b and the distance L from the displacement sensor 2 to the position P1y that is dy away from the center-of-gravity position WG1 in the Y direction 1c are measured. dx > dP and dy > dP. dP is the diameter of the hole of the workpiece W.

[0074] L 1a and L 1b and L 1c When the difference between the median value and the minimum value among them is below the threshold value, the three points can be determined to be positions other than the hole. Therefore, the control device 4 can set the average value of L 1a and L 1b and L 1c as L1. L 1a and L 1b and L 1c When the difference between the median value and the minimum value among them exceeds the threshold value, the position with the maximum value can be determined to be the position of the hole. Therefore, the control device 4 can set the average value of the minimum value and the median value as L1.

[0075] The control device 4 also detects L2 to L 10 from the distances to three points for each of the workpieces WK2 to WK10 by the same method.

[0076] As described above, according to this embodiment, similar to Embodiment 1, the three-dimensional position of workpieces loaded on a pallet can be detected without using a three-dimensional camera, and the vertical position of the workpieces can be detected more accurately.

[0077] The various aspects of this disclosure are summarized below as an appendix. (Note 1) A method for detecting the three-dimensional position of multiple workpieces arranged horizontally on the top surface of a pallet and stacked vertically, The upper surface of the pallet is a quadrilateral having a first side, a second side perpendicular to the first side, and a first vertex which is the intersection of the first side and the second side. The steps include roughly detecting the horizontal position of the first vertex by scanning a displacement sensor located above the workpiece in the horizontal direction, The steps include: detecting the horizontal position of the uppermost first workpiece, which is stacked in the region of the first corner of the upper surface of the pallet including the first vertex, using a two-dimensional camera positioned above the workpiece; A step of calculating the horizontal position of one or more workpieces on the uppermost layer, which are stacked in multiple areas other than the first corner area on the upper surface of the pallet, based on the horizontal position of the first workpiece and the size of the workpiece; A method for detecting the three-dimensional position of a workpiece, comprising the steps of scanning the displacement sensor to the horizontal position of each uppermost workpiece and measuring the vertical distance from the displacement sensor to each uppermost workpiece.

[0078] (Note 2) The step of roughly detecting the horizontal position of the first vertex is: The steps include roughly detecting the horizontal position of a first edge point on the first side by scanning the displacement sensor from a first point outside the first side to the inside of the pallet in the direction where the first side exists, The steps include roughly detecting the horizontal position of the second edge point on the first side by scanning the displacement sensor from a second point outside the first side to the inside of the pallet in the direction where the first side exists, The steps include roughly detecting the horizontal position of the third edge point on the second side by scanning the displacement sensor from a third point outside the second side to the inside of the pallet in the direction where the second side exists, The steps include roughly detecting the horizontal position of the fourth edge point on the second side by scanning the displacement sensor from a fourth point outside the second side to the inside of the pallet in the direction where the second side exists, A method for detecting the three-dimensional position of a workpiece as described in Appendix 1, comprising the step of estimating the intersection point of a first virtual line passing through the first edge point and the second edge point and a second virtual line passing through the third edge point and the fourth edge point as the horizontal position of the first vertex.

[0079] (Note 3) The method for detecting the three-dimensional position of a workpiece as described in Appendix 2, further comprising the step of notifying an error when the angle between the first virtual line and the second virtual line is outside a predetermined range.

[0080] (Note 4) The step of detecting the horizontal position of the first workpiece is: The steps include generating a two-dimensional image by capturing a region including the first vertex with the two-dimensional camera, The steps include detecting the horizontal position of the vertex of the first workpiece in the two-dimensional image, The steps include detecting the horizontal rotation angle of the first workpiece based on the position of the edges present in the two-dimensional image, A method for detecting the three-dimensional position of a workpiece according to Appendix 1, comprising the step of detecting the position of the center of gravity of the first workpiece based on the horizontal position of the vertex of the first workpiece and the rotation angle.

[0081] (Note 5) The workpiece has a plurality of holes, The step of detecting the horizontal position of the vertices of the first workpiece in the two-dimensional image is: The steps include detecting the horizontal position of the first hole located closest to the first vertex in the two-dimensional image, A method for detecting the three-dimensional position of a workpiece according to Appendix 4, comprising the step of detecting the horizontal position of the vertex of the first workpiece based on the horizontal position of the first hole in the two-dimensional image.

[0082] (Note 6) The step of detecting the horizontal rotation angle of the first workpiece is: The steps include detecting edges that connect to the vertices of the first workpiece in the two-dimensional image, A method for measuring the three-dimensional position of a workpiece according to Appendix 5, comprising the step of calculating the direction of the edge in the two-dimensional image as the horizontal rotation angle of the first workpiece.

[0083] (Note 7) The method for measuring the three-dimensional position of a workpiece as described in Appendix 1, further comprising the step of calculating the relative number of layers of workpieces stacked in each region based on the vertical distance to each uppermost workpiece and the thickness of the workpiece.

[0084] (Note 8) The step of measuring the vertical distance from the displacement sensor to each uppermost workpiece is as follows: The steps include measuring the vertical distance from the displacement sensor to multiple positions on each uppermost workpiece, A method for measuring the three-dimensional position of a workpiece according to Appendix 1, comprising the step of calculating the vertical distance from the displacement sensor to each uppermost workpiece using the distances to the aforementioned multiple positions.

[0085] (Note 9) A robot for loading and unloading multiple workpieces that are horizontally arranged on the top surface of a pallet and stacked vertically, A hand configured to grasp the aforementioned workpiece, A displacement sensor attached to the hand, A two-dimensional camera attached to the aforementioned hand, A robot comprising a control device that performs a three-dimensional position detection method for a workpiece as described in any one of the appendices 1 to 8.

[0086] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended. [Explanation of symbols]

[0087] 1. Robot, 2. Displacement sensor, 3. 2D camera, 4. Control device, 5. Notification device, 6. Magnet, 7. Hand. WK1~WK10 Workpiece, PL Pallet, SP1~SP10 Region, SD1~SD4 Edges, E,V Vertices.

Claims

1. A method for detecting the three-dimensional position of multiple workpieces arranged horizontally on the top surface of a pallet and stacked vertically, The upper surface of the pallet is a quadrilateral having a first side, a second side perpendicular to the first side, and a first vertex which is the intersection of the first side and the second side. The steps include roughly detecting the horizontal position of the first vertex by scanning a displacement sensor located above the workpiece in the horizontal direction, The steps include: detecting the horizontal position of the uppermost first workpiece, which is stacked in the region of the first corner of the upper surface of the pallet including the first vertex, using a two-dimensional camera positioned above the workpiece; A step of calculating the horizontal position of one or more workpieces on the uppermost layer, which are stacked in multiple areas other than the first corner area on the upper surface of the pallet, based on the horizontal position of the first workpiece and the size of the workpiece; A method for detecting the three-dimensional position of a workpiece, comprising the steps of scanning the displacement sensor to the horizontal position of each uppermost workpiece and measuring the vertical distance from the displacement sensor to each uppermost workpiece.

2. The step of roughly detecting the horizontal position of the first vertex is: The steps include roughly detecting the horizontal position of the first edge point on the first side by scanning the displacement sensor from a first point outside the first side to the inside of the pallet in the direction where the first side exists, The steps include roughly detecting the horizontal position of the second edge point on the first side by scanning the displacement sensor from a second point outside the first side to the inside of the pallet in the direction where the first side exists, The steps include roughly detecting the horizontal position of the third edge point on the second side by scanning the displacement sensor from a third point outside the second side to the inside of the pallet in the direction where the second side exists, The steps include roughly detecting the horizontal position of the fourth edge point on the second side by scanning the displacement sensor from a fourth point outside the second side to the inside of the pallet in the direction where the second side exists, A method for detecting the three-dimensional position of a workpiece according to claim 1, comprising the step of estimating the intersection point of a first virtual line passing through the first edge point and the second edge point and a second virtual line passing through the third edge point and the fourth edge point as the horizontal position of the first vertex.

3. The method for detecting the three-dimensional position of a workpiece according to claim 2, further comprising the step of notifying an error when the angle between the first virtual line and the second virtual line is outside a predetermined range.

4. The step of detecting the horizontal position of the first workpiece is: The steps include generating a two-dimensional image by capturing a region including the first vertex with the two-dimensional camera, The steps include detecting the horizontal position of the vertex of the first workpiece in the two-dimensional image, The steps include detecting the horizontal rotation angle of the first workpiece based on the position of the edges present in the two-dimensional image, A method for detecting the three-dimensional position of a workpiece according to claim 1, comprising the step of detecting the position of the center of gravity of the first workpiece based on the horizontal position of the vertex of the first workpiece and the rotation angle.

5. The workpiece has a plurality of holes, The step of detecting the horizontal position of the vertices of the first workpiece in the two-dimensional image is: The steps include detecting the horizontal position of the first hole located closest to the first vertex in the two-dimensional image, A method for detecting the three-dimensional position of a workpiece according to claim 4, comprising the step of detecting the horizontal position of the vertex of the first workpiece based on the horizontal position of the first hole in the two-dimensional image.

6. The step of detecting the horizontal rotation angle of the first workpiece is: The steps include detecting edges that connect to the vertices of the first workpiece in the two-dimensional image, A method for measuring the three-dimensional position of a workpiece according to claim 5, comprising the step of calculating the direction of the edge in the two-dimensional image as the horizontal rotation angle of the first workpiece.

7. A method for measuring the three-dimensional position of a workpiece according to claim 1, further comprising the step of calculating the relative number of layers of workpieces stacked in each region based on the vertical distance to each uppermost workpiece and the thickness of the workpiece.

8. The step of measuring the vertical distance from the displacement sensor to each uppermost workpiece is as follows: The steps include measuring the vertical distance from the displacement sensor to multiple positions on each uppermost workpiece, A method for measuring the three-dimensional position of a workpiece according to claim 1, comprising the step of calculating the vertical distance from the displacement sensor to each uppermost workpiece using the distances to the aforementioned plurality of positions.

9. A robot for loading and unloading multiple workpieces that are horizontally arranged on the top surface of a pallet and stacked vertically, A hand configured to grasp the aforementioned workpiece, A displacement sensor attached to the hand, A two-dimensional camera attached to the aforementioned hand, A robot comprising a control device that performs a three-dimensional position detection method for a workpiece according to any one of claims 1 to 8.

Images