3D Measuring System

Abstract

The three-dimensional measurement system 1 pertaining to a first embodiment of the present disclosure, whereby the three-dimensional shape of a measurement object can be accurately measured regardless of the position of the measurement object, comprises a three-dimensional measurement unit 32 that measures the position and shape of the measurement object using a three-dimensional sensor 20 that measures a distance to the measurement object, a robot 10 that moves the three-dimensional sensor 20 or the measurement object, a distance measurement unit 33 that measures the distance between the three-dimensional sensor 20 and the measurement object, and a measurement orientation setting unit 34 that sets a measurement orientation of the robot 10 such that the distance between the three-dimensional sensor 20 and the measurement object is a prescribed reference distance on the basis of the distance between the three-dimensional sensor 20 and the measurement object, measured by the distance measurement unit 33.

Description

[Technical field]

[0001] The present invention relates to a three-dimensional measuring system. [Background technology]

[0002] For example, in a system in which a robot picks up workpieces that are placed in random positions and orientations, a 3D measurement system may be used to determine the position and orientation of the workpiece, which measures the 3D shape of the object using a 3D sensor that detects the distance to the object and surrounding objects for each position in 2D vision.

[0003] In order to grasp the three-dimensional shape of the object to be measured more accurately, it has been proposed to measure the three-dimensional shape of the object from different directions using a three-dimensional sensor, and to complement the shape of the back side of the object, which cannot be confirmed in a single three-dimensional measurement, based on the results of other three-dimensional measurements (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-184340 A Summary of the Invention [Problem to be solved by the invention]

[0005] A 3D sensor may have a relatively narrow distance range for accurate detection. In that case, if the distance between the 3D sensor and the measurement object is inappropriate, the 3D shape of the measurement object cannot be accurately measured. For example, if multiple workpieces are randomly overlapped, it may not be possible to simultaneously fit both the topmost and bottommost workpieces within the appropriate distance range of the 3D sensor. For this reason, a 3D measurement system that can accurately measure the 3D shape of the measurement object regardless of the position of the measurement object is desired. [Means for solving the problem]

[0006] A three-dimensional measurement system according to a first aspect of the present disclosure comprises a three-dimensional measurement unit that uses a three-dimensional sensor to measure the distance to a measurement object and measures the position and shape of the measurement object, a robot that moves the three-dimensional sensor or the measurement object, a distance measurement unit that measures the distance between the three-dimensional sensor and the measurement object, and a measurement attitude setting unit that sets a measurement attitude of the robot such that the distance between the three-dimensional sensor and the measurement object becomes a predetermined reference distance based on the distance between the three-dimensional sensor and the measurement object measured by the distance measurement unit.

[0007] A three-dimensional measurement system according to a second aspect of the present disclosure comprises a three-dimensional measurement unit that measures the position and shape of the measurement object using a three-dimensional sensor that measures the distance to the measurement object, a robot that moves the three-dimensional sensor or the measurement object, a reference posture setting unit that sets a reference posture of the robot, a shape memory unit that pre-stores the shape of the measurement object, a match calculation unit that calculates the degree of match between the shape of the measurement object measured by the three-dimensional measurement unit and the shape of the measurement object stored in the shape memory unit, a deviation calculation unit that calculates the amount of deviation of the distance between the three-dimensional sensor and the measurement object from a predetermined reference distance based on the position and shape of the measurement object measured by the three-dimensional measurement unit, and a corrected posture setting unit that sets a corrected posture of the robot by moving the three-dimensional sensor by the amount of deviation if the degree of match does not meet a predetermined judgment threshold. Effect of the Invention

[0008] The three-dimensional measuring system according to the present disclosure can accurately measure the three-dimensional shape of an object regardless of the position of the object. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing a configuration of a three-dimensional measuring system according to a first embodiment of the present invention. [Diagram 2] 4 is a flowchart showing the procedure of an initial setting process in the three-dimensional measuring system of FIG. 1. [Diagram 3]4 is a flowchart showing a procedure for a workpiece detection process in the three-dimensional measuring system of FIG. 1. [Figure 4] FIG. 11 is a schematic diagram showing the configuration of a three-dimensional measuring system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of a three-dimensional measuring system according to the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic diagram showing the configuration of a three-dimensional measuring system 1 according to a first embodiment of the present disclosure.

[0011] The three-dimensional measuring system 1 includes a robot 10, a three-dimensional sensor 20, and a control device 30. The three-dimensional measuring system 1 is a system that measures the position and shape of a measurement target such as a workpiece W, and is configured to be used also as a pick-up system that picks up the workpieces W one by one.

[0012] In the three-dimensional measuring system 1, the workpieces W are placed on a reference plane S, such as the surface of a table, the belt surface of a belt conveyor, the bottom surface of a container, etc. The workpieces W can be placed on the reference plane S so as to overlap each other.

[0013] The robot 10 has a hand 11 that holds a workpiece W at its tip so that the three-dimensional measuring system 1 can be used as a pickup system. The robot 10 positions the three-dimensional sensor 20 at a position where the workpiece W (the workpiece W before being held by the hand 11) can be measured. The robot 10 can be a vertical articulated robot as exemplified in Fig. 1, but is not limited to this and may be, for example, a Cartesian coordinate robot, a SCARA robot, a parallel link robot, or the like.

[0014] The hand 11 may be any type capable of holding the workpiece W, and may be configured to include a plurality of finger-shaped members for gripping the workpiece W, a vacuum pad for adsorbing the workpiece W, or the like.

[0015] The three-dimensional sensor 20 measures the distance to the measurement target and other objects. More specifically, the three-dimensional sensor 20 detects the distance to the measurement target and other objects for each two-dimensional position in the field of view, that is, for each position in a plane perpendicular to the central axis of the measurement range. In other words, the three-dimensional sensor 20 acquires information that can create a three-dimensional image such as a distance image of the measurement target and point cloud data. The measurement target may be not only the workpiece W, but also an initial setting tool for performing initial setting that determines conditions under which accurate measurement can be performed in the three-dimensional measurement system 1.

[0016] The 3D sensor 20 is guaranteed to measure the distance to the object to a specified accuracy only within a certain distance range, as illustrated by hatching in the figure; outside this distance range, the measurement error may become large or it may become impossible to obtain a measurement value.

[0017] The three-dimensional sensor 20 can be configured to have two two-dimensional cameras 21, 22 that capture two-dimensional images of the measurement target, and a projector 23 that projects an image including grid-like reference points onto the measurement target. Such a three-dimensional sensor 20 captures the measurement target onto which grid-like reference points are projected by the two two-dimensional cameras 21, 22, and makes it possible to calculate the distance from the three-dimensional sensor 20 to each grid point based on the positional shift of the grid caused by the parallax of the images captured by the two two-dimensional cameras 21, 22.

[0018] The control device 30 includes a shape memory unit 31, a three-dimensional measurement unit 32, a distance measurement unit 33, a measurement attitude setting unit 34, a reference attitude setting unit 35, a degree of match calculation unit 36, a deviation amount calculation unit 37, a corrected attitude setting unit 38, an initial setting control unit 39, a work detection control unit 40, and a work removal control unit 41. The control device 30 can be realized by causing one or more computer devices having, for example, a memory, a CPU, an input / output interface, etc. to execute an appropriate program. Each component of the control device 30 is classified according to its function, and does not necessarily have to be clearly distinguished in terms of physical structure and program structure.

[0019] The shape memory unit 31 pre-stores the shapes of the measurement target (workpiece W, an initial setting jig not shown, etc.) In the three-dimensional measuring system 1, an object having a shape matching the shape stored in the shape memory unit 31 is recognized as the measurement target.

[0020] The 3D measurement unit 32 measures the position and shape of the measurement target and surrounding objects using the 3D sensor 20. That is, a detection trigger is given to the 3D sensor 20, and 3D data of the surface shape of objects such as the measurement target in the coordinate system of the 3D sensor 20 is created based on the detection data of the 3D sensor 20.

[0021] The distance measurement unit 33 measures the distance between the three-dimensional sensor 20 and the measurement object (hereinafter referred to as the initial placement distance). The distance measurement unit 33 may be configured to use a dedicated sensor, but may be configured to measure the initial placement distance between the three-dimensional sensor 20 and the measurement object using one of the two two-dimensional cameras 21, 22 of the three-dimensional sensor 20. As an example, the distance measurement unit 33 may be configured to calculate the initial placement distance between the three-dimensional sensor 20 and the measurement object based on a plurality of two-dimensional images obtained by changing the posture of the robot 10 and capturing the measurement object from different positions by the two-dimensional cameras 21, 22.

[0022] The measurement attitude setting unit 34 sets a measurement attitude of the robot 10 such that the predicted value of the distance between the three-dimensional sensor 20 and the measurement object becomes a predetermined reference distance based on the initial placement distance between the three-dimensional sensor 20 and the measurement object measured by the distance measurement unit 33. The reference distance is a distance at which the three-dimensional sensor 20 can accurately detect the distance. In other words, the "measurement attitude" is an attitude in which the robot 10 positions the three-dimensional sensor 20 at a position where the measurement by the three-dimensional measurement unit 32 is accurate. Note that the "measurement attitude" is determined based on one measurement value of the initial placement distance, which may include an error, and does not necessarily make the actual distance between the three-dimensional sensor 20 and the measurement object exactly match the reference distance.

[0023] The reference attitude setting unit 35 sets a reference attitude of the robot 10 such that the distance between the three-dimensional sensor 20 and the measurement object (hereinafter referred to as the measurement arrangement distance) calculated based on the position and shape of the measurement object measured by the three-dimensional measurement unit 32 with the robot 10 in the measurement attitude coincides with the reference distance. In other words, the "reference attitude" is an attitude that is considered to more accurately coincide the actual distance between the three-dimensional sensor 20 and the measurement object with the reference distance, based on the measurement arrangement distance between the three-dimensional sensor 20 and the measurement object measured relatively accurately by the three-dimensional measurement unit 32 in the measurement attitude.

[0024] The degree of match calculation unit 36 ​​calculates the degree of match between the shape of the measurement target measured by the 3D measurement unit 32 and the shape of the measurement target stored in the shape memory unit 31. When the shape of the measurement target is constant, this "degree of match" decreases due to measurement errors of the 3D measurement unit 32 as the deviation between the actual distance between the 3D sensor 20 and the measurement target and the reference distance increases.

[0025] The deviation amount calculation unit 37 calculates the deviation amount of the measurement arrangement distance between the 3D sensor 20 and the measurement object from the reference distance, that is, the difference between the measurement arrangement distance and the reference distance, based on the position and shape of the measurement object measured by the 3D measurement unit 32. Since the measurement object is placed on the reference surface, the measurement arrangement distance becomes shorter than the reference distance when the measurement objects are placed overlapping each other. For this reason, it is simple to set the deviation amount to a value obtained by subtracting the measurement arrangement distance from the reference distance.

[0026] When the degree of match calculated by the degree of match calculation unit 36 ​​does not meet a predetermined judgment threshold, the corrected attitude setting unit 38 sets a corrected attitude of the robot by moving the 3D sensor 20 by the amount of deviation calculated by the deviation amount calculation unit. A large decrease in the degree of match may occur when the measurement objects are piled up after the reference attitude setting unit 35 sets the reference attitude, and the actual distance to the measurement object becomes small. For this reason, when the degree of match does not meet the judgment threshold, a corrected attitude is set by moving the 3D sensor 20 by the amount of deviation. In other words, the corrected attitude setting unit 38 sets a corrected attitude that allows the 3D sensor 20 to measure the position and shape of the measurement object more accurately by bringing the actual distance between the 3D sensor 20 and the measurement object closer to the reference distance.

[0027] The initial setting control unit 39 performs an initial setting process to set a reference posture, which is the posture of the robot 10 used as the initial setting when detecting the position and orientation of the workpiece W in the 3D measurement system 1, by causing the 3D measurement unit 32, the distance measurement unit 33, the measurement posture setting unit 34 and the reference posture setting unit 35 to function in an appropriate order.

[0028] 2 shows the procedure of the initial setting process by the initial setting control unit 39. The initial setting process includes an initial positioning step (step S01), a distance measurement step (step S02), a measurement attitude setting step (step S03), a measurement attitude positioning step (step S04), a three-dimensional measurement step (step S05), and a reference attitude setting step (step S06). The initial setting process may be performed using the workpiece W, but is preferably performed by placing an initial setting tool (not shown) having a shape and color that can be accurately detected by the three-dimensional measurement unit 32 and the distance measurement unit 33 on the reference surface S. It is preferable to use an initial setting tool that has a shape feature at approximately the same height position as the workpiece W when it is placed on the reference surface S so as not to overlap it. It is preferable that the initial setting tool has feature points that can be detected by the three-dimensional measurement unit 32 and the distance measurement unit 33 arranged rotationally asymmetrically.

[0029] In the initial positioning step of step S01, the posture of the robot 10 is determined to be a posture that allows the distance to a measurement target to be measured by the distance measurement unit 33. This posture may be set in advance as a posture of the robot 10 that sufficiently separates the three-dimensional sensor 20 from the reference plane S, for example, or may be set manually by an operator or manager. In other words, the initial positioning step may be a step of requesting the operator or manager to position the robot 10 and receiving an input confirming completion of positioning.

[0030] In the distance measurement process of step S02, distance measurement unit 33 is caused to measure the initial placement distance between 3D sensor 20 and the measurement object. Specifically, the distance measurement process may include, for example, a process of having 2D camera 21 capture an image of the measurement object, a process of changing the posture of robot 10 to move 3D sensor 20, a process of having 2D camera 21 capture an image of the measurement object again, and a process of calculating the initial placement distance between 3D sensor 20 and the measurement object from changes in the position, arrangement, shape, etc. of the measurement object in the two captured images.

[0031] In the measurement attitude setting process of step S03, the measurement attitude setting unit 34 sets a measurement attitude of the robot 10 such that the distance between the 3D sensor 20 and the measurement object becomes a predetermined reference distance, based on the initial placement distance measured in the distance measurement process.

[0032] In a measurement posture positioning step S04, the posture of the robot 10 is changed to the assumed posture set in the measurement posture setting step.

[0033] In the three-dimensional measurement process of step S05, three-dimensional measurement unit 32 uses three-dimensional sensor 20 to measure the position and shape of the measurement object, and calculates the measurement arrangement distance between three-dimensional sensor 20 and the measurement object.

[0034] In the reference attitude setting process of step S06, based on the measurement arrangement distance measured in the 3D measurement process, the reference attitude setting unit 35 sets a reference attitude of the robot 10 such that the distance between the 3D sensor 20 and the measurement object is a predetermined reference distance.

[0035] The workpiece detection control unit 40 performs a workpiece detection process to detect the position and orientation of the workpiece W by causing the 3D measurement unit 32, the matching degree calculation unit 36, the deviation amount calculation unit 37 and the corrected posture setting unit 38 to function in an appropriate order.

[0036] 3 shows the procedure of the work detection process by the work detection control unit 40. The work detection process includes a reference posture positioning step (step S11), a first measurement step (step S12), a first work shape recognition step (step S13), a matching degree calculation step (step S14), a matching degree confirmation step (step S15), a deviation amount calculation step (step S16), a corrected posture setting step (step S17), a corrected posture positioning step (step S18), a second measurement step (step S19), a second work shape recognition step (step S20), and a work position identification step (step S21).

[0037] In the reference posture positioning step of step S11, the robot 10 is operated so as to assume a reference posture position that has been set in advance in the initial setting process.

[0038] In the first measurement process of step S12, the three-dimensional measuring unit 32 measures the position and shape of the measurement target and surrounding objects using the three-dimensional sensor 20. If the workpiece W is placed under the same conditions as the measurement target used in the initial setting process, typically without overlapping on the reference surface S, the shape measured in this first measurement process will include a portion that matches the shape of the workpiece W relatively accurately.

[0039] In the first workpiece shape recognition process of step S13, a portion having a shape similar to the shape of the workpiece W stored in the shape memory unit 31 is identified from the shape measured in the first measurement process using a well-known pattern recognition technique.

[0040] In the match calculation process of step S14, the match calculation unit 36 ​​calculates a match indicating the degree of match between the part identified in the first work shape recognition process and the shape of the work W stored in the shape memory unit 31.

[0041] In the matching degree confirmation step of step S15, it is confirmed whether the matching degree calculated in the matching degree calculation step is equal to or greater than a preset judgment threshold. If the matching degree is equal to or greater than the predetermined judgment threshold, the process proceeds to step S21, and if the matching degree is less than the predetermined judgment threshold, the process proceeds to step S16. A case in which the matching degree is low may be when the workpieces W are stacked and the topmost workpiece W is outside the optimal distance range of the three-dimensional sensor 20.

[0042] In the deviation amount calculation process of step S16, the deviation amount calculation unit 37 calculates the amount of deviation from a reference distance between the 3D sensor 20 and the measurement object (the shape portion whose matching degree was calculated in the matching degree calculation process) based on the position and shape of the measurement object measured in the first measurement process.

[0043] In the corrected attitude setting step of step S17, corrected attitude setting unit 38 is caused to set a corrected attitude of the robot that moves 3D sensor 20 backward from the reference attitude by the amount of deviation calculated in the deviation amount calculation step.

[0044] In the corrected attitude positioning step of step S18, the robot 10 is operated so as to assume the corrected attitude set in the corrected attitude setting step. This makes it possible to bring the distance to the workpiece W identified by the three-dimensional sensor 20 in the first workpiece shape recognition step closer to a reference distance at which appropriate measurement is possible.

[0045] In the second measurement step of step S19, the three-dimensional measuring unit 32 measures the positions and shapes of the measurement target and surrounding objects again. That is, in the second measurement step, the position and shape of the measurement target are measured again in the corrected attitude.

[0046] In the second workpiece shape recognition process of step S20, similar to the first workpiece shape recognition process, a portion having a shape similar to the shape of the workpiece W stored in the shape memory unit 31 is identified from the shape measured in the second measurement process.

[0047] In the work position identification process of step S21, if the degree of match in the match confirmation process is equal to or greater than the judgment threshold, it is determined that the work W is present in a position and orientation that matches the shape portion identified in the first work shape recognition process, and if the degree of match in the match confirmation process is less than the judgment threshold, it is determined that the work W is present in a position and orientation that matches the shape portion identified in the second work shape recognition process, and the position and orientation of the work W are identified.

[0048] The workpiece removal control unit 41 executes a workpiece removal process that operates the robot 10 to grasp and remove the workpiece W identified by the workpiece detection process by the workpiece detection control unit 40 with the hand 11 using a conventional method.

[0049] In the three-dimensional measuring system 1, when a plurality of workpieces W are arranged on the reference plane S, the workpieces W can be picked up one by one by alternately repeating the workpiece detection process and the workpiece pick-up process.

[0050] As described above, the three-dimensional measuring system 1 can accurately measure the three-dimensional shape of the measurement object regardless of the position of the measurement object. Therefore, the three-dimensional measuring system 1 can reliably pick up the workpiece W.

[0051] Fig. 4 shows the configuration of a three-dimensional measurement system 1A according to a second embodiment of the present disclosure. In the three-dimensional measurement system 1A in Fig. 4, components similar to those in the three-dimensional measurement system 1 in Fig. 1 are denoted by the same reference numerals and will not be described repeatedly.

[0052] The three-dimensional measurement system 1A includes a robot 10 having a hand 11, a three-dimensional sensor 20, and a control device 30. In the three-dimensional measurement system 1A, the three-dimensional sensor 20 is fixed above the robot 10, for example to a ceiling, beams, or the like of a room in which the three-dimensional measurement system 1A is installed.

[0053] This three-dimensional measuring system 1A measures the position and shape of the workpiece W while the workpiece W is held by the hand 11, thereby confirming a deviation in the holding position of the workpiece W by the hand 11. Therefore, in the three-dimensional measuring system 1A, the robot 10 moves the measurement target held by the hand 11 relative to the three-dimensional sensor 20.

[0054] Like the 3D measurement system 1 of FIG. 1, the control device 30 of the 3D measurement system 1A of FIG. 4 also includes a shape memory unit 31, a 3D measurement unit 32, a distance measurement unit 33, a measurement attitude setting unit 34, a reference attitude setting unit 35, a match degree calculation unit 36, a deviation amount calculation unit 37, a corrected attitude setting unit 38, an initial setting control unit 39, a work detection control unit 40, and a work removal control unit 41.

[0055] In the 3D measurement system 1A of Figure 4, the robot 10 enters the field of view of the 2D cameras 21, 22 during measurement by the 3D measurement unit 32 and measurement by the distance measurement unit 33, but the procedures for the initial setting process and work detection process are the same as those of the 3D measurement system 1 of Figure 1.

[0056] 4 can accurately measure the three-dimensional shape of the measurement target regardless of the position at which the measurement target is held by the hand 11. As a result, the three-dimensional measurement system 1A can correct any deviation in the position at which the hand 11 holds the workpiece W, and allow the robot 10 to accurately position the workpiece W relative to, for example, a processing device.

[0057] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment. Furthermore, the effects described in the present embodiment are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the present embodiment.

[0058] In the three-dimensional measuring system according to the present invention, in the workpiece detection process, the positions of the three-dimensional sensor and the measurement object measured by the distance measurement unit are confirmed as in the initial setting process, a measurement posture is set in which the distance between the three-dimensional sensor and the measurement object is a reference distance, and the position and orientation of the workpiece may be specified based on the measurement results of the position and shape of the measurement object by the three-dimensional measuring unit in this measurement posture. In this case, in the second and subsequent initial positioning steps, it is preferable to automatically reproduce the posture of the robot set in the first initial positioning step.

[0059] Furthermore, in the three-dimensional measuring system according to the present invention, the reference posture may be set manually and the workpiece detection process may be performed without performing the initial setting process.

[0060] Furthermore, in the 3D measuring system of the present invention, the correction posture setting unit may judge the accuracy of the measurement results of the 3D measuring unit based on the number of 3D points that the 3D measuring unit was able to measure, the number of works detected by the work detection control unit, etc., instead of or in addition to the degree of match, and set the correction posture if the measurement results are determined to be inaccurate. [Explanation of symbols]

[0061] 1 3D Measuring System 10. Robot 11 Hands 20 3D Sensor 21,22 2D Camera 23 Projector 30 Control device 31 Shape memory section 32 3D Measurement Section 33 Distance measurement unit 34 Measurement posture setting section 35 Reference posture setting section 36 Matching degree calculation section 37 Displacement calculation unit 38 Correction posture setting section 39 Initial setting control section 40 Work detection control unit 41 Work removal control section S reference plane W work

Claims

1. a three-dimensional measuring unit that measures a position and a shape of a measurement object using a three-dimensional sensor that measures a distance to the measurement object; a robot that moves the three-dimensional sensor or the measurement object; a distance measurement unit that measures a distance between the three-dimensional sensor and the measurement object; a measurement attitude setting unit that sets a measurement attitude of the robot such that the distance between the three-dimensional sensor and the measurement object becomes a predetermined reference distance based on the distance between the three-dimensional sensor and the measurement object measured by the distance measurement unit; Equipped with the three-dimensional sensor includes a two-dimensional camera; the distance measurement unit calculates a distance between the three-dimensional sensor and the measurement object based on a plurality of two-dimensional images obtained by changing a posture of the robot and photographing the measurement object from different positions by the two-dimensional camera; The three-dimensional measuring system is configured so that the three-dimensional measuring unit measures the position and shape of the measurement object in one of the measurement postures.

2. 2. The three-dimensional measurement system according to claim 1, further comprising a reference posture setting unit that sets a reference posture of the robot such that the distance between the three-dimensional sensor and the object to be measured coincides with the reference distance, based on the position and shape of the object to be measured measured by the three-dimensional measurement unit when the robot is in the measurement posture.

3. A shape storage unit that stores the shape of the measurement object in advance; a matching degree calculation unit that calculates a matching degree between the shape of the measurement object measured by the three-dimensional measurement unit and the shape of the measurement object stored in the shape memory unit; a deviation amount calculation unit that calculates a deviation amount of a distance between the three-dimensional sensor and the measurement object from the reference distance based on the position and shape of the measurement object measured by the three-dimensional measurement unit; a corrected posture setting unit that sets a corrected posture of the robot by moving the three-dimensional sensor by the deviation amount when the degree of match does not satisfy a predetermined judgment threshold value; The three-dimensional measurement system of claim 2 , further comprising:

Images