Workpiece inspection system

Abstract

A workpiece detection system that correctly and quickly detects a workpiece includes a three-dimensional sensor that measures a three-dimensional shape of an object within a field of view, and an information processing device that sets a visually recognizable shape of a three-dimensional model at each viewpoint as a search target shape, searches for a coincident partial shape that coincides with the search target shape from the three-dimensional shape measured by the three-dimensional sensor, and outputs the found coincident partial shape as a search result, wherein the information processing device excludes a part of the search target shape or a part of the search result based on a condition set in advance.

Description

Technical Field

[0001] This invention relates to a workpiece inspection system. Background Technology

[0002] In, for example, a robot operation processing system that retrieves randomly supplied workpieces utilizes a workpiece detection system that uses a three-dimensional sensor to acquire the three-dimensional shape of an object region and determines the position and orientation of the workpiece by a matching process that determines the shape of a consistent portion that matches the shape of a three-dimensional model.

[0003] The computational load of matching such three-dimensional shapes is very high, and it may take a long time to process without the use of high-performance computing devices. Therefore, the following solution is proposed: extract the planar portion from the three-dimensional shape of the object, and confirm the consistency between the shape of the extracted planar portion and the shape of the plane of the workpiece, thereby detecting the workpiece (see, for example, Patent Document 1).

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-97889 Summary of the Invention

[0007] The problem the invention aims to solve

[0008] When the features of the target shape in the 3D model are insufficient or inappropriate, incorrect consistent shapes may sometimes be output as search results. Furthermore, if incorrect consistent shapes are calculated as search results using insufficient or inappropriate target shapes, these results cannot be used in subsequent processing, resulting in wasted processing time. Additionally, there are cases where the features of the consistent shape in the search results are insufficient due to single or combined factors such as errors or omissions in 3D sensor measurements, or the state of the target shape or its features; in such cases, the search results will mostly be incorrect.

[0009] Solution for solving the problem

[0010] One aspect of the workpiece inspection system disclosed herein includes: a three-dimensional sensor that measures the three-dimensional shape of an object within its field of view; and an information processing device that sets the visually recognizable shape of a three-dimensional model at each viewpoint as a search target shape, searches for a consistent portion shape that matches the search target shape from the three-dimensional shapes measured by the three-dimensional sensor, and outputs the found consistent portion shape as a search result, wherein the information processing device excludes a portion of the search target shape or a portion of the search result based on pre-set conditions.

[0011] The effects of the invention

[0012] The workpiece inspection system disclosed herein can accurately and quickly inspect workpieces. Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating the structure of a workpiece inspection system according to the first embodiment of this disclosure.

[0014] Figure 2 This is a schematic diagram showing the visually recognizable shape of a cylindrical 3D model.

[0015] Figure 3 It is shown Figure 2 A schematic diagram of the visually recognizable shape of the 3D model viewed from the Z direction.

[0016] Figure 4 It is shown Figure 2 A schematic diagram of the visually recognizable shape of the 3D model when viewed from the X direction. Detailed Implementation

[0017] The embodiments of this disclosure will now be described with reference to the accompanying drawings. Figure 1 This is a block diagram illustrating the structure of a workpiece inspection system 1 according to a first embodiment of this disclosure. The workpiece inspection system 1 detects the position and orientation of the workpiece and outputs the position and orientation of the workpiece to, for example, a robot control device.

[0018] The workpiece inspection system 1 includes: a three-dimensional sensor 10, which measures the three-dimensional shape of an object within its field of view; and an information processing device 20, which sets the visually recognizable shape of a pre-set three-dimensional model at each viewpoint as the search target shape, searches for a matching part shape that matches the search target shape from the three-dimensional shapes measured by the three-dimensional sensor 10, and outputs the found matching part shape as the search result.

[0019] The 3D sensor 10 detects the distance to objects (not only workpieces, but also other objects, workbenches, belt conveyors, etc.) present within its field of view at each two-dimensional position, that is, at each position in the plane perpendicular to the central axis of the measurement range. In other words, the 3D sensor 10 measures the three-dimensional shape of the surface of an object within its field of view, on the side that can be visually recognized from the 3D sensor 10. The 3D sensor 10 can be, for example, a stereo camera or a laser scanner. The measured three-dimensional shape is transmitted to the information processing device 20 as information capable of creating a 3D image, such as distance image data or point group data.

[0020] The information processing device 20 excludes a portion of the search target shape or a portion of the search results based on preset conditions. Examples of conditions related to excluding a portion of the search target shape or a portion of the search results include, for example, the size of a planar region in a visually identifiable shape of a 3D model or a consistent part of a measured result, and the symmetry of the shape of the 3D model.

[0021] The information processing device 20 can be implemented by executing appropriate information processing programs, for example, using a computer device equipped with a memory, a processor (CPU), an input / output interface, etc. The information processing device 20 may also include a display device such as a display panel, an input device such as a keyboard, a touch pad, etc., and can be configured to display information using an external display device and input information using an external input device via the input / output interface. Furthermore, the information processing device 20 may be integrated with a computer device that manages the entire equipment using a workpiece inspection system 1, such as an operation processing system or a machining system, or a computer device that controls other components within the equipment.

[0022] Specifically, the information processing device 20 can be configured to include a 3D model setting unit 21, a 3D shape acquisition unit 22, a matching processing unit 23, a detection result output unit 24, a viewpoint exclusion processing unit 25, a unit shape exclusion processing unit 26, and a result exclusion processing unit 27. These components can also be components that are obtained by classifying the functions of the information processing device 20 and cannot be clearly distinguished in terms of physical structure and program structure.

[0023] The 3D model setting unit 21 sets a 3D model for determining the 3D shape of the workpiece to be inspected. The 3D model of the workpiece can be determined by, for example, CAD data. Therefore, the 3D model setting unit 21 can be configured to acquire 3D model information from CAD via a communication or recording medium. In addition, the 3D model setting unit 21 may also have the function of constructing or modifying the 3D model.

[0024] The three-dimensional shape acquisition unit 22 acquires data on the three-dimensional shape of the object within the field of view of the three-dimensional sensor 10 as measured by the three-dimensional sensor 10.

[0025] The matching processing unit 23 performs the following matching process: It searches for a matching portion shape from the three-dimensional shapes acquired by the three-dimensional shape acquisition unit 22 that matches the shape of the visually recognizable area of ​​the three-dimensional model of the workpiece set by the three-dimensional model setting unit 21 at each viewpoint, i.e., the surface shape of the three-dimensional model that can be visually recognized from the viewpoint. It then determines the position and orientation of this matching portion shape that matches the search target shape. For example, when viewed radially from a cylindrical three-dimensional model, the visually recognizable shape is a half-pipe shape.

[0026] The matching process can be performed by repeatedly performing the following calculations in a three-dimensional scanning manner, with slight shifts in position: For a specified size of the matching portion shape of the three-dimensional shape acquired by the three-dimensional shape acquisition unit 22, the similarity degree between this portion shape and the target search shape is calculated. If the similarity degree exceeds a threshold, it is determined that the shape matches the target search shape. Alternatively, if a matching portion shape matching a visually recognizable shape is found through the search process, the matching processing unit 23 can use an algorithm such as ICP (Iterative Closest Point) to accurately determine the position and orientation of the matching portion shape.

[0027] The detection result output unit 24 outputs the position and orientation of the matching part shape that the matching processing unit 23 determines is consistent with the search target shape as the search result to the robot control device, etc. Regarding a three-dimensional shape acquired by the three-dimensional shape acquisition unit 22, the search result can include multiple matching part shapes. Furthermore, the search result can include information indicating that no matching part shape was found. The detection result output unit 24 is not limited to immediately outputting the search result upon obtaining it; it can also store the search result and output it in response to a request.

[0028] The viewpoint exclusion processing unit 25 excludes visually recognizable shapes from the search target shapes from viewpoints where the ratio of the projected area of ​​the planar region to the overall projected area of ​​the visually recognizable shape is greater than a predetermined reference value. The planar region is not limited to a completely flat plane, and can be, for example, a region with a separation distance from the virtual plane of a set value or less, or a region with a curvature of a set value or more.

[0029] As an example, consider the following: Figure 2 As shown, the 3D model of the workpiece obtained by modeling in XYZ space (orthogonal coordinate space) is a cylinder extending in the Z direction. When viewing the 3D model from the Z-axis direction, as... Figure 3 As shown, only the planar end face can be visually identified. In this case, the projected area of ​​the planar region is 100% of the projected area of ​​the overall visually recognizable shape. Therefore, the visually recognizable shape from this viewpoint is excluded from the search target shape. Additionally, as... Figure 4 As shown, when viewing the 3D model from the X direction or any direction parallel to the XY plane, its end face cannot be visually identified; only the cylindrical surface can be visually identified. Therefore, the projected area ratio of the completely flat planar region is 0% (the projected area of ​​the planar region defined by the separation distance from the aforementioned virtual plane can be a valid value). When viewing the 3D model from a direction inclined relative to the XY plane, the projected area ratio of the planar region is a value between 0% and 100%, depending on the inclination angle. The viewpoint exclusion processing unit 25 can be configured to exclude visually recognizable shapes from the search target shapes from viewpoints where the projected area ratio of the planar region is greater than or equal to a reference value, that is, viewpoints where the angle of inclination of the viewing direction relative to the XY plane is greater than or equal to a certain angle.

[0030] The viewpoint exclusion processing unit 25 can also calculate the proportion of the projected area of ​​the planar region in the visually recognizable shape of the 3D model only for the largest planar region. When the visually recognizable shape has multiple planar regions, the possibility that multiple planar regions exist due to measurement errors of the 3D sensor 10, and that the positional relationship of these multiple planar regions matches the positional relationship of the multiple planar regions of the visually recognizable shape, is extremely small. Therefore, if only visually recognizable shapes whose projected area proportion is greater than or equal to a reference value are excluded from the search target shapes, the processing speed can be improved without unnecessarily reducing the probability (detection rate) of detecting an actual workpiece.

[0031] Preferably, the viewpoint exclusion processing unit 25 is configured to provide an interface that allows the user to arbitrarily set a reference value for the proportion of the projected area of ​​a planar region, and a setting value for an indicator that determines a plane (such as the allowable separation distance from the lower plane). In this case, to facilitate appropriate settings by the user, the viewpoint exclusion processing unit 25 is more preferably configured to output information about visually recognizable shapes to be excluded from the search target shape, and to display, for example, an image, the coordinate position of the viewpoint, and the proportion of the excluded visually recognizable shapes on a display screen.

[0032] Furthermore, the viewpoint exclusion processing unit 25 can also be configured to exclude only visually identifiable shapes from the search target shapes if not only the proportion of the projected area of ​​the planar region is greater than a predetermined reference value, but also if the outline of the visually identifiable shape of the 3D model under the viewpoint is unclear. The clarity of the outline of the visually identifiable shape can be judged based on predetermined indicators such as the curvature of the outer edge of the visually identifiable shape and the change in the position of the viewing direction near the outer edge of the visually identifiable shape. When the clarity of the outline of the visually identifiable shape is high, the measurement error of the portion corresponding to the outer edge of the visually identifiable shape in the measurement results of the 3D sensor 10 is smaller, so the consistency threshold in the matching processing unit 23 can be set higher. Thus, even if the consistency is high due to measurement errors in the planar region, it is possible to avoid misdetecting it as a shape consistent with the search target shape, so it is not necessary to exclude it from the search target shapes.

[0033] As an example, when the 3D model of the workpiece is a precise cube, the projected area ratio of even the largest planar region is more than 1 / 3. However, since the outer edge of the visually recognizable shape is the edge between faces, the curvature near the outer edge of the visually recognizable shape is infinitely large, and the rate of change of the viewing direction position near the outer edge of the visually recognizable shape is fixed. Since such a visually recognizable shape can be evaluated as having a well-defined outline, the possibility of false detection is small even if it is not excluded from the search target shape. On the other hand, when the 3D model of the workpiece is a cube with its edges and corners rounded, if the curvature of the chamfer becomes smaller, there is a possibility that the measurement error of the part corresponding to the outer edge of the visually recognizable shape will increase, so it is not advisable to set the consistency threshold too high. In such a 3D model, the curvature of the outer edge of the visually recognizable shape becomes a relatively small value, and the rate of change of the viewing direction position near the outer edge of the visually recognizable shape changes (the second derivative value becomes the effective value). Therefore, by using metrics obtained by quantifying the features of the outer edge or near the outer edge of a visually recognizable shape, the parts of the visually recognizable shape with unclear outlines can be identified. At the same time, only visually recognizable shapes with a large proportion of the projected area of ​​the planar region can be added to the objects excluded from the search target shape. This enables the detection process to be accelerated while ensuring the detection rate and suppressing false detections.

[0034] When a 3D model has multiple unit shapes arranged around an axis of symmetry, the unit shape exclusion processing unit 26 excludes a portion of these unit shapes from the visually recognizable shapes that serve as the search target shape. Typically, it retains only one of the multiple mutually symmetrical unit shapes arranged around an axis of symmetry and excludes the others. This reduces the computational load for finding consistent partial shapes that match the search target shape by compressing the information in the search target shape after excluding unit shapes. Furthermore, by deleting duplicate unit shapes, it prevents the same shape portion from being repeatedly identified as multiple visually recognizable shapes with different starting points.

[0035] Preferably, the unit shape exclusion processing unit 26 is configured such that the user can specify the start and end points of the unit shape using two planes containing the axis of symmetry. Specifically, the unit shape exclusion processing unit 26 can be configured to provide a user interface that displays the overall shape of the 3D model and two planes representing the start and end points of the unit shapes, which the user can rotate around the axis of symmetry using, for example, dragging. When the 3D model has a solid shape of revolution, such as a cylinder, it can be divided into any number of unit shapes. Therefore, by appropriately selecting the size of the unit shapes, sufficient detection accuracy can be ensured, and consistent portions of the shape matching the target shape can be quickly detected.

[0036] The result exclusion processing unit 27 excludes matching portions of the same shape whose projected area of ​​the planar region relative to the overall projected area of ​​the matching portion shape is greater than a predetermined benchmark value from the search results. In other words, the result exclusion processing unit 27 re-confirms the shapes that the matching processing unit 23 has determined to be matching portions of the search target shape, and excludes them from the search results if the projected area of ​​the planar region is large. Therefore, the detection result output unit 34 only outputs the search results not excluded by the result exclusion processing unit 27 as the final detection result.

[0037] As an example, in the case of a prismatic object, even if the 3D sensor 10 fails to accurately measure the shape of the object and the matching processing unit 23 determines that it matches the visually recognizable shape of a cylindrical 3D model, the proportion of the projected area of ​​the planar region will be significantly larger compared to the case of measuring a cylindrical workpiece. Therefore, the result exclusion processing unit 27 can exclude the matching part with the large proportion of the projected area of ​​the planar region from the search results, thereby preventing false detection. Furthermore, in the result exclusion processing unit 27, similar to the viewpoint exclusion processing unit 25, the proportion of the projected area of ​​the planar region can be calculated only for the largest plane.

[0038] The result exclusion processing unit 27 can also exclude only the consistent shapes in the large projected area of ​​the planar region whose contours or contour regions do not meet the reference value for consistency with the contours or contour regions of the search target shape from the search results. When the proportion of the planar region is large but the consistency of the contour region is high, it can be assumed that the workpiece actually exists. The contour region can be appropriately set, for example, to be a region within a certain distance range from the position corresponding to the outer edge of the search target shape.

[0039] The result exclusion processing unit 27 can also be configured to provide an interface for the user to set the proportion of the projected area of ​​the planar region of the consistent shape excluded from the search results, the consistency of the outline region, etc. Furthermore, it is preferable that the result exclusion processing unit 27 is configured to provide an interface that can output information about the shape of the consistent parts excluded from the search results, typically by displaying it on a screen. Since the user can confirm the actual shape of the excluded consistent parts, the settings used for excluding from the search results can be easily optimized.

[0040] The embodiments of this disclosure have been described above, but the present invention is not limited to the embodiments described above. Furthermore, the effects described in the above embodiments are merely examples of the best effects produced by the present invention, and the effects of the present invention are not limited to those described in the above embodiments.

[0041] In the workpiece inspection system disclosed herein, the information processing device only needs to be able to exclude a portion of the target shape or a portion of the search results based on preset conditions, and may not have a portion or all of the viewpoint exclusion processing unit, unit shape exclusion processing unit and result exclusion processing unit in the above embodiments.

[0042] Explanation of reference numerals in the attached figures

[0043] 1: Workpiece inspection system; 10: 3D sensor; 20: Information processing device; 21: 3D model setting unit; 22: 3D shape acquisition unit; 23: Matching processing unit; 24: Inspection result output unit; 25: Viewpoint exclusion processing unit; 26: Unit shape exclusion processing unit; 27: Result exclusion processing unit.

Claims

1. A workpiece inspection system, comprising: A three-dimensional sensor that measures the three-dimensional shape of objects within its field of view; and An information processing device sets the visually recognizable shapes of a 3D model from various viewpoints as the search target shapes, searches for matching portions of the 3D shapes measured by the 3D sensor, and outputs the found matching portions as the search results. in, The information processing device has a viewpoint exclusion processing unit, which excludes visually recognizable shapes from the search target shapes from viewpoints where the ratio of the projected area of ​​the planar region of the visually recognizable shape to the overall projected area of ​​the visually recognizable shape is greater than a predetermined reference value.

2. A workpiece inspection system, comprising: A three-dimensional sensor that measures the three-dimensional shape of objects within its field of view; and An information processing device sets the visually recognizable shapes of a 3D model from various viewpoints as the search target shapes, searches for matching portions of the 3D shapes measured by the 3D sensor, and outputs the found matching portions as the search results. in, The information processing device has a viewpoint exclusion processing unit, which excludes visually identifiable shapes whose lower contours are unclear from the search target shapes when the ratio of the projected area of ​​the planar region of the visually identifiable shape to the overall projected area of ​​the visually identifiable shape is greater than a predetermined reference value.

3. The workpiece inspection system according to claim 1 or 2, wherein, The proportion of the projected area of ​​the planar region of the visually recognizable shape is calculated only for the largest plane.

4. The workpiece inspection system according to claim 1 or 2, wherein, The viewpoint exclusion processing unit is configured to output information about the visually recognizable shapes to be excluded from the search target shapes.

5. A workpiece inspection system, comprising: A three-dimensional sensor that measures the three-dimensional shape of objects within its field of view; and An information processing device sets the visually recognizable shapes of a 3D model from various viewpoints as the search target shapes, searches for matching portions of the 3D shapes measured by the 3D sensor, and outputs the found matching portions as the search results. in, The information processing device has a result exclusion processing unit, which excludes from the search results any matching partial shape whose projected area of ​​a planar region relative to the overall projected area of ​​the matching partial shape is greater than a predetermined baseline value.

6. A workpiece inspection system, comprising: A three-dimensional sensor that measures the three-dimensional shape of objects within its field of view; and An information processing device sets the visually recognizable shapes of a 3D model from various viewpoints as the search target shapes, searches for matching portions of the 3D shapes measured by the 3D sensor, and outputs the found matching portions as the search results. in, The information processing device has a result exclusion processing unit, which excludes from the search results any matching partial shapes that are consistent with the search target shape, where the projected area of ​​the planar region of the matching partial shape is larger in proportion to the overall projected area of ​​the matching partial shape, and the consistency of its outline region with the outline region of the search target shape does not reach a benchmark value.

7. The workpiece inspection system according to claim 5 or 6, wherein, The proportion of the projected area of ​​the planar region of the uniform partial shape is calculated only for the largest plane.

8. The workpiece inspection system according to claim 5 or 6, wherein, The result exclusion processing unit is configured to output information about the shape of the consistent portion that has been excluded from the search results.

9. A workpiece inspection system, comprising: A three-dimensional sensor that measures the three-dimensional shape of objects within its field of view; and An information processing device sets the visually recognizable shapes of a 3D model from various viewpoints as the search target shapes, searches for matching portions of the 3D shapes measured by the 3D sensor, and outputs the found matching portions as the search results. in, The information processing device includes a unit shape exclusion processing unit, which excludes a portion of the unit shapes from the visually recognizable shapes that are the search target shapes when the shape of the three-dimensional model has multiple unit shapes that are mutually symmetrical about an axis of symmetry. The unit shape exclusion processing unit is configured so that the user can specify the start and end points of the unit shape using two planes containing the axis of symmetry.

Images