Work detection system

Abstract

A workpiece detection system capable of detecting a workpiece accurately and rapidly comprises: a three-dimensional sensor that measures a three-dimensional shape of an object in a field of view; and an information processing device that uses a visually recognizable shape in each point of view of a three-dimensional model as a search target shape, and, when a matching partial shape that matches the search target shape is found from the three-dimensional shape measured by the three-dimensional sensor, outputs the matching partial shape as a search result. The information processing device excludes a part of the search target shape or a part of the search result on the basis of a preset condition.

Description

Technical Field

[0001] The present invention relates to a work detection system.

Background Art

[0002] For example, in a robot handling system that picks up randomly supplied workpieces, a three-dimensional sensor is used to acquire the three-dimensional shape of the target area, and a matching process is performed to identify the matching partial shape whose shape matches the three-dimensional model, thereby identifying the position and orientation of the workpiece. A workpiece detection system is used.

[0003] Such a three-dimensional shape matching process has a very large computational load, and it may take a long time to process without using a high-performance computing device. Therefore, it has been proposed to detect a workpiece by extracting a flat portion from the three-dimensional shape of the target and checking the degree of coincidence between the shape of the extracted flat portion and the shape of the flat surface of the workpiece (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When the features of the search target shape of the three-dimensional model are scarce or inappropriate, an incorrect matching partial shape may be output as the search result. In addition, when an incorrect matching partial shape is calculated as the search result using a search target shape with scarce features or an inappropriate search target shape, this search result cannot be utilized in subsequent processing, and as a result, it will require wasted processing time. In addition, due to single or combined factors such as measurement errors and measurement omissions of the three-dimensional sensor, and the state of the search target shape and its features, the features of the matching partial shape of the search result may be scarce, and in that case, it is often an incorrect search result.

Means for Solving the Problem

[0006] A work detection system according to an aspect of the present disclosure includes a three-dimensional sensor that measures the three-dimensional shape of an object within a field of view, and searches for a visible shape at each viewpoint of a three-dimensional model as a target shape to be searched, and searches for a matching partial shape that matches the target shape to be searched from among the three-dimensional shapes measured by the three-dimensional sensor, and outputs the discovered matching partial shape as a search result. The information processing device excludes a part of the target shape to be searched or a part of the search result based on a preset condition.

Advantages of the Invention

[0007] According to the work detection system according to the present disclosure, a work can be detected accurately and quickly.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Modes for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a work detection system 1 according to the first embodiment of the present disclosure. The work detection system 1 detects the position and orientation of a work, and outputs the position and orientation of the work to, for example, a robot control device or the like.

[0010] The work detection system 1 includes a three-dimensional sensor 10 that measures the three-dimensional shape of an object within the field of view, and an information processing device 20 that searches for a matching partial shape that matches the search target shape, which is the visible shape at each viewpoint of a preset three-dimensional model, among the three-dimensional shapes measured by the three-dimensional sensor 10, and outputs the discovered matching partial shape as a search result.

[0011] The three-dimensional sensor 10 detects the distance to an object existing within the field of view (which may include not only the work but also articles other than the work, a table on which the work or the like is placed, a belt conveyor, etc.) for each two-dimensional position within its field of view, that is, the position in the plane direction perpendicular to the central axis of the measurement range. That is, the three-dimensional sensor 10 measures the three-dimensional shape of the surface on the visible side of the object within the field of view from the three-dimensional sensor 10. As the three-dimensional sensor 10, for example, a stereo camera, a laser scanner, etc. can be used. The measured three-dimensional shape is transmitted to the information processing device 20 as information capable of creating a three-dimensional image such as distance image data, point cloud data, etc.

[0012] The information processing device 20 excludes a part of the search target shape or a part of the search result based on preset conditions. Examples of the conditions regarding the exclusion of a part of the search target shape or a part of the search result include, for example, the size of the planar region in the visible shape of the three-dimensional model or the matching partial shape of the measurement result, and the objectivity of the shape of the three-dimensional model.

[0013] The information processing device 20 can be realized by, for example, causing a computer device having a memory, a processor (CPU), an input / output interface, etc. to execute an appropriate information processing program. The information processing device 20 may further include a display device such as a display panel, an input device such as a keyboard, a touch pad, etc., and may be configured to enable information display using an external display device and information input using an external input device via the input / output interface. Also, the information processing device 20 may be integrally configured with a computer device that manages the entire facility such as a handling system, a processing system, etc. that uses the work detection system 1 or a computer device that controls other components within the facility.

[0014] Specifically, the information processing apparatus 20 may be configured to include a three-dimensional model setting unit 21, a three-dimensional shape acquisition unit 22, a matching processing unit 23, a detection result output unit 24, a viewpoint exclusion processing unit 25, a unit shape exclusion processing unit 26, and a result exclusion processing unit 27. These components classify the functions of the information processing apparatus 20, and do not necessarily have to be clearly distinguishable in terms of physical configuration and program configuration.

[0015] The three-dimensional model setting unit 21 sets a three-dimensional model that specifies the three-dimensional shape of the workpiece to be detected. The three-dimensional model of the workpiece can be specified by, for example, CAD data or the like. Therefore, the three-dimensional model setting unit 21 may be configured to acquire the information of the three-dimensional model from CAD via communication or a recording medium. Further, the three-dimensional model setting unit 21 may have a function of modeling or modifying the three-dimensional model.

[0016] The three-dimensional shape acquisition unit 22 acquires the data of the three-dimensional shape of the object within the field of view of the three-dimensional sensor 10 measured by the three-dimensional sensor 10.

[0017] The matching processing unit 23 searches for a matching partial shape that matches the search target shape, that is, the visible shape at each viewpoint of the three-dimensional model of the workpiece set in the three-dimensional model setting unit 21, that is, the shape of the region visible from the viewpoint among the surface shapes of the three-dimensional model, and performs a matching process to specify the position and orientation of the matching partial shape that matches the search target shape. As an example, when viewed from the radial direction of a cylindrical three-dimensional model, the visible shape is a half-pipe-shaped curved surface.

[0018] The matching process can be performed by a method of repeatedly calculating the degree of match with the search target shape for a matching partial shape of a predetermined size of the three-dimensional shape acquired by the three-dimensional shape acquisition unit 22, and shifting the position little by little so as to scan three-dimensionally and determining that it matches the search target shape when the degree of match exceeds a threshold value. When the matching processing unit 23 discovers a matching partial shape that matches the visible shape by the search processing, it may accurately specify the position and orientation of the matching partial shape using, for example, an ICP (Iterative Closest Point) algorithm or the like.

[0019] The detection result output unit 24 outputs the position and orientation of the matching partial shape determined to match the search target shape by the matching processing unit 23 to a robot control device or the like as a search result. For one three-dimensional shape acquired by the three-dimensional shape acquisition unit 22, the search result may include a plurality of matching partial shapes. Further, the search result may include information indicating that no matching partial shape was found. The detection result output unit 24 is not limited to immediately outputting the search result when the search result is obtained, and may store the search result and output the search result in response to a request.

[0020] The viewpoint exclusion processing unit 25 excludes the visible shape at a viewpoint where the ratio of the projected area of the planar region to the total projected area of the visible shape is greater than a predetermined reference value from the search target shape. The planar region is not limited to a perfect plane, and can be, for example, a region where the separation distance from a virtual plane is equal to or less than a set value, a region where the curvature is equal to or greater than a set value, and the like.

[0021] As an example, consider the case where the three-dimensional model of the workpiece modeled in the XYZ space (orthogonal coordinate space) shown in FIG. 2 is cylindrical and extends in the Z direction. When the three-dimensional model is viewed from the Z-axis direction, as shown in FIG. 3, only the end face, which is a plane, is visible. In this case, the ratio of the projected area of the plane region to the total projected area of the visible shape is 100%. Therefore, the visible shape at this viewing point is excluded from the search target shape. Also, as shown in FIG. 4, when the three-dimensional model is viewed from the X direction or any direction parallel to the XY plane, the end face cannot be seen and only the cylindrical surface is visible, so the ratio of the projected area of the region that is a complete plane is 0% (the projected area of the plane region defined by the distance from the virtual plane described above can be a non-zero value). When the three-dimensional model is viewed from a direction inclined with respect to the XY plane, the ratio of the projected area of the plane region becomes a value between 0% and 100% depending on the inclination angle. The viewing point exclusion processing unit 25 may be configured to exclude from the search target shape the visible shape at a viewing point where the ratio of the projected area of the plane region is equal to or greater than a reference value, that is, at a viewing point where the inclination angle with respect to the XY plane in the line-of-sight direction is equal to or greater than a certain angle.

[0022] The viewing point exclusion processing unit 25 may calculate the ratio of the projected area of the plane region in the visible shape of the three-dimensional model only for the largest plane. When the visible shape has a plurality of plane regions, it is extremely unlikely that there are a plurality of plane regions due to measurement errors of the three-dimensional sensor 10 and that the positional relationship of these plurality of plane regions matches the positional relationship of the plurality of plane regions of the visible shape. Therefore, by excluding only the visible shapes for which the ratio of the projected area of the largest plane region is equal to or greater than the reference value from the search target shape, the processing speed can be improved without unnecessarily reducing the probability (detection rate) of detecting an actual workpiece.

[0023] The viewpoint exclusion processing unit 25 is preferably configured to provide an interface that allows a user to arbitrarily set a reference value for the ratio of the projected area of the planar region, a set value for an index for determining a plane (such as an allowable separation distance from the lower plane), and the like. In this case, in order to make it easier for the user to make appropriate settings, the viewpoint exclusion processing unit 25 is configured to be able to output information about the visible shape to be excluded from the search target shape. More preferably, for example, an image, the coordinate position of the viewpoint, the ratio of the visible shape to be excluded, etc. can be displayed on a display or the like.

[0024] In addition to the ratio of the projected area of the planar region being larger than a predetermined reference value, the viewpoint exclusion processing unit 25 may be configured to exclude only those with unclear contours of the visible shape of the three-dimensional model at that viewpoint from the search target shape. The clarity of the contour of the visible shape can be determined based on predetermined indices such as the curvature at the outer edge of the visible shape and the change in the line-of-sight direction position near the outer edge of the visible shape. When the clarity of the contour of the visible shape is high, the measurement error in the portion corresponding to the outer edge of the visible shape in the measurement result of the three-dimensional sensor 10 becomes small, so that the threshold value of the degree of coincidence in the matching processing unit 23 can be set high. As a result, even if the degree of coincidence becomes high due to a measurement error or the like in the planar region, it can be prevented from being erroneously detected as matching the search target shape, so it may not be excluded from the search target shape.

[0025] For example, when the three-dimensional model of the workpiece is an exact cube, even only the largest planar area has a projected area ratio of 1 / 3 or more. However, since the outer edge of the visible shape is the ridge line between the surfaces, the curvature in the vicinity of the outer edge of the visible shape is infinite, and the change rate of the line-of-sight direction position in the vicinity of the outer edge of the visible shape is constant. Such a visible shape can be evaluated as having a clear contour, so the possibility of false detection is small even if it is not excluded from the search target shape. On the other hand, when the three-dimensional model of the workpiece is a shape in which the ridge lines and corners of the cube are rounded and chamfered, if the curvature of the chamfering becomes small, the measurement error in the portion corresponding to the outer edge of the visible shape may increase. Therefore, it is not preferable to set a high threshold for the degree of coincidence. In such a three-dimensional model, the curvature at the outer edge of the visible shape becomes a certain small value, and the change rate of the line-of-sight direction position in the vicinity of the outer edge of the visible shape changes (the second derivative value becomes non-zero). Therefore, based on an index obtained by quantifying the features of the outer edge or the vicinity of the outer edge of such a visible shape, a visible shape with an unclear contour is identified, and at the same time, in addition to excluding only those with a large projected area ratio of the planar area from the search target shape, by adding them to the objects to be excluded, it is possible to achieve both rapid detection processing and ensuring the detection rate and suppressing false detection.

[0026] When the shape of the three-dimensional model has a plurality of unit shapes around the axis of symmetry, the unit shape exclusion processing unit 26 excludes some of the unit shapes from the visible shape of the search target shape. Typically, only one of the plurality of symmetric unit shapes arranged around the axis of symmetry is left, and the other unit shapes are excluded. In this way, by using the search target shape in which the unit shapes are excluded from the visible shape to compress the amount of information, the computational load for finding the matching partial shape that matches the search target shape can be suppressed. Also, by deleting the overlapping unit shapes, it is possible to prevent the same shape portion from being recognized as overlapping with a plurality of visible shapes with different starting points.

[0027] The unit shape exclusion processing unit 26 is preferably configured such that the user can specify the start point and the end point of the unit shape by two planes including the axis of symmetry respectively. Specifically, the unit shape exclusion processing unit 26 displays the overall shape of the three-dimensional model and also displays two planes indicating the start point and the end point of the unit shape, and can be configured to provide a user interface that allows the user to rotate each plane around the axis of symmetry by, for example, a drag operation or the like. When the three-dimensional model has a rotational body shape such as a cylinder, it can be divided into any number of unit shapes. Therefore, by appropriately selecting the size of the unit shape, it is possible to quickly detect a matching partial shape that matches the search target shape while ensuring sufficient detection accuracy.

[0028] The result exclusion processing unit 27 excludes from the search results a matching partial shape in which the ratio of the projected area of the planar region to the projected area of the entire matching partial shape is greater than a predetermined reference value among the matching partial shapes that match the search target shape. That is, the result exclusion processing unit 27 rechecks the shape of the matching part determined by the matching processing unit 23 to match the search target shape, and excludes it from the search results if the projected area ratio of the planar region is large. For this reason, the detection result output unit 34 outputs only the search results not excluded by the result exclusion processing unit 27 as the final detection results.

[0029] As an example, when there is a prismatic object, even when the matching processing unit 23 determines that it matches the visible shape of a cylindrical three-dimensional model because the three-dimensional sensor 10 cannot correctly measure the shape of the object, the projected area ratio of the planar region can be significantly larger than when measuring a cylindrical workpiece. Therefore, false detection can be prevented by excluding from the search results a matching partial shape with a large projected area ratio of the planar region by the result exclusion processing unit 27. Note that also in the result exclusion processing unit 27, similar to the viewpoint exclusion processing unit 25, the ratio of the projected area of the planar region may be calculated only for the plane with the largest area.

[0030] The result exclusion processing unit 27 may exclude from the search results only those matching partial shapes with a large projected area of the planar region whose degree of match with the search target shape of its contour or contour region does not meet the reference value. Even if the ratio of the planar region is large, if the degree of match of the contour region is high, it is considered highly likely that the workpiece actually exists. The contour region can be set as appropriate. For example, it can be a region within a certain distance range from a position corresponding to the outer edge of the search target shape.

[0031] The result exclusion processing unit 27 may be configured to provide an interface through which the user can set the ratio of the projected area of the planar region of the matching partial shape to be excluded from the search results, the degree of match of the contour region, etc. Further, the result exclusion processing unit 27 is preferably configured to be able to provide an interface for outputting information about the matching partial shape excluded from the search results, typically for display on a screen. By allowing the user to confirm the matching partial shape actually excluded, the setting values for exclusion from the search results can be easily optimized.

[0032] As described above, the embodiments of the present disclosure have been explained. However, the present invention is not limited to the above-described embodiments. Also, the effects described in the above-described embodiments are merely an enumeration of the most suitable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the above-described embodiments.

[0033] In the workpiece detection system according to the present disclosure, the information processing apparatus only needs to be able to exclude a part of the search target shape or a part of the search results based on preset conditions, and may not have some or all of the viewpoint exclusion processing unit, the unit shape exclusion processing unit, and the result exclusion processing unit in the above-described embodiments.

Explanation of Reference Numerals

[0034] 1 Workpiece detection system 10 Three-dimensional sensor 20 Information processing apparatus 21 Three-dimensional model setting unit 22 Three-dimensional shape acquisition unit 23 Matching Processing Unit 24 Detection Result Output Unit 25 Viewpoint Exclusion Processing Unit 26 Unit Shape Exclusion Processing Unit 27 Result Exclusion Processing Unit

Claims

1. A three-dimensional sensor for measuring the three-dimensional shape of an object within a field of view, An information processing apparatus that searches for one or more visible shapes at each viewpoint of a three-dimensional model as search target shapes, searches for matching partial shapes that match the search target shapes from among the three-dimensional shapes measured by the three-dimensional sensor, and outputs the discovered one or more matching partial shapes as search results, Comprising: The information processing apparatus excludes, from the search target shape at that viewpoint, the visible shape among the one or more visible shapes whose ratio of the projected area of the planar region of the visible shape to the projected area of the visible shape on a plane perpendicular to the line-of-sight direction of the entire one or more visible shapes is greater than a predetermined reference value. A work detection system.

2. A three-dimensional sensor for measuring the three-dimensional shape of an object within a field of view, An information processing apparatus that searches for one or more visible shapes at each viewpoint of a three-dimensional model as search target shapes, searches for matching partial shapes that match the search target shapes from among the three-dimensional shapes measured by the three-dimensional sensor, and outputs the discovered one or more matching partial shapes as search results, Comprising: The information processing apparatus excludes, from the search target shape at that viewpoint, the visible shape among the one or more visible shapes whose ratio of the projected area of the planar region of the visible shape to the projected area of the visible shape on a plane perpendicular to the line-of-sight direction of the entire one or more visible shapes is greater than a predetermined reference value, and whose contour is determined to be unclear based on at least one of a curvature at the outer edge of the visible shape and a change in the line-of-sight direction position near the outer edge of the visible shape according to a predetermined index. A work detection system.

3. The ratio of the projected area of the planar region of the visible shape to the projected area of the visible shape on a plane perpendicular to the line-of-sight direction is calculated only for the planar region having the largest area among the one or more planar regions. The work detection system according to Claim 1 or 2.

4. The information processing apparatus is configured to be able to output information about the visible shape to be excluded from the search target shape. The work detection system according to any one of Claims 1 to 3.

5. A three-dimensional sensor for measuring the three-dimensional shape of an object within a field of view, Using one or more visible shapes at each viewpoint of the three-dimensional model as the search target shape, searching for a matching partial shape that matches the search target shape from among the three-dimensional shapes measured by the three-dimensional sensor, and outputting the discovered one or more matching partial shapes as search results, an information processing apparatus; comprising; The information processing apparatus has a result exclusion processing unit that excludes, from the search results at that viewpoint, a matching partial shape among the one or more matching partial shapes that match the search target shape, where the ratio of the projected area of the planar region of the matching partial shape onto a plane perpendicular to the line-of-sight direction of the entire one or more matching partial shapes is greater than a predetermined reference value. A work detection system.

6. A three-dimensional sensor that measures the three-dimensional shape of an object within the field of view, Using one or more visible shapes at each viewpoint of the three-dimensional model as the search target shape, searching for a matching partial shape that matches the search target shape from among the three-dimensional shapes measured by the three-dimensional sensor, and outputting the discovered one or more matching partial shapes as search results, an information processing apparatus; comprising; The information processing apparatus has a result exclusion processing unit that excludes, from the search results at that viewpoint, a matching partial shape among the one or more matching partial shapes that match the search target shape, where the ratio of the projected area of the planar region of the matching partial shape onto a plane perpendicular to the line-of-sight direction of the entire one or more matching partial shapes is greater than a predetermined reference value and the degree of coincidence of its contour region with the search target shape is less than the reference value. A work detection system.

7. The ratio of the projected area of the planar region of the matching partial shape onto a plane perpendicular to the line-of-sight direction is calculated only for the planar region with the largest area. The work detection system according to claim 5 or 6.

8. The result exclusion processing unit is configured to be able to output information about the matching partial shape excluded from the search results. The work detection system according to any one of claims 5 to 7.

9. A three-dimensional sensor that measures the three-dimensional shape of an object within the field of view, Using one or more visible shapes at each viewpoint of the three-dimensional model as the search target shape, searching for a matching partial shape that matches the search target shape from among the three-dimensional shapes measured by the three-dimensional sensor, and outputting the discovered one or more matching partial shapes as search results, an information processing apparatus; comprising; when the shape of the three-dimensional model has a plurality of unit shapes around the axis of symmetry, the information processing apparatus has a unit shape exclusion processing unit that excludes other unit shapes from the visible shapes of the search target shape, leaving only one of the plurality of unit shapes that are symmetric to each other. A workpiece detection system.

10. The unit shape exclusion processing unit displays the overall shape of the three-dimensional model, and can display two planes including the axis of symmetry rotatably around the axis of symmetry. The intersection of one of the two planes and the three-dimensional model is the starting point of the unit shape, and the intersection of the other of the two planes and the three-dimensional model is the end point of the unit shape. The workpiece detection system according to claim 9, configured as such.

11. A matching processing unit that uses one or more visible shapes at each viewpoint of the three-dimensional model as the search target shape and searches for a matching partial shape that matches the search target shape from among the three-dimensional shapes measured by the three-dimensional sensor; a viewpoint exclusion processing unit that excludes, from the search target shape at that viewpoint, a visible shape among the one or more visible shapes whose ratio of the projected area of the visible shape on a plane perpendicular to the line-of-sight direction of the one or more visible shapes to the projected area of the visible shape on a plane perpendicular to the line-of-sight direction is greater than a predetermined reference value; a detection result output unit that outputs the discovered one or more matching partial shapes as search results; An information processing apparatus comprising.

Images