Positional deviation detection system

Abstract

To improve detection accuracy of positional deviation of a camera.SOLUTION: A positional deviation detection system of a camera in inspection using the camera comprises: a determination unit that determines which of a circle or an ellipse a contour shape of a marker image in an image imaged by the camera of a marker, in which the contour shape is circle, is; and a detection unit that utilizes positions of two focal points of the ellipse in matrix calculation when the contour shape of the marker image is the ellipse.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a positional deviation detection system. [Background technology]

[0002] For example, Patent Document 1 discloses a technique for detecting a positional deviation of a camera by using a marker image with an elliptical contour obtained when a marker with a circular contour is photographed from an oblique direction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 07-098208 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology of Patent Document 1 leaves room for improvement in the accuracy of detecting camera positional deviation. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, one embodiment of a positional misalignment detection system is a system for detecting positional misalignment of a camera in an inspection using the camera, and includes a judgment unit that determines whether the outline shape of a marker image captured by the camera of a marker with a circular outline shape is a circle or an ellipse, and a detection unit that, if the outline shape of the marker image is an ellipse, uses the positions of the two foci of the ellipse for matrix calculation. [Effects of the Invention]

[0006] According to the positional deviation detection system of one embodiment, it is possible to improve the accuracy of detecting the positional deviation of the camera. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of a positional deviation detection system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of a functional configuration of a positional deviation detection system according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a contour shape of a marker image. [Figure 4] FIG. 10 is a diagram illustrating an example of positional deviation of a marker image. [Figure 5] FIG. 10 is a diagram illustrating an example of distortion. [Figure 6] 10 is a flowchart illustrating an example of processing performed by a positional deviation detection system according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] An embodiment of the present invention will be described below with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, and redundant description will be omitted. In the drawings shown below, an XYZ Cartesian coordinate system may be used to represent directions.

[0009] <Configuration of positional deviation detection system according to one embodiment> Fig. 1 is a schematic diagram showing an example of the configuration of a positional deviation detection system 1 according to an embodiment. Fig. 2 is a block diagram showing an example of the functional configuration of the positional deviation detection system 1. Fig. 3 is a diagram showing an example of the contour shape of a marker image 30. Fig. 4 is a diagram showing an example of positional deviation of a marker 3. Fig. 5 is a diagram showing an example of distortion.

[0010] The positional deviation detection system 1 is a system that detects positional deviation of the camera 2 during inspection using the camera 2. For example, in inspection using the camera 2, the camera 2 is mounted on an industrial robot. The industrial robot moves the camera 2 with six degrees of freedom in three-dimensional space. The six degrees of freedom include three degrees of freedom in which the camera 2 moves in directions along the X-axis, Y-axis, and Z-axis, respectively, and three degrees of freedom in which the camera 2 rotates around each of the X-axis, Y-axis, and Z-axis. The camera 2 stops at a predetermined shooting position and captures an image of the object while stopped. The object is inspected based on the image of the object captured by the camera 2. The object is, for example, a part processed by a casting method.

[0011] In an inspection using a camera 2, if the camera 2 moves and stops at a predetermined shooting position that is a target position and the position of the camera 2 is deviated from the predetermined shooting position, a deviation occurs in the image captured by the camera 2 in accordance with this positional deviation. The deviation in the captured image leads to a decrease in inspection accuracy, etc. When machine learning is used in an inspection using the camera 2, the deviation in the captured image also leads to an increase in learning variations, etc. The positional deviation detection system 1 is used to reduce the positional deviation of the camera 2, thereby improving inspection accuracy and reducing learning variations in machine learning, etc.

[0012] 1, the positional displacement detection system 1 is communicably connected to a camera 2 via a wired or wireless connection. The camera 2 has a lens 21 and an imaging unit 22 such as a CCD that captures an image through the lens 21. The positional displacement detection system 1 receives an image Im of a marker 3 captured by the camera 2 as an input.

[0013] The markers 3 are graphics used to detect positional deviation of the camera 2. The markers 3 shown in FIG. 1 are four markers 3, each with a circular outline. However, the number of markers 3 is not limited to four and may be two or more. The markers 3 are placed on the placement surface 5 at a predetermined calibration position by, for example, attaching a printed matter on which an image of the marker 3 is printed, or by imprinting the marker 3. The placement surface 5 may be a floor, a wall, or a surface on a workbench. The predetermined calibration position may be the same as the predetermined shooting position. The camera 2 captures the markers 3 placed at the predetermined calibration position.

[0014] The positional deviation detection system 1 is an electronic circuit such as a central processing unit (CPU), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0015] As shown in FIG. 2, the positional displacement detection system 1 includes a determination unit 12 that determines whether the contour shape of a marker image 30 in an image Im captured by a camera 2 of a marker 3 with a circular contour shape is a circle or an ellipse. The positional displacement detection system 1 also includes a detection unit 13 that, when the contour shape of the marker image 30 is an ellipse, uses the positions of two focal points 32 of the ellipse for matrix calculation. Furthermore, in the example shown in FIG. 2, the positional displacement detection system 1 also includes a correction unit 11 that corrects distortion in the captured image Im, and an output unit 14 that outputs the detection result of the detection unit 13 to a system or device other than the positional displacement detection system 1. In the following description, for simplicity, systems or devices other than the positional displacement detection system 1 will be referred to as external devices.

[0016] The positional deviation detection system 1 executes various processes by executing instruction codes stored in a memory or by being designed as a circuit for a special purpose, thereby realizing the functions of the correction unit 11, the determination unit 12, the detection unit 13, and the output unit 14. However, some of the above functions of the positional deviation detection system 1 may be realized by an external device such as a PC (Personal Computer) or a server, or may be realized by distributed processing between the positional deviation detection system 1 and the external device.

[0017] In Fig. 3, marker image 30a is a marker image 30 with a circular outline. Marker image 30b is a marker image with an elliptical outline. Note that marker image 30 is a collective notation for marker image 30a and marker image 30b. For this reason, in Fig. 3, the reference numeral for marker image 30 is written alongside the reference numerals for marker image 30a and marker image 30b.

[0018] When the optical axis of the lens 21 is nearly perpendicular to the placement surface 5 on which the marker 3 is placed, in other words, when the camera 2 is not rotating around either the X-axis or the Y-axis, a circular image such as marker image 30a is obtained in the captured image Im. On the other hand, when the camera 2 is rotating around at least one of the X-axis and the Y-axis and the optical axis of the lens 21 is not perpendicular to the placement surface 5, an elliptical image such as marker image 30b is obtained in the captured image Im. The center of gravity 31 is the center of gravity of each of the marker images 30a and 30b. The focal points 32 are the two focal points of the marker image 30b.

[0019] For example, if only the position of the center of gravity 31 of the marker image 30 is used, the positional displacement detection system can detect the positional displacement of the camera 2 on a plane, but cannot detect the positional displacement of the camera 2 in three-dimensional space. The positional displacement of the camera 2 on a plane is, for example, a translational positional displacement in directions along each of the X-axis, Y-axis, and Z-axis. The positional displacement of the camera 2 in three-dimensional space is, for example, a rotational positional displacement of the camera 2 around each of the X-axis, Y-axis, and Z-axis. The inability to detect the positional displacement of the camera 2 in three-dimensional space may result in a decrease in the accuracy of detecting the positional displacement of the camera 2.

[0020] In the positional deviation detection system 1, the determination unit 12 determines whether the outline shape of the marker image 30 is a circle or an ellipse, and if the outline shape of the marker image 30 is an ellipse, the detection unit 13 uses the positions of the two focal points 32 of the ellipse for matrix calculation. This allows the positional deviation detection system 1 to detect not only the positional deviation of the camera 2 on a plane but also the positional deviation of the camera 2 in three-dimensional space, thereby increasing the accuracy of detecting the positional deviation of the camera 2.

[0021] From another perspective, the positional deviation detection system 1 can detect a composite positional deviation of the camera 2 in six degrees of freedom. From yet another perspective, the positional deviation detection system 1 can detect a composite positional deviation of the camera 2 in six axes, including three axes along the X-axis, Y-axis, and Z-axis, respectively, and three axes of rotation about the X-axis, Y-axis, and Z-axis, respectively.

[0022] Based on the detection result by the positional deviation detection system 1, the positional deviation of the camera 2 is reduced, thereby improving the accuracy of the inspection using the camera 2. When machine learning is used for the inspection using the camera 2, the learning variation is reduced and the inspection accuracy by the machine learning is improved.

[0023] The determination unit 12 determines whether the contour shape of the marker image 30 is a circle or an ellipse, thereby determining whether the camera 2 is misaligned in three-dimensional space. For example, the determination unit 12 extracts the contour of the marker image 30 through image processing and detects the maximum deviation of the extracted contour shape from a circle. The maximum deviation from a circle can be determined, for example, by the difference between the maximum width of the contour shape and the diameter of the circle. The determination unit 12 determines that the contour shape of the marker image 30 is a circle if the maximum deviation from a circle is smaller than a predetermined threshold, and determines that the contour shape of the marker image 30 is an ellipse if the deviation from a circle is equal to or greater than the predetermined threshold. However, the determination method used by the determination unit 12 may be a method other than the above method using the maximum deviation from a circle.

[0024] When the contour shape of the marker image 30 is an ellipse, the detection unit 13 performs a matrix calculation based on the positions of the two focal points 32 of the ellipse, for example, the marker image 30b. This matrix calculation is, for example, an affine transformation or an inverse affine transformation. The affine transformation and inverse affine transformation are coordinate transformations that use matrices to perform operations such as scaling, rotation, and translation of an image.

[0025] The detection of positional misalignment of the camera 2 will be explained in more detail. Fig. 4 shows states P1 to P6 corresponding to six states of positional misalignment of the camera 2. In Fig. 4, the reference image 30S virtually represents the ideal position of the marker image without positional misalignment. The marker image 30 indicated by the dashed line represents the misaligned marker image.

[0026] In state P1, the marker image 30 is displaced in both the X and Y directions relative to the reference image 30S. In state P2, the marker image 30 is displaced in both the X and Z directions relative to the reference image 30S. In state P3, the marker image 30 is displaced in both the Y and Z directions relative to the reference image 30S. In states P2 and P3, the camera 2 is displaced in the Z direction, which changes the imaging magnification of the lens 21, and the marker image 30 becomes larger than the reference image 30S.

[0027] In state P4, the marker image 30 is rotated around the Z axis relative to the reference image 30S. In state P5, the marker image 30 is rotated around the Y axis relative to the reference image 30S. Due to the rotation around the Y axis, the contour shape of the marker image 30 becomes an ellipse with the Y direction as the major axis. In state P6, the marker image 30 is rotated around the X axis relative to the reference image 30S. Due to the rotation around the X axis, the contour shape of the marker image 30 becomes an ellipse with the X direction as the major axis.

[0028] The detection unit 13 performs matrix calculations based on the marker images 30, for example, in each state from state P1 to state P6, or in a state where states P1 to P6 are combined. This allows the detection unit 13 to detect the positional displacement of the camera 2 on a plane and the positional displacement of the camera 2 in three-dimensional space. The detection unit 13 outputs the positional displacement detection result of the camera 2 to an external device via the output unit 14. Note that the positional displacement states of the camera 2 are not limited to states P1 to P6 shown in FIG. 4, and may be various states.

[0029] If the contour shape of the marker image 30 is a circle, the detection unit 13 performs matrix calculations to perform an affine transformation or an inverse affine transformation based on the position of the center of gravity 31 of the circle, for example, the marker image 30a. This allows the detection unit 13 to detect a positional deviation on a plane of the camera 2. If the contour shape of the marker image 30 is a circle, the detection unit 13 may calculate a rotational positional deviation of the camera 2 about the Z axis as the center of rotation. Furthermore, if the contour shape of the marker image 30 is a circle, the detection unit 13 can treat the rotational positional deviation of the camera 2 about each of the X axis and the Y axis as being approximately zero.

[0030] On the other hand, the image Im captured by the camera 2 may suffer from distortion as shown in FIG. 5 depending on the specifications of the lens 21, etc. In FIG. 5, the rectangular image 40 indicated by the dashed line is a captured image without distortion. The barrel-shaped image 41 indicated by the solid line is an image in which the rectangular image 40 has been deformed like a barrel due to negative distortion. The pincushion-shaped image 42 indicated by the solid line is an image in which the rectangular image 40 has been deformed like a pincushion due to positive distortion. For example, the wider the angle of view of the lens 21, the greater the distortion. Note that distortion can also be referred to as image distortion.

[0031] When the contour shape of the marker image 30 is deformed by distortion, the accuracy of determining whether the contour shape of the marker image 30 is a circle or an ellipse decreases. This decrease in determination accuracy may decrease the accuracy of detecting positional deviation of the camera 2. In particular, when a wide-angle lens or a fisheye lens, which has large distortion, is used as the lens 21, the decrease in detection accuracy may be significant.

[0032] The positional deviation detection system 1 corrects distortion in the captured image Im using the correction unit 11. Furthermore, the positional deviation detection system 1 determines, using the determination unit 12, whether the contour shape of the marker image 30 in the captured image Im, in which the distortion has been corrected by the correction unit 11, is a circle or an ellipse. Note that the correction unit 11 can use various distortion correction algorithms for the distortion correction process.

[0033] By using the captured image Im in which distortion has been corrected, the determination unit 12 can accurately determine whether the contour shape of the marker image 30 is a circle or an ellipse. This makes it possible to detect the positional deviation of the camera 2 on a plane and in three-dimensional space, thereby improving the accuracy of detecting the positional deviation of the camera 2.

[0034] In particular, when a wide-angle lens, a fisheye lens, or the like that has large distortion is used as the lens 21, the decrease in detection accuracy is suppressed and the accuracy of detecting misalignment is increased. By performing inspection using a wide-angle lens, a fisheye lens, or the like as the lens 21, stable imaging is possible even for objects that have an uneven shape with large height differences. As a result, inspection of objects that have an uneven shape with large height differences using the camera 2 can be realized with high reliability.

[0035] <Processing by the positional deviation detection system according to one embodiment> 6 is a flowchart showing an example of processing by the positional deviation detection system 1 according to an embodiment. The positional deviation detection system 1 starts the processing shown in FIG. 6 when a captured image Im is input from the camera 2, which is a starting condition.

[0036] First, in step 1, the positional deviation detection system 1 causes the correction unit 11 to correct distortion in the captured image Im input from the camera 2.

[0037] Next, in step 2, the positional misalignment detection system 1 binarizes the captured image Im after distortion correction by image processing using the correction unit 11. The correction unit 11 passes the binarized captured image Im to the determination unit 12. Note that the positional misalignment detection system 1 may perform step 2 using a functional configuration other than the correction unit 11, such as the determination unit 12 or the detection unit 13.

[0038] Next, in step 3, the positional displacement detection system 1 determines, by the determination unit 12, whether or not the contour shape of the marker image 30 in the captured image Im is an ellipse.

[0039] In step 3, if it is determined that the marker image 30 is an ellipse (step 3, YES), in step 4, the positional deviation detection system 1 uses the detection unit 13 to determine the positions of the two focal points 32 in the ellipse of the marker image 30.

[0040] Next, in step 5, the positional deviation detection system 1 uses the positions of the two focal points 32 of the ellipse for matrix calculation by the detection unit 13. This allows the detection unit 13 to detect the positional deviation of the camera 2 in three-dimensional space in addition to the positional deviation of the camera 2 on a plane. The detection unit 13 passes the detection results of the positional deviation of the camera 2 on a plane and the positional deviation of the camera 2 in three-dimensional space to the output unit 14.

[0041] On the other hand, if it is determined in step 3 that the marker image 30a is not an ellipse (step 3, NO), in step 6, the positional deviation detection system 1 detects only the positional deviation on the plane of the camera 2 using the detection unit 13. Specifically, the detection unit 13 calculates the position of the center of gravity 31 of the marker image 30a. Based on the calculated position of the center of gravity 31, the detection unit 13 calculates the movement positional deviation of the camera 2 in the directions along each of the X-axis, Y-axis, and Z-axis. The detection unit 13 may also calculate the rotational positional deviation of the camera 2 around the Z-axis as the center of rotation. Furthermore, the detection unit 13 can treat the rotational positional deviation of the camera 2 around each of the X-axis and Y-axis as approximately zero. The detection unit 13 passes the positional deviation detection result to the output unit 14.

[0042] Next, in step 7, the positional deviation detection system 1 outputs the detection result by the detection unit 13 to an external device via the output unit 14. After outputting the detection result, the positional deviation detection system 1 ends the process.

[0043] In this manner, the positional deviation detection system 1 can execute the process of detecting the positional deviation of the camera 2 during inspection using the camera 2.

[0044] Although a preferred embodiment has been described in detail above, the present invention is not limited to the above-described embodiment of the present invention, and various modifications and substitutions can be made to the above-described embodiment of the present invention without departing from the scope of the claims.

[0045] All ordinal numbers, quantitative numbers, and other figures used in the description of one embodiment of the present invention are provided as examples to specifically explain the technology of the present invention, and the present invention is not limited to the illustrated figures. Furthermore, the connection relationships between components are provided as examples to specifically explain the technology of the present invention, and do not limit the connection relationships that realize the functions of the present invention. [Explanation of symbols]

[0046] 1. Position deviation detection system 11 Correction section 12 Judgment Department 13 Detection unit 14 Output section 2 Cameras 21 Lens 22 Imaging unit 3 Markers 30, 30a, 30b Marker images 30S reference image 31 Center of gravity 32 focus 40 Rectangular Images 41 Barrel Image 42 Pincushion Images Im photo P1, P2, P3, P4, P5, P6 states

Claims

1. A system for detecting positional deviation of a camera in an inspection using the camera, a determination unit that determines whether the contour shape of a marker image is circular or elliptical in an image of the marker captured by the camera, the contour shape of the marker being circular; a detection unit that, when the contour shape of the marker image is an ellipse, uses the positions of two foci of the ellipse for matrix calculation.

2. a correction unit that corrects distortion of the captured image, The positional displacement detection system according to claim 1 , wherein the determining unit determines whether the contour shape of the marker image in the captured image in which the distortion has been corrected by the correcting unit is a circle or an ellipse.

3. The misalignment detection system according to claim 1 or 2, wherein the inspection is an inspection based on machine learning.

Images