Position offset detection system

Abstract

The invention relates to a position offset detection system, which can improve the detection precision of the position offset of a camera. A system for detecting positional displacement of a camera in an inspection using the camera, the system comprising: a determination unit for determining whether the contour shape of a mark image is a circle or an ellipse in a captured image of the camera of a mark having a circular contour shape; and a detection unit that, when the contour shape of the mark image is an ellipse, uses the positions of the two focal points of the ellipse for matrix calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to a position shift detection system. BACKGROUND

[0002] For example, in Patent Literature 1, a technique is disclosed in which a position shift of a camera is detected using a marker image in which a marker having a circular outline shape becomes an elliptical shape when the marker is imaged from a tilt direction.

[0003] [Patent Literature]

[0004] [Patent Literature]

[0005] [Patent Literature 1] Japanese Patent Application Laid-Open (kokai) No. 07-098208 SUMMARY

[0006] [Problems to be Solved by the Invention]

[0007] However, the technique of Patent Literature 1 has room for improvement in terms of detection accuracy of a position shift of a camera.

[0008] [Means for Solving the Problems]

[0009] To solve the above problems, a position shift detection system according to an embodiment is a position shift detection system used in inspection of a camera, in which there are a judging section that judges whether a marker image has a circular or elliptical outline shape in an imaged image of a marker having a circular outline shape, and a detecting section that uses positions of two foci of the elliptical marker image for matrix calculation in a case where the marker image has an elliptical outline shape.

[0010] [Effects of the Invention]

[0011] According to the position shift detection system according to an embodiment, it is possible to improve detection accuracy of a position shift of a camera. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a schematic view showing an example of a configuration of a position shift detection system according to an embodiment.

[0013] Figure 2 is a block diagram showing an example of a functional configuration of a position shift detection system according to an embodiment.

[0014] Figure 3 is a view showing an example of an outline shape of a marker image;

[0015] Figure 4 is a view showing an example of a position shift of a marker image.

[0016] Figure 5is a diagram showing an example of distortion.

[0017] Figure 6 is a flowchart showing an example of processing of the position shift detection system according to an embodiment. DETAILED DESCRIPTION

[0018] An embodiment of the present application will be described below with reference to the drawings. In the present specification and the drawings, structural elements having substantially the same function are denoted with the same reference numerals, and repeated description is omitted. In the drawings shown below, XYZ orthogonal coordinate system is sometimes used as a directional representation.

[0019] Structure of the position shift detection system according to an embodiment

[0020] Figure 1 is a diagram showing an example of the structure of the position shift detection system 1 according to an embodiment. Figure 2 is a block diagram showing an example of the functional structure of the position shift detection system 1. Figure 3 is a diagram showing an example of the outline shape of the marker image 30. Figure 4 is a diagram showing an example of the position shift of the marker 3. Figure 5 is a diagram showing an example of distortion.

[0021] The position shift detection system 1 is a system that detects the position shift of the camera 2 in an inspection using the camera 2. For example, in the inspection using the camera 2, the camera 2 is disposed on an industrial robot. The camera 2 is moved in a three-dimensional space with six degrees of freedom by the industrial robot. The six degrees of freedom include three degrees of freedom in which the camera 2 moves in the directions along the X axis, the Y axis, and the Z axis, respectively, and three degrees of freedom in which the camera 2 rotates with the X axis, the Y axis, and the Z axis as the centers of rotation, respectively. The camera 2 stops at a prescribed shooting position, and an object is photographed in a stopped state. Based on the photographed image of the object by the camera 2, the inspection of the object is performed. The object is, for example, a member processed by a casting method.

[0022] In the inspection using the camera 2, when the camera 2 is moved and stopped at a prescribed shooting position as a target position, if the position of the camera 2 is shifted with respect to the prescribed shooting position, the photographed image of the camera 2 is shifted depending on the position shift. The shift of the photographed image leads to a decrease in inspection accuracy and the like. In a case where machine learning is used in the inspection using the camera 2, the shift of the photographed image also leads to an increase in learning variation and the like. The position shift detection system 1 is used to reduce the position shift of the camera 2, and to achieve an improvement in inspection accuracy, a reduction in learning variation in machine learning, and the like.

[0023] As Figure 1As shown, the position offset detection system 1 is communicatively connected to the camera 2 via wired or wireless means. The camera 2 has an image-capturing unit 22, such as a lens 21 and a CCD that captures the image of the lens 21. The position offset detection system 1 is input with an image Im of the marker 3 captured by the camera 2.

[0024] Marker 3 is a graphic used to detect the positional offset of camera 2. Figure 1 The markers 3 shown are four markers, each with a circular outline. However, the number of markers 3 is not limited to four; there can be two or more. The markers 3 are arranged by pasting printed materials with images of the markers 3 onto a surface 5 at a designated correction position, or by engraving the markers 3, etc. The surface 5 can be a surface such as a ground, wall, or workbench. The designated correction position can also be the same as the designated shooting position. The camera 2 photographs the markers 3 arranged at the designated correction position.

[0025] The position offset detection system 1 is an electronic circuit such as a central processing unit (CPU), a field-programmable gate array (FPGA), and an application-specific integrated circuit (ASIC).

[0026] like Figure 2 As shown, the position offset detection system 1 includes a judgment unit 12, which determines whether the outline shape of the marker image 30 in the image Im captured by the camera 2, which has a circular outline shape, is a circle or an ellipse. Furthermore, the position offset detection system 1 includes a detection unit 13, which, when the outline shape of the marker image 30 is an ellipse, uses the positions of the two foci 32 of the ellipse for matrix calculation. And, in Figure 2 In the example shown, the positional offset detection system 1 includes a correction unit 11 that corrects distortion in the captured image Im, and an output unit 14 that outputs the detection result of the detection unit 13 to a system or device other than the positional offset detection system 1. Furthermore, in the following description, for the sake of simplicity, the system or device other than the positional offset detection system 1 will be referred to as an external device.

[0027] The position offset detection system 1 executes command codes stored in memory or performs various processes through circuit design for special purposes, thereby realizing the functions of the correction unit 11, the judgment unit 12, the detection unit 13, and the output unit 14. However, some of the functions of the position offset detection system 1 can be implemented by external devices such as PCs (Personal Computers) or servers, or by decentralized processing of the position offset detection system 1 and external devices.

[0028] exist Figure 3In the present embodiment, the marker image 30a is a marker image 30 of which the outline shape is a circle. The marker image 30b is a marker image of which the outline shape is an ellipse. In addition, the marker image 30 is a collective marker of the marker image 30a and the marker image 30b. Therefore, in the present embodiment, the marker image 30 is a marker image of which the outline shape is a circle or an ellipse. Figure 3 In the present embodiment, the symbol of the marker image 30 is shown together with the symbol of each of the marker image 30a and the marker image 30b.

[0029] In a case where the optical axis of the lens 21 is substantially orthogonal with respect to the arrangement surface 5 on which the marker 3 is arranged, in other words, in a case where the camera 2 is not rotated with the X-axis and the Y-axis as the centers of rotation, an image of a circle such as the marker image 30a is obtained in the captured image Im. On the other hand, in a case where the camera 2 is rotated with at least one of the X-axis and the Y-axis as the center of rotation, in a case where the optical axis of the lens 21 is not perpendicular to the arrangement surface 5, an image of an ellipse such as the marker image 30b is obtained in the captured image Im. The center of gravity 31 is the center of gravity of each of the marker image 30a and the marker image 30b. The foci 32 are the two foci of the marker image 30b.

[0030] For example, if only the position of the center of gravity 31 of the marker image 30 is used, the position shift detection system is able to detect the position shift in the plane of the camera 2, but is unable to detect the position shift in the three-dimensional space of the camera 2. The position shift on the plane of the camera 2 is, for example, a shift in the position in the directions along the X-axis, the Y-axis, and the Z-axis, respectively. The position shift in the three-dimensional space of the camera 2 is, for example, a shift in the position of the camera 2 in which the X-axis, the Y-axis, and the Z-axis are the centers of rotation, respectively. Since the position shift of the camera 2 in the three-dimensional space cannot be detected, the detection accuracy of the position shift of the camera 2 sometimes becomes low.

[0031] The position shift detection system 1 determines, by the determination section 12, whether the outline shape of the marker image 30 is a circle or an ellipse, and in a case where the outline shape of the marker image 30 is an ellipse, the positions of the two foci 32 of the ellipse are used for the matrix operation by the detection section 13. Thus, the position shift detection system 1 is able to detect the position shift of the camera 2 in the three-dimensional space in addition to the position shift on the plane of the camera 2, and is able to improve the detection accuracy of the position shift of the camera 2.

[0032] In another aspect, the position shift detection system 1 is able to detect the position shift of the camera 2 in which six degrees of freedom are compounded. In yet another aspect, the position shift detection system 1 is able to detect the position shift of the camera 2 in which six axes including three axes along the X-axis, the Y-axis, and the Z-axis, respectively, and three axes of rotation with the X-axis, the Y-axis, and the Z-axis as the centers of rotation, respectively, are compounded.

[0033] The positional displacement of the camera 2 is reduced based on the detection result of the positional displacement detection system 1, and thus the accuracy of the inspection using the camera 2 is increased. In the case where machine learning is used in the inspection using the camera 2, the learning variation is reduced, and the inspection accuracy of the machine learning is increased.

[0034] The determination section 12 determines whether the positional displacement of the camera 2 in the three-dimensional space has occurred by determining whether the outline shape of the marker image 30 is a circle or an ellipse. For example, the determination section 12 extracts the outline of the marker image 30 by image processing, and detects the maximum displacement amount of the extracted outline shape with respect to a circle. The maximum displacement amount with respect to a circle can be the difference between the maximum width of the outline shape and the diameter of a circle, or the like. The determination section 12 determines that the outline shape of the marker image 30 is a circle in the case where the maximum displacement amount with respect to a circle is less than a predetermined threshold value, and determines that the outline shape of the marker image 30 is an ellipse in the case where the displacement with respect to a circle is equal to or greater than the predetermined threshold value. However, the determination method of the determination section 12 can also be a method other than the above-described method using the maximum displacement amount with respect to a circle.

[0035] When the outline shape of the marker image 30 is an ellipse, the detection section 13 performs matrix calculation based on the ellipse, for example, the positions of the two foci 32 of the marker image 30b. The matrix calculation performs, for example, affine transformation or projective transformation. Affine transformation and projective transformation are coordinate transformations that perform enlargement / reduction, rotation, parallel movement, and the like of an image using a matrix.

[0036] The positional displacement detection of the camera 2 will be described in more detail. Figure 4 The states P1 to P6 corresponding to the six states of the positional displacement of the camera 2 are shown. In Figure 4 The reference image 30S virtually shows the position of an ideal marker image without positional displacement. The marker image 30 shown by a dashed line shows a marker image with positional displacement.

[0037] The state P1 is a state in which the marker image 30 is displaced in position in the X direction and the Y direction, respectively, with respect to the reference image 30S. The state P2 is a state in which the marker image 30 is displaced in position in the X direction and the Z direction, respectively, with respect to the reference image 30S. The state P3 is a state in which the marker image 30 is displaced in position in the Y direction and the Z direction, respectively, with respect to the reference image 30S. In the states P2 and P3, the positional displacement of the camera 2 occurs in the Z direction, and thus the imaging magnification of the lens 21 changes, and the marker image 30 becomes larger with respect to the reference image 30S.

[0038] The state P4 is a state in which the marker image 30 has a rotational positional deviation with respect to the reference image 30S with the Z axis as the center of rotation. The state P5 is a state in which the marker image 30 has a rotational positional deviation with respect to the reference image 30S with the Y axis as the center of rotation. By the rotation with the Y axis as the center of rotation, the outline shape of the marker image 30 becomes an ellipse with the Y direction as the long axis. The state P6 is a state in which the marker image 30 has a rotational positional deviation with respect to the reference image 30S with the X axis as the center of rotation. By the rotation with the X axis as the center of rotation, the outline shape of the marker image 30 becomes an ellipse with the X direction as the long axis.

[0039] For example, the detection section 13 performs matrix calculation based on the marker image 30 in each state from the state PI to the state P6 or a state in which the states from the state PI to the state P6 are combined. Thereby, the detection section 13 can detect the positional deviation of the camera 2 on the plane and the positional deviation of the camera 2 in the three-dimensional space. The detection section 13 outputs the positional deviation detection result of the camera 2 to the external device via the output section 14. In addition, the state of the positional deviation of the camera 2 is not limited to the states PI to P6 shown in the drawing, and can be various states. Figure 4 The state of the positional deviation of the camera 2 is not limited to the states PI to P6 shown in the drawing, and can be various states.

[0040] In a case where the outline shape of the marker image 30 is a circle, the detection section 13 performs matrix calculation for performing affine transformation or affine inverse transformation based on the position of the center of gravity 31 of the marker image 30a, for example, of the circle. Thereby, the detection section 13 can detect the positional deviation of the camera 2 on the plane. In a case where the outline shape of the marker image 30 is a circle, the detection section 13 can also calculate the rotational positional deviation of the camera 2 with the Z axis as the center of rotation. In addition, in a case where the outline shape of the marker image 30 is a circle, the detection section 13 can approximately regard the rotational positional deviation of the camera 2 with the X axis and the Y axis as the center of rotation as zero.

[0041] On the other hand, in the captured image Im of the camera 2, depending on the specifications of the lens 21 and the like, sometimes a distortion occurs Figure 5 as shown in the drawing. In Figure 5 the drawing, the rectangular image 40 shown by the broken line is a captured image without distortion. The barrel-type image 41 shown by the solid line is an image in which the rectangular image 40 is deformed like a barrel due to negative distortion. The roll-type image 42 shown by the solid line is an image in which the rectangular image 40 is deformed like a roll due to positive distortion. For example, the wider the photographic field angle of the lens 21, the greater the distortion. In addition, the distortion can also be referred to as distortion of an image.

[0042] When the outline shape of the marker image 30 is deformed due to distortion, the determination accuracy of whether the outline shape of the marker image 30 is a circle or an ellipse decreases. Due to the decrease in the determination accuracy, the detection accuracy of the positional shift of the camera 2 sometimes decreases. In particular, in a case where a wide-angle lens or a fisheye lens, or the like having large distortion is used as the lens 21, the decrease in the detection accuracy can become significant.

[0043] The positional shift detection system 1 corrects the distortion of the captured image Im by the correction section 11. Further, in the positional shift detection system 1, the determination section 12 determines whether the outline shape of the marker image 30, the outline shape of which is corrected by the correction section 11, is a circle or an ellipse. In addition, the correction section 11 can use various correction algorithms for the correction process of the distortion.

[0044] By using the captured image Im whose distortion is corrected, the determination section 12 can accurately determine whether the outline shape of the marker image 30 is a circle or an ellipse. Thereby, the positional shift on the plane of the camera 2 and the positional shift in the three-dimensional space of the camera 2 can be detected, and the detection accuracy of the positional shift of the camera 2 becomes high.

[0045] In particular, in a case where a wide-angle lens or a fisheye lens, or the like having large distortion is used as the lens 21, the decrease in the detection accuracy is suppressed, and the detection accuracy of the positional shift becomes high. By using a wide-angle lens or a fisheye lens, or the like as the lens 21, even an object having a large uneven shape can be stably captured. As a result, the inspection of an object having a large uneven shape using the camera 2 can be performed with high reliability.

[0046] <Process of the positional shift detection system of one embodiment>

[0047] Figure 6 is a flowchart indicating one example of the process of the positional shift detection system 1 of one embodiment. The positional shift detection system 1 starts the process shown in FIG. 8 with the captured image Im input from the camera 2 as a start condition, and starts the process of step 1. Figure 6 The process shown in FIG. 8 is performed.

[0048] First, in step 1, the positional shift detection system 1 corrects the distortion of the captured image Im input from the camera 2 by the correction section 11.

[0049] Next, in step 2, the positional shift detection system 1 binarizes the captured image Im whose distortion is corrected by the correction section 11 by image processing. The correction section 11 hands over the binarized captured image Im to the determination section 12. Note that the positional shift detection system 1 can perform step 2 by a functional structure other than the correction section 11, such as the determination section 12 or the detection section 13.

[0050] Next, in step 3, the position shift detection system 1 judges, by the judging section 12, whether the outline shape of the marker image 30 in the captured image Im is an ellipse.

[0051] In the case where it is judged to be an ellipse in step 3 (step 3, Yes), the position shift detection system 1 obtains, in step 4, the positions of the two foci 32 in the ellipse of the marker image 30 by the detecting section 13.

[0052] Next, in step 5, the position shift detection system 1 uses, by the detecting section 13, the positions of the two foci 32 of the ellipse for matrix operation. Thereby, the detecting section 13 can detect the position shift in the stereoscopic space of the camera 2 in addition to the position shift in the plane of the camera 2. The detecting section 13 hands over the detection results of the position shift in the plane of the camera 2 and the position shift in the stereoscopic space of the camera 2 to the output section 14.

[0053] On the other hand, in the case where it is judged not to be an ellipse in step 3 (step 3, No), the position shift detection system 1 detects, in step 6, only the position shift on the plane of the camera 2 by the detecting section 13. Specifically, the detecting section 13 calculates the position of the center of gravity 31 of the marker image 30a. The detecting section 13 calculates the movement position shift of the camera 2 in the directions along the X axis, the Y axis, and the Z axis, respectively, based on the calculated position of the center of gravity 31. The detecting section 13 can also calculate the rotational position shift of the camera 2 with the Z axis as the center of rotation. In addition, the detecting section 13 can regard the rotational position shifts of the camera 2 with the X axis and the Y axis as the centers of rotation as approximately zero. The detecting section 13 hands over the detection results of the position shift to the output section 14.

[0054] Next, in step 7, the position shift detection system 1 outputs, by the output section 14, the detection results of the detecting section 13 to an external device. After the detection results are output, the position shift detection system 1 ends the process.

[0055] As described above, the position shift detection system 1 can perform the process of detecting the position shift of the camera 2 in the inspection using the camera 2.

[0056] The above-described embodiment of the present application is not limited to the above-described embodiment of the present application, and various modifications and substitutions can be made to the above-described embodiment of the present application without departing from the scope recited in the claims.

[0057] The numerals, such as ordinal numbers, quantities, and the like, used in the description of the embodiment of the present application are all exemplified for specifically explaining the technology of the present application, and the present application is not limited to the exemplified numerals. In addition, the connection relationship between the constituent elements is exemplified for specifically explaining the technology of the present application, and the connection relationship for realizing the function of the present application is not limited thereto.

[0058] [Reference Signs List]

[0059] 1 Position offset detection system

[0060] 11 Correction unit

[0061] 12 Determination unit

[0062] 13 Detection unit

[0063] 14 Output unit

[0064] 2 Camera

[0065] 21 Lens

[0066] 22 Imaging section

[0067] 3 Mark

[0068] 30, 30a, 30b Mark image

[0069] 30S Reference image

[0070] 31 Center of gravity

[0071] 32 Focal point

[0072] 40 Rectangular image

[0073] 41 Barrel image

[0074] 42 Spiral image

[0075] Im Captured image

[0076] P1, P2, P3, P4, P5, P6 State

Claims

1. A position shift detection system for use in an inspection using a camera, wherein having: a determination section that determines whether the outline shape of a marker image is a circle or an ellipse in a captured image of the camera of a marker whose outline shape is a circle; and a detection section that uses the positions of the two foci of the ellipse for matrix calculation in the case where the outline shape of the marker image is an ellipse.

2. The position shift detection system according to claim 1, further having a correction section that corrects distortion of the captured image, the determination section determining whether the outline shape of the marker image is a circle or an ellipse in the captured image whose distortion is corrected by the correction section.

3. The position shift detection system according to claim 1 or 2, the inspection being an inspection based on machine learning.

Images