Calibration method for camera

Abstract

To provide a calibration method for a camera, capable of improving accuracy of calibration of a camera.SOLUTION: A calibration method for a camera 12 includes: an image acquisition step of acquiring a plurality of calibration images capturing a calibration pattern including an ultra-small target T in a defocus state; a pattern information setting step of setting pattern information regarding calibration; a feature point detection step of detecting, as a feature point, a centroid position of a defocused image of the target from a calibration image; and a parameter calculation step of calculating a camera parameter, according to a location of the feature point and the pattern information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a method for calibrating a camera. [Background technology]

[0002] Generally, when a camera is used for object detection, the camera is calibrated. In camera calibration, a planar calibration pattern having multiple feature points aligned two-dimensionally is prepared, and the calibration pattern is photographed from multiple directions by the camera. Then, feature points are detected from the multiple images photographed by the camera, and the camera parameters are estimated using the coordinates of the detected feature points. The camera parameters include the camera's internal parameters (focal length, optical center) or distortion parameters (distortion coefficients).

[0003] For example, Patent Document 1 discloses a method for calibrating a camera using a dot pattern in which a plurality of dots (black circles) are arranged in a lattice as a calibration pattern. In this method, the positions of the centers of gravity of dots included in the dot pattern are detected as feature points on an image (calibration image) captured by the camera, and camera parameters are obtained.

[0004] Patent Document 1 also discloses a grid pattern and a checkered pattern as calibration patterns other than the dot pattern. When these calibration patterns are used, intersections (corners) between straight lines are detected as feature points. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2022-30807 Summary of the Invention [Problem to be solved by the invention]

[0006] When the grid pattern shown in XIA of FIG. 11 or the checker pattern shown in XIB of FIG. 11 among the calibration patterns disclosed in Patent Document 1 is used, the intersection (corner) H1 or H2 of the straight lines of each pattern on the calibration image is detected as a feature point (that is, edge detection). In such edge detection, it is necessary to use the luminance gradient (differential value) of the edge. Therefore, the detection position (coordinates) of the intersection H1 or H2 is easily affected by luminance noise, and there is a problem that the detection accuracy of the feature point is not good compared with the dot pattern.

[0007] On the other hand, when the dot pattern shown in XIC of FIG. 11 is used, since the position of the center of gravity of the dot D of the dot pattern on the calibration image is detected as a feature point, it is less affected by luminance noise compared with the grid pattern or the checker pattern, but there are problems as described below.

[0008] FIGS. 12 and 13 show an example of photographing the dot D on the dot pattern by the camera CAM. In FIG. 12, O c indicates the origin (camera origin) of the camera coordinate system with respect to the camera CAM. Also, in FIGS. 12 and 13, the U axis and the V axis are coordinate axes that constitute an image coordinate system (two-dimensional orthogonal coordinate system) defined on the image plane M of the camera CAM.

[0009] In an image captured using a camera with a normal lens (a non - telecentric optical system), there is a characteristic where the magnification changes depending on the distance from the camera. That is, the farther the distance from the camera, the smaller the resulting image, and the closer the distance, the larger the resulting image. Therefore, as shown in FIG. 12, when the dot D on the dot pattern is photographed by the camera CAM from an oblique direction, as shown in FIG. 13, the image of the dot D (hereinafter referred to as the "dot image") DI on the image captured by the camera CAM has an oval shape distorted in a specific direction according to the positional relationship with the camera CAM. That is, in the dot image DI on the image captured by the camera CAM, the part on the front side (the side with a short distance from the camera CAM, the +V side) is relatively smaller than the part on the back side (the side with a long distance from the camera CAM, the -V side). Therefore, when trying to obtain the dot centroid position two - dimensionally from the image captured by the camera CAM, due to the influence of the non - telecentric optical system, the dot centroid position may be obtained at a position deviated from the original position.

[0010] FIGS. 14 and 15 are diagrams for explaining the influence of the inclination of the camera CAM with respect to the dot pattern (dot D) on the detection result of the dot centroid position.

[0011] FIG. 14 shows a state where the camera CAM is facing the dot pattern (dot D) directly (the optical axis AX of the camera CAM is perpendicular to the dot D, that is, the state where the camera CAM is not inclined with respect to the dot pattern). In this case, since the dot image DI on the image plane M of the camera CAM and the dot D before projection are similar in shape, the dot centroid position G1 on the image plane M (the centroid position obtained two - dimensionally from the shape of the dot image DI on the image plane M) coincides with the actual dot centroid position G0 projected onto the image plane M (the projection point of the centroid position g of the dot D before projection onto the image plane M), and the dot centroid position can be detected without error.

[0012] FIG. 15 shows a state in which the camera CAM is tilted with respect to the dot pattern (dot D). In this case, due to the influence of the non-telecentric optical system, the dot image DI on the image plane M has an asymmetric shape with one side (the left side in FIG. 13) and the other side (the right side in FIG. 13) sandwiching the actual dot center-of-gravity position G0 projected onto the image plane M (the projection point obtained by projecting the center-of-gravity position g of the dot D before projection onto the image plane M). Therefore, the dot center-of-gravity position G1 on the image plane M does not coincide with the actual dot center-of-gravity position G0 projected onto the image plane M, resulting in a detection error in the dot center-of-gravity position.

[0013] The occurrence of such a detection error in the dot center-of-gravity position causes a decrease in the calibration accuracy of the camera.

[0014] The present invention has been made in view of such circumstances, and an object thereof is to provide a camera calibration method capable of improving the calibration accuracy of a camera.

Means for Solving the Problems

[0015] The present invention comprises the following aspects in order to achieve the above object.

[0016] The camera calibration method according to the first aspect of the present invention includes an image acquisition step of acquiring a plurality of calibration images obtained by photographing a calibration pattern including a minute target in a defocused state, a pattern information setting step of setting pattern information related to calibration, a feature point detection step of using, as a feature point, the center-of-gravity position of the defocused image of the target from the calibration images, and a parameter calculation step of calculating camera parameters based on the pattern information and the position of the feature point.

[0017] The camera calibration method according to the second aspect of the present invention is, in the first aspect, the size of the target is determined according to at least one of the focal length of the camera that photographs the target, the distance between the camera and the target, and the pixel size of the imaging element of the camera.

[0018] In the calibration method of a camera according to the third aspect of the present invention, in the first aspect, the target is a size such that the size of the image on the imaging element of the camera obtained when shooting in the in-focus state is 1 pixel or less. However, depending on the required tolerance, a target having a size exceeding 1 pixel for the size of the image on the imaging element of the camera obtained when shooting in the in-focus state may be used.

[0019] In the calibration method of a camera according to the fourth aspect of the present invention, in any one of the first aspect to the third aspect, the pattern information includes information regarding the arrangement of the targets in the calibration pattern.

[0020] In the calibration method of a camera according to the fifth aspect of the present invention, in any one of the first aspect to the fourth aspect, the targets are in a shape that is line-symmetric or point-symmetric in two directions orthogonal to each other.

[0021] In the calibration method of a camera according to the sixth aspect of the present invention, in the fifth aspect, the targets are dot-shaped.

[0022] In the calibration method of a camera according to the seventh aspect of the present invention, in the fifth aspect, the targets are point light sources.

Advantages of the Invention

[0023] According to the present invention, the accuracy of camera calibration can be improved.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0025] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0026] 〔Projection Model of Camera〕 First, the projection model of the camera in the present embodiment will be described. FIG. 1 is an explanatory diagram for explaining the projection model of the camera in the present embodiment.

[0027] As shown in FIG. 1, the world coordinate system (global coordinate system) is a coordinate system representing positions in a three-dimensional space (real space), with the origin at O w and with the X w axis, Y w axis, and Z w axis as the coordinate axes, forming a three-dimensional orthogonal coordinate system. Note that any coordinate system may be used as long as it can specify positions (three-dimensional positions) in the three-dimensional space. The camera coordinate system has the center O c of the optical axis of the camera as the origin, with the right direction from the origin O c as the X c axis, the downward direction as the Y c axis, and the optical axis direction as the Z c axis, forming a three-dimensional orthogonal coordinate system. The image coordinate system has the upper left corner of the image plane IP, which is at a focal length f away from the origin O c of the camera coordinate system in the Z c direction, as the origin, and has a two-dimensional orthogonal coordinate system (pixel coordinate system) with the U axis and V axis in directions parallel to the X c axis and Y c axis respectively.

[0028] First, the coordinates (x w , y w , z w ) of a point P (object point) in the three-dimensional space in the world coordinate system can be converted into the coordinates (x, y, z) in the camera coordinate system using the rotation matrix R and translation vector t of the camera as shown in the following equation (1).

[0029]

Equation

[0030] Here, [R|t] is the matrix for converting from the world coordinate system to the camera coordinate system (external parameter matrix), representing the pose and position of the camera in the world coordinate system. Each component r 11 , r 12 , …, r 33 , t x , t y , t z of [R|t] are called the external parameters of the camera.

[0031] Next, when the coordinates (pixel coordinates) of the projection point Q obtained by projecting the point P at (x, y, z) as seen from the camera coordinate system onto the image plane IP are (u, v), the relational expressions shown in the following formulas (2) to (7) hold.

[0032]

Number

[0033]

Number

[0034]

Number

[0035]

Number

[0036]

Number

[0037]

Number

[0038] Here, (x´, y´) represents the coordinates of the projection point obtained by projecting the point P at (x, y, z) as seen from the camera coordinate system onto the normalized image plane (z = 1). Also, (x´´, y´´) represents the coordinates of the projection point (distorted point) obtained by projecting the point P onto the normalized image plane when considering the lens distortion of the camera.

[0039] Also, f x and f y represent the focal lengths in the x - direction and y - direction expressed in pixel units. Also, c x and cy indicates the optical center in the image coordinate system (the position where the optical axis of the camera intersects the image plane IP, the optical center in pixel units). Also, k1, k2, and k3 are the radial distortion coefficients, and p1 and p2 are the tangential distortion coefficients. In this specification, the focal length f x , f y , the optical center c x , c y are referred to as the internal parameters of the camera, and the distortion coefficients k1, k2, k3, p1, and p2 are referred to as the distortion parameters of the camera.

[0040] 〔Calibration Device〕 FIG. 2 is a block diagram showing an example of the schematic configuration of the calibration device 10 of the present embodiment. As shown in FIG. 2, the calibration device 10 includes an arithmetic control unit 20 and a storage unit 22. In addition, a camera 12, an operation unit 14, and an output unit 16 are connected to the calibration device 10.

[0041] The camera 12 generates an image IM (calibration image) obtained by photographing a calibration pattern CA (see FIG. 4) from different directions and outputs it to the calibration device 10. The camera 12 used in the present embodiment has a non - telecentric optical system (an optical system having non - telecentricity on the subject side), and photographing is performed by an imaging element via a lens group as the non - telecentric optical system.

[0042] The operation unit 14 includes operation members such as a keyboard and a mouse, and receives input of various operations by an operator.

[0043] The output unit 16 is a device for outputting the calculation result etc. by the arithmetic control unit 20. The output unit 16 includes, for example, an operation UI (User Interface) and a monitor (e.g., a liquid crystal display etc.) for displaying the calculation result. Also, in addition to or instead of the monitor, the output unit 16 may include a printer or a speaker etc.

[0044] The arithmetic control unit 20 controls the operation of the calibration device 10. The arithmetic control unit 20 is composed of an arithmetic device such as a personal computer, and includes an arithmetic circuit composed of various processors and memories. The various processors include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and programmable logic devices [such as SPLD (Simple Programmable Logic Devices), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Arrays)]. Note that the various functions of the calibration device 10 may be realized by one processor, or may be realized by a plurality of processors of the same type or different types.

[0045] The storage unit 22 stores a control program and various data. The storage unit 22 is composed of, for example, a hard disk drive (HDD) or a semiconductor storage device (SSD). The storage unit 22 may include a temporary storage element composed of a RAM (Random Access Memory) such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory), and may function as a work area for the arithmetic control unit 20.

[0046] A plurality of calibration images IM captured by the camera 12 are temporarily stored in the storage unit 22. Further, the camera parameters CP obtained by the arithmetic control unit 20 are stored in the storage unit 22. The camera parameters CP include the internal parameters (focal length, optical center) of the camera 12 and distortion parameters (radial and tangential distortion coefficients).

[0047] By reading and executing the control program stored in the storage unit 22, the arithmetic control unit 20 functions as an image acquisition unit 30, a pattern information setting unit 32, a feature point detection unit 34, and a parameter calculation unit 36. The functions of each unit constituting the arithmetic control unit 20 will be described later.

[0048] In the present embodiment, calibration processing of the camera 12 is performed based on a plurality of calibration images IM obtained by photographing a very small target in a defocused state with the camera 12. FIG. 3 is a diagram showing an example of photographing a calibration pattern CA with the camera 12, and FIG. 4 is a plan view of the calibration pattern CA.

[0049] As shown in FIG. 3, the camera 12 includes a lens (objective lens) 120 and an image sensor (including, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), etc.) 122. In the example shown in FIG. 3, for simplicity, only two targets T1 and T2 are illustrated, representing a plurality of targets T on the calibration pattern CA. Reference numerals PX1 and PX2 in FIG. 3 are the optical axes of light from the targets T1 and T2 toward the image sensor 122, respectively. In the example shown in FIG. 3, the optical axis PX1 coincides with the optical axis AX of the lens 120. Also, in FIG. 3, the in-focus position where the images of the targets T1 and T2 are in focus through the lens 120 is designated as F1, and among the defocus positions where they are out of focus, the defocus position deeper (opposite to the lens 120 side) than the in-focus position F1 is designated as F2, and the defocus position closer (to the lens 120 side) than the in-focus position F1 is designated as F3. Also, in FIG. 3, as an example, the case where the image sensor 122 is arranged at the defocus position F2 is illustrated. Note that reference numerals f´ and f in the figure indicate the front focal position and the rear focal position of the lens 120, respectively.

[0050] The target T on the calibration pattern CA is an extremely small dot (circle marker). The shape of this dot in a plan view is approximately a perfect circle. Here, the size (diameter) of the target T is determined in relation to at least one of, for example, the focal length of the camera 12 (the focal length of the lens 120), the distance between the camera 12 and the target T (the distance along the optical axis between the lens 120 and the target T), the pixel size of the imaging element 122 of the camera 12, and the allowable error of calibration (pixel error). Note that the target T is not limited to an extremely small dot and may be an extremely small point light source.

[0051] Note that the size (diameter) of the target T on the calibration pattern CA is preferably such that when the target T is photographed by the imaging element 122 at the in-focus position F1, the size of the region occupied by the image of the target T on the imaging element 122 is 1 pixel or less. For example, when the focal length of the lens 120 (rear focal length) = 8 mm, the center-to-center distance between the camera 12 and the target T = 300 mm, and the pixel size of the imaging element 122 = 2.74 μm, the diameter φ of the target T is preferably 102.75 μm or less. However, depending on the allowable error of calibration, the size of the target T may be made larger than the above example.

[0052] When a lens in which the lens 120 is appropriately designed and manufactured is used (specifically, when the optical axis AX of the lens 120 and the center of the aperture stop AP of the lens 120 coincide), when photographing the target T, the height of the incident light at the position of the lens 120 is Δ1 = Δ2. In this case, the image height (the dimension in the Y direction of the defocused image of the target T) at the defocus position F2 is also δ1 = δ2. Although not shown, it is the same as the defocus position F2 also at the defocus position F3. C direction dimension) is also δ1 = δ2. Although illustration is omitted, it is the same as the defocus position F2 also at the defocus position F3.

[0053] VA in FIG. 5 shows an image (Example) of target T captured by image sensor 122 disposed at defocus position F2. In VA of FIG. 5, as described in FIG. 3, the center line of the image of target T overlaps and substantially coincides with optical axis PX of light from target T. The same applies to the case where target T is captured by image sensor 122 disposed at defocus position F3.

[0054] VB in FIG. 5 shows an image (Comparative Example) of a dot (a target that is not extremely small) whose image size captured in the in-focus state is about the same as that of VA in FIG. 5. In this case, as shown in the enlarged view in VC of FIG. 5, the center line L1 of the dot image is shifted with respect to line L2 passing through the center of gravity of the dot image.

[0055] On the other hand, when an extremely small target T is captured in a defocus state, since the center line of the image of target T coincides with the optical axis (AX or AX1), the center (feature point) of target T can be detected with high accuracy.

[0056] In this embodiment, since target T on calibration pattern CA is minimized, even when calibration pattern CA is captured while being tilted with respect to camera 12, the position of the feature point (the center of gravity position of target T) can be detected with high accuracy without being affected by the non-telecentric optical system. On the other hand, although minimizing target T makes it more susceptible to the influence of luminance noise, in this embodiment, since target T is captured in a defocus state, the position of the feature point can be detected while effectively suppressing the influence of luminance noise.

[0057] Note that in this embodiment, the extremely small target T is substantially circular (dot-shaped), but it is not limited thereto. In the defocus image, any shape may be used as long as the center of gravity position and the center position of target T coincide with each other. For example, a shape that is line-symmetric or point-symmetric in two orthogonal directions such as an elliptical shape may be used.

[0058] Hereinafter, the processing procedure of the calibration process (an example of a calibration method) executed by the calibration apparatus 10 of the present embodiment will be described. FIG. 6 is a flowchart showing the overall process of the calibration process executed by the calibration apparatus 10 of the present embodiment. At the start of the flowchart shown in FIG. 6, it is assumed that various initial setting processes such as operation confirmation of each part of the calibration apparatus 10 have been performed.

[0059] (Step S10: Image acquisition step) First, the image acquisition unit 30 acquires a plurality of calibration images IM captured by the camera 12 and stores them in the storage unit 22. In the present embodiment, as the calibration pattern CA, a calibration pattern in which extremely small targets T (dots or point light sources) satisfying the above size conditions are arranged vertically and horizontally is used.

[0060] The image acquisition unit 30 may acquire each calibration image IM from the camera 12 through a cable or a recording medium. Further, each calibration image IM may be stored in an external storage device such as an external server installed outside the calibration apparatus 10, and the image acquisition unit 30 may acquire each calibration image IM from the external storage device via a wired or wireless network.

[0061] The plurality of calibration images IM are those obtained by photographing the calibration pattern with the camera 12 from different directions. The number of calibration images IM is at least 2 or more, preferably 5 or more, and more preferably 10 or more. Note that the calibration pattern may be photographed by the camera 12 as long as it is performed at least before the image acquisition step is performed.

[0062] (Step S12: Pattern information setting step) Next, the pattern information setting unit 32 sets pattern information including information regarding the arrangement of the targets T in the calibration pattern CA. For example, the pattern information includes the number of targets in the vertical direction (column direction) and the horizontal direction (row direction) of the calibration pattern CA. In the example shown in FIG. 4, the number of targets in the vertical direction of the calibration pattern CP is 5, and the number of targets in the horizontal direction is 8. Further, the pattern information includes the total lengths S1 and S2 (see FIG. 4) in the vertical and horizontal directions of the calibration pattern CA in addition to the number of targets in the vertical and horizontal directions of the dot pattern DP. Note that instead of the total lengths S1 and S2 in the vertical and horizontal directions of the calibration pattern CP, pitches (intervals) P1 and P2 in the vertical and horizontal directions of the calibration pattern CP may be included. Thereby, it becomes possible to specify the relative positional relationship of the targets T on the calibration pattern CA. Note that the pattern information is information used to specify a point (object point) in the three-dimensional space corresponding to a feature point (image point) in the image coordinate system in the feature point detection unit 34 described later.

[0063] The pattern information setting unit 32 may, for example, acquire the pattern information input by the operator via the operation unit 14. Alternatively, the pattern information may be stored in advance in the storage unit 22, and the pattern information setting unit 32 may acquire it from the storage unit 22. Note that the pattern information setting step only needs to be performed at least before the parameter calculation step described later. For example, it may be performed before the image acquisition step, or may be performed after the feature point detection step.

[0064] (Step S14: Feature Point Detection Step) Next, the feature point detection unit 34 executes a feature point detection process for detecting a plurality of feature points for each of the plurality of calibration images IM captured by the camera 12. Specifically, the feature point detection unit 34 sequentially reads the plurality of calibration images IM stored in the storage unit 22. Then, for each of the read calibration images IM, the feature point detection unit 34 performs predetermined image processing (for example, binarization processing or grayscale conversion) on the calibration image IM, and then detects each feature point (image point) from the calibration image IM, and obtains the coordinates (pixel coordinates) of each feature point in the image coordinate system. In the present embodiment, the center of gravity position of the image of the target T is detected as a feature point from the calibration image IM obtained by photographing the target T on the calibration pattern CA in a defocused state by the camera 12. The coordinates of the feature points detected by the feature point detection unit 34 are temporarily stored in the storage unit 22.

[0065] If there is a calibration image IM for which the detection of feature points has failed among the plurality of calibration images IM, the feature point detection unit 34 performs an exclusion process of excluding the calibration image IM for which the detection of feature points has failed from the target of the parameter calculation process described later. As a result, in the parameter calculation process, the camera parameters CP can be calculated based on the calibration images IM after the exclusion process (that is, the calibration images IM for which the detection of feature points has succeeded) among the plurality of calibration images IM.

[0066] (Step S16: Parameter calculation step) Next, the parameter calculation unit 36 executes the calculation process of the camera parameters CP. Specifically, the parameter calculation unit 36 calculates the camera parameters CP based on the positions of the feature points on each calibration pattern image IM detected by the feature point detection unit 34 and the pattern information set by the pattern information setting unit 32. The camera parameters CP can be calculated using a known method (for example, Zhang's method). Zhang's method is a method of optimizing parameters so that the position of the feature point (image point) on the captured image (calibration image IM) and the position of the point (object point, known) in the three-dimensional space corresponding to the feature point have a correct correspondence relationship (Z. Zhang, "A flexible new technique for camera calibration", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, No.11, pp.1330-1334, 2000.).

[0067] The camera parameters CP calculated by the parameter calculation unit 36 are the internal parameters of the camera 12 (focal lengths f x 、f y 、optical centers c x 、c y ) and distortion parameters (distortion coefficients k1, k2, k3, p1, p2). The parameter calculation unit 36 stores the calculated camera parameters CP in the storage unit 22.

[0068] (Step S18: Output step) Next, the parameter calculation unit 36 outputs the calculation result of the camera parameters CP to the output unit 16. As a result, the operator can obtain the camera parameters CP, and thus it becomes possible to perform image correction and the like on the images captured by the camera 12 based on the camera parameters CP.

[0069] Thus, the flowchart of the entire calibration process executed by the calibration device 10 ends.

[0070] 〔Example〕 FIG. 7 is a graph showing the tilt resistance of the calibration method. The horizontal axis in FIG. 7 indicates the tilt angle (degree) of the camera with respect to the calibration pattern, and the vertical axis indicates the pixel error (pixel).

[0071] The example (defocus) shows an example in which calibration is performed by photographing the target according to the above-described embodiment in a defocus state. Comparative Example 1 shows an example in which calibration is performed using the dot pattern illustrated in XIC of FIG. 11, and Comparative Example 2 shows an example in which calibration is performed using the checker pattern illustrated in XIB of FIG. 11.

[0072] In the case of Comparative Example 1, as shown in FIG. 13, since an object closer to the camera is photographed larger, the pixel error (absolute value) increases as the tilt angle (absolute value) with respect to the dot pattern increases, and the tilt resistance deteriorates.

[0073] In the example, the pixel error (absolute value) is small regardless of the tilt angle (absolute value) of the camera 12 with respect to the target. Also, in Comparative Example 2, the pixel error (absolute value) is small regardless of the tilt angle (absolute value) of the camera with respect to the checker pattern.

[0074] FIGS. 8 to 10 are graphs showing the luminance noise resistance of the calibration method, and correspond to the example (defocus), Comparative Example 1 (dot pattern), and Comparative Example 2 (checker pattern), respectively. The horizontal axis in FIGS. 8 to 10 indicates the sample number, and the vertical axis indicates the pixel error (pixel).

[0075] In FIGS. 8 to 10, the luminance noise added during the photographing of the calibration pattern (CA in FIG. 4, XIC and XIB in FIG. 11) is increased in the order of Noise = 0.0, 4.0, 8.0.

[0076] In Comparative Example 2 (Fig. 10), since the luminance gradient (differential value) is used when detecting the edge of the checker pattern, the pixel error becomes significantly large according to the luminance noise, and it can be seen that it is the most sensitive to the luminance noise among the examples shown in Figs. 8 to 10.

[0077] On the other hand, the dot pattern according to Comparative Example 1 (Fig. 9) has higher luminance noise resistance than the checker pattern.

[0078] Furthermore, in the example (Fig. 8), the luminance noise resistance is higher than in either of Comparative Examples 1 and 2.

[0079] Summarizing the above results in a table gives the following table. In the example, both the slope resistance and the luminance noise resistance are good.

[0080]

Table 1

[0081] 〔Effect〕 According to the present embodiment, calibration processing is executed based on a plurality of calibration images IM obtained by photographing a calibration pattern CA including a minimum target T in a defocused state by the camera 12. Therefore, even when the calibration pattern CA is photographed while being tilted with respect to the camera 12, the position of the feature point (the centroid position of the target T) can be detected with high accuracy without being affected by the non-telecentric optical system. Furthermore, by photographing the target T in a defocused state, the position of the feature point can be detected in a state where the influence of the luminance noise associated with the minimization of the target T is effectively suppressed. Therefore, it becomes possible to accurately detect the feature points of the calibration image IM, and the calibration accuracy of the camera 12 can be improved.

Explanation of Signs

[0082] 10…Calibration device, 12…Camera, 14…Operation unit, 16…Output unit, 20…Arithmetic control unit, 22…Memory unit, 30…Image acquisition unit, 32…Pattern information setting unit, 34…Feature point detection unit, 36…Parameter calculation unit, 120…Lens, 122…Image sensor, T, T1, T2…Target, F1…In-focus position, F2…Defocus position, F3…Defocus position

Claims

1. An image acquisition step of acquiring a plurality of calibration images by photographing a calibration pattern including a very small target in a defocused state; A pattern information setting step of setting pattern information regarding calibration; A feature point detection step of using, as a feature point, the barycentric position of the defocused image of the target from the calibration image; A parameter calculation step of calculating camera parameters based on the pattern information and the position of the feature point; A camera calibration method including the above steps.

2. The size of the target is determined according to at least one of the focal length of the camera that photographs the target, the distance between the camera and the target, and the pixel size of the imaging element of the camera. The calibration method according to Claim 1.

3. The target is sized such that the size of the image on the imaging element of the camera obtained when photographed in the in-focus state is 1 pixel or less. The calibration method according to Claim 1.

4. The pattern information includes information regarding the arrangement of the target in the calibration pattern. The calibration method according to any one of Claims 1 to 3.

5. The target has a shape that is line-symmetric or point-symmetric in two directions orthogonal to each other. The calibration method according to any one of Claims 1 to 3.

6. The target is dot-shaped. The calibration method according to Claim 5.

7. The target is a point light source. The calibration method according to Claim 5.

Images