Calibration method and calibration device for a camera

Abstract

A camera calibration method includes: an image acquisition step of acquiring a plurality of calibration images obtained by capturing a dot pattern in which a plurality of dots are periodically arranged by a camera; a pattern information setting step of setting pattern information including information about arrangement of the dots; a feature point detection step of detecting a feature point indicating a position of the dot from the calibration image; and a parameter calculation step of calculating a parameter of the camera based on the feature point and the pattern information. In the feature point detection step, a dot center position is detected as the feature point based on a projected image of an outline of a dot image (DI) on an image plane (IP) of the calibration image projected to a virtual plane (VP) set in a three-dimensional space.

Description

Technical Field

[0001] This invention relates to a camera calibration method and calibration apparatus. Background Technology

[0002] Generally, when a camera is used for object detection, camera calibration (correction) is performed. In camera calibration, a planar calibration pattern with multiple feature points arranged in two dimensions is prepared, and the calibration pattern is captured by the camera from multiple directions. Furthermore, feature points are detected based on the multiple images captured by the camera, and the coordinates of the detected feature points are used to estimate the camera's parameters. These camera parameters include the camera's internal parameters (focal length, optical center) or distortion parameters (distortion coefficients).

[0003] For example, Patent Document 1 discloses a method for calibrating a camera using a grid pattern of multiple dots (black circles) arranged as a calibration pattern. In this method, camera parameters are determined by detecting the positions of the centroids of the dots in the dot pattern as feature points on an image (calibration image) captured by the camera.

[0004] In addition, Patent Document 1 discloses grid patterns and checkerboard patterns as calibration patterns other than dot patterns. When using these calibration patterns, the intersections (corners) of straight lines are detected as feature points.

[0005] Prior art literature

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2022-30807 Summary of the Invention

[0008] -The problem the invention aims to solve-

[0009] Using, for example, the calibration pattern disclosed in Patent Document 1 Figure 25 The grid pattern shown in 1025A or Figure 25 In the case of the checkerboard pattern shown in 1025B, the intersection points (corners) H1 or H2 of the straight lines in each pattern are detected as feature points (i.e., edge detection) on the calibrated image. In such edge detection, the brightness gradient (differential value) of the edge is required. Therefore, the detection position (coordinates) of the intersection point H1 or H2 is easily affected by brightness noise, resulting in poor feature point detection accuracy compared to the dot pattern.

[0010] On the other hand, in use Figure 25In the case of the dot pattern shown in 1025C, since the centroid position of the dot D of the dot pattern is detected as a feature point on the calibration image, it is less susceptible to brightness noise compared to grid patterns or checkerboard patterns, but the following problems exist.

[0011] Figure 26 as well as Figure 27 An example is shown where dot D on a dot pattern is captured using a camera (CAM). Figure 26 In the middle, O c This represents the origin of the camera coordinate system (camera origin) based on the camera CAM. Additionally, in Figure 26 as well as Figure 27 In this context, the U-axis and V-axis are the coordinate axes of the image coordinate system (two-dimensional orthogonal coordinate system) defined on the image plane M of the camera CAM.

[0012] In images captured using a camera with a standard lens (non-telecentric optical system), there is a characteristic that the magnification varies with the distance from the camera. That is, the greater the distance to the camera, the smaller the image; the shorter the distance, the larger the image. Therefore, as... Figure 26 As shown, when dot D on the dot pattern is photographed from an oblique direction using a camera CAM, as... Figure 27 As shown, the image of a dot D in an image captured by a camera CAM (hereinafter referred to as the "dot image") DI becomes an elliptical shape with increased distortion in a specific direction based on its positional relationship with the camera CAM. That is, in the dot image DI in the image captured by the camera CAM, the portion on the near side (the side shorter from the camera CAM, +V side) is relatively smaller compared to the portion on the inner side (the side longer from the camera CAM, -V side). Therefore, when it is necessary to determine the centroid position of the dot in two dimensions based on the shape of the dot image DI (the image captured by the camera CAM), due to the influence of the non-telecentric optical system, the centroid position of the dot is sometimes determined at a position deviating from its original position.

[0013] Figure 28 as well as Figure 29 This is a diagram used to illustrate the effect of the tilt of the camera CAM relative to the dot pattern (dot D) on the detection results of the dot's center of gravity position.

[0014] Figure 28The diagram illustrates a state where the camera CAM is directly facing the dot pattern (dot D) (the optical axis AX of the camera CAM is orthogonal to dot D, i.e., the camera CAM is not tilted relative to the dot pattern). In this case, since the dot image DI on the image plane M of the camera CAM has a similar shape to the dot D before projection, the centroid position G1 of the dot on the image plane M (the centroid position calculated two-dimensionally based on the shape of the dot image DI on the image plane M) is consistent with the actual centroid position G0 of the dot projected onto the image plane M (the projection point where the centroid position g of the dot D before projection is projected onto the image plane M). Therefore, the centroid position of the dot can be detected without error.

[0015] Figure 29 This illustrates the state where the camera CAM is tilted towards the dot pattern (dot D). In this case, the dot image DI on the image plane M, due to the influence of the non-telecentric optical system, becomes sandwiched between the actual centroid position G0 projected onto the image plane M (the projection point where the centroid position g of the dot D before projection is projected onto the image plane M) and one side ( Figure 29 (left side) and the other side ( Figure 29 The right side of the circle is an asymmetrical shape. Therefore, the centroid position G1 of the circle on the image plane M is inconsistent with the actual centroid position G0 of the circle projected onto the image plane M, which will produce a detection error in the centroid position of the circle.

[0016] When such a detection error occurs in the position of the center of gravity of the dot, it becomes a factor leading to a decrease in the accuracy of camera calibration.

[0017] The present invention was made in view of the following circumstances, and therefore aims to provide a camera calibration method and calibration apparatus that can improve the accuracy of camera calibration.

[0018] -Methods for solving problems-

[0019] To achieve the above objectives, the present invention includes the following methods.

[0020] The camera calibration method involved in the first method includes: an image acquisition step, acquiring multiple calibration images obtained by the camera capturing a dot pattern in which multiple dots are periodically arranged; a pattern information setting step, setting pattern information containing information related to the arrangement of dots; a feature point detection step, detecting feature points representing the positions of dots from the calibration images; and a parameter calculation step, calculating camera parameters based on the feature points and the pattern information. In the feature point detection step, the position of the centroid of the dots is detected as a feature point based on a projection image of the outline of the dots on the image plane of the calibration image projected onto a virtual plane set in three-dimensional space.

[0021] In the camera calibration method involved in the second method, in the first method, in the feature point detection step, a virtual plane optimized to fit the circle with the projected image is obtained, and the position of the centroid of the circle is detected as a feature point based on the projected image projected onto the obtained virtual plane.

[0022] In the camera calibration method involved in the third method, in the first method, in the feature point detection step, when the circle obtained by processing a group of points composed of multiple projection points is set as an approximate circle, a virtual plane optimized to make the evaluation value representing the fitting error of the approximate circle relative to the group of points minimum or below a threshold is obtained, and the position of the point on which the center of the approximate circle on the optimized virtual plane is projected onto the image plane is set as the position of the centroid of the circle, wherein the multiple projection points are obtained by projecting multiple contour points representing the contour of the circle on the image plane onto the virtual plane.

[0023] In the camera calibration method involved in the fourth method, in the third method, the feature point detection step includes: a search step, which searches for an optimized virtual plane from multiple virtual planes with different poses in three-dimensional space based on an evaluation value representing the fitting error of the approximate circle relative to the point group.

[0024] In the camera calibration method involved in the fifth method, in the fourth method, in the search step, while changing the virtual plane to multiple poses by rotating the virtual plane around an axis parallel to the major axis direction of the dots on the image plane, an evaluation value is calculated for each pose, and an optimized virtual plane is determined based on the evaluation value calculated for each pose.

[0025] In the camera calibration method involved in the sixth method, in the fourth or fifth method, the feature point detection step includes: a contour point extraction step, which extracts multiple contour points of the circle from the image plane; a ray group generation step, which generates a ray group consisting of multiple rays that take the optical center of the camera as the starting point and pass through multiple contour points of the circle on the image plane; and an approximate circle calculation step, which calculates an approximate circle based on the intersection group of the ray group and the virtual plane.

[0026] The camera calibration apparatus according to the seventh method includes: an image acquisition unit that acquires multiple calibration images obtained by the camera capturing a dot pattern in which multiple dots are periodically arranged; a pattern information setting unit that sets pattern information including information related to the arrangement of dots in the dot pattern; a feature point detection unit that detects feature points from the calibration images; and a parameter calculation unit that calculates camera parameters based on the feature points and the pattern information. The feature point detection unit detects the position of the centroid of the dots as a feature point based on a projection image of the outline of the dots on the image plane of the calibration image projected onto a virtual plane set in three-dimensional space.

[0027] The camera calibration method according to the eighth aspect of the present invention includes: an image acquisition step, acquiring multiple calibration images obtained by capturing a calibration pattern containing a tiny target in a defocused state; a pattern information setting step, setting pattern information related to calibration; a feature point detection step, detecting the centroid position of the defocused image of the target as a feature point from the calibration image; and a parameter calculation step, calculating camera parameters based on the pattern information and the position of the feature point.

[0028] In the camera calibration method according to the ninth aspect of the present invention, in the eighth aspect, the size of the target is determined based on at least one of the following: the focal length of the camera that captures the target, the distance between the camera and the target, and the pixel size of the camera's image sensor.

[0029] In the camera calibration method according to the tenth aspect of the present invention, in the eighth aspect, the target is a target whose size is less than 1 pixel when the camera is captured in a focused state. However, depending on the required tolerance, a target whose size is greater than 1 pixel when the camera is captured in a focused state may also be used.

[0030] In the camera calibration method according to the 11th aspect of the present invention, in any of the 8th to 10th aspects, the pattern information includes information related to the configuration of the target in the calibration pattern.

[0031] In the camera calibration method according to the 12th aspect of the present invention, in any one of the 8th to 11th aspects, the target is a shape that is linearly symmetrical or point-symmetrical in two mutually orthogonal directions.

[0032] In the camera calibration method according to the 13th aspect of the present invention, in the 12th aspect, the target is a dot.

[0033] In the camera calibration method according to the 14th aspect of the present invention, in the 12th aspect, the target is a point light source.

[0034] The camera calibration method according to the 15th aspect of the present invention comprises: an image acquisition step, which captures reflected light from illumination light illuminating a calibration device from a light source by a camera positioned optically conjugate to the light source, and acquires a calibration image, the calibration device including a reflector having the property of reflecting illumination light back in the incident direction; a pattern information setting step, which sets pattern information related to the calibration device; a feature point detection step, which uses the focusing position of the reflected light as a feature point from the calibration image; and a parameter calculation step, which calculates camera parameters based on the pattern information and the position of the feature point.

[0035] In the camera calibration method according to the 16th aspect of the present invention, in the 15th aspect, in the image acquisition step, the camera captures the reflected light from at least two reflectors of the calibration device.

[0036] In the camera calibration method according to the 17th aspect of the present invention, in the 15th or 16th aspect, the pattern information includes information related to the configuration and spacing of reflectors in the calibration apparatus.

[0037] In the camera calibration method according to the 18th aspect of the present invention, in any one of the 15th to 17th aspects, the reflector is spherical and arranged in an array on the calibration device.

[0038] In the camera calibration method according to the 19th aspect of the present invention, in the 18th aspect, the surface of the reflector is a mirror surface, or a rough surface that can be obtained to receive diffusely reflected light and identifiable orthogonal reflected light incident non-perpendicularly on the surface of the reflector.

[0039] In the camera calibration method according to the 20th aspect of the present invention, in the 18th aspect, the reflector is made of glass or sapphire.

[0040] -Invention Effects-

[0041] According to the present invention, the accuracy of camera calibration can be improved. Attached Figure Description

[0042] Figure 1 This is an explanatory diagram used to illustrate the projection model of a camera.

[0043] Figure 2 This is a block diagram illustrating an example of the general structure of a calibration device.

[0044] Figure 3 This is a flowchart illustrating the overall process of calibration performed by the calibration apparatus of the first embodiment.

[0045] Figure 4 This is a diagram illustrating an example of the calibration pattern (dot pattern) used in the first embodiment.

[0046] Figure 5 This is an explanatory diagram used to illustrate the detection principle of feature point detection processing.

[0047] Figure 6 This is a flowchart illustrating an example of the steps involved in feature point detection processing.

[0048] Figure 7 This is a flowchart illustrating an example of the steps involved in feature point detection processing.

[0049] Figure 8 This is a diagram used to illustrate the effects of the first embodiment.

[0050] Figure 9 This is an image illustrating an example of using a camera to photograph a target.

[0051] Figure 10 This is a top view showing an example of a calibration pattern.

[0052] Figure 11 These are images obtained by photographing the target (examples and comparative examples).

[0053] Figure 12 This is a flowchart illustrating the overall process of calibration performed by the calibration apparatus of the second embodiment.

[0054] Figure 13 This is a graph showing the tilt resistance of the calibration method.

[0055] Figure 14 This is a graph illustrating the brightness noise tolerance of the calibration method (example).

[0056] Figure 15 This is a graph showing the brightness noise tolerance of the calibration method (Comparative Example 1).

[0057] Figure 16 This is a graph showing the brightness noise tolerance of the calibration method (Comparative Example 2).

[0058] Figure 17 This is a three-dimensional view showing the calibration device.

[0059] Figure 18 yes Figure 17 AA-AA sectional view.

[0060] Figure 19 This is a diagram illustrating an example of calibration using a calibration instrument.

[0061] Figure 20 This is a flowchart illustrating the overall process of calibration performed by the calibration apparatus of the third embodiment.

[0062] Figure 21 This is a graph showing the tilt resistance of the calibration method.

[0063] Figure 22 This is a graph illustrating the brightness noise tolerance of the calibration method (example).

[0064] Figure 23 This is a graph showing the brightness noise tolerance of the calibration method (Comparative Example 1).

[0065] Figure 24 This is a graph showing the brightness noise tolerance of the calibration method (Comparative Example 2).

[0066] Figure 25 This is a diagram showing an example of a calibration pattern.

[0067] Figure 26 This is an image showing an example of using a camera to capture a dot.

[0068] Figure 27 This is an image showing an example of using a camera to capture a dot.

[0069] Figure 28 This is a diagram used to illustrate the effect of the camera's tilt relative to the dot pattern on the detection results.

[0070] Figure 29 This is a diagram used to illustrate the effect of the camera's tilt relative to the dot pattern on the detection results. Detailed Implementation

[0071] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0072] [Camera projection model]

[0073] First, the projection model of the camera will be explained. Figure 1 This is an explanatory diagram used to illustrate the projection model of a camera. Furthermore, Figure 1 The projection model of the camera shown is common to all embodiments described below.

[0074] like Figure 1 As shown, the world coordinate system is a coordinate system that represents a position in three-dimensional space (actual space), with the origin set at O. w , make X orthogonal to each other w Axis, Y w Axis, Z w The axes are set in a three-dimensional orthogonal coordinate system. Furthermore, any coordinate system can be used as long as the position in three-dimensional space (three-dimensional position) can be determined. The camera coordinate system is set with the center O of the camera's optical axis... c Let O be the origin. c Let's set the rightward direction as X. c Axis, set the downward direction to Y c Axis, set the optical axis direction to Z c A three-dimensional orthogonal coordinate system of axes. The image coordinate system is derived from the origin O of the camera coordinate system. c Along Z c The image plane IP, whose focal length f is separated by direction, is set to its upper left as the origin, and is perpendicular to the X-axis. c axis and Y cA two-dimensional orthogonal coordinate system (pixel coordinate system) with U-axis and V-axis parallel to each other.

[0075] First, the coordinates (x, y) of point P (target point) in the world coordinate system in three-dimensional space. w y w , z w The camera can be transformed into coordinates (x, y, z) in the camera coordinate system using the camera's rotation matrix R and translation vector t, as shown in equation (1) below.

[0076] [Mathematical Expression 1]

[0077]

[0078] Here, [R|t] is the matrix that transforms the world coordinate system to the camera coordinate system (external parameter matrix), representing the pose and position of the camera in the world coordinate system. The components r of [R|t]... 11 r 12 ... r 33 t x t y t z These are called the camera's external parameters.

[0079] Next, if the coordinates (pixel coordinates) of the projection point Q, which is projected from the camera coordinate system at (x, y, z) onto the image plane IP, are set to (u, v), the following equations (2) to (7) hold.

[0080] [Mathematical Expression 2]

[0081]

[0082] [Mathematical Expression 3]

[0083]

[0084] [Mathematical Expression 4]

[0085]

[0086] [Mathematical Expression 5]

[0087]

[0088] [Mathematical Expression 6]

[0089]

[0090] [Mathematical Expression 7]

[0091]

[0092] Here, (x´, y´) represents the coordinates of the projection point of point P, which is located at (x, y, z) as observed from the camera coordinate system, onto the normalized image plane (z=1). Additionally, (x´´, y´´) represents the coordinates of the projection point (distorted point) of point P onto the normalized image plane, taking into account camera lens distortion.

[0093] In addition, f x f y This represents the focal length in the x and y directions, expressed in pixels. Additionally, c... x c y This represents the optical center in the image coordinate system (the position where the camera's optical axis intersects the image plane IP, the optical center per pixel). Additionally, k1, k2, and k3 are the distortion coefficients in the radial direction, and p1 and p2 are the distortion coefficients in the tangential direction. In this specification, the focal length f is... x f y Optical Center c x c y These are called the camera's internal parameters, and the distortion coefficients k1, k2, k3, p1, and p2 are called the camera's distortion parameters.

[0094] <First Implementation>

[0095] [Calibration device]

[0096] Figure 2 This is a block diagram illustrating an example of the general structure of the calibration device 10 according to the first embodiment. Figure 2 As shown, the calibration device 10 includes an arithmetic control unit 20 and a storage unit 22. Additionally, a camera 12, an operation unit 14, and an output unit 16 are connected to the calibration device 10.

[0097] The camera 12 generates an image IM (calibration image) obtained by shooting calibration patterns from different directions and outputs it to the calibration device 10. The camera 12 has a non-telecentric optical system (an optical system that is non-telecentric on the subject side) and takes pictures through the imaging element via a lens group that is a non-telecentric optical system.

[0098] The operation unit 14 is configured to include operating components such as a keyboard and mouse, and accepts input of various operations performed by the operator.

[0099] The output unit 16 is a device for outputting calculation results, etc., based on the calculation control unit 20. The output unit 16 may include, for example, a monitor (e.g., an LCD) for displaying the operation UI (User Interface) and the calculation results. Alternatively, the output unit 16 may include, in addition to a monitor, a printer or a speaker, or other devices that replace the monitor.

[0100] The arithmetic control unit 20 controls the operation of the calibration device 10. The arithmetic control unit 20 is, for example, a computing device such as a personal computer, and has arithmetic circuits composed of various processors and memory. These processors include CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and programmable logic devices [such as SPLDs (Simple Programmable Logic Devices), CPLDs (Complex Programmable Logic Devices), and FPGAs (Field Programmable Gate Arrays)]. Furthermore, the various functions of the calibration device 10 can be implemented by a single processor or by multiple processors of the same or different types.

[0101] The storage unit 22 stores control programs and various data. The storage unit 22 may be composed of, for example, a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 22 may include temporary storage elements composed of RAM (Random Access Memory) such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory), and may also function as the operating area of ​​the computing control unit 20.

[0102] The storage unit 22 temporarily stores multiple calibration images IM captured by the camera 12. Additionally, the storage unit 22 stores camera parameters CP calculated by the arithmetic control unit 20. The camera parameters CP include the camera 12's internal parameters (focal length, optical center) and distortion parameters (distortion coefficients in the radial and tangential directions).

[0103] The arithmetic control unit 20 functions as the image acquisition unit 30, the pattern information setting unit 32, the feature point detection unit 34, and the parameter calculation unit 36 ​​by reading and executing the control program stored in the storage unit 22. The functions of each part constituting the arithmetic control unit 20 will be described later.

[0104] Hereinafter, the processing steps (an example of a calibration method) of the calibration process performed by the calibration device 10 of the first embodiment will be described. Figure 3 This is a flowchart illustrating the overall process of the calibration procedure performed by the calibration apparatus 10 of the first embodiment. Furthermore, it is set to begin... Figure 3 The flowchart shown illustrates the initial setup process, including confirming the operation of each part of the calibration device 10.

[0105] (Step S10: Image acquisition step)

[0106] First, the image acquisition unit 30 acquires multiple calibration images IM captured by the camera 12 and stores them in the storage unit 22. In this embodiment, as a calibration pattern, such as Figure 4 As shown in the example, a dot pattern DP (also known as a "circular grid") is used, in which multiple dots D (black circles) are arranged horizontally and vertically. Furthermore, the shape or arrangement of each dot D in the dot pattern DP is not limited to any particular shape or configuration, as long as the feature points are easily detected using existing search methods. Figure 4 The example shown.

[0107] Multiple calibration images IM are obtained by the camera 12 capturing images of the dot pattern DP, which serves as the calibration pattern, from different directions. The number of calibration images IM is at least 2, preferably 5 or more, and more preferably 10 or more. Furthermore, the camera 12 only needs to capture the calibration pattern (dot pattern DP) at least before the image acquisition step.

[0108] The image acquisition unit 30 can acquire each calibration image IM from the camera 12 via a cable or recording medium. Alternatively, each calibration image IM can be pre-stored on an external storage device such as an external server located outside the calibration device 10, and the image acquisition unit 30 can acquire each calibration image IM from the external storage device via a wired or wireless network.

[0109] (Step S12: Pattern Information Setting Step)

[0110] Next, the pattern information setting unit 32 sets pattern information including information related to the arrangement of dots D in the dot pattern DP. For example, the pattern information includes the number of dots in the vertical (column direction) and horizontal (row direction) directions of the dot pattern DP. Figure 4In the example shown, the number of dots in the vertical direction of the dot pattern DP is 5, and the number of dots in the horizontal direction is 8. Furthermore, the pattern information includes not only the number of dots in the vertical and horizontal directions of the dot pattern DP, but also the total lengths S1 and S2 of the dot pattern DP in both directions. Alternatively, the vertical and horizontal spacings P1 and P2 of the dot pattern DP can be included instead of the total lengths S1 and S2. This allows the relative positional relationships of the dots D on the dot pattern DP to be determined. Moreover, the pattern information is used to determine the points (target points) in three-dimensional space corresponding to the feature points (image points) in the image coordinate system in the feature point detection unit 34 described later.

[0111] The pattern information setting unit 32 can, for example, acquire pattern information input by the operator via the operation unit 14. Alternatively, pattern information can be pre-stored in the storage unit 22, and the pattern information setting unit 32 can retrieve it from the storage unit 22. Furthermore, the pattern information setting step only needs to be performed at least before the parameter calculation step described later; for example, it can be performed before the image acquisition step or after the feature point detection step.

[0112] (Step S14: Feature point detection step)

[0113] Next, the feature point detection unit 34 performs feature point detection processing for each of the multiple calibration images IM captured by the camera 12, detecting multiple feature points. Specifically, the feature point detection unit 34 sequentially reads the multiple calibration images IM stored in the storage unit 22. Furthermore, for each read calibration image IM, the feature point detection unit 34 performs a given image processing (grayscale transformation, etc.) on the calibration image IM, and then detects each feature point (image point) based on the calibration image IM, calculating the coordinates (pixel coordinates) of each feature point in the image coordinate system. In this embodiment, as feature points on the calibration image IM, the position of the centroid of each dot D (dot image DI) is detected (dot centroid position). The coordinates of the feature points detected by the feature point detection unit 34 are temporarily stored in the storage unit 22. The details of the feature point detection processing performed by the feature point detection unit 34 will be described later.

[0114] In the case where there is a calibration image IM in which feature point detection fails among multiple calibration images IM, the feature point detection unit 34 performs an exclusion process to exclude the calibration image IM in which feature point detection failed from the parameter calculation process described later. As a result, in the parameter calculation process, the camera parameter CP can be calculated based on the calibration image IM after the exclusion process among multiple calibration images IM (i.e., the calibration image IM in which feature point detection is successful).

[0115] (Step S16: Parameter Calculation Step)

[0116] Next, the parameter calculation unit 36 ​​performs the calculation process for the camera parameters CP. Specifically, the parameter calculation unit 36 ​​calculates the camera parameters CP based on the positions (centroid positions of dots) of feature points on each calibration image IM detected by the feature point detection unit 34 and the pattern information set by the pattern information setting unit 32. The camera parameters CP can be calculated using known methods (e.g., Zhang's method). Zhang's method refers to a method of optimizing parameters so that the positions of feature points (image points) on the captured image (calibration image IM) and the positions of points (target points, known) in three-dimensional space corresponding to those feature points are correctly correlated (Z. Zhang, "A flexible new technique for camera calibration", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 11, pp. 1330-1334, 2000.).

[0117] The camera parameter CP calculated by parameter calculation unit 36 ​​is the internal parameter of camera 12 (focal length f). x f y Optical Center C x c y The parameter calculation unit 36 ​​stores the calculated camera parameters CP in the storage unit 22, along with distortion parameters (distortion coefficients k1, k2, k3, P1, P2).

[0118] (Step S18: Output step)

[0119] Next, the parameter calculation unit 36 ​​outputs the calculated result of the camera parameter CP to the output unit 16. Therefore, since the operator can obtain the camera parameter CP, image correction and other actions can be performed based on the camera parameter CP on the image captured by the camera 12.

[0120] The flowchart of the overall process of calibration performed by calibration device 10, as shown above, ends here.

[0121] [Feature Point Detection and Processing]

[0122] Next, the detailed process of feature point detection performed in the feature point detection step will be explained. In the first embodiment, the feature point detection process involves detecting the centroid position of a dot as a feature point on the calibration image IM. Furthermore, it is assumed that the feature point detection process is performed sequentially for each calibration image IM.

[0123] First, refer to Figure 5 The detection principle of feature point detection processing is explained. Figure 5 This is an explanatory diagram used to illustrate the detection principle of feature point detection processing.

[0124] When the dot pattern DP is captured at an angle relative to the camera 12, due to the non-telecentric optical system of the camera 12, the dots D on the dot pattern DP are distorted and projected onto the image plane IP of the camera 12. Therefore, when it is desired to directly detect the centroid position of the dots on the dot image DI in the calibration image IM in two dimensions, there is a problem that the detected centroid position deviates from its original position.

[0125] Therefore, in the first embodiment, a virtual plane VP, as described below, is used to perform feature point detection processing in three dimensions. That is, in this feature point detection processing, as... Figure 5 As shown, when a virtual plane is defined in three-dimensional space as a virtual plane VP, the virtual plane VP is obtained by optimizing it through fitting techniques such as least squares so that the outline of the dot image DI on the image plane IP of the calibration image IM is projected onto the virtual plane VP and the projected image is close to a circle.

[0126] Specifically, it generates the origin O of the camera coordinate system. c Starting from the optical center of camera 12, a ray group B is formed by multiple rays passing through multiple contour points of the dotted image DI on the image plane IP. The intersection group T of the ray group B and the virtual plane VP is then determined. Furthermore, if the circle obtained by approximating the intersection group T using the least squares method is considered an approximate circle, the virtual plane VP is optimized to reduce the evaluation value representing the fitting error of the approximate circle relative to the intersection group T.

[0127] Here, the virtual plane VP is closer to the dot pattern DP in three-dimensional space ( Figure 5 (Not illustrated) A parallel state indicates a smaller evaluation value of the fitting error of the approximate circle on the virtual plane VP relative to the intersection point group T. Therefore, a virtual plane VP optimized to reduce the evaluation value of the fitting error of the approximate circle on the virtual plane VP relative to the intersection point group T can be considered as a pose substantially parallel to the dot pattern DP in three-dimensional space. Furthermore, "substantially parallel" includes not only the case of being strictly parallel but also the case of being approximately parallel.

[0128] Furthermore, when the center VC of the approximate circle on the optimized virtual plane VP is projected onto the image plane IP, the position of the projection point IC on the image plane IP represents the centroid position of the circle on the image plane IP where the distortion error caused by the projection is minimized.

[0129] In the first embodiment, within three-dimensional space, from multiple virtual planes VP with different poses, an optimized virtual plane VP, as described above, is searched to minimize the evaluation value representing the fitting error. Furthermore, the position of the projection point IC on the image plane IP, onto which the center VC of the approximate circle on the optimized virtual plane VP is projected, is set as the centroid position of the circle. As described above, since the optimized virtual plane VP can be considered substantially parallel to the dot pattern DP in three-dimensional space, by setting the projection point IC on the image plane IP, onto the center VC of the approximate circle on the optimized virtual plane VP, the coordinates (pixel coordinates) of the centroid position where the distortion error caused by projection is minimized can be obtained.

[0130] Figure 6 as well as Figure 7 This is a flowchart illustrating an example of the steps involved in feature point detection processing. The following is a description of the steps involved in feature point detection processing. Figure 6 as well as Figure 7 The flowchart shown illustrates an example of the steps involved in feature point detection processing.

[0131] (Step S20: Contour point extraction step)

[0132] First, the feature point detection unit 34 reads the calibration image IM. After performing given image processing (grayscale transformation, etc.) on the calibration image IM, it extracts multiple contour points representing the contour of the dot image DI on the image plane IP of the calibration image IM. At this time, the number of extracted contour points is at least three. The extraction of the contour (multiple contour points) of the dot image DI uses known methods such as edge detection and binarization. From this, the coordinates of the multiple contour points (contour pixels) representing the contour of the dot image DI on the image plane IP are obtained.

[0133] (Step S22: Ray swarm generation step)

[0134] Next, the feature point detection unit 34 generates ray group B, which is as follows: Figure 5 As shown, the origin O of the camera coordinate system is... c It consists of multiple rays that start from the contour point and pass through each contour point obtained by the contour point extraction step.

[0135] (Step S24: Calculate the direction of the major axis of the dot)

[0136] Next, the feature point detection unit 34 calculates the major axis direction of the dot image DI on the image plane IP (hereinafter referred to as the "major axis direction of the dot").

[0137] (Step S26: Angle initialization step)

[0138] Next, the feature point detection unit 34 initializes the angle θ of the virtual plane VP to 0°. Here, the angle θ of the virtual plane VP represents the angle of rotation position when the virtual plane VP is rotated around an axis parallel to the major axis of the dot in the loop processing described later. In this embodiment, when the virtual plane VP is in the initial rotation position (θ = 0°), the virtual plane VP is configured in three-dimensional space at a position parallel to the image plane IP, but it is not limited to this and the virtual plane VP can be configured at any position.

[0139] (Step S28: Loop the start step)

[0140] The loop start step is the beginning of a loop process (loop 1) that repeats the processes of steps S30 to S56 described later N times. Furthermore, this loop process is an example of the search step of the present invention.

[0141] (Step S30: Calculation of positive and side rotation angle)

[0142] Next, as shown in equation (8) below, the feature point detection unit 34 calculates the preset step angle δ. θ The value obtained by adding the angle θ of the virtual plane VP is taken as the first angle θ1 (where δ θ >0. (The same applies below).

[0143] [Mathematical Expression 8]

[0144]

[0145] (Step S32: Virtual plane rotation step)

[0146] Next, the feature point detection unit 34 performs a process to rotate the virtual plane VP around an axis parallel to the major axis of the dot so that the virtual plane VP becomes the first angle θ1. Hereinafter, the virtual plane VP that becomes the first angle θ1 will be referred to as "the first virtual plane VP1".

[0147] (Step S34: Intersection group calculation steps)

[0148] Next, the feature point detection unit 34 calculates the first intersection point group T1 of the ray group B and the first virtual plane VP1.

[0149] (Step S36: Circle approximation step)

[0150] Next, the feature point detection unit 34 performs a circle approximation process on the first intersection group T1 based on the least squares method, and calculates the position and shape (center position and radius) of the approximate circle on the first virtual plane VP1.

[0151] (Step S38: Calculation of the first evaluation value)

[0152] Next, the feature point detection unit 34 calculates an evaluation value ε1 representing the fitting error of the approximate circle on the first virtual plane VP1 relative to the first intersection point group T1. For example, the sum of squared residuals or the sum of absolute residuals is used as the evaluation value calculated as the first evaluation value ε1. The sum of squared residuals is the sum of the squares of the residuals between the distances from the center position of the approximate circle to each point of the first intersection point group T1 and the radius of the approximate circle, and the sum of absolute residuals is the sum of the absolute values ​​of the residuals.

[0153] (Step S40: Negative side angle calculation steps)

[0154] Next, as shown in equation (9) below, the feature point detection unit 34 calculates the angle θ of the virtual plane VP minus the preset step angle δ. θ The obtained value is taken as the second angle θ2.

[0155] [Mathematical Expression 9]

[0156]

[0157] (Step S42: Virtual plane rotation step)

[0158] Next, the feature point detection unit 34 performs a process to rotate the virtual plane VP around an axis parallel to the major axis of the dot so that the virtual plane VP becomes the second angle θ2. Hereinafter, the virtual plane VP that becomes the second angle θ2 will be referred to as "the second virtual plane VP2".

[0159] (Step S44: Intersection group calculation steps)

[0160] Next, the feature point detection unit 34 calculates the second intersection point group T2 of the ray group B and the second virtual plane VP2.

[0161] (Step S46: Circle approximation step)

[0162] Next, the feature point detection unit 34 performs a circle approximation process on the second intersection group T2 based on the least squares method, and calculates the position and shape (center position and radius) of the approximate circle on the second virtual plane VP2.

[0163] (Step S48: Calculation of the second evaluation value)

[0164] Next, the feature point detection unit 34 calculates an evaluation value ε2 representing the fitting error of the approximate circle on the second virtual plane VP2 relative to the second intersection point group T2. The evaluation value calculated as the second evaluation value ε2 is the same as the first evaluation value ε1 mentioned above, using the sum of squared residuals or the sum of absolute residuals.

[0165] (Step S50: Evaluation value comparison step)

[0166] Next, the feature point detection unit 34 compares the first evaluation value ε1 and the second evaluation value ε2. If the first evaluation value ε1 is smaller than the second evaluation value ε2 ("yes"), the process proceeds to step S52; if the first evaluation value ε1 is larger than or equal to the second evaluation value ε2 ("no"), the process proceeds to step S54. Alternatively, if the first evaluation value ε1 and the second evaluation value ε2 are equal, the process may proceed to step S52 instead of step S54.

[0167] (Steps S52 and S54: Angle update steps)

[0168] Next, if the first evaluation value ε1 is smaller than the second evaluation value ε2, the feature point detection unit 34 updates the angle θ of the virtual plane VP to the first angle θ1 (step S52), and if the first evaluation value ε1 is larger than or equal to the second evaluation value ε2, the feature point detection unit 34 updates the angle θ of the virtual plane VP to the second angle θ2 (step S54).

[0169] (Step S56: Step Angle Update Step)

[0170] Next, the feature point detection unit 34 will step by an angle δ θ Updated to δ θ ×α. Where the coefficient α is a real number 0 < α ≤ 1. For example, the coefficient α is set empirically based on required computational accuracy. Therefore, since the coefficient α is set to less than 1, the step angle δ increases with each increase in the number of iterations. θ The step angle gradually (in stages) decreases, thus enabling the determination of the optimized virtual plane VP, which minimizes the evaluation value representing the fitting error of the circle relative to the intersection group T, with good accuracy. Furthermore, the step angle update step can be performed only when the smaller of the first evaluation value ε1 and the second evaluation value ε2 is less than a pre-set threshold (evaluation value threshold). Therefore, the step angle δ can be adjusted as the evaluation value decreases. θ By performing progressively smaller calculations, the processing speed of feature point detection can be improved.

[0171] (Step S58: End of loop)

[0172] The loop termination step marks the end of the aforementioned loop processing (loop 1). In this embodiment, the loop processing ends when the processing from steps S30 to S56 is repeated N times. Furthermore, the loop termination condition can be either the number of loop processing iterations reaching an upper limit value N (where N is an integer greater than or equal to 2) (the first termination condition) or the updated step angle δ.θ The loop process ends when either of the following conditions is met: the value of the evaluation ε1 becomes smaller than the preset threshold (angle threshold) (second termination condition). Alternatively, the loop process can also end when the smaller of the evaluation values ​​ε1 and ε2 becomes smaller than the threshold (evaluation value threshold) (third termination condition).

[0173] By performing the above iterative process, it is possible to search, from multiple virtual planes VP with different poses in three-dimensional space, for a virtual plane VP whose evaluation value representing the fitting error of the approximate circle on the virtual plane VP relative to the intersection group T is minimized or below a threshold.

[0174] (Step S60: Intersection group calculation steps)

[0175] Next, the feature point detection unit 34 calculates the intersection group T of the virtual plane VP, which has been optimized to angle θ through the above-mentioned iterative processing, and the ray group B.

[0176] (Step S62: Circle approximation process)

[0177] Next, the feature point detection unit 34 performs circle approximation processing on the intersection group T obtained by the intersection group calculation step (step S60) based on the least squares method, and calculates the position and shape (center position and radius) of the approximate circle on the virtual plane VP.

[0178] (Step S64: Ray generation step)

[0179] Next, the feature point detection unit 34 generates the origin O of the camera coordinate system. c A ray CL is taken as the starting point and passes through the center VC of the approximate circle obtained by the circle approximation process step (step S62).

[0180] (Step S66: Calculation of intersection points of image planes)

[0181] Next, the feature point detection unit 34 calculates the pixel position (pixel coordinates) of the intersection point of the ray CL and the image plane IP. This intersection point becomes the projection point IC that projects the center VC of the approximate circle on the virtual plane VP onto the image plane IP.

[0182] (Step S68: Feature point determination step)

[0183] Next, the feature point detection unit 34 determines the position (center of gravity) of the feature points of the calibration image IM as the position (pixel coordinates) of the intersection point (i.e., the projection point IC) on the image plane IP obtained in step S66.

[0184] The flowchart for dot detection processing is now complete.

[0185] [Effects of the first embodiment]

[0186] Next, the effects of the first embodiment will be explained.

[0187] Figure 8 This is a diagram used to illustrate the effects of the first embodiment. Figure 8 In the figure, the horizontal axis represents the tilt angle of the dot pattern DP relative to the camera 12 (in degrees (°)), and the vertical axis represents the detection error of the center position of the dot (in pixels). Figure 8 The graph, represented by solid lines, illustrates the detection error when the feature point detection process in the first embodiment detects the centroid position of a circle. Additionally, Figure 8 The chart shown in the middle, indicated by dashed lines, serves as a comparative example, illustrating the detection error when directly detecting the centroid position of a dot in two dimensions based on a calibration image.

[0188] Depend on Figure 8 It can be seen that, in the comparative example, the larger the tilt angle (absolute value) of the dot pattern DP relative to the camera 12, the larger the detection error (absolute value) of the dot's center of gravity position.

[0189] In contrast, in the feature point detection process of the first embodiment, an optimized virtual plane VP is obtained by fitting a projected image obtained by projecting a circle onto the outline of a dot image DI on the image plane IP. Furthermore, since the position of the projection point on the image plane IP from the center of the circle (approximate circle) on the optimized virtual plane VP is set as the centroid position of the dot, the coordinates (pixel coordinates) of the centroid position of the dot that minimizes the distortion error caused by the projection can be obtained.

[0190] Therefore, in the first embodiment, the detection error of the dot's center of gravity is generally smaller compared to the comparative example. In particular, in the comparative example, there is a tendency for the detection error of the dot's center of gravity to increase when the tilt angle (absolute value) of the camera 12 relative to the dot pattern DP increases; however, in the first embodiment, even when the tilt angle (absolute value) increases, the detection error of the dot's center of gravity can be suppressed within a certain range (in...). Figure 8 In the example shown, the detection error (absolute value) is within 0.1 pixels.

[0191] Therefore, according to the first embodiment, since the centroid position of the dot can be detected without being affected by the non-telecentric optical system, the tilt resistance is excellent, and the position of the feature point (centroid position of the dot) on the calibration image IM can be determined with good accuracy. As a result, the calibration accuracy of the camera 12 can be improved.

[0192] Furthermore, according to the first embodiment, due to its excellent tilt resistance, the camera 12 has a high degree of freedom in its posture and position relative to the dot pattern DP, which is very convenient for the user.

[0193] Furthermore, according to the first embodiment, when searching for the optimized virtual plane VP in three-dimensional space, it is performed by rotating around an axis parallel to a direction orthogonal to the direction in which the dot image DI on the image plane IP of the calibration image IM is prone to distortion (the direction of the major axis of the dot). Therefore, by rotating the virtual plane VP around one axis, the search for the optimized virtual plane VP can be performed efficiently. Moreover, the search for the optimized virtual plane VP can also be performed by rotating the virtual plane VP around any two or three axes, and is not limited to one axis.

[0194] <Second Implementation Method>

[0195] Next, the second embodiment will be described. Furthermore, regarding the structure of the calibration device 10, since it differs from the first embodiment (see...), Figure 2 The same applies to ( ), therefore the explanation is omitted.

[0196] In the second embodiment, the calibration process of the camera 12 is performed based on multiple calibration images IM obtained by the camera 12 capturing a very small target in a defocused state. Figure 9 This is a diagram illustrating an example of capturing a calibration pattern CA using camera 12. Figure 10 This is a top view of the calibration pattern CA.

[0197] like Figure 9 As shown, the camera 12 includes: a lens (objective lens) 120 and an imaging element (e.g., including a CCD (Charge-Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor)) 122. Figure 9 In the example shown, for simplicity, only two targets T1 and T2 are represented by multiple targets T on the calibration pattern CA. Figure 9 The reference numerals PX1 and PX2 in the attached diagram represent the optical axes of light traveling from targets T1 and T2 to the imaging element 122, respectively. Figure 9 In the example shown, the optical axis PX1 is aligned with the optical axis AX of lens 120. Additionally, in Figure 9In this configuration, the in-focus position where the images of targets T1 and T2 are brought into focus by lens 120 is set to F1. The defocus position, which is further inward than the in-focus position F1 (opposite to the lens 120 side), is set to F2. The defocus position, which is closer to the front (lens 120 side) than the in-focus position F1, is set to F3. Additionally, in... Figure 9 As an example, the figure shows the image sensor 122 positioned at the defocus position F2. Furthermore, the reference numerals f' and f in the figure indicate the front and rear focus positions of the lens 120, respectively.

[0198] The target T on the calibration pattern CA is a tiny dot (circular mark). The top view of this dot is approximately circular. Here, the size (diameter) of the target T is determined, for example, by at least one of the following relationships: the focal length of the camera 12 (the focal length of the lens 120), the distance between the camera 12 and the target T (the distance along the optical axis between the lens 120 and the target T), the pixel size of the imaging element 122 of the camera 12, and the tolerance for calibration error (pixel error). Furthermore, the target T is not limited to a tiny dot; it can also be a tiny point light source.

[0199] Furthermore, regarding the size (diameter) of the target T on the calibration pattern CA, it is preferable that, when the target T is captured by the imaging element 122 at the focus position F1, the size of the area occupied by the image of the target T on the imaging element 122 is 1 pixel or less. For example, if the focal length (rear focal length) of the lens 120 is set to 8 mm, the center-to-center distance between the camera 12 and the target T is set to 300 mm, and the pixel size of the imaging element 122 is set to 2.74 μm, the diameter φ of the target T is preferably 102.75 μm or less. However, depending on the allowable calibration error, the size of the target T may be larger than the example described above.

[0200] When lens 120 is a properly designed and manufactured lens (specifically, when the optical axis AX of lens 120 coincides with the center of the aperture stop AP of lens 120), the height of the incident light at the position of lens 120 is Δ1 = Δ2 when photographing the target T. In this case, the image height (the dimension of the defocused image of target T in the YC direction) at the defocused position F2 is also δ1 = δ2. Furthermore, although the illustration is omitted, the same applies at the defocused position F3 as at the defocused position F2.

[0201] Figure 11 Example 1011A shows an image of target T captured by imaging element 122 positioned at defocus position F2 (example). Figure 11 In 1011A, such as Figure 9As explained, the center line of the image of target T overlaps and is approximately aligned with the optical axis PX of the light from target T. Furthermore, the same applies to the case where the image of target T is captured by the imaging element 122 positioned at defocus F3.

[0202] Figure 11 The 1011B shows that the image captured in focus mode becomes the same as... Figure 11 An image of a dot (not a very small target) of the same size as the 1011A (comparative example). In this case, as... Figure 11 As shown in the enlarged view of 1011C, the center line L1 of the dot image is deviated from the line L2 that passes through the centroid of the dot image.

[0203] In contrast, when shooting a very small target T in a defocused state, since the center line of the image of the target T is aligned with the optical axis (AX or AX1), the center (feature point) of the target T can be detected with good accuracy.

[0204] In the second embodiment, the target T on the calibration pattern CA is minimized, so even when the calibration pattern CA is photographed at an angle relative to the camera 12, it is not affected by the non-telecentric optical system, and the position of the feature point (the centroid position of the target T) can be detected with high accuracy. On the other hand, although the minimization of the target T makes it susceptible to brightness noise, in the second embodiment, since the target T is photographed in a defocused state, the position of the feature point can be detected while effectively suppressing the influence of brightness noise.

[0205] Furthermore, in the second embodiment, the extremely small target T is set to a roughly circular shape (dot-like shape), but it is not limited to this. In the defocused image, any shape in which the centroid position and the center position of the target T are the same can be used. For example, it can also be an elliptical shape or a shape that is linearly symmetrical or point-symmetrical in two mutually orthogonal directions.

[0206] Hereinafter, the processing steps (an example of a calibration method) of the calibration process performed by the calibration device 10 of the second embodiment will be described. Figure 12 This is a flowchart illustrating the overall process of the calibration procedure performed by the calibration apparatus 10 of the second embodiment. Furthermore, at the beginning... Figure 12 As shown in the flowchart, it is assumed that various initial setting processes, such as confirming the operation of each part of the calibration device 10, have been performed.

[0207] (Step S110: Image acquisition step)

[0208] First, the image acquisition unit 30 acquires multiple calibration images IM captured by the camera 12 and stores them in the storage unit 22. In the second embodiment, the calibration pattern CA is a calibration pattern using extremely small targets T (dots or point light sources) arranged in a horizontal and vertical pattern that meets the above-mentioned size requirements.

[0209] The image acquisition unit 30 can acquire each calibration image IM from the camera 12 via a cable or recording medium. Alternatively, each calibration image IM can be pre-stored on an external storage device such as an external server located outside the calibration device 10, and the image acquisition unit 30 can acquire each calibration image IM from the external storage device via a wired or wireless network.

[0210] Multiple calibration images IM are images obtained by the camera 12 capturing the calibration pattern from different directions. The number of calibration images IM is at least 2, preferably 5 or more, and more preferably 10 or more. Furthermore, the camera 12 only needs to capture the calibration pattern at least before the image acquisition step.

[0211] (Step S112: Pattern Information Setting Step)

[0212] Next, the pattern information setting unit 32 sets pattern information including information related to the configuration of the targets T in the calibration pattern CA. For example, the pattern information includes the number of targets in the vertical (column direction) and horizontal (row direction) directions of the calibration pattern CA. Figure 10 In the example shown, the vertical target number of the calibration pattern CA is 5, and the horizontal target number is 8. Additionally, the pattern information, besides the vertical and horizontal target numbers of the calibration pattern CA, also includes the total lengths S1 and S2 of the calibration pattern CA in both directions (see reference). Figure 10 Furthermore, the longitudinal and transverse lengths S1 and S2 of the calibration pattern CA can be replaced by the longitudinal and transverse spacings (intervals) P1 and P2 of the calibration pattern CA. This allows the relative positional relationships of the targets T on the calibration pattern CA to be determined. Furthermore, the pattern information is used in the feature point detection unit 34, described later, to determine the points (target points) in three-dimensional space corresponding to feature points (image points) in the image coordinate system.

[0213] The pattern information setting unit 32 can, for example, acquire pattern information input by the operator via the operation unit 14. Alternatively, pattern information can be pre-stored in the storage unit 22, and the pattern information setting unit 32 can retrieve it from the storage unit 22. Furthermore, the pattern information setting step only needs to be performed at least before the parameter calculation step described later; for example, it can be performed before the image acquisition step or after the feature point detection step.

[0214] (Step S114: Feature point detection step)

[0215] Next, the feature point detection unit 34 performs feature point detection processing for each of the multiple calibration images IM captured by the camera 12, detecting multiple feature points. Specifically, the feature point detection unit 34 sequentially reads the multiple calibration images IM stored in the storage unit 22. Then, for each calibration image IM read in, the feature point detection unit 34 performs a given image processing (e.g., binarization or grayscale transformation), and then detects each feature point (image point) from the calibration image IM, calculating the coordinates (pixel coordinates) of each feature point in the image coordinate system. In this embodiment, the centroid position of the image of the target T detected from the calibration image IM obtained by the camera 12 in a defocused state, on the calibration pattern CA, is used as the feature point. The coordinates of the feature points detected by the feature point detection unit 34 are temporarily stored in the storage unit 22.

[0216] In the case where there is a calibration image IM in which feature point detection fails among multiple calibration images IM, the feature point detection unit 34 performs an exclusion process to exclude the calibration image IM in which feature point detection failed from the parameter calculation process described later. As a result, in the parameter calculation process, the camera parameter CP can be calculated based on the calibration image IM after the exclusion process among multiple calibration images IM (i.e., the calibration image IM in which feature point detection is successful).

[0217] (Step S116: Parameter Calculation Step)

[0218] Next, the parameter calculation unit 36 ​​performs the calculation process for the camera parameters CP. Specifically, the parameter calculation unit 36 ​​calculates the camera parameters CP based on the positions of feature points on each calibration pattern image IM detected by the feature point detection unit 34 and the pattern information set by the pattern information setting unit 32. The camera parameters CP can be calculated using known methods (e.g., Zhang's method) similarly to those in the first embodiment.

[0219] The camera parameter CP calculated in parameter calculation unit 36 ​​is the internal parameter of camera 12 (focal length f). x f y Optical Center C x c y The parameter calculation unit 36 ​​stores the calculated camera parameters CP in the storage unit 22, along with distortion parameters (distortion coefficients k1, k2, k3, P1, P2).

[0220] (Step S118: Output step)

[0221] Next, the parameter calculation unit 36 ​​outputs the calculated result of the camera parameter CP to the output unit 16. Therefore, since the operator can obtain the camera parameter CP, image correction and other actions can be performed based on the camera parameter CP on the image captured by the camera 12.

[0222] The flowchart of the overall calibration process performed by the calibration device 10 is now complete.

[0223] [Example of the second embodiment]

[0224] Figure 13 This is a graph showing the tilt resistance of the calibration method. Figure 13 The horizontal axis represents the tilt angle (degrees) of the camera relative to the calibration pattern, and the vertical axis represents the pixel error (pixels).

[0225] An embodiment of the second implementation (defocus) illustrates an example of photographing and calibrating the target involved in the second implementation described above in a defocused state. Comparative Example 1 illustrates the use of Figure 25 An example of calibrated dot patterns as illustrated in 1025C, and Comparative Example 2 shows the use of... Figure 25 The checkerboard pattern illustrated in 1025B is an example of calibration.

[0226] In the case of Comparative Example 1, such as Figure 27 As shown, since the part closer to the camera is captured more, the larger the tilt angle (absolute value) relative to the dot pattern, the larger the pixel error (absolute value) and the worse the tilt resistance.

[0227] In this embodiment, the pixel error (absolute value) is reduced regardless of the tilt angle (absolute value) of the camera 12 relative to the target. Similarly, in Comparative Example 2, the pixel error (absolute value) is reduced regardless of the tilt angle (absolute value) of the camera relative to the checkerboard pattern.

[0228] Figures 14 to 16 The graphs show the brightness noise resistance of the calibration method, corresponding to Example 1 (defocus), Comparative Example 1 (dot pattern), and Comparative Example 2 (checkerboard pattern). Figures 14 to 16 The horizontal axis represents the sample number, and the vertical axis represents the pixel error (pixels).

[0229] exist Figures 14 to 16 In the process of shooting the calibration pattern ( Figure 10 CA, Figure 25 The additional luminance noise of 1025C and (1025B) increases in the order of noise = 0.0, 4.0, and 8.0.

[0230] In Comparative Example 2 ( Figure 16In the example, because the brightness gradient (differential value) is used when detecting the edge of the checkerboard pattern, the pixel error increases significantly due to the brightness noise. This indicates that... Figures 14 to 16 The example shown is the most sensitive to luminance noise.

[0231] On the other hand, Comparative Example 1 ( Figure 15 Compared to the checkerboard pattern, the dot pattern has higher resistance to brightness noise.

[0232] Furthermore, in the embodiments ( Figure 14 In the case of Comparative Examples 1 and 2, the light noise resistance is higher.

[0233] The results summarized in the table below show that the embodiment exhibits good tilt resistance and brightness noise resistance.

[0234] [Table 1]

[0235]

[0236] [Effects of the second embodiment]

[0237] According to the second embodiment, calibration processing is performed based on multiple calibration images IM obtained by capturing a calibration pattern CA containing a tiny target T in a defocused state using the camera 12. Therefore, even when the calibration pattern CA is captured at an angle relative to the camera 12, it is not affected by the non-telecentric optical system, and the position of the feature point (the centroid position of the target T) can be detected with high accuracy. Furthermore, by capturing the target T in a defocused state, the position of the feature point can be detected while effectively suppressing the influence of brightness noise accompanying the minimization of the target T. Therefore, feature points in the calibration images IM can be detected with good accuracy, improving the calibration accuracy of the camera 12.

[0238] <Third Implementation Method>

[0239] In addition to the issues addressed in this invention as described above, there are other aspects to consider. Specifically, when using planar patterns such as dot patterns, grid patterns, and checkerboard patterns as calibration devices, it is sometimes impossible to accurately determine the positional errors of the feature points on each pattern. Therefore, the difficulty in calibrating the calibration device becomes a factor leading to a decrease in the calibration accuracy of the camera.

[0240] Therefore, in the third embodiment, the objective is to provide a camera calibration method that can improve the accuracy of camera calibration.

[0241] The third embodiment will be described below. Furthermore, regarding the calibration device 10, since it differs from the first embodiment (see...) Figure 2 The same applies, therefore the description is omitted. Furthermore, in the third embodiment, the calibration device MR (see below) is used instead of the calibration pattern described above. Figure 17 as well as Figure 18 ).

[0242] Figure 17 This is a three-dimensional view showing the calibration device MR. Figure 18 yes Figure 17 AA-AA sectional view.

[0243] The calibration device MR is a device (spherical target) in which multiple spheres B are arranged on the housing F. The spacing and diameter of the spheres B are known. Information related to the calibration device MR (e.g., information related to the type of calibration device MR, the arrangement, spacing and diameter of the spheres B, hereinafter referred to as pattern information) is stored in the storage unit 22 of the calibration device 10.

[0244] The shell F is, for example, a planar (plate-like) component formed from a material with high rigidity and long-term stability (e.g., metal or ceramic). In the shell F, multiple recesses H are formed in an array.

[0245] Sphere B is, for example, a spherical component formed from a material with high rigidity and long-term stability (e.g., metal). Sphere B is fixed to the housing F by being embedded in the recess H. Sphere B is an example of a reflector, possessing the reflective property of reflecting illumination light incident perpendicularly to its surface in its incident direction (normal reflection). Sphere B has a high sphericity (e.g., refer to Japanese Industrial Standard JIS B 1501:2009), and its surface is mirror-finished. Here, the surface roughness (arithmetic mean roughness) Ra (Japanese Industrial Standard JIS B0601:2001) of the mirror surface is, for example, 0.2 μm or less.

[0246] Furthermore, as long as the reflected light described later can be detected, the surface of sphere B may not be a mirror surface but a rough surface. Here, a rough surface refers to a surface that is rougher than a mirror surface (in one example, Ra>0.2μm) but satisfies at least one of the following conditions.

[0247] (A) A surface capable of reflecting positively reflected light (L21 and L22) of a given intensity or higher.

[0248] (B) A surface can be obtained from diffusely reflected light that is incident non-perpendicularly to the surface of sphere B from light source 50 and diffusely reflected, and a surface of identifiable (e.g., stronger than diffusely reflected light) orthogonal reflected light (L21 and L22) (the difference between the intensity of the reflected light of the illumination light incident perpendicularly to the surface of sphere B and the intensity of the diffusely reflected light diffusely reflected around the perpendicularly incident position is a threshold above).

[0249] Alternatively, sphere B can be a sphere made of glass or sapphire with retroreflective properties (refer to Japanese Industrial Standard JIS Z8713:1995).

[0250] In addition, Figure 17 as well as Figure 18 In the example shown, the spheres B are arranged at equal intervals, but this is not a limitation. For example, the intervals d1 and d2 of the spheres B only need to be known, so the intervals d1 and d2 of the spheres B can be different, or they can be non-equally spaced. In addition, the diameters of the spheres B can be different from each other. Furthermore, the correction device MR can be, for example, a one-dimensional corrector (sphere / rod type corrector) in which multiple spheres B are arranged in a straight housing (rod), or a corrector in which multiple spheres B are arranged in an L-shaped housing (rod).

[0251] The calibration apparatus MR, constructed as described above, can determine the position (three-dimensional coordinates) of each sphere B disposed on the housing F using a high-precision measuring device (a traceable three-dimensional coordinate measuring device). Therefore, traceable calibration of the actual calibration apparatus MR can be performed. Consequently, high-precision calibration of the camera 12 can be performed based on high-precision pattern information (including information related to the arrangement, spacing, and diameter of the spheres B).

[0252] Furthermore, in the third embodiment, sphere B is shown as an example of a reflector, but it is not limited to this. For example, a correction device equipped with a retroreflector (refer to Japanese Industrial Standard JIS Z8713:1995; for example, a corner cube prism, etc.) can also be used. Moreover, compared to sphere B, the range of tilt angles from which retroreflective performance can be obtained for incident light is sometimes narrower with a retroreflector, but a retroreflector can be used when using an imaging system that allows for a tilt angle.

[0253] To simplify the explanation, the following case will be used as an example: Spheres B are arranged at equal intervals along two mutually orthogonal directions, and the intervals d1 and d2 between adjacent spheres B along these two directions are equal.

[0254] Figure 19 This is a diagram illustrating an example of calibration using a calibration apparatus (MR). Furthermore, in Figure 19 In the example shown, for the sake of simplifying the illustration, a light source 50 is arranged on the perpendicular bisecting line of the line segment connecting the centers of spheres B1 and B2, but it is not limited to this.

[0255] like Figure 19As shown, when photographing the calibration apparatus MR, illumination light (coaxial illumination) is irradiated onto the calibration apparatus MR coaxially with the lens 124 of the camera 12. The imaging system of the calibration apparatus MR includes a light source 50, a semi-reflective mirror 52, and a camera 12. The camera 12 includes a lens (objective lens) 124 and an imaging element (e.g., including a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor)) 126. The calculation and control unit 20 of the calibration device 10 (see reference) Figure 2 It performs light emission control of light source 50 and image capture control of camera 12, etc.

[0256] The light source 50 is a device that illuminates the MR calibration apparatus with visible light, and may include, for example, a light-emitting diode. The light source 50 is positioned optically conjugate to the camera origin (object-side principal point) Oc via the semi-reflective mirror 52. Here, the camera origin Oc is, for example, located at the center (optical axis center) of the lens 124 of the camera 12.

[0257] The components L11 and L12 of the illumination light from light source 50 that are perpendicularly incident on the surfaces of spheres B1 and B2 are reflected (retroreflected) along their respective incident light paths toward light source 50. Furthermore, the reflected light L21 and L22, reflected from the surfaces of spheres B1 and B2, are reflected by the semi-reflective mirror 52, pass through the lens 124, and are imaged onto the imaging element 126. Thus, an image of the retroreflected light L21 and L22 from spheres B1 and B2 is obtained.

[0258] Figure 19 Reference numerals P1 and P2 indicate the positions (focusing positions) where reflected light L21 and L22 from spheres B1 and B2 are imaged in the imaging element 126, and reference numeral P0 indicates the conjugate image of the light source 50.

[0259] The image acquisition unit 30 acquires a calibration image IM of images P1 and P2, which include reflected light L21 and L22, captured by the imaging element 126.

[0260] The pattern information setting unit 32 obtains pattern information related to the calibration instrument MR from the storage unit 22.

[0261] The feature point detection unit 34 detects the images P1 and P2 (bright field observation images) of reflected light L21 and L22, and detects the centroid position of each image P1 and P2.

[0262] The parameter calculation unit 36 ​​calculates the camera parameter CP based on the position of the feature points (the centroids of the images P1 and P2 of the reflected light L21 and L22) detected by the feature point detection unit 34 and the pattern information set by the pattern information setting unit 32 (for example, information related to the configuration and spacing of spheres B1 and B2 in the calibration device MR).

[0263] As described above, in the third embodiment, the light source 50 is positioned conjugate to the camera origin Oc (object-side principal point) via the semi-reflective mirror 52. Therefore, the imaging center (center of gravity) of the light source 50 reflected by the correction device MR falls on the line connecting the viewpoint (camera origin Oc) and the center of the correction device MR.

[0264] In the third embodiment, reflected light L21 and L22, which are perpendicularly reflected from the surfaces of spheres B1 and B2, are observed. Therefore, according to the third embodiment, distortion and pixel errors caused by pattern tilt are not generated, and the detection pixel position accuracy of the calibration device MR is not affected by the orientation of the calibration device MR. Thus, high-precision calibration of the camera 12 can be performed. Furthermore, a traceable calibration device can be provided.

[0265] In addition, Figure 19 In the example shown, reflected light L21 and L22 from two spheres B1 and B2 were captured, but the number of spheres used in the calibration is not limited to two; more than three spheres can also be used.

[0266] In addition, Figure 19 In the example shown, the illumination light from the transmissive semi-reflector 52 reaches spheres B1 and B2, and the reflected light from the semi-reflector 52 reaches the camera 12. However, the configuration of the optical system in the camera system is not limited to this. For example, the illumination light reflected by the semi-reflector 52 may reach spheres B1 and B2, and the reflected light from the transmissive semi-reflector 52 may reach the camera 12.

[0267] Hereinafter, the processing steps (an example of a calibration method) of the calibration process performed by the calibration device 10 of the third embodiment will be described. Figure 20 This is a flowchart illustrating the overall process of the calibration procedure performed by the calibration apparatus 10 of the third embodiment. Furthermore, at the beginning... Figure 20 As shown in the flowchart, it is assumed that various initial setting processes, such as confirming the operation of each part of the calibration device 10, have been performed.

[0268] (Step S210: Image acquisition step)

[0269] First, the image acquisition unit 30 acquires multiple calibration images IM captured by the camera 12 and stores them in the storage unit 22.

[0270] The image acquisition unit 30 can acquire each calibration image IM from the camera 12 via a cable or recording medium. Alternatively, each calibration image IM can be pre-stored on an external storage device such as an external server located outside the calibration device 10, and the image acquisition unit 30 can acquire each calibration image IM from the external storage device via a wired or wireless network.

[0271] (Step S212: Pattern Information Setting Step)

[0272] Next, the pattern information setting unit 32 sets pattern information related to the calibration apparatus MR. The pattern information includes, for example, information related to the arrangement and spacing of spheres B1 and B2 in the calibration apparatus MR. The pattern information is used to define points (target points) in three-dimensional space corresponding to feature points (image points) in the image coordinate system.

[0273] The pattern information setting unit 32 can, for example, acquire pattern information input by the operator via the operation unit 14. Alternatively, pattern information can be pre-stored in the storage unit 22, and the pattern information setting unit 32 can retrieve it from the storage unit 22. Furthermore, the pattern information setting step only needs to be performed at least before the parameter calculation step described later; for example, it can be performed before the image acquisition step or after the feature point detection step.

[0274] (Step S214: Feature point detection step)

[0275] Next, the feature point detection unit 34 performs feature point detection processing for each of the multiple calibration images IM captured by the camera 12, detecting multiple feature points. Specifically, the feature point detection unit 34 sequentially reads the multiple calibration images IM stored in the storage unit 22. Furthermore, for each read calibration image IM, the feature point detection unit 34 performs a given image processing (e.g., binarization or grayscale transformation), then detects each feature point (image point) from the calibration image IM and calculates the coordinates (pixel coordinates) of each feature point in the image coordinate system. In this embodiment, for example, from... Figure 19 The bright-field observation images of spheres B1 and B2 shown are used to detect the reflected light L21 and L22 images P1 and P2, and the centroid positions of each image P1 and P2 are detected as feature points. The coordinates of the feature points detected by the feature point detection unit 34 are temporarily stored in the storage unit 22.

[0276] In the case where there is a calibration image IM in which feature point detection fails among multiple calibration images IM, the feature point detection unit 34 performs an exclusion process to exclude the calibration image IM in which feature point detection failed from the parameter calculation process described later. As a result, in the parameter calculation process, the camera parameter CP can be calculated based on the calibration image IM after the exclusion process among multiple calibration images IM (i.e., the calibration image IM in which feature point detection is successful).

[0277] (Step S216: Parameter Calculation Step)

[0278] Next, the parameter calculation unit 36 ​​performs the calculation process for the camera parameters CP. Specifically, the parameter calculation unit 36 ​​calculates the camera parameters CP based on the positions of feature points on each calibration image IM detected by the feature point detection unit 34 and the pattern information set by the pattern information setting unit 32. The camera parameters CP can be calculated using known methods (e.g., Zhang's method) similarly to those in the first embodiment.

[0279] The camera parameter CP calculated in parameter calculation unit 36 ​​is the internal parameter of camera 12 (focal length f). x f y Optical Center C x c y The parameter calculation unit 36 ​​stores the calculated camera parameters CP in the storage unit 22, along with distortion parameters (distortion coefficients k1, k2, k3, P1, P2).

[0280] (Step S218: Output step)

[0281] Next, the parameter calculation unit 36 ​​outputs the calculated result of the camera parameter CP to the output unit 16. Therefore, since the operator can obtain the camera parameter CP, image correction and other actions can be performed based on the camera parameter CP on the image captured by the camera 12.

[0282] The flowchart of the overall calibration process performed by the calibration device 10 is now complete.

[0283] [Example of the third embodiment]

[0284] Figure 21 This is a graph showing the tilt resistance of the calibration method. Figure 21 The horizontal axis represents the tilt angle (degrees) of the camera relative to the calibration device (MR) or calibration pattern, and the vertical axis represents the pixel error (pixels).

[0285] The third embodiment illustrates an example of calibration using a calibration apparatus MR (sphere target). Comparative Example 1 illustrates the use of... Figure 25An example of calibration using the dot pattern exemplified in 1025C, Comparative Example 2 shows the use of... Figure 25 An example of calibration using the checkerboard pattern illustrated in 1025B.

[0286] In the case of Comparative Example 1, such as Figure 27 As shown, since the part closer to the camera is captured more, the larger the tilt angle (absolute value) relative to the dot pattern, the larger the pixel error (absolute value) and the worse the tilt resistance.

[0287] In this embodiment, the pixel error (absolute value) is reduced regardless of the tilt angle (absolute value) of the camera 12 relative to the calibration device MR. Similarly, in Comparative Example 2, the pixel error (absolute value) is reduced regardless of the tilt angle (absolute value) of the camera relative to the checkerboard pattern.

[0288] Figures 22 to 24 The graphs show the resistance to brightness noise of the calibration method, corresponding to the example (spherical target), comparative example 1 (dot pattern), and comparative example 2 (checkerboard pattern), respectively. Figures 22 to 24 The horizontal axis represents the sample number, and the vertical axis represents the pixel error (pixels). Furthermore, to prevent chart compression, only the chart is narrowed. Figure 22 The range of the vertical axis.

[0289] exist Figures 22 to 24 In this process, the added luminance noise during the capture of the calibration instrument MR or calibration pattern increases in the order of noise = 0.0, 4.0, and 8.0.

[0290] In Comparative Example 2 ( Figure 24 In the example, because the brightness gradient (differential value) is used when detecting the edge of the checkerboard pattern, the pixel error increases significantly due to the brightness noise. This indicates that... Figures 22 to 24 The example shown is the most sensitive to luminance noise.

[0291] On the other hand, Comparative Example 1 ( Figure 23 Compared to the checkerboard pattern, the dot pattern has higher resistance to brightness noise.

[0292] Furthermore, in the embodiments ( Figure 22 As can be clearly seen from the range of the vertical axis, compared with either Comparative Example 1 or 2, the light noise resistance is higher.

[0293] The results summarized in the table below show that the embodiment exhibits good tilt resistance and brightness noise resistance.

[0294] [Table 2]

[0295]

[0296] [Effects of the third embodiment]

[0297] According to the third embodiment, illumination light (coaxial illumination light) is emitted from a light source 50 located at a position conjugate to the camera origin Oc, illuminating a calibration apparatus MR containing a reflector (sphere B or retroreflector) that has reflective properties that reflect illumination light back in the incident direction. The camera 12 then captures the reflected light from the reflector to obtain a calibration image IM. Furthermore, by detecting the position of the reflected light as a feature point based on this calibration image IM, high-precision calibration with excellent tilt resistance and brightness noise resistance can be achieved.

[0298] Furthermore, according to the third embodiment, by using the calibration apparatus MR as described above, the position (three-dimensional coordinates) of the reflector (sphere B or retroreflector) can be measured using a high-precision measuring device (a traceable three-dimensional coordinate measuring device). Thus, traceable calibration of the calibration apparatus MR can be performed, thereby improving calibration accuracy.

[0299] The embodiments of the present invention have been described above. However, the present invention is not limited to the above examples. Of course, various improvements and modifications can be made without departing from the spirit of the present invention.

[0300] -Explanation of Figure Markers-

[0301] 10…calibration device, 12…camera, 14…operation unit, 16…output unit, 20…computation control unit, 22…storage unit, 30…image acquisition unit, 32…pattern information setting unit, 34…feature point detection unit, 36…parameter calculation unit, 50…light source, 52…semi-reflective mirror, 120…lens, 122…image sensor, 124…lens, 126…image sensor, IM…calibration image, IP…image plane, VP…virtual plane, DI…dot image, VC…center, IC…projection point, B…ray group, T…intersection group, CL…straight line, DP…dot pattern, D…dot, T, T1, T2…target, F1…focus position, F2…defocus position, F3…defocus position, MR…correction device, B…sphere, F…shell.

Claims

1. A method for calibrating a camera, characterized in that, Include: The image acquisition step involves acquiring multiple calibration images obtained by capturing a dot pattern with multiple dots periodically arranged by a camera. The pattern information setting step involves setting pattern information that includes information related to the configuration of the dots. The feature point detection step involves detecting feature points representing the positions of the dots from the calibration image. and The parameter calculation step calculates the camera parameters based on the feature points and the pattern information. In the feature point detection step, the position of the centroid of the circle is detected as the feature point by projecting the outline of the circle on the image plane of the calibration image onto a projection image set in a virtual plane in three-dimensional space.

2. The camera calibration method according to claim 1, wherein, In the feature point detection step, a virtual plane optimized to fit the circle to the projected image is obtained. Based on the projected image projected onto the obtained virtual plane, the position of the centroid of the circle is detected as the feature point.

3. The camera calibration method according to claim 1, wherein, In the feature point detection step, when a circle obtained by approximating a group of points composed of multiple projection points is defined as an approximate circle, a virtual plane optimized to minimize or reduce the evaluation value representing the fitting error of the approximate circle relative to the group of points is determined. The position of the point on the optimized virtual plane that projects the center of the approximate circle onto the image plane is defined as the position of the centroid of the circle. The multiple projection points are obtained by projecting multiple contour points representing the contour of the circle on the image plane onto the virtual plane.

4. The camera calibration method according to claim 3, wherein, The feature point detection step includes: a search step, which searches for the optimized virtual plane in three-dimensional space from multiple virtual planes with different poses, based on an evaluation value representing the fitting error of the approximate circle relative to the point group.

5. The camera calibration method according to claim 4, wherein, In the search step, while changing the virtual plane to multiple poses by rotating the virtual plane around an axis parallel to the major axis of the dots on the image plane, an evaluation value is calculated for each pose, and the optimized virtual plane is determined based on the evaluation value calculated for each pose.

6. The camera calibration method according to claim 4, wherein, The feature point detection step includes: The contour point extraction step involves extracting the plurality of contour points of the circle from the image plane. The ray group generation step generates a ray group, which is composed of multiple rays that take the optical center of the camera as the starting point and pass through the multiple contour points of the circle on the image plane respectively. and The approximate circle calculation step involves calculating the approximate circle based on the intersection group of the ray group and the virtual plane.

7. A camera calibration device, characterized in that, have: The image acquisition unit acquires multiple calibration images obtained by a camera capturing a dot pattern in which multiple dots are periodically arranged. The pattern information setting unit sets pattern information that includes information related to the arrangement of the dots in the dot pattern; The feature point detection unit detects feature points from the calibration image; and The parameter calculation unit calculates the parameters of the camera based on the feature points and the pattern information. The feature point detection unit detects the position of the centroid of the dot as the feature point by projecting the outline of the dot on the image plane of the calibration image onto a projection image set in a virtual plane in three-dimensional space.

8. A method for calibrating a camera, characterized in that, Include: The image acquisition step involves acquiring multiple calibration images obtained by capturing a calibration pattern containing a tiny target in a defocused state. The pattern information setting steps involve setting the pattern information related to calibration. The feature point detection step involves detecting the centroid position of the defocused image of the target as a feature point from the calibration image. and The parameter calculation step calculates the camera parameters based on the pattern information and the positions of the feature points.

9. The camera calibration method according to claim 8, wherein, The size of the target is determined by at least one of the following: the focal length of the camera that captures the target, the distance between the camera and the target, and the pixel size of the camera's image sensor.

10. The camera calibration method according to claim 8, wherein, The target is such that the size of the image on the camera's image sensor obtained when shooting in focus mode is less than 1 pixel.

11. The camera calibration method according to any one of claims 8 to 10, wherein, The pattern information includes information related to the configuration of the target in the calibration pattern.

12. The camera calibration method according to any one of claims 8 to 10, wherein, The target is linearly symmetrical or point-symmetrical in two mutually orthogonal directions.

13. The camera calibration method according to claim 12, wherein, The target is in the shape of a round dot.

14. The camera calibration method according to claim 12, wherein, The target is a point light source.

15. A method for calibrating a camera, characterized in that, have: The image acquisition step involves capturing reflected light from the illumination light illuminating the calibration apparatus from the light source using a camera positioned optically conjugate with the light source, thereby acquiring a calibration image. The calibration apparatus includes a reflector having reflective properties that reflect the illumination light back in the incident direction. The pattern information setting step involves setting pattern information related to the calibration device. The feature point detection step involves identifying the focal point of the reflected light from the calibration image as a feature point. and The parameter calculation step calculates the camera parameters based on the pattern information and the positions of the feature points.

16. The camera calibration method according to claim 15, wherein, In the image acquisition step, the camera captures the reflected light from at least two reflectors of the calibration device.

17. The camera calibration method according to claim 15, wherein, The pattern information includes information relating to the configuration and spacing of the reflectors in the correction device.

18. The camera calibration method according to any one of claims 15 to 17, wherein, The reflectors are spherical in shape and are arranged in an array on the correction device.

19. The camera calibration method according to claim 18, wherein, The surface of the reflector is either a mirror or a rough surface that allows for diffuse reflection and identifiable orthogonal reflection when light is incident on the surface of the reflector non-perpendicularly.

20. The camera calibration method according to claim 18, wherein, The reflector is made of glass or sapphire.

Images