Calibration method and calibration system

Abstract

To provide a calibration method and a calibration system that can easily execute an imaging device.SOLUTION: A calibration method using a display 3, comprises: an acquisition step; a calculation step; a first leading step; a second leading step; and an optimization step. The acquisition step uses an imaging device 2 as an object of a calibration to acquire an image obtained by imaging a pattern 31 displayed onto the display 3. The calculation step calculates a homography matrix on the basis of a display condition when displaying the pattern 31 onto the display 3, and an imaging shape of a pattern image in the image. The first leading step leads an image center in the imaging device 2 on the basis of the imaging shape. The second leading step leads a focal distance on the basis of the image center.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This invention relates to a calibration method and a calibration system.

Background Art

[0002] Conventionally, as calibration methods for imaging devices using a display, technologies such as those disclosed in Patent Document 1 and Non-Patent Document 1 have been proposed.

[0003] The camera calibration device disclosed in Patent Document 1 includes an image acquisition circuit, a first posture information acquisition circuit, a second posture information acquisition circuit, a calculation circuit, and a correction circuit. The image acquisition circuit acquires a captured image of a calibration marker provided on a display device (such as a display) equipped with an attitude sensor, taken by a camera. The first posture information acquisition circuit acquires the first posture information of the calibration marker at the time of capturing the captured image. The second posture information acquisition circuit acquires the second posture information of the mounting device (such as a vehicle) of the camera. The calculation circuit calculates a calibration amount corresponding to the camera posture information of the camera derived based on the calibration marker included in the captured image. The correction circuit corrects the calibration amount calculated by the calculation circuit based on the difference between the first posture information and the second posture information.

[0004] Non-Patent Document 1 discloses a method of performing calibration based on the result of estimating camera parameters using a virtual pattern displayed on a display for which the relative position from the camera has been specified.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Here, Patent Document 1 assumes correction based on the difference between the first posture information of the configuration marker in the display device and the second posture information of the mounting device of the camera. Therefore, the calibration of the camera itself is not considered.

[0008] Also, in Non-Patent Document 1, it is necessary to grasp in advance the relative positions of the camera and the display, and it cannot be used when the relationship between the postures is unclear as in Patent Document 1. Therefore, a method that can easily perform calibration of imaging devices such as cameras is desired.

[0009] Therefore, the present invention has been devised in view of the above problems, and its object is to provide a calibration method and a calibration system that can easily perform calibration of an imaging device.

Means for Solving the Problems

[0010] The calibration method according to the first invention is a calibration method using a display, comprising: an acquisition step of acquiring an image obtained by imaging a pattern displayed on the display using an imaging device to be calibrated; a calculation step of obtaining a homography matrix based on the display conditions when the pattern is displayed on the display and the imaging shape of the pattern image in the image; a first derivation step of deriving the image center in the imaging device based on the imaging shape; a second derivation step of deriving the focal length based on the image center; and an optimization step of solving a non-linear equation using the homography matrix, the image center, and the focal length to derive the lens distortion of the imaging device.

[0011] The calibration method according to the second invention is characterized in that, in the first invention, prior to the acquisition step, it further comprises an installation step of arranging the planar display surface on the display at an inclination greater than 0° and less than 90° with respect to the optical axis of the imaging device.

[0012] The calibration method according to the third invention is characterized in that, in the second invention, the acquisition step includes acquiring the image obtained by imaging an auxiliary pattern displayed on an auxiliary display different from the display in addition to the pattern, and the display surface of the auxiliary display is arranged in a direction different from the display surface of the display.

[0013] The calibration method according to the fourth invention is characterized in that, in any one of the first to third inventions, it further comprises a luminance correction step of correcting the luminance of the display based on the image.

[0014] The calibration method according to the fifth invention is characterized in that, in the fourth invention, it further comprises a pattern correction step of correcting the imaging shape to be circular.

[0015] The calibration method according to the sixth invention is, in the fifth invention, characterized in that the optimization step includes a re-acquisition step of obtaining a new image by changing the positions of the patterns displayed on the display and combining the results of imaging a plurality of times, and a distortion derivation step of solving a non-linear equation using the new image, the homography matrix, the image center, and the focal length to derive the lens distortion of the imaging device.

[0016] The calibration system according to the seventh invention is a calibration system using a display, comprising: an acquisition unit that acquires an image obtained by imaging a pattern displayed on the display using an imaging device to be calibrated; a calculation unit that obtains a homography matrix based on the display conditions when the pattern is displayed on the display and the imaging shape of the pattern image in the image; a first derivation unit that derives the image center in the imaging device based on the imaging shape; a second derivation unit that derives the focal length based on the image center; and an optimization unit that solves a non-linear equation using the homography matrix, the image center, and the focal length to derive the lens distortion of the imaging device.

Advantages of the Invention

[0017] According to the first to sixth inventions, the acquisition step acquires an image obtained by imaging a pattern displayed on a display using an imaging device to be calibrated. Further, the first derivation step derives the image center in the imaging device based on the imaging shape of the pattern image in the image. Further, the second derivation step derives the focal length based on the image center. Therefore, the image center and the focal length necessary for calibration can be derived without using the relative positions of the imaging device and the display. As a result, it becomes possible to easily perform calibration of the imaging device.

[0018] In particular, according to the second invention, in the installation step, the planar display surface in the display is arranged at an inclination greater than 0° and less than 90° with respect to the optical axis of the imaging device. Therefore, as information on the pattern image captured in the image, three-dimensional position information can be included. This makes it possible to improve the derivation accuracy of the focal length.

[0019] In particular, according to the third invention, the acquisition step includes acquiring an image obtained by imaging an auxiliary pattern displayed on an auxiliary display in addition to the pattern. Therefore, by using the position information of the pattern and the auxiliary pattern captured in the image, the influence of noise can be reduced. This makes it possible to improve the derivation accuracy of various parameters.

[0020] In particular, according to the fourth invention, in the luminance correction step, the luminance of the display is corrected based on the image. Therefore, when imaging the pattern, the variation in luminance values across the entire pattern can be suppressed. This makes it possible to improve the processing accuracy using the pattern image.

[0021] In particular, according to the fifth invention, in the pattern correction step, the imaging shape is corrected to be circular. This makes it possible to improve the calibration accuracy.

[0022] In particular, according to the sixth invention, in the re-acquisition step, a new image is acquired by changing the positions of the patterns to be displayed on the display respectively and combining the results of imaging multiple times. This makes it possible to further improve the accuracy when deriving lens distortion.

[0023] According to the seventh invention, the acquisition unit acquires an image obtained by imaging a pattern displayed on a display using an imaging device to be calibrated. Further, the first derivation unit derives the image center in the imaging device based on the imaging shape of the pattern image in the image. Further, the second derivation unit derives the focal length based on the image center. Therefore, the image center and the focal length necessary for calibration can be derived without using the relative position between the imaging device and the display. As a result, it becomes possible to easily perform calibration of the imaging device.

Brief Description of Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0025] Hereinafter, as an embodiment of the present invention, an example of a calibration method and a calibration system will be described with reference to the drawings. In the following description, coordinates based on the arrangement of the imaging device (world coordinates) and coordinates based on the image captured using the imaging device (image coordinates) will be used.

[0026] The world coordinates indicate coordinates in which the directions intersecting each other along the optical axis of the imaging device are the first direction Xw and the second direction Yw, and the direction along the optical axis is the third direction Zw. Each of the directions Xw, Yw, and Zw may be perpendicular to each other. Also, either the right-handed system or the left-handed system of the world coordinates is arbitrary. The image coordinates indicate coordinates in which the directions perpendicular to each other along the imaging plane in the image are the first image direction x and the second image direction y.

[0027] (First Embodiment) FIG. 1 is a schematic diagram showing an example of the configuration of a calibration system 100 in the present embodiment. The calibration system 100 in the present embodiment includes an imaging device 2 and a display 3, and may include, for example, a control device 1. The imaging device 2 represents a known camera and is used, for example, for appearance inspection in the manufacturing process or character recognition in the logistics process. The display 3 displays a pattern 31 used for calibration of the imaging device 2, as shown in FIG. 2, for example. Display conditions such as the shape and display range of the pattern 31 can be arbitrarily set. The control device 1 represents a known electronic device such as a personal computer and controls the driving of the imaging device 2 and the display 3. Details of the control device 1 will be described later.

[0028] The calibration method in the present embodiment is implemented, for example, using the above-described calibration system 100. By the calibration method, calibration of the imaging device 2 can be easily performed.

[0029] The calibration method includes, for example, as shown in FIG. 3, an acquisition step S110, a calculation step S120, a first derivation step S130, a second derivation step S140, and an optimization step S150. Note that the calibration method may include an installation step, for example, before the acquisition step S110.

[0030] <Installation step> In the installation step, for example, as shown in FIG. 1, a display 3 is installed on the optical axis of the imaging device 2. At this time, the position of the display 3 and the pattern 31 displayed on the display 3 can be indicated using, for example, world coordinates, but it is not necessary to grasp the exact coordinates at this point.

[0031] For example, the planar display surface of the display 3 is installed at an inclination greater than 0° and less than 90° with respect to the optical axis of the imaging device 2. In this case, when deriving the positions of the display 3 and the pattern 31, the element in the third direction Zw can be included. Note that when the display surface of the display 3 is curved, it may be installed other than the above inclination.

[0032] <Acquisition step S110> In the acquisition step S110, for example, as shown in FIG. 4(a), an image 21f obtained by imaging the pattern 31 displayed on the display 3 is acquired using the imaging device 2 to be calibrated. The image 21f includes the imaged pattern image 31f and may include, for example, the imaged display image 3f.

[0033] The pattern 31 is displayed in a shape arranged parallel to the display surface of the display 3, and may also be displayed in a shape with an inclination provided with respect to the display surface. The display conditions when displaying the pattern 31 on the display 3 can be set arbitrarily.

[0034] <Calculation step S120> Calculation step S120 obtains a homography matrix H based on the display conditions when displaying pattern 31 on display 3 and the imaging shape of pattern image 31f in image 21f. The homography matrix H can be obtained using, for example, known projective transformation techniques.

[0035] By using the homography matrix H, for example, as shown in FIG. 4(b), it is possible to convert from image 21f to converted image 21s. At this time, the converted image 21s may include a pattern-converted image 31s obtained by converting the pattern image 31f, and may also include, for example, a display-converted image 3s obtained by converting the display image 3f. Note that, for example, the pattern-converted image 31s may be included in the converted image 21s with a part of the pattern image 31f deleted. Also, by obtaining the homography matrix H, the positions of the display 3 and the pattern 31 in the world coordinates can be specified.

[0036] <First Derivation Step S130> The first derivation step S130 derives the image center in the imaging device 2 based on the imaging shape of the pattern image 31f. The image center can be derived, for example, by calculating the degree of distortion in the imaging shape of the pattern image 31f. For example, as shown in FIG. 5, six specific two-point distances in the pattern image 31f (distances d11, d12, d13, d21, d22, d23 in FIG. 5) are calculated, and the intersection point 21c where the minimum two-point distances in the first image direction x and the second image direction y (distances d11 and d21 in FIG. 5) intersect is specified, and the intersection point 21c can be derived as the image center.

[0037] In addition to the above, for example, the image center can be derived using OpenCV (registered trademark). In this case, by using half of the values in the size of the image 21f (for example, in the case of an image size of 1280 × 1024, (640, 512)) as the initial value of the image center and using the lens design value (for example, f = 12.8 mm or 8 mm, etc.) as the initial value of the focal length, the image center can be derived. Note that in the first derivation step S130, instead of the above-described pattern image 31f, for example, the image center may be derived using the pattern conversion image 31s.

[0038] <Second Derivation Step S140> In the second derivation step S140, the focal length is derived based on the image center. In the second derivation step S140, the focal length can be derived using, for example, OpenCV (registered trademark) with the image center derived in the first derivation step S130. Note that, for example, the first derivation step S130 and the second derivation step S140 may be performed at once using OpenCV (registered trademark).

[0039] <Optimization Step S150> The optimization step S150 solves a non-linear equation using the homography matrix H, the image center, and the focal length, and derives the lens distortion of the imaging device 2. The optimization step S150 can be performed using, for example, OpenCV (registered trademark). Note that as the non-linear equation, a known equation described in, for example, Equation (10) of Non-Patent Document 1 can be used.

[0040] For example, in the optimization step S150, a new image 21 may be acquired before solving the non-linear equation. In this case, by acquiring a new image 21 with a wider imaging range than the image 21f acquired in the acquisition step S110, for example, it becomes possible to derive the lens distortion with high accuracy.

[0041] By performing each of the above steps, the calibration method in the present embodiment is completed. In the calibration method, the number of times of performing each of the above steps and the timing of performing them can be arbitrarily set.

[0042] According to the present embodiment, in the acquisition step S110, an image 21f obtained by imaging a pattern 31 displayed on the display 3 is acquired using the imaging device 2 to be calibrated. Further, in the first derivation step S130, the image center in the imaging device 2 is derived based on the imaging shape of the pattern image 31f in the image 21f. Further, in the second derivation step S140, the focal length is derived based on the image center. For these reasons, the image center and the focal length necessary for calibration can be derived without using the relative position between the imaging device 2 and the display 3. Thereby, it becomes possible to easily perform the calibration of the imaging device 2.

[0043] Further, according to the present embodiment, in the installation step, the planar display surface of the display 3 is arranged at an inclination greater than 0° and less than 90° with respect to the optical axis of the imaging device 2. For this reason, three-dimensional position information can be included as information on the pattern image 31f imaged in the image 21f. Thereby, it becomes possible to improve the derivation accuracy of the focal length.

[0044] (Second Embodiment) Next, the calibration method and the calibration system 100 in the present embodiment will be described. The difference between the above-described embodiment and the present embodiment is that an auxiliary display 3b is used for calibration. Note that descriptions of the same content as in the above-described embodiment will be omitted.

[0045] The calibration system 100 in the present embodiment includes two displays 3 (main display 3a and auxiliary display 3b). As each of the displays 3a and 3b, the same product may be used, or different products may be used.

[0046] In the above case, for example, in the installation step, each of the displays 3a and 3b is installed at a different angle. At this time, the display surface of the auxiliary display 3b is arranged in a direction different from that of the display surface of the main display 3a.

[0047] Further, the acquisition step S110 acquires an image 21f obtained by imaging an auxiliary pattern 31b displayed on the auxiliary display 3b in addition to the main pattern 31a displayed on the main display 3a. Note that, as each of the patterns 31a and 31b, the same display conditions may be used, or different display conditions may be used. Thereafter, by performing each step similar to the above-described embodiment, the calibration method in the present embodiment is completed.

[0048] According to the present embodiment, the acquisition step S110 includes acquiring an image 21f obtained by imaging an auxiliary pattern 31b displayed on the auxiliary display 3b in addition to the pattern (main pattern 31a). Therefore, by using the position information of the main pattern 31a and the auxiliary pattern 31b imaged in the image 21f, the influence of noise can be reduced. Thereby, it becomes possible to improve the accuracy of deriving various parameters. Note that, in the above description, one auxiliary display 3b has been described, but two or more auxiliary displays 3b may be used. Also in this case, by using the position information of the main pattern 31a and the plurality of auxiliary patterns 31b imaged in the image 21f, the influence of noise can be reduced. Thereby, it becomes possible to further improve the accuracy of deriving various parameters.

[0049] (Third Embodiment) Next, the calibration method and the calibration system 100 in the present embodiment will be described. The difference between the above-described embodiment and the present embodiment is that the present embodiment further includes a luminance correction step. Note that descriptions of the same content as in the above-described embodiment will be omitted.

[0050] Here, known display devices used for the display 3 often have directivity. For this reason, the appearance of the pattern 31 may vary depending on the installation angle of the display 3 and the position within the display surface. Also, there may be defective pixels in a part of the display surface. In these cases, when imaging the pattern 31, variations in luminance values occur depending on the position of the pattern 31, leading to concerns about a decrease in the processing accuracy using the pattern image 31f. In particular, when the display surface is installed with an inclination greater than 0° and less than 90° with respect to the optical axis of the imaging device 2, there is a high possibility of being greatly affected by directivity.

[0051] In contrast, the luminance correction step in this embodiment is implemented to suppress variations in luminance values across the entire pattern 31. Thereby, it is possible to improve the processing accuracy using the pattern image 31f. The details of the luminance correction step will be described below.

[0052] <Luminance Correction Step> The luminance correction step corrects the luminance of the display 3 based on the said image. The luminance correction step first causes the display 3 to display the pattern 31. At this time, for example, by displaying the pattern 31 across the entire display surface, it is possible to perform luminance correction for the entire display surface. Note that, as the pattern 31, for example, a chessboard that is resistant to color changes and is used in the imaging device 2 is used. Note that, as the feature detection algorithm, a known image recognition technique is used and can be arbitrarily set according to the application.

[0053] Next, the luminance correction step uses the imaging device 2 to image the pattern 31 and obtains the image 21. Note that the image 21 may use the image 21f obtained in the acquisition step S110.

[0054] Next, the brightness correction step obtains the imaging position information of the pattern image 31f from the feature points detected based on the pattern image 31f captured in the image 21. Then, the brightness of the display 3 is adjusted so that the entire pattern 31 is displayed in white. At this time, the brightness value (first brightness value) of the darkest pixel is calculated among the pixels of the image 21 that captured the pattern 31.

[0055] Then, in the brightness correction step, the brightness of the display 3 is gradually decreased, and the pattern 31 displayed for each gradually decreased brightness is imaged. From the plurality of images 21 thus obtained, the brightness value for each pixel is calculated. At this time, the brightness of the display 3 when the brightness value of each pixel matches the above-described first brightness value is specified. Thereby, the brightness of the display 3 suitable for each pixel can be associated, and for example, a conversion table for making the brightness values of the pixels of the image 21 uniform can be generated. Note that in the brightness correction step, when imaging the pattern 31 displayed for each brightness of the display 3, the pattern 31 may be imaged multiple times in addition to being imaged once for each brightness of the display 3. In this case, the average value for each pixel may be calculated using the average of the plurality of images 21 obtained by imaging multiple times.

[0056] Thereafter, the brightness adjustment step adjusts the brightness of the display 3 based on, for example, the above-described conversion table. Note that the adjustment of the brightness of the display 3 may be performed each time the brightness value of each pixel is detected, for example, without using the conversion table. Further, the brightness correction step can be performed before and after the above-described acquisition step S110, calculation step S120, first derivation step S130, and second derivation step S140.

[0057] According to the present embodiment, the brightness correction step corrects the brightness of the display 3. Therefore, when imaging the pattern 31, the variation in the brightness values over the entire pattern 31 can be suppressed. Thereby, it becomes possible to improve the processing accuracy using the pattern image 31f.

[0058] (Fourth Embodiment) Next, the calibration method and the calibration system 100 in the present embodiment will be described. The difference between the above-described embodiment and the present embodiment is that the pattern correction step is further provided. Note that the description of the same content as the above-described embodiment will be omitted.

[0059] Here, when a perfect circle is used as the shape of the pattern 31, the calibration accuracy can be improved. However, when the circular pattern 31 displayed on the display is imaged, the image 21 may include an elliptical pattern image 31f. For this reason, compared with the case where a circular pattern image is used, a decrease in calibration accuracy is a concern.

[0060] On the other hand, the pattern correction step in the present embodiment is performed to realize calibration using a circular pattern image 31f or the like. Thereby, the calibration accuracy can be improved. The details of the pattern correction step will be described below.

[0061] <Pattern Correction Step> The pattern correction step corrects the imaged shape of the imaged pattern 31 to be circular. The pattern correction step is performed, for example, after the luminance correction step.

[0062] For example, as shown in FIG. 7(a), the transformed image 21s obtained by projective transformation using the homography matrix H may include an elliptical pattern transformed image 31s. In this case, the pattern correction step obtains a shape homography matrix h for making the imaged shape of the pattern 31 circular. At this time, the position of the pattern 31 displayed on the display 3 is set in world coordinates, and the shape homography matrix h can be obtained using a known projective transformation technique. Then, by applying a transformation matrix (=HhH -1 ) to the image 21, for example, a corrected image 21t shown in FIG. 7(b) and a pattern corrected image 31t included in the corrected image 21t can be obtained.

[0063] According to this embodiment, the pattern correction step corrects the imaging shape so as to be circular. Thereby, it becomes possible to improve the calibration accuracy.

[0064] Note that the pattern correction step may correct the shape of the pattern 31 displayed on the display 3 based on, for example, the imaging shape of the pattern conversion image 31s imaged in the converted image 21s. Even in this case, a pattern correction image 31t can be obtained. Also, the pattern correction step can be performed before and after the above-described acquisition step S110, calculation step S120, first derivation step S130, and second derivation step S140.

[0065] (Fifth Embodiment) Next, the calibration method and the calibration system 100 in this embodiment will be described. The difference between the above-described embodiment and this embodiment is that the optimization step S150 includes a re-acquisition step and a distortion derivation step. Note that descriptions of the same content as in the above-described embodiment will be omitted.

[0066] <Re-acquisition step> In the re-acquisition step, a new image is acquired by combining the results of imaging a plurality of times while changing the positions of the patterns 31 displayed on the display 3. For example, as shown in FIG. 8, each pattern correction image 31t is obtained by superimposing the results of imaging at different display positions as one new image. For example, when each pattern 31 is circular, it is preferably displayed in a state of being arranged at equal intervals. Therefore, the pattern image imaged in the new image is different from the pattern image 31f imaged in the acquisition step S110.

[0067] <Distortion derivation step> The distortion derivation step solves a non-linear equation using a new image, a homography matrix H, the image center, and the focal length to derive the lens distortion of the imaging device 2. The distortion derivation step can be implemented using, for example, OpenCV (registered trademark) in the same manner as the optimization step S150 described above. When the pattern correction step is performed, the distortion derivation step may, in addition to the above, solve a non-linear equation using the shape homography matrix h to derive the lens distortion of the imaging device 2.

[0068] According to this embodiment, the re-acquisition step acquires a new image by combining the results of imaging a plurality of times with the positions of the pattern 31 displayed on the display 3 changed respectively. Thereby, it becomes possible to further improve the accuracy when deriving the lens distortion.

[0069] Here, it is better that there are many patterns 31 uniformly throughout the image. However, for the following reasons, it is difficult to use a pattern 31 with small circles arranged in a fine manner. · When the circles of the pattern 31 are too small, the circles cannot be recognized. · When the circles of the pattern 31 are large, they overlap with the adjacent circles. On the other hand, in the re-acquisition step described above, after imaging one pattern 31 with an appropriate circle size to obtain the dot positions, shifting the position of the pattern 31 displayed on the display 3, imaging again to obtain the dot positions, and combining them, many dots can be obtained and it can be used as if a pattern 31 with fine circles was photographed. Thereby, it becomes possible to further improve the accuracy when deriving the lens distortion.

[0070] <Control device 1> The control device 1 included in the calibration system 100 in the above-described embodiment includes, for example, an acquisition unit, a calculation unit, a first derivation unit, a second derivation unit, and an optimization unit. The acquisition unit can perform the acquisition step S110 described above. The calculation unit can perform the calculation step S120 described above. The first derivation unit can perform the first derivation step S130 described above. The second derivation unit can perform the second derivation step S140 described above. The optimization unit can perform the optimization step S150 described above. Note that the control device 1 may include an auxiliary unit that can perform at least any one of the above-described luminance correction step, pattern correction step, re-acquisition step, and distortion correction step.

[0071] The control device 1 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and a storage unit. The CPU controls the entire control device 1. The ROM stores the operation code of the CPU. The RAM is a work area used during the operation of the CPU. Various information such as the image 21 and functions is stored in the storage unit. As the storage unit, for example, in addition to an HDD (Hard Disk Drive), a data storage device such as an SSD (Solid State Drive) is used. The acquisition unit and the calculation unit are realized, for example, when the CPU executes a program stored in a storage unit or the like using the RAM as a work area.

[0072] In the calibration system 100, for example, at least any one of the acquisition unit, the calculation unit, the first derivation unit, the second derivation unit, and the optimization unit of the control device 1 described above may be included in the imaging device 2 or the display 3.

[0073] Although some embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

Explanation of Signs

[0074] 1: Control device 2: Imaging device 3: Display 3f: Display image 3s: Display conversion image 21: Image 21f: Image 21s: Converted image 21t: Corrected image 31: Pattern 31a: Main pattern 31b: Auxiliary pattern 31f: Pattern image 31s: Pattern conversion image 31t: Pattern corrected image 100: Calibration system S110: Acquisition step S120: Calculation step S130: First derivation step S140: Second derivation step S150: Optimization step Xw: First direction Yw: Second direction Zw: Third direction x: First image direction y: Second image direction

Claims

1. A calibration method using a display, comprising: an acquisition step of acquiring an image obtained by imaging a pattern displayed on the display using an imaging device to be calibrated; a calculation step of obtaining a homography matrix based on the display conditions when the pattern is displayed on the display and the imaging shape of the pattern image in the image; a first derivation step of deriving an image center in the imaging device based on the imaging shape; a second derivation step of deriving a focal length based on the image center; an optimization step of solving a non-linear equation using the homography matrix, the image center, and the focal length to derive lens distortion of the imaging device; comprising A calibration method characterized by the above.

2. Before the acquisition step, further comprising an installation step of arranging a planar display surface on the display at an inclination greater than 0° and less than 90° with respect to the optical axis of the imaging device The calibration method according to claim 1, characterized by the above.

3. The acquisition step includes acquiring the image obtained by imaging an auxiliary pattern displayed on an auxiliary display different from the display in addition to the pattern, wherein a display surface of the auxiliary display is arranged in a direction different from a display surface of the display The calibration method according to claim 2, characterized by the above.

4. Further comprising a luminance correction step of correcting the luminance of the display based on the image The calibration method according to any one of claims 1 to 3, characterized by the above.

5. Further comprising a pattern correction step of correcting so that the imaging shape becomes circular The calibration method according to claim 4, characterized by the above.

6. The optimization step includes: a re-acquisition step of changing the positions of the patterns to be displayed on the display respectively, and combining the results of imaging a plurality of times to obtain a new image; a distortion derivation step of solving a non-linear equation using the new image, the homography matrix, the image center, and the focal length to derive lens distortion of the imaging device including The calibration method according to claim 5, characterized by the above.

7. A calibration system using a display, comprising: An acquisition unit that acquires an image obtained by imaging a pattern displayed on the display using an imaging device to be calibrated; An arithmetic unit that obtains a homography matrix based on the display conditions when the pattern is displayed on the display and the imaging shape of the pattern image in the image; A first derivation unit that derives the image center in the imaging device based on the imaging shape; A second derivation unit that derives the focal length based on the image center; An optimization unit that solves a non-linear equation using the homography matrix, the image center, and the focal length, and derives the lens distortion of the imaging device; Comprising A calibration system characterized by the above.

Images