Camera calibration device

Abstract

A camera calibration device according to the present invention comprises: a light source unit for emitting light toward a camera; a diffraction unit which is arranged between the camera and the light source unit and which diffracts the light emitted from the light source unit; a driving unit for tilting the camera; and a calculation unit which receives an image from the light incident on the camera, and which calculates extrinsic parameters of the camera through the image, wherein the driving unit tilts the camera within the range in which an incident angle of the light diffracted by the diffraction unit is incident on the tilted camera becomes the angle of view of the camera.

Description

Camera Calibration Device

[0001] The present invention relates to a camera calibration device.

[0002] A camera is a device that captures images or videos of a subject, and is installed in portable devices, drones, vehicles, etc.

[0003] Camera devices or camera modules may show differences from the actual image depending on various variables in the process of converting a three-dimensional object into 2D image coordinates. To correct this, the camera can be calibrated by illuminating the camera with light from various angles through a diffractive optical element (DOE).

[0004] However, when the camera's field of view is large, the size of the DOE increases, the diffraction angle of the DOE also increases correspondingly to the camera's field of view, and the distance between the camera and the DOE must be very close, making it difficult to apply universally, and the calibration process is somewhat complicated, so a means to solve this problem is needed.

[0005] The present invention is an invention devised to solve the problems of the above-described prior art, and has as its task the calibration of a camera with a wide field of view even at a small diffraction angle of DOE.

[0006] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned herein will be clearly understood by those skilled in the art from the description below.

[0007] According to an embodiment of the present invention for achieving the above-described purpose, a camera calibration device includes a light source unit that irradiates light toward a camera, a diffraction unit that is disposed between the camera and the light source unit and diffracts the light irradiated from the light source unit, a driving unit that tilts the camera, and a calculating unit that receives an image from light incident on the camera and calculates an extrinsic parameter of the camera through the image, wherein the driving unit tilts the camera within a range in which an angle of incidence at which light diffracted by the diffraction unit is incident on the tilted camera becomes an angle of view of the camera.

[0008] Here, the driving unit can tilt the camera so that the plane on which the diffraction unit faces the camera is inclined based on the plane on which the diffraction unit faces the camera.

[0009] Alternatively, the driving unit may tilt the camera based on one side of the camera facing the diffraction unit.

[0010] In addition, the driving unit can tilt the camera with respect to the center of the camera on the optical axis along which the light moves.

[0011] Alternatively, the driving unit may tilt the camera based on one side of the camera facing the diffraction unit and the other side facing the opposite direction.

[0012] Meanwhile, the tilt angle by which the driving unit tilts the camera relative to the axis along which the light moves may be smaller than the angle of view of the camera.

[0013] Here, the tilt angle by which the driving unit tilts the camera may have a range of differences between the angle of view of the camera and the diffraction angle of light diffracted by the diffraction unit.

[0014] Additionally, the diffraction angle of light diffracted by the diffraction section may be smaller than the angle of view of the camera.

[0015] Meanwhile, the output unit can receive images for at least two positions during the process in which the driving unit tilts the camera.

[0016] Here, the above-mentioned output unit receives information about the tilt angle at which the camera is tilted from the above-mentioned driving unit, and can output the above-mentioned extrinsic parameter through the tilt angle and images for at least two positions.

[0017] A camera calibration device according to an embodiment of the present invention for solving the above problem can be effective in calibrating a camera with a wide field of view even with a small diffraction angle of DOE.

[0018] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

[0019] In addition, the effects of the present invention may be described in more detail in the detailed description of the present invention, and may not necessarily be limited to what is presented above.

[0020] The summary set forth above, as well as the detailed description of preferred embodiments of the present application described below, will be better understood when read in conjunction with the accompanying drawings.

[0021] For the purpose of illustrating the present invention, preferred embodiments are shown in the drawings.

[0022] However, it should be understood that the present application is not limited to the precise arrangements and means illustrated.

[0023] FIG. 1 is a drawing illustrating a part of a calibration process of a camera calibration device according to one embodiment of the present invention;

[0024] FIG. 2 is a drawing illustrating a conventional camera calibration device according to an embodiment of the present invention;

[0025] FIG. 3 is a drawing illustrating the overall arrangement of a camera calibration device according to one embodiment of the present invention;

[0026] FIG. 4 is a drawing illustrating a general description of a camera calibration device according to one embodiment of the present invention;

[0027] FIG. 5 is a diagram illustrating a comparison of a camera angle of view and a diffraction angle of a camera calibration device according to one embodiment of the present invention; and

[0028] FIG. 6 is a drawing illustrating a driving unit of a camera calibration device according to one embodiment of the present invention.

[0029] The present invention is susceptible to various modifications and embodiments, and specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. In describing the present invention, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the present invention.

[0030] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0031] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0032] Additionally, throughout the specification, when we say "connected," this does not only mean that two or more components are directly connected, but also that two or more components are indirectly connected through other components, that they are electrically connected as well as physically connected, or that they are referred to by different names depending on location or function but are one.

[0033] Additionally, when described as being formed or arranged "above or below" each component, "above" or "below" includes not only cases where the two components are in direct contact with each other, but also cases where one or more other components are formed or arranged between the two components. Furthermore, when expressed as "above" or "below," it can include the meaning of a downward direction as well as an upward direction based on one component.

[0034] Hereinafter, preferred embodiments of the present invention, in which the purpose of the present invention can be specifically realized, will be described with reference to the attached drawings.

[0035] First, reference may be made to FIGS. 1 and 2 to explain the overall background of the camera calibration device according to an embodiment of the present invention.

[0036] Specifically, FIG. 1 is a drawing illustrating a part of a calibration process of a camera calibration device according to one embodiment of the present invention, and FIG. 2 is a drawing illustrating a conventional technique of a camera calibration device according to one embodiment of the present invention.

[0037] First, the camera (C) can be calibrated by considering the extrinsic parameters of the camera (C) according to external factors such as the angle of incidence of light and refraction, and the accessories of the camera (C), and the intrinsic parameters according to the unique performance of the camera (C), in the process of correcting a three-dimensional object to a two-dimensional plane.

[0038] Specifically, as illustrated in FIG. 1, light (L) is irradiated toward a camera (C), a diffraction unit (100) is placed between the camera (C) and the light source unit irradiating the light (L), and the light (L) is diffracted at various angles through the diffraction unit (100) so that it can be incident toward the camera (C). Here, the diffracted light (LD) may have a different diffraction angle depending on the size and area of ​​the diffraction unit (100).

[0039] For example, if the area and size of the diffraction unit (100) are large, the camera (C) with a large angle of view can be calibrated based on the high diffraction angle to correspond to the angle of view of the camera (C). However, if the angle of view of the camera (C) is large, for example, if the angle of view exceeds 120 degrees, the distance (A) between the camera (C) and the diffraction unit (100) can become narrow.

[0040] This is because the incident light (L) diffracted through the diffraction unit (100) as shown in FIG. 1 is incident on the camera (C) in correspondence to the angle of view in one direction as shown in FIG. 1 (a) and in correspondence to the angle of view in the other direction as shown in FIG. 1 (b), so the camera (C) is an area where the diffracted light (LD) is incident on the camera (C) in an overlapping area (O) where the incident area for the diffracted light (LD) in one direction and the incident area for the diffracted light (LD) in the other direction intersect each other at the same position, so the extrinsic parameter of the camera (C) can be calculated based on the diffracted light (LD) incident on the above-described overlapping area (O).

[0041] However, as described above, when the angle of view of the camera (C) increases, the distance between the camera (C) and the diffraction unit (100) must be too close, and the size of the diffraction unit (100) must be relatively large in order to form diffracted light (LD) having a high diffraction angle, so there may be a disadvantage in that the calibration process of the camera (C) is not universal.

[0042] As illustrated in FIG. 2, when light (L) is incident and diffracted by the diffraction unit (100) and the diffracted light (LD) is incident on the camera (C), if the diffraction angle is relatively low, the angle of view of the camera (C) is relatively small, so the distance (A) between the camera (C) and the diffraction unit (100) can be positioned far away, as in FIG. 2 (a). However, if the angle of view of the camera (C) is large, as in FIG. 2 (b), the area of ​​the diffraction unit (100) becomes larger than that of the camera (C), and the distance (A) between the camera (C) and the diffraction unit (100) must be relatively close so that an overlap area (O) is formed between the diffracted light (LD) in one direction and the diffracted light (LD) in the other direction, so that the calibration of the camera (C) can be performed completely.

[0043] To explain in another way, when the size of the diffraction section (100), i.e., the area where the light (L) is incident, is narrow, the diffraction angle is large, but since an overlap area (O) between the diffracted light (LD) in one direction and the diffracted light (LD) in the other direction is not formed, the camera (C) must be moved in one direction or the other direction to take pictures at the position of an individual area to perform calibration for the camera (C), and this may also have the disadvantage of requiring precise adjustment of the distance (A) between the area where the camera (C) is moved and the diffracted light (LD) incident on the camera (C).

[0044] In addition, since the diffracted light (LD) has a diffraction angle depending on the incident light (L) and the area of ​​the diffraction unit (100), it can be used universally, but performing calibration by moving the camera (C) in one direction and the other direction is difficult to apply to universal situations, such as when the distance to be moved is large depending on the intrinsic parameters of the camera (C) or the distance (A) between the diffraction unit (100) and the camera (C) must be narrowed, and there may be the inconvenience of having to create different environments for each camera (C).

[0045] In addition, no matter how large the diffraction angle of the diffraction section (100) is made, the angle of view of the camera (C) being studied recently becomes larger than the diffraction angle of the diffracted light (LD), so it may be difficult to perform specific calibration work on the spare angle of view of the camera (C), i.e., the residual angle of view.

[0046] Therefore, the device according to the embodiment of the present invention can perform calibration as shown in FIGS. 3 to 6 to universally perform calibration corresponding to the angle of view of the camera (C) while effectively solving the above-described problem.

[0047] Specifically, FIG. 3 is a drawing for explaining the overall arrangement of a camera calibration device according to an embodiment of the present invention, FIG. 4 is a drawing for explaining the overall arrangement of a camera calibration device according to an embodiment of the present invention, FIG. 5 is a drawing for explaining by comparing a camera angle of view and a diffraction angle of a camera calibration device according to an embodiment of the present invention, and FIG. 6 is a drawing for explaining a driving unit of a camera calibration device according to an embodiment of the present invention.

[0048] First, a device according to an embodiment of the present invention may include a light source unit that irradiates light (L) toward a camera (C), a diffraction unit (100) that is arranged between the camera (C) and the light source unit and diffracts the light (L), a driving unit that tilts the camera (C), and a calculation unit (200) that receives an image from the light (L) incident on the camera (C) and calculates an extrinsic parameter of the camera (C) through the image. Here, the light (L) irradiated by the light source unit may be parallel light (L).

[0049] In addition, although not shown, the distance (A) between the light source and the diffraction unit (100) can be varied in the process of calibrating the camera (C) or can be arranged to have a predetermined distance as needed.

[0050] In addition, the diffraction unit (100) is arranged between the light source unit and the camera (C), and the diffraction unit (100) may be formed to have a preset pattern, and specifically, may be formed to have different patterns on one side facing the light source unit and the other side facing the camera (C), or may be formed to have a pattern only on one side, or may be formed to have a pattern only on the other side, and may not necessarily be limited to what has been mentioned.

[0051] Here, the diffraction section (100) can be formed to have a relatively smaller area than the diffraction section (100) according to the prior art, and accordingly, can have a relatively low diffraction angle.

[0052] Meanwhile, the driving unit tilts the camera (C) based on a reference axis formed in the direction from the light source unit toward the camera (C), and the reference point may vary depending on the situation, which will be explained in more detail through the drawings described later.

[0053] Here, the driving unit can tilt the camera (C) only at a preset position without moving in the left and right directions or upward and downward directions based on Fig. 3, and this can be the same even if the models of the cameras (C) are different. This will be explained in more detail with reference to the drawings to be described later.

[0054] Meanwhile, the output unit (200) may be electrically connected to the camera (C) or connected via wireless communication, and may receive an image formed when diffracted light (LD) incident through the lens of the camera (C) is incident on the image sensor, and may calculate the extrinsic parameters of the camera (C) based on the received image.

[0055] However, in the device according to the embodiment of the present invention, since the camera (C) is tilted through the driving unit, the tilt angle (AT) at which the camera (C) is tilted with respect to the above-described reference axis can be transmitted together to perform calibration.

[0056] As a result, the device according to the embodiment of the present invention can perform non-chart calibration by utilizing an image captured through the camera (C) and the tilt angle (AT) of the camera (C), and can have the advantage of universally utilizing the calibration of the camera (C) by changing the tilt angle (AT) through the driving unit without changing the area of ​​the diffraction unit (100) despite a change in the model of the camera (C) or a high angle of view.

[0057] Specifically, as illustrated in FIG. 4, when light (L) is irradiated from the light source toward the diffraction unit (100), the light (L) incident on the diffraction unit (100) is diffracted through the diffraction unit (100), and the diffraction angle may be smaller than the angle of view of the camera (C). In addition, an image can be transmitted to the output unit (200) from the diffraction light (LD) incident on the camera (C) through the overlap area (O) between the diffraction light (LD) in one direction and the diffraction light (LD) in the other direction.

[0058] At this time, the light (L) diffracted through the diffraction unit (100) is described as being unidirectional in one direction or the other direction, but the diffracted light (LD) diffracted in the diffraction unit (100) may be radially irradiated to have diffracted light (LD) in both directions centered on a point between the point of incidence or the point passing through the diffraction unit (100), and various methods may be attempted, such as adjusting the diffracted angle by adjusting the wavelength of the light (L) source as described above, and may not necessarily be limited to what has been described above.

[0059] However, accurate calibration is possible through the image transmitted from the camera (C) to the output unit (200) only when the diffracted light (LD) corresponding to the overlap area (O) is incident on the camera (C), and the overlap area (O) can be defined as the area where the light (L) of (a) and (b) of Fig. 1 overlap each other, and more accurate calibration can be possible when the diffracted light (LD) incident on the camera (C) is placed in the overlap area (O).

[0060] Here, the driving unit tilts the camera (C) in one direction and the other direction at least once each, and can transmit an image to the output unit (200) based on at least two positions during the process of tilting in one direction and the other direction.

[0061] For example, the driving unit can transmit an image for a first position based on a maximum tilt angle (AT) in one direction and an image for a second position based on a maximum tilt angle (AT) in the other direction to the calculating unit (200), and since this is based on a minimum position, more positions can be provided between the first position for the maximum tilt angle (AT) in one direction and the second position for the maximum tilt angle (AT) in the other direction, and images for the multiple positions can be transmitted to the calculating unit (200) to perform accurate calibration.

[0062] Meanwhile, the driving unit may include a separate control unit to input the angle of view of the camera (C) and the diffraction angle of the diffraction unit (100) to calculate the maximum tilt angle (AT) and automatically transmit the image for the preset position to the output unit (200), or the operator may manually tilt the camera (C) using the angle of view of the camera (C) and the diffraction angle of the diffraction unit (100), and the present invention may not necessarily be limited to what has been mentioned.

[0063] Here, the driving unit tilts the diffraction unit (100) so that the front side (Ca) facing the camera (C) and the side facing the camera (C) toward the diffraction unit (100) form an angle with each other, and the angle between the front side (Ca) facing the camera (C) and the side facing the diffraction unit (100) may be equal to the tilt angle (AT). This will be described in more detail with reference to the drawings to be described later.

[0064] Meanwhile, in the case of tilting the camera (C) through a driving unit, as in the device according to an embodiment of the present invention, the camera (C) can be tilted by the driving unit so that the incident angle (AI) at which the diffracted light (LD) is incident on the front (Ca) of the camera (C) corresponds to the maximum angle of view of the camera (C), as shown in FIG. 5, and the tilt angle (AT) of the camera (C) can be smaller than the angle of view.

[0065] In addition, the incident angle (AI) incident on the camera (C) may be the sum of the angle between the plane on which the camera (C) faces the diffraction section (100) and the plane on which the diffraction section (100) faces the camera (C) and the diffraction angle, or more simply, the sum of the tilt angle (AT) tilted with respect to the above-described reference axis and the diffraction angle. This can be mathematically proven, but since it is self-evident that the tilt angle (AT) and the angle between the tilted plane and the reference plane are equal, the proof will be omitted.

[0066] To summarize the above, in the case of a calibration device according to the prior art, as the diffraction angle of the diffraction section (100) increases, the area of ​​the diffraction section (100) increases, and there is a disadvantage in that the camera (C) and the diffraction section (100) must be taken too close together to form an overlap area (O).

[0067] However, since the device according to the embodiment of the present invention can tilt the camera (C) in the driving unit so that the incident angle (AI) corresponds to the maximum angle of view even if the diffraction angle diffracted by the diffraction unit (100) is smaller than the maximum angle of view of the camera (C) as shown in FIG. 5, there is no need to separately vary the distance between the camera (C) and the diffraction unit (100), there is no need to move the camera (C) in both directions, up and down, forward and backward, etc., and there is no need to separately adjust the distance (A) between the camera (C) and the diffraction unit (100), there may be an advantage in that it can be used more universally.

[0068] That is, even if the camera (C) model has different angles of view, there may be an advantage in that calibration can be performed more easily by adjusting only the tilt angle (AT) of the camera (C) without having to adjust the diffraction angle of the diffraction section (100) and the area of ​​the diffraction section (100).

[0069] In addition, since the only variable added in the process of performing calibration of the camera (C) in the output unit (200) is the tilt angle (AT) formed by tilting the camera (C), there may also be an advantage in that more effective calibration of the camera (C) is possible because no complex calculation process is added.

[0070] Accordingly, the diffraction angle of light (L) diffracted by the diffraction unit (100) is smaller than the angle of view of the camera (C), the tilt angle (AT) by which the driving unit tilts the camera (C) is smaller than the angle of view of the camera (C), and as a result, it may be desirable for the tilt angle (AT) to tilt in a range corresponding to the value of the difference between the angle of view and the diffraction angle of the camera (C).

[0071] That is, the output unit (200) can output extrinsic parameters through information on the tilt angle (AT) at which the camera (C) is tilted in the image and driving unit for at least two positions.

[0072] Meanwhile, the driving unit can be tilted in various ways according to the reference point of the camera (C) based on the reference axis as illustrated in Fig. 6.

[0073] Specifically, the driving unit can tilt the camera (C) based on one side of the camera (C) facing the diffraction unit (100), tilt the camera (C) based on the center of the camera (C), or tilt the camera (C) based on the other side opposite to one side of the camera (C) facing the diffraction unit (100).

[0074] More specifically, as shown in (a) of FIG. 6, the camera (C) can be tilted based on the front reference point (Cf) of the camera (C), and when the camera (C) is tilted based on the front reference point (Cf) of the camera (C), the other side of the camera (C) can be tilted over a wider range, and when the camera (C) is tilted based on the center reference point (Cc) of the camera (C), as shown in (b) of FIG. 6, one side of the camera (C) and the other side of the camera (C) can be tilted so as to have ranges corresponding to opposite directions, and when the camera (C) is tilted based on the rear reference point (Cb) of the camera (C), as shown in (c) of FIG. 6, one side of the camera (C) can be tilted over a wider range.

[0075] That is, tilting the camera (C) in FIG. 6 can be done based on various reference points, but preferably, a virtual line crossing one side and the other side passes through the center of the camera (C), and it may be desirable to tilt based on one of the parts where the virtual line and the camera (C) overlap.

[0076] In addition, the driving unit may be formed in various ways, such as holding only the side of the camera (C), supporting part of it and holding part of it, or being formed in the form of a stage. These are only examples for carrying out embodiments of the present invention and may not necessarily be limited to those mentioned.

[0077] Having described preferred embodiments of the invention, it will be apparent to those skilled in the art that the invention may be embodied in other specific forms without departing from the spirit or scope thereof, in addition to the embodiments described above.

[0078] Therefore, the above-described embodiments should be considered as illustrative rather than restrictive, and accordingly, the present invention is not limited to the above description but may be modified within the scope of the appended claims and their equivalents.

Claims

1. A light source that irradiates light toward the camera; A diffraction unit disposed between the camera and the light source unit and diffracting light irradiated from the light source unit; A driving unit that tilts the above camera; and It includes a calculation unit that receives an image from light incident on the camera and calculates an extrinsic parameter of the camera through the image, The above driving unit is a camera calibration device that tilts the camera within a range where the angle of incidence at which light diffracted by the diffraction unit is incident on the tilted camera becomes the angle of view of the camera.

2. In paragraph 1, The above driving unit is a camera calibration device that tilts the camera so that the plane facing the diffraction unit is inclined based on the plane facing the camera.

3. In paragraph 1, The above driving unit is a camera calibration device that tilts the camera based on one side of the camera facing the diffraction unit.

4. In paragraph 2, The above driving unit is a camera calibration device that tilts the camera with respect to the center of the camera on the optical axis along which the light moves.

5. In paragraph 2, The above driving unit is a camera calibration device that tilts the camera based on one side of the camera facing the diffraction unit and the other side facing the opposite direction.

6. In paragraph 1, A camera calibration device in which the tilt angle of the driving unit tilting the camera relative to the axis along which the light moves is smaller than the angle of view of the camera.

7. In paragraph 6, A camera calibration device in which the tilt angle at which the driving unit tilts the camera has a range of differences between the angle of view of the camera and the diffraction angle of light diffracted by the diffraction unit.

8. In paragraph 1, A camera calibration device in which the diffraction angle of light diffracted in the above diffraction section is smaller than the angle of view of the camera.

9. In paragraph 1, The above-mentioned output unit is a camera calibration device that receives images for at least two positions during the process in which the above-mentioned driving unit tilts the camera.

10. In paragraph 9, The above output unit receives information about the tilt angle at which the camera is tilted from the driving unit, A camera calibration device that calculates the extrinsic parameter through images for the tilt angle and at least two positions.

Images