Imaging device, control method, and program
The imaging device enhances calibration efficiency by calculating and presenting deviation angles and distances to align the camera with a calibration chart, addressing the lack of detailed operational guidance in existing methods.
Patent Information
- Application Number
- JP2022007274
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-01-20
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2042-01-20
AI Technical Summary
Existing methods for aligning a camera with a calibration chart do not provide sufficient detail on the required operations, leading to time-consuming adjustments.
An imaging device that calculates and presents deviation angles and distances to align the camera with a calibration chart, using markers and optical information to facilitate precise positioning.
Reduces the effort required for aligning the camera with the chart by providing detailed operational information, enhancing calibration efficiency.
Smart Images

Figure 0007580415000001 
Figure 0007580415000002 
Figure 0007580415000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a technique for adjusting a positional relationship with respect to a calibration chart. [Background technology]
[0002] There is a technology called Visual Effects (hereinafter referred to as VFX). VFX is a technology that combines computer graphics (hereinafter referred to as CG) with real-life images that have been shot.
[0003] When compositing live-action images and CG in VFX, if the distortion of the camera lens used to capture the images is not taken into consideration, a difference in size will occur between the live-action image and the CG, so it is necessary to correct the distortion of the live-action image before compositing it with the CG. In this case, since distortion differs depending on the image height, it is necessary to correct the distortion according to the image height.
[0004] If information regarding distortion is added as metadata to the lens attached to the camera, it is possible to correct the distortion according to the image height, but some lenses do not have information regarding distortion.
[0005] If information about distortion is not attached to the lens, it is necessary to obtain information about the lens distortion by photographing a calibration chart (hereinafter referred to as the chart) for correcting distortion. However, when photographing a chart, it is necessary to face the camera lens directly against the chart, align the optical center of the camera with the center of the chart, and adjust the angle of view to the chart.
[0006] Patent Documents 1 and 2 describe a method of presenting a direction in which a user should rotate or move a camera in order to point the camera directly at an object. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] JP 2020-193820 A [Patent Document 2] Patent Publication No. 2021-027584 Summary of the Invention [Problem to be solved by the invention]
[0008] Patent documents 1 and 2 do not provide any information on the amount of rotation or movement required to face the camera directly against the object, to align the optical center of the camera with the center of the object, or to adjust the angle of view to the object, which may result in a lot of work being required to adjust the positional relationship between the camera and the object.
[0009] The present invention has been made in view of the above-mentioned problems, and has as its object to realize a technique that can present more detailed information regarding operations required by a user during calibration. [Means for solving the problem]
[0010] In order to solve the above problems and achieve the object, the present invention provides an imaging device for imaging a chart for correcting lens distortion, comprising: a first acquisition means for determining a deviation angle indicating an angle by which the imaging device deviates from a positional relationship in which the imaging device faces the chart directly, based on first distance information from the optical center of the imaging device to the center of the chart and second distance information to an edge of the chart, and angle of view information of the imaging device; a second acquisition means for determining a deviation distance indicating a distance deviated from the directly facing positional relationship, based on the deviation angle and the first distance information; and a third acquisition means for determining an appropriate distance between the imaging device and the chart in the directly facing positional relationship, based on a size of the chart and the angle of view information. Until the deviation distance and the deviation angle become less than a predetermined determination threshold value. The device has a presentation means for presenting information based on the deviation angle and the deviation distance so as to achieve the directly facing positional relationship, and for presenting information regarding the appropriate distance when the deviation distance and the deviation angle are less than a predetermined judgment threshold value. Effect of the Invention
[0011] According to the present invention, since more detailed information regarding operations required by the user during calibration can be presented, the effort required for adjustment during calibration can be reduced. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a block diagram showing an apparatus configuration according to an embodiment of the present invention. [Diagram 2] FIG. 4 is a plan view showing a schematic positional relationship between a camera and a calibration chart according to the embodiment. [Diagram 3] 6A and 6B are diagrams showing examples of display of information presented during calibration in the present embodiment. [Figure 4] 5 is a flowchart showing a processing procedure during calibration according to the present embodiment. [Diagram 5] 5 is a flowchart showing a correction process to a normal facing state in step S402 of FIG. 4. [Figure 6] 6 is a flowchart showing the angle variable calculation process in step S508 of FIG. 5. [Figure 7] 6 is a plan view showing a schematic positional relationship between the camera and the calibration chart in the correction process to the normal facing state in FIG. 5. [Figure 8] 5 is a flowchart showing the process of correcting the distance to an appropriate distance in step S403 of FIG. 4. [Figure 9] 5 is a plan view showing a schematic diagram of the positional relationship between the camera and the calibration chart in the correction process to the appropriate distance in step S403 in FIG. 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0014] Hereinafter, an embodiment in which the imaging device of the present invention is applied to a digital video camera will be described in detail with reference to the attached drawings. Note that the imaging device is not limited to a digital video camera, but can also be applied to a digital single-lens reflex camera with an interchangeable lens or an electronic device having a camera function to which a lens can be attached.
[0015] <Apparatus Configuration> First, the configuration and functions of the imaging apparatus of this embodiment will be described with reference to FIG.
[0016] The imaging device (hereinafter, camera) 100 of this embodiment includes a lens control device 110 and a camera control device 130. The lens control device 110 is detachable from the camera control device 130. The lens control device 110 is mechanically connected to the camera control device 130 via a lens mount (not shown) and is electrically connected to the camera control device 130 via electrical contacts 150, which will be described later.
[0017] Lens control device 110 is a lens unit including a fixed lens group 111, a zoom lens 112, an aperture 113, an image stabilization lens 114, and a focus lens 115, which form an imaging optical system. Lenses 111, 112, 114, and 115 are usually made up of multiple lenses, but for simplicity, only one lens is shown here.
[0018] The lens control device 110 has a lens control unit 121 that communicates with a camera control unit 138 of the camera control device 130 via an electrical contact 150. The lens control unit 121 is communicatively connected to a zoom driving unit 116, an aperture driving unit 117, an image stabilization driving unit 118, and a focus driving unit 119 via a bus 122.
[0019] The lens operation unit 120 includes operation members such as ring members for operating the zoom lens 112, the aperture 113, and the focus lens 115, and a switch for setting the image stabilization function to enabled or disabled. When the operation members are operated by a user operation, the lens operation unit 120 outputs operation information according to the operation type to the lens control unit 121. The lens control unit 121 performs control according to the operation information received from the lens operation unit 120.
[0020] The lens control unit 121 performs calculation processing for controlling the lens control device 110. The lens control unit 121 includes a processor such as a CPU that controls the components of the lens control device 110.
[0021] The lens control unit 121 communicates with the camera control unit 138 of the camera control device 130 via electrical contacts, receives control information from the camera control unit 138, and transmits lens information (such as optical information) held by the lens control device 110 to the camera control device 130 in response to a transmission request from the camera control unit 138 (hereinafter, communication between the lens control unit 121 and the camera control unit 138 is referred to as lens communication).
[0022] Furthermore, the lens control unit 121 controls the zoom driving unit 116, the aperture driving unit 117, and the focus driving unit 119 in accordance with operation information from the lens operation unit 120. Furthermore, the lens control unit 121 controls the zoom driving unit 116, the aperture driving unit 117, the image stabilization driving unit 118, and the focus driving unit 119 in accordance with control information from the camera control unit 138 of the camera control device 130.
[0023] The zoom driver 116 changes the focal length by driving the zoom lens 112. The aperture driver 117 drives the aperture 113 to adjust the aperture diameter of the aperture 113 and adjust the amount of light during shooting. The image stabilization driver 118 reduces camera shake by driving the image stabilization lens 114 in accordance with the shake of the lens control device 110. The focus driver 119 drives the focus lens 115 to control the focus state.
[0024] The camera control device 130 is a camera body that captures an image of a subject passing through the lens control device 110 and generates image data.
[0025] The camera control unit 138 performs calculation processing for controlling the camera control device 130. The camera control unit 138 includes a processor such as a CPU that controls the components of the camera control device 130.
[0026] The camera control unit 138 communicates with the lens control unit 121 of the lens control device 110 via the electrical contacts 150. The camera control unit 138 transmits control signals to the lens control unit 121 to drive the lens and aperture of the lens control device 110, and transmits lens information (optical information, etc.) held by the lens control device 110. Lens control unit 121 Receive from . electric The electrical contact 150 has a pair of communication terminals (a camera-side terminal and a lens-side terminal) corresponding to a pair of communication lines that enable bidirectional communication between the lens control device 110 and the camera control device 130.
[0027] The light beam transmitted through the lens control device 110 is focused by the imaging optical system and received as an optical image of a subject (subject image) by the image sensor 131. The subject image received by the image sensor 131 is converted into an electric signal by a photoconversion element such as a CMOS of the image sensor 131. The electric signal generated by the image sensor 131 is processed as an image signal (image data) by the imaging signal processing unit 134.
[0028] The image sensor 131 has a plurality of pixels arranged two-dimensionally. Each of the plurality of pixels includes a pair of photoelectric conversion units (photodiodes) and one microlens provided for these photoelectric conversion units. Light incident on each pixel is pupil-divided by the microlens, and a pair of subject images are formed on the pair of photoelectric conversion units. Each of the pair of photoelectric conversion units accumulates electric charge by photoelectric conversion.
[0029] The divided image generating unit 132 reads out output signals having voltages corresponding to the charges accumulated in each of the pair of photoelectric conversion units of each pixel as focus detection signals (A image signal and B image signal). The divided image generating unit 132 also combines the A image signal and B image signal read out from each pixel of the image sensor 131 for each pixel. The A image signal and B image signal are used for focus detection by a phase difference detection method. A signal obtained by combining the A image signal and B image signal is used to generate an image signal.
[0030] The focus detection unit 133 performs a correlation calculation on the A and B image signals to calculate the phase difference between the A and B image signals. The focus detection unit 133 also calculates a defocus amount, which indicates the focus state of the imaging optical system, from the phase difference between the A and B image signals.
[0031] The camera control unit 138 performs AF (autofocus) processing to drive the focus lens 115 by controlling the lens control unit 121 and the focus driving unit 119 of the lens control device 110 based on the defocus amount calculated by the focus detection unit 133. This provides a focused state of the imaging optical system.
[0032] The image data output from the imaging signal processing unit 134 is output to the sensor control unit 135 and temporarily stored in the volatile memory 140. The image data is also corrected and compressed by the image processing unit 139, and then recorded on a recording medium 145 such as a memory card.
[0033] In parallel with the AF process, the display control unit 143, under the control of the camera control unit 138, reduces and enlarges the image data stored in the volatile memory 140 to an optimal size for the display unit 142 built into or external to the camera control device 130. The image data processed to the optimal size is temporarily stored again in an area of the volatile memory 140 different from the area before the processing. The display control unit 143 also superimposes various image information such as shooting settings on the image data with characters, icons, etc., and outputs the superimposed image information to the display unit 142, and the display unit 142 displays the image data with the various information superimposed on a display device composed of a liquid crystal panel, an organic EL, or the like. This allows the user to monitor the image captured by the image sensor 131 in real time (live view image).
[0034] Image stabilization control unit 137 controls image stabilization drive unit 136 under the control of camera control unit 138 to displace image sensor 131 in a direction that reduces camera shake. Image stabilization drive unit 136 can also drive image sensor 131 and image stabilization lens 114 in conjunction with each other in cooperation with image stabilization drive unit 118 of lens control device 110. In this case, greater image stabilization can be achieved than when image sensor 131 and image stabilization lens 114 are driven individually.
[0035] Operation unit 144 is an operation member such as a switch, a button, a ring, a lever, etc. that accepts user operation, and outputs an operation signal corresponding to the operation member operated by the user to camera control unit 138. Camera control unit 138 outputs a control signal to each component of camera control device 130 based on the operation signal to control them. The operation member includes, for example, a touch panel that is integrally configured with display unit 142.
[0036] The volatile memory 140 is, for example, a RAM, and is used not only to temporarily store image data, but also as a work area for temporarily storing data used in the control processing of each component of the camera control device 130, lens information obtained from the lens control device 110, etc.
[0037] Non-volatile memory 141 stores a control program necessary for the operation of camera 100. When the power is turned on by a user operation and camera 100 is started, the control program stored in non-volatile memory 141 is read (loaded) into a part of volatile memory 140. Camera control unit 138 controls the operation of camera 100 in accordance with the control program loaded into volatile memory 140.
[0038] A calibration chart (hereinafter, chart) photographed by camera 100 to correct lens distortion aberration has markers 303, 304a, and 304b for position detection at the center and at the ends on either side of the center in a direction horizontal to the ground, as in image (captured image) 300 of a photographed chart shown in Fig. 3. The markers 303, 304a, and 304b only need to be able to detect their positions, and for example, a method may be used in which specific patterns are formed at the center and ends of the chart and detected. A chart with such characteristics is placed horizontally to the ground and photographed for calibration.
[0039] Next, a process relating to the calibration of the camera 100 of this embodiment will be described.
[0040] FIG. 2 is a plan view showing a schematic positional relationship between the camera and the chart in this embodiment.
[0041] In FIG. 2, the optical center (optical axis) of the camera 100 is A, the angle of view of the camera 100 is Θ, and the center of the chart 200 is O. In this embodiment, the user can easily correct the positional relationship between the camera 100 and the chart 200 from a state 202 in which the camera 100 and the chart 200 are not facing each other directly to a state 201 in which the camera 100 and the chart 200 are facing each other directly. For this reason, in this embodiment, the angle Φ formed by the line AO connecting the optical center A of the camera 100 to the chart center O and the normal line Z extending from the chart center O, and the distance H between the optical center A of the camera 100 and the normal line Z are calculated. Then, in this embodiment, the deviation angle Φ is calculated based on the angle Φ. ΦBy presenting information indicating the amount and direction to rotate and move the camera 100 based on the deviation movement amount H, the user can easily correct the positional relationship between the camera 100 and the chart 200 to a position where they face each other directly. In this embodiment, the positional relationship where the camera 100 and the chart 200 face each other directly is a state where the angle Φ between the line AO connecting the optical center A of the camera 100 to the chart center O and the normal line Z extending from the chart center O, and the distance H between the optical center A of the camera 100 and the normal line Z are each less than a predetermined determination threshold value.
[0042] FIG. 4 is a flowchart showing a processing procedure during calibration in this embodiment.
[0043] 4 is realized by the camera control unit 138 of the camera control device 130 executing a program stored in the non-volatile memory 141 and controlling each component of the camera control device 130. The process of Fig. 4 is started in step S401 when the user operates the operation unit 144 to input an instruction to start calibration to the camera 100.
[0044] In step S402, a process is performed to correct the camera 100 to a state facing directly toward the chart 200, and the process proceeds to step S403. The process in step S402 will be described later with reference to FIG.
[0045] In step S403, a process is carried out to correct the distance between the optical center of the camera 100 and the center of the chart 200 in a facing state to an appropriate distance, and the process proceeds to step S404. The process of step S403 will be described later with reference to FIGS.
[0046] In step S404, calibration is performed.
[0047] As described above, the processes in steps S402 and S403 allow the camera 100 to face the chart 200 directly and capture the chart 200 within the range (angle of view) that the camera 100 can capture, allowing calibration to be performed with high accuracy.
[0048] Next, the correction process to the normal facing state in step S402 in FIG. 4 will be described with reference to FIG.
[0049] FIG. 5 is a flowchart showing the correction process to the facing state in step S402 of FIG.
[0050] When the process starts in step S501, the process proceeds to step S502.
[0051] In step S502, the camera control unit 138 executes AF processing using an image (captured image) of the chart 200 captured by the image sensor 131.
[0052] In step S503, the camera control unit 138 performs a chart center detection process by detecting a center detection marker (303 in FIG. 3) indicating the center of the chart in the captured image acquired from the image processing unit 139.
[0053] In step S504, the camera control unit 138 determines whether or not the distance between the center of the chart acquired in step S503 and the center of the captured image (center of the screen) is less than a predetermined determination threshold. In order to facilitate the operation of aligning the camera 100 and the chart 200, assistance such as displaying a mark at the center of the screen by the display control unit 143 may be implemented. If the camera control unit 138 determines that the distance between the center of the chart and the center of the screen is equal to or greater than the predetermined determination threshold, the process proceeds to step S505. If the camera control unit 138 determines that the distance between the center of the chart and the center of the screen is less than the predetermined determination threshold, the process proceeds to step S506.
[0054] In step S505, the camera control unit 138 controls the display control unit 143 to display, on the display unit 142, first information that prompts the user to move the camera 100 so that the screen center approaches the chart center. After that, the camera control unit 138 returns the process to step S502, and repeats the processes of steps S502 to S505 until the distance between the chart center and the screen center becomes less than a predetermined determination threshold in step S504.
[0055] In step S506, the camera control unit 138 performs chart edge detection processing by detecting edge detection markers (304a and 304b in FIG. 3) that indicate the chart edges in the captured image acquired from the image processing unit 139, and stores the number of detected chart edges in the volatile memory 140.
[0056] In step S507, the camera control unit 138 acquires the defocus amount of each pixel from the focus detection unit 133, and calculates the focus drive amount for driving the focus lens 115 so that the imaging optical system is in focus based on the defocus amount. The camera control unit 138 also calculates the subject distance based on the focus drive amount and the current position information and focal length information of the focus lens. This makes it possible to acquire a defocus map indicating the subject distance at each position of the captured image.
[0057] In step S508, the camera control unit 138 performs a process of calculating an angle variable Ψ, which will be described later with reference to FIG.
[0058] In step S509, the camera control unit 138 calculates the deviation angle Φ based on the angle variable Ψ calculated in step S508.
[0059] FIG. 7 is a plan view showing a schematic positional relationship between the camera and the chart in the correction process to the normal facing state of FIG. 5. FIG. 7(a) is a diagram showing the positional relationship between the camera 100 and the chart 200 when the screen edges (both ends of the angle of view) of the camera 100 are within the chart 200. FIG. 7(b) is a diagram showing the positional relationship between the camera 100 and the chart 200 when one end of the angle of view of the camera 100 is within the chart 200. FIG. 7(c) is a diagram showing the positional relationship between the camera 100 and the chart 200 when one end of the angle of view of the camera 100 is within the chart 200 and the other end is not. FIG. 7(d) is a diagram showing the positional relationship between the camera 100 and the chart 200 when the chart 200 is within the angle of view of the camera 100. In the example of FIG. 7, the optical center (optical axis of the imaging optical system) of the camera 100 is A, the chart center is O, the limit position where the subject distance can be detected by the distance measurement of the camera 100 is B, and the angle variable Ψ is <OABであるとする。
[0060] If the distance from the optical center A of the camera 100 to the center O of the chart is D, the focal length of the camera 100 is 1 [mm], and the image sensor size of the camera 100 is s [mm], the angle of view information Θ of the camera 100 can be calculated by the following equation 1. (Formula 1) Θ=2atan(s / 2l) Furthermore, when attention is focused on the triangle OAB, the law of cosines gives the following relational expression using the angle variable Ψ for the angle Θc between AO and the chart 200: (Formula 2) OB=√(AB 2 +AO 2 -2AB×AO×cosΨ) Θc=acos((OB 2 +AO 2 -AB 2 ) / (2OB×AO)) Φ=90[deg]-Θc In this case, the deviation angle Φ can be calculated by substituting the distance information DL and D for AB and AO in the above formula 2. The distance information D is information corresponding to the distance from the optical center A of the camera 100 to the chart center O. The distance information DL is information corresponding to the distance from the optical center A of the camera 100 to the viewing angle end B. For AB, if the end detection marker can be detected, a value calculated from the defocus amount at the chart end is substituted, and if not, a value calculated from the defocus amount at the end of the captured image (screen end) is substituted. For AO, since the center position detection marker can be detected, the subject distance at the chart center O is substituted.
[0061] 5, in step S510, the camera control unit 138 calculates the distance H to the chart 200. The distance H is calculated from the following formula 3. (Formula 3) H=AO×sinΦ By substituting the distance D and the deviation angle Φ into AO in the above formula 3, the deviation distance H can be calculated.
[0062] In step S511, the camera control unit 138 controls the display control unit 143 to display on the display unit 142 information based on the deviation angle Φ and the deviation distance H calculated in steps S509 and S510.
[0063] 3 illustrates an example of information based on the deviation angle Φ and deviation distance H displayed on the display unit 142 in step S511. On the screen of the display unit 142, an image 301 captured of the chart 200 and information 302 indicating the amount and direction of rotation and movement of the camera 100 based on the deviation angle Φ and deviation distance H required to correct the attitude of the camera 100 so that the camera 100 and the chart 200 face each other squarely are displayed. The user can fine-tune the attitude of the camera 100 while looking at the direction of rotation and movement presented in the information 302 to bring the camera 100 and the chart 200 into a positional relationship where they face each other squarely.
[0064] In step S512, the camera control unit 138 determines whether or not the number of chart ends detected in step S506 is two. If the camera control unit 138 determines that the number of chart ends is one or less, it determines that the chart 200 is not within the angle of view (captured image) of the camera 100, and proceeds to step S513. If the camera control unit 138 determines that the number of chart ends is two, it determines that the chart 200 is within the angle of view (captured image) of the camera 100, and proceeds to step S514.
[0065] In step S513, the camera control unit 138 controls the display control unit 143 to display, on the display unit 142, second information that prompts the user to move the camera 100 so that the two chart ends are within the angle of view (captured image) of the camera 100. After that, the camera control unit 138 returns the process to step S502, and repeats the processes of steps S502 to S513 until the two chart ends are within the angle of view (captured image) of the camera 100 in step S512.
[0066] In step S514, the camera control unit 138 determines whether the deviation angle Φ and the deviation distance H are less than the respective thresholds. If the camera control unit 138 determines that the deviation angle Φ and the deviation distance H are less than the respective thresholds, the process ends at step S516, and if the camera control unit 138 determines that the deviation angle Φ and the deviation distance H are equal to or greater than the respective thresholds, the process proceeds to step S515.
[0067] In step S515, camera control unit 138 controls display control unit 143 to display third information indicating the amount and direction of rotation and movement of camera 100 based on the deviation angle Φ and deviation distance H calculated in steps S509 and S510 on display unit 142. After that, camera control unit 138 returns the process to step S502, and repeats the processes of steps S502 to S515 until deviation angle Φ and deviation distance H become less than their respective determination thresholds in step S514.
[0068] The determination threshold values in steps S504 and S514 may be changeable by the user via the operation unit 144, and the smaller the value, the more accurately the camera 100 and the chart can be positioned to face each other.
[0069] Fig. 8 is a flowchart showing the process of correcting to the appropriate distance in step S403 in Fig. 4. Fig. 9 is a plan view showing a schematic diagram of the positional relationship between the camera and the chart in the process of correcting to the appropriate distance in step S403 in Fig. 4.
[0070] As shown in Fig. 9, the optical center of the camera 100 before correction in Fig. 8 is A, the left edge of the chart 200 is B, the center of the chart is O, and the angle of view of the camera 100 is Θ. The process shown in Fig. 8 is a process for correcting the distance between the camera 100 and the chart 200 to an appropriate value by moving the camera 100 in a direction (front-back direction) perpendicular to the center of the chart (normal direction).
[0071] When the process starts in step S801 in FIG. 8, the process proceeds to step S802.
[0072] In step S802, the camera control unit 138 executes AF processing using an image (captured image) of the chart 200 captured by the image sensor 131.
[0073] In step S803, the camera control unit 138 performs a chart center detection process by detecting a center detection marker (303 in FIG. 3) indicating the center of the chart in the captured image acquired from the image processing unit 139.
[0074] In step S804, the camera control unit 138 determines whether or not the distance between the center of the chart acquired in step S803 and the center of the captured image (center of the screen) is less than a predetermined determination threshold. In order to facilitate the operation of aligning the camera 100 and the chart 200, assistance such as displaying a mark at the center of the screen by the display control unit 143 may be implemented. If the camera control unit 138 determines that the distance between the center of the chart and the center of the screen is equal to or greater than the predetermined determination threshold, the process proceeds to step S805. If the camera control unit 138 determines that the distance between the center of the chart and the center of the screen is less than the predetermined determination threshold, the process proceeds to step S806.
[0075] In step S805, the camera control unit 138 controls the display control unit 143 to display fourth information, which prompts the user to move the camera 100 so that the screen center approaches the chart center, on the display unit 142. After that, the camera control unit 138 returns the process to step S802, and repeats the processes of steps S802 to S805 until the distance between the chart center and the screen center becomes less than the predetermined determination threshold in step S804.
[0076] In step S806, the camera control unit 138 performs chart edge detection processing by detecting edge detection markers (304a and 304b in FIG. 3) that indicate the chart edges in the captured image acquired from the image processing unit 139, and stores the number of detected chart edges in the volatile memory 140.
[0077] In step S807, the camera control unit 138 acquires the defocus amount of each pixel from the focus detection unit 133, and calculates the focus drive amount for driving the focus lens 115 so that the imaging optical system is in focus based on the defocus amount. The camera control unit 138 also calculates the subject distance based on the focus drive amount and the current position information and focal length information of the focus lens. This makes it possible to acquire a defocus map indicating the subject distance at each position of the captured image.
[0078] In step S808, the camera control unit 138 calculates distance information D from the optical center of the camera 100 to the chart center O and distance information DL to the chart edge B, as shown in Fig. 9, from the chart center position, chart edge position, and defocus amount acquired in steps S803, S806, and S807. Note that although the chart edge B on the left side is selected in the example of Fig. 9, the chart edge on the right side may also be selected.
[0079] In step S809, the camera control unit 138 calculates the chart size L using the distance information DL and D calculated in step S808. In Fig. 9, the chart size L is calculated by the following formula 4 from Pythagoras' theorem for the triangle AOB. (Formula 4) L = √(DL 2 -D 2 ) In step S810, the camera control unit 138 calculates an appropriate distance D1 for the subject distance D at the chart center O. The appropriate distance D1 is calculated by the following formula 5. (Formula 5) D1 = √(DL 2 -D 2 ) / tan(Θ / 2) In this case, the movement distance ΔD from the subject distance D at the chart center O to the appropriate distance D1 is calculated by the following formula 6. (Formula 6) ΔD=D-√(DL 2 -D 2 ) / tan(Θ / 2) In step S811, the camera control unit 138 controls the display control unit 143 to 1 Information relating to the movement amount ΔD of camera 100 to appropriate distance D1 calculated in step 2 is displayed on display unit 142. This allows the user to fine-tune the position of camera 100 while viewing the information relating to the movement amount ΔD.
[0080] In step S812, the camera control unit 138 determines whether the camera 100 and the chart 200 have approached the appropriate distance D1, that is, whether the moving distance ΔD is less than a predetermined determination threshold value. When the camera control unit 138 determines that the camera 100 and the chart 200 have approached the appropriate distance D1, that is, the moving distance ΔD is less than the predetermined determination threshold value, the process proceeds to step S813. When the camera control unit 138 determines that the camera 100 and the chart 200 have not approached the appropriate distance D1, that is, the moving distance ΔD is greater than or equal to the predetermined determination threshold value, the process returns to step S802, and the processes from step S802 to S812 are repeated until the camera 100 and the chart 200 approach the appropriate distance D1, that is, until the moving distance ΔD becomes less than the predetermined determination threshold value.
[0081] In step S813, the camera control unit 138 controls the display control unit 143 to display fifth information indicating that the camera 100 and the chart 200 have approached the appropriate distance D1 on the display unit 142, and ends the process in step S814.
[0082] Note that the determination threshold values in steps S804 and S81 2 may be changeable by the user via the operation unit 144, and the smaller the value, the more accurately the camera 100 and the chart can be brought closer to the appropriate distance D1.
[0083] Here, with reference to FIG. 6, the process of calculating the angle variable Ψ (<OAB) used for calculating the deviation angle Φ in step S508 of FIG. 5 will be described.
[0084] FIG. 6 is a flowchart showing the angle variable calculation process in step S508 of FIG. 5.
[0085] When the process starts in step S601 of FIG. 6, it proceeds to step S602.
[0086] In step S602, the camera control unit 138 refers to the number of chart edges detected in step S506 of Fig. 5 and stored in the volatile memory 140, and determines whether the number of chart edges is two or not. If the camera control unit 138 determines that the number of chart edges is two, it returns to step S60. 4 If it is determined that the number of chart edges is one or less, the process proceeds to step S60. 3 Proceed to next step.
[0087] Step S60 3 Then, the camera control unit 138 assigns half the value of the angle of view Θ (Θ / 2, first angle of view information) to the angle variable Ψ. This corresponds to the cases in Figures 7(a) to 7(c) (where the end of the angle of view B is located within the chart), for example.
[0088] Step S60 4 Then, the camera control unit 138 calculates the approximate angle λ (second angle of view information) and assigns it to the angle variable Ψ. This corresponds to the case of FIG. 7(d) (where both ends of the angle of view are located beyond the ends of the chart), for example. In the example of FIG. 7(d), since both ends of the angle of view are not located within the chart, it is not possible to set the angle variable Ψ to half the angle of view Θ (Θ / 2). For this reason, the camera control unit 138 calculates the approximate angle λ using the ratio between the distance OC from the center of the angle of view (chart center O) to the edge C of the angle of view and the distance OB from the center of the angle of view (chart center O) to the edge B of the chart in the captured image, according to the following formula 5. (Formula 5) λ=(Θ / 2)×(OC / OB) In step S605, the camera control unit 138 assigns the approximate angle λ calculated in step S604 to the angle variable Ψ, and proceeds to step S509 in FIG.
[0089] As described above, according to this embodiment, information on the direction and amount of rotation and movement with respect to the chart center O in order to correct the attitude of the camera 100 to a state in which it faces the chart 200 directly is presented to the user. Furthermore, according to this embodiment, in order to correct the distance between the camera 100 and the chart 200 to the appropriate distance D1 when the camera 100 and the chart 200 face each other directly, information on the amount of movement ΔD of the camera 100 to the appropriate distance D1 is presented to the user. In this way, by presenting more detailed information on the operations required by the user during calibration, it is possible to reduce the effort required for the operations required to align the camera 100 and the chart 200 during calibration.
[0090] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of each embodiment to a system or device via a network or a storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.
[0091] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]
[0092] 100: camera (imaging device), 110: lens control device, 130: camera control device, 138: camera control unit, 200: calibration chart
Claims
1. 1. An imaging device for capturing an image of a chart for correcting lens distortion, comprising: a first acquisition means for determining a deviation angle indicating an angle by which the imaging device deviates from a positional relationship in which the imaging device faces the chart directly, based on first distance information from an optical center of the imaging device to a center of the chart and second distance information to an edge of the chart, and information on an angle of view of the imaging device; a second acquisition means for acquiring a deviation distance indicating a distance deviated from the directly facing positional relationship based on the deviation angle and the first distance information; a third acquisition means for acquiring an appropriate distance between the imaging device and the chart in the facing positional relationship based on the size of the chart and the angle of view information; a presentation means for presenting information based on the deviation angle and the deviation distance so as to maintain the directly facing positional relationship until the deviation distance and the deviation angle become less than a predetermined judgment threshold, and for presenting information regarding the appropriate distance when the deviation distance and the deviation angle become less than a predetermined judgment threshold.
2. a fourth acquisition means for acquiring a distance from the imaging device to the chart based on a defocus amount determined from a captured image generated by capturing an image of the chart and position information of the lens; a fifth obtaining means for obtaining a size of the chart based on the first distance information and the second distance information; and a sixth acquisition means for acquiring a distance from the imaging device to the appropriate distance based on a distance from the imaging device to the chart and the appropriate distance, The imaging device according to claim 1, characterized in that the presentation means presents information based on the deviation distance and the deviation angle until the deviation distance and the deviation angle become less than a predetermined judgment threshold, and presents the distance to the appropriate distance of the imaging device when the deviation distance and the deviation angle become less than a predetermined judgment threshold.
3. a first detection means for detecting the center of the chart from the captured image; a second detection means for detecting an edge of the chart from the captured image; a calculation unit for calculating the defocus amount from the captured image, the second detection means detects an edge of the chart when a distance between a center of the captured image and a center of the chart is less than a predetermined determination threshold; 3. The imaging apparatus according to claim 2, wherein the fourth acquisition means acquires the distance to the chart based on a subject distance in a focus state obtained from the defocus amount and position information of the lens.
4. the ends of the chart are provided on both ends of the chart that sandwich the center of the chart in a direction horizontal to the center of the chart, 4. The imaging device according to claim 3, wherein the first acquisition means calculates the deviation angle using first angle-of-view information in which the angle of view is halved when one or fewer chart edges are detected from the captured image, and calculates the deviation angle using second angle-of-view information approximated by a distance from a center of the chart to both ends of the angle of view and a distance from the center of the chart to the chart edges when two chart edges are detected from the captured image and both ends of the angle of view are not located within the chart.
5. 5. The imaging device according to claim 1, wherein the presenting means displays first information that prompts a user to move the imaging device so that the ends of the two charts are within an angle of view of the imaging device until the ends of the two charts are detected in an image generated by imaging the chart.
6. the presenting means displays second information presenting to a user a prompt to move the imaging device so as to bring the imaging device closer to the center of the chart until a distance between a center of an image generated by imaging the chart in the facing positional relationship and a center of the chart becomes less than a predetermined determination threshold; The imaging device according to any one of claims 1 to 5, characterized in that when the center of the captured image and the center of the chart are less than a predetermined judgment threshold, information regarding the appropriate distance obtained by the third acquisition means is presented.
7. The imaging device according to claim 6, characterized in that the presentation means presents third information indicating that the distance to the appropriate distance of the imaging device has become less than a predetermined judgment threshold when the center of the captured image and the center of the chart become less than a predetermined judgment threshold.
8. 8. The imaging device according to claim 1, wherein the directly opposed positional relationship is a state in which the angle between a line connecting the optical center of the imaging device and the center of the chart and a normal line extending from the center of the chart, and the distance between the optical center of the imaging device and the normal line are each less than a predetermined judgment threshold value.
9. 1. A method for controlling an imaging device that captures an image of a chart for correcting lens distortion, comprising the steps of: calculating a deviation angle indicating an angle by which the imaging device deviates from a positional relationship in which the imaging device faces the chart directly, based on first distance information from an optical center of the imaging device to a center of the chart, second distance information from an optical center of the imaging device to an edge of the chart, and field of view information of the imaging device; calculating a deviation distance indicating a distance deviated from the directly facing positional relationship based on the deviation angle and the first distance information; determining an appropriate distance between the imaging device and the chart in the facing-to-the-camera positional relationship based on the size of the chart and the angle of view information; presenting information based on the deviation angle and the deviation distance so as to maintain the directly facing positional relationship until the deviation distance and the deviation angle become less than a predetermined judgment threshold, and presenting information regarding the appropriate distance when the deviation distance and the deviation angle become less than a predetermined judgment threshold.
10. A program for causing a computer to function as each of the means of the imaging apparatus according to any one of claims 1 to 8.
Citation Information
Patent Citations
JP193820A
Crack width measuring system
JP2017049152A
Information processor and information processing method and system
JP2020123042A
Measurement device, imaging device, control method, and program
JP2020193820A
Image processing device, image processing method, and program
JP2021027584A
