Imaging device, computer program, and imaging method

Abstract

To provide an imaging apparatus capable of performing focus detection with high accuracy with less time lag and at a high frame rate.SOLUTION: The imaging apparatus includes focus position detection means for calculating a focus position by using an image through an imaging optical system, aberration information acquisition means for acquiring the aberration information of the imaging optical system, color information acquisition means for acquiring the color information of a subject in a focus detection area, first correction value calculation means for calculating a first correction value for correcting the focus position on the basis of the aberration information, second correction value calculation means for calculating a second correction value for correcting the focus position on the basis of the first correction value and the color information, and correction means for correcting the focus position by using the second correction value.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an imaging device and the like, and more particularly to an imaging device and the like capable of detecting a focal position. [Background technology]

[0002] In imaging devices, contrast detection and phase difference detection methods are used to perform autofocus detection using images obtained from an image sensor. In both methods, the output of the image sensor is filtered with a bandpass filter to use frequency components with a predetermined narrow band lower than the Nyquist frequency, thereby improving the signal-to-noise ratio and reducing memory and computational requirements. Furthermore, during focus detection, pixels are added within the image sensor to lower the sampling rate. As described above, the frequency band used for focus detection of a captured image is different from the frequency band of the image for display or recording.

[0003] On the other hand, lens aberrations can cause the focus lens position at which contrast is highest for each spatial frequency to shift slightly. Because lens aberrations depend on the optical design, the aberration characteristics of interchangeable lens imagers change depending on the lens attached. Furthermore, even with the same interchangeable lens, the specifications of the imager vary from product to product, resulting in different frequency characteristics for focus detection.

[0004] For example, Patent Document 1 discloses a method in which the relationship between aberration and spatial frequency is stored on the lens side or the imaging device side as a set of characteristic curves for each aperture and focus position, a curve is selected according to the conditions, and a correction value is calculated. In this way, correction is possible even if there are many combinations of interchangeable lenses and imaging devices.

[0005] Furthermore, Patent Document 2 discloses a method for detecting and correcting information obtained from a captured image regarding focal position shifts that occur in accordance with the color, frequency, edge direction, etc. of a subject. This makes it possible to correct focal position shifts caused by, for example, extremely red or blue subjects. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-222411 [Patent Document 2] Japanese Patent Application Publication No. 2017-142333 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the conventional technology disclosed in the aforementioned Patent Document 2 requires solving a long polynomial using floating-point numbers, which increases the amount of calculation and increases the time lag between when the focus detection results are reflected and when the lens is driven. This increases the release time lag between when the user presses the release button and when image capture begins, increasing the possibility of missing a photo opportunity.

[0008] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an imaging apparatus capable of highly accurate focus detection with little time lag and at a high frame rate. [Means for solving the problem]

[0009] In order to achieve the above object, an imaging device of the present invention comprises: a focal position detection means for calculating a focal position using an image obtained through an imaging optical system; an aberration information acquisition means for acquiring aberration information of the imaging optical system; a color information acquisition means for acquiring color information of a subject within a focus detection area; a first correction value calculation means for calculating a first correction value for correcting the focal position based on the aberration information; a second correction value calculation means for calculating a second correction value for correcting the focus position based on the first correction value and the color information; a correction means for correcting the focal position using the second correction value; With death, The focus position detection means detects the focus position based on the image of a predetermined frame after exposure of the image of the frame is completed, and the first correction value calculation means calculates the first correction value during exposure of the image of the frame. It is characterized by: [Effects of the Invention]

[0010] According to the present invention, an imaging apparatus can be realized that is capable of highly accurate focus detection with a small time lag and at a high frame rate. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram of an imaging apparatus according to a first embodiment. [Figure 2] 1 is a cross-sectional view of a pixel of an image sensor according to a first embodiment. [Figure 3] FIG. 10 is a diagram illustrating the relationship between image contrast and focus position. [Figure 4] FIG. 10 is a diagram illustrating aberrations for each color. [Figure 5] 1 is a flowchart of a process for calculating a correction value according to the prior art. [Figure 6] 1 is a timing chart of an imaging device according to the prior art. [Figure 7] 4 is a flowchart of focus detection in the first embodiment. [Figure 8] 8 is a flowchart showing step S104 of FIG. 7 in detail. [Figure 9] 8 is a flowchart showing step S105 of FIG. 7 in detail. [Figure 10] 3 is a timing chart of the imaging apparatus of the first embodiment. [Figure 11] FIG. 10 is a block diagram of an imaging apparatus according to a second embodiment. [Figure 12]10 is a diagram showing a detection frame detected by the subject detection unit 1001. FIG. [Figure 13] 10 is a flowchart of focus detection in the second embodiment. [Figure 14] 10 is a timing chart of the imaging apparatus of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, preferred embodiments of the present invention will be described by way of example with reference to the accompanying drawings. In each drawing, the same members or elements are designated by the same reference numerals, and duplicated descriptions will be omitted or simplified. In the embodiments, the present invention will be described as being applied to a digital still camera as an imaging device, but imaging devices also include electronic devices with imaging functions, such as digital movie cameras, smartphones with cameras, tablet computers with cameras, network cameras, and cameras mounted on moving objects. Example 1

[0013] FIG. 1 is a block diagram of an image pickup apparatus according to the first embodiment, and FIG. 2 is a cross-sectional view of a pixel of an image pickup element according to the first embodiment. The overall operation of the imaging apparatus is controlled by a microcomputer (not shown) that executes a computer program stored in a memory (not shown).

[0014] The lens 301 is, for example, a lens arranged in an interchangeable lens barrel that can be attached to and detached from the imaging device, and an image formed by the lens 301 on the imaging surface of the imaging element 302 is read out as an electrical signal from the imaging element 302. Each pixel of the imaging element 302 has a configuration as shown in FIG.

[0015] 2 is a cross-sectional view of a pixel of the image sensor of Example 1, in which 201 denotes a microlens arranged on a pixel, 204 denotes a color filter, and 202 and 203 denote photoelectric conversion units. Light collected by the microlens 201 passes through one color filter 204 of R (red), G (green), B (blue), etc., and is incident on the photoelectric conversion units 202 and 203, where it is converted into an electrical signal. The R, G, B, etc. color filters are arranged in an array such as a Bayer array for the multiple pixels of the image sensor 302.

[0016] The photoelectric conversion units 202 and 203 form two signals having parallax by respectively receiving light that has passed through different pupil positions of the lens 301. Here, it is assumed that the photoelectric conversion unit 202 outputs an A signal as a photoelectric conversion signal, and the photoelectric conversion unit 203 outputs a B signal as a photoelectric conversion signal. A collection of A signals from multiple pixels forms an A image signal, and a collection of B signals from multiple pixels forms a B image signal. The A and B image signals are two image signals having parallax, and the distance to the subject and the amount of defocus can be determined by calculating the relative phase difference between the two.

[0017] A method of detecting focus that applies this principle is known as the image plane phase difference method. On the other hand, the A+B signal, which is the sum of the A and B signals from each pixel, can be treated as a normal pixel signal, and the collection of A+B signals from multiple pixels can be displayed or recorded as a normal image signal. That is, the A and B signals of each pixel output from the image sensor 302 in FIG. 1 are added by an A+B adder 303 to become a normal pixel signal, which is then converted into a color video signal by a signal processing circuit 306 and transferred to the subsequent stage.

[0018] Although the blocks subsequent to the signal processing circuit 306 are omitted, the signal is supplied to a recording circuit, a display circuit, etc. and is treated as a normal image signal (video signal) in the imaging device. The color information acquisition unit 309 acquires statistical information about the color of the subject in the image signal based on the output of the A+B addition unit 303 .

[0019] The A and B signals of each pixel output from the image sensor 302 are also input to an AB separation unit 304, where an A image signal made up of A signals from multiple pixels and a B image signal made up of B signals from multiple pixels are each input to a BPF (band pass filter) 305. These signals are then processed into signals that contain only a specific frequency band lower than the Nyquist frequency. The output from the BPF 305 is input to a correlation calculation unit 307, which calculates the amount and direction of the correlation phase difference between the A and B image signals.

[0020] The focus detection unit 308 calculates the defocus amount using the output of the correlation calculation unit 307 and the correction value calculated by the correction value calculation unit 310, and sends a control signal to the lens control unit 311 to drive the lens 301. Then, for example, the focus detection unit 308 adjusts the focal position of the lens for exposure (capturing) of the next frame. Here, the focus detection unit 308 functions as a focal position detection means that calculates the focal position using an image that has passed through the imaging optical system.

[0021] The correction value calculation unit 310 calculates a correction value using lens aberration information acquired from the lens control unit 311 and color information acquired from the color information acquisition unit 309. Here, the correction value calculation unit 310 functions as an aberration information acquisition unit that acquires aberration information of the imaging optical system, and the color information acquisition unit functions as a color information acquisition unit that acquires color information of the subject within the focus detection area.

[0022] The first embodiment is characterized by the calculation process in the correction value calculation unit 310 and the like. As described above, Example 1 uses an interchangeable lens barrel, and the lens 301 and lens control unit 311 are disposed inside the interchangeable lens barrel. Therefore, if the lens barrel is replaced, the optical characteristics such as lens aberration will change.

[0023] Therefore, in the first embodiment, an in-lens memory (not shown) that stores optical characteristic information such as lens aberration is provided in the lens control unit 311. Then, the main body of the imaging unit and the lens control unit 311 communicate with each other via a communication path (not shown), so that the correction value calculation unit 310 and the focus detection unit 308 can acquire optical characteristic information such as lens aberration information from the in-lens memory.

[0024] FIG. 3 is a diagram showing the relationship between image contrast and focus position. FIG. 3A is a diagram showing the relationship between the contrast of an image obtained through a predetermined lens A and the focus lens position, with the vertical axis representing the contrast and the horizontal axis representing the focus lens position. Reference numeral 401 denotes a contrast curve of a first frequency band having a predetermined bandwidth lower than the Nyquist frequency, which is band-limited by the BPF 305, and the position 402 has the highest contrast.

[0025] Therefore, if the focusing lens is brought to position 402, a focused image should be obtained. On the other hand, reference numeral 403 denotes a contrast curve of a second frequency band extending up to the vicinity of the Nyquist frequency of the video signal of the signal processing circuit 306. That is, the second frequency band for the video signal is wider than the first frequency band.

[0026] The output of the signal processing circuit 306 has a wider bandwidth than the BPF 305, and includes frequencies from low to high, and a state in which the contrast is high near the Nyquist frequency corresponds to a visually in-focus state. Therefore, if the focus lens is positioned at position 404, which is the peak of the curve, the image will visually appear to be in focus.

[0027] In other words, if the focus lens is controlled to position 402, which is the focus position calculated based on the correlation phase difference obtained from the correlation calculation unit 307, a slightly out-of-focus image will be obtained from the signal processing circuit 306. In this embodiment, the difference between 402 and 404 is corrected, and a correction value for moving the lens to position 404 is calculated by the correction value calculation unit 310.

[0028] 3B is a diagram showing the relationship between contrast and focus lens position for lens B, which is different from lens A. Reference numeral 501 denotes the contrast curve of the output of the BPF 305, and 503 denotes the contrast curve of the output of the signal processing circuit 306. Reference numerals 502 and 504 denote the peaks of the contrast curves 501 and 503, respectively.

[0029] 3B, a peak position 502 of a contrast curve 501 of the output of the BPF 305 of lens B is different from a peak position 402 of the contrast curve of the output of the BPF 305 of lens A. In other words, since lenses B and A have different optical characteristics, an image with correctly detected focus cannot be obtained unless a correction value is calculated according to the characteristics of each lens.

[0030] FIG. 4 is a diagram explaining aberration for each color. FIG. 4(A) shows the relationship between contrast and focus lens position for the same lens A as FIG. 3(A). FIG. 4(B) shows the characteristics of curve 403 in FIG. 4(A) broken down into smaller sections for each color. 405, 406, and 407 show the relationship between contrast and focus position for the R, G, and B wavelengths, respectively. The peak positions of R, G, and B contrast curves 405 to 407 are 409 to 411, respectively, and curve 408 is a combination of R, G, and B contrast curves 405 to 407.

[0031] Although the peak position (focus position) of the contrast curve shifts slightly for each color due to the influence of aberration, it is generally best to use the peak position 410 of the contrast curve 408 because the image contains various color components. However, if the color of the subject is extremely red or blue, for example, when photographing a red flower, the focus will be blurred unless the focus is at position 409 rather than 410. In this embodiment, even in such a case, a good correction value can be obtained.

[0032] FIG. 5 is a flowchart of the correction value calculation process according to the prior art, and FIG. 5(A) is a flowchart of the entire focus detection operation. After focus detection is started in step S101, exposure (image capture) operation for, for example, one frame is performed in the image sensor in step S102, and then the image is read out.

[0033] In step S103, the focus detection unit 308 calculates the defocus amount using the result of the correlation calculation unit 307. In step S501, a correction value for correcting the focus position according to the subject color is calculated, and in step S106, the lens is driven to the corrected focusing position based on the correction value. After that, the process returns to step S101 and the focus detection operation is repeated.

[0034] Figure 5(B) is a flowchart showing in detail the correction value calculation flow in step S501 in Figure 5(A). In step S502, lens status information is acquired. The lens status information includes the current focus lens position, aperture value, etc. Next, in step S503, lens aberration information is acquired from the memory in the lens barrel.

[0035] Next, in step S504, color information of the subject is acquired from color information acquisition unit 309. Then, in step S505, a predetermined calculation is performed based on the information acquired in steps S502 to S504 to determine the correction value. The process performed in step S505 will be described in detail below. While aberration exhibits different characteristics in the concentric and radial directions, the pixels are rectangular and comprised of a plurality of pixels arranged in the horizontal and vertical directions.

[0036] Furthermore, multiple optical elements such as lenses of different sizes are supported by a frame or the like within the lens barrel and arranged side by side along the optical axis, and the light paths that reach each coordinate on the imaging surface of the image sensor are different. Furthermore, the shape of the optical diaphragm, which is the shape of light restricted by the frame or the like within the lens barrel, also changes smoothly depending on the location. Therefore, the aberration information takes on a complex shape rendered from these optical calculations, but in order to store this shape using a small amount of memory, it is stored as a curve-fitted polynomial calculation formula.

[0037] For example, information MTF_P_RH about the position of the focus lens 104, which indicates the maximum value of the defocus MTF, is expressed by the following equation 1. That is, for example, for each of six combinations of color and vertical and horizontal directions, it is expressed by the following equation 1, with the spatial frequency f and the position coordinates (x, y) of the focus detection area on the image sensor as variables.

[0038] MTF_P_RH(f,x,y)= (rh(0)×x+rh(1)×y +rh(2))×f2 +(rh(3)×x +rh(4)×y +rh(5))×f +(rh(6)×x +rh(7)×y +rh(8))...Equation (1)

[0039] Note that while Equation 1 represents the focus lens position information MTF_P_RH, which indicates the maximum value of the defocus MTF for each spatial frequency corresponding to the horizontal (H) direction for the red (R) signal, similar equations can be used for other combinations. That is, in this embodiment, rh(n) (0≦n≦8) is stored in advance in a memory within the lens barrel, and the imaging device acquires rh(n) (0≦n≦8) by requesting it from the CPU within the lens barrel. However, rh(n) (0≦n≦8) may also be stored in a memory within the imaging device.

[0040] The coefficients (rv, gh, gv, bh, bv) for each combination of red and vertical (MTF_P_RV), green and horizontal (MTF_P_GH), green and vertical (MTF_P_GV), blue and horizontal (MTF_P_BH), and blue and vertical (MTF_P_BV) can also be stored and retrieved in the same way. By inputting the spatial frequency f and the position coordinates (x, y) of the focus detection area as variables, the characteristics of that frequency at that position can be obtained. By obtaining such curves for each color (R, G, B), for each vertical and horizontal direction, and for each frequency, and then combining them, the correction value can be obtained.

[0041] If the spatial frequency is set to 10 discretely, the calculation will be based on a total of 60 curves, which are vertical and horizontal (2) x RGB (3) x spatial frequency (10). Moreover, since this will be a very long polynomial, the calculation will be done using floating point numbers.

[0042] These curves change when the lens aperture, zoom position, or focus position (distance to the pupil) changes, so they must be calculated in real time.

[0043] When the position coordinates (x, y) of the focus detection area for calculating the correction value are determined, the characteristics for each spatial frequency are expressed by the following equation 2. MTF_P_RH(f)=Arh×f2+Brh×f+Crh...Equation (2) Calculate MTF_P_RV(f), MTF_P_GH(f), MTF_P_GV(f), MTF_P_BH(f), and MTF_P_BV(f) in the same way. Next, the following calculation is performed for each spatial frequency in accordance with the characteristics of the pupil-separated AF signal based on the following equation 3. That is, the spatial frequency characteristic MTF_P_AF(f) for focus detection is found using equation 3.

[0044] MTF_P_AF(f)= K_AF_R×K_AF_H×MTF_P_RH(f) +K_AF_R×K_AF_V×MTF_P_RV(f) +K_AF_G×K_AF_H×MTF_P_GH(f) +K_AF_G×K_AF_V×MTF_P_GV(f) +K_AF_B×K_AF_H×MTF_P_BH(f) +K_AF_B×K_AF_V×MTF_P_BV(f)...Equation (3)

[0045] Here, K_AF_R, K_AF_G, and K_AF_B are information indicating the magnitude of weighting for focus detection for R, G, and B. Also, K_AF_H and K_AF_V are information indicating the magnitude of weighting for focus detection for the contrast direction (horizontal, vertical), and this information is stored in advance in memory inside the lens barrel or the imaging device.

[0046] Similarly, the spatial frequency characteristic MTF_P_IMG(f) for the captured image is calculated using the following equation 4. MTF_P_IMG(f)= K_IMG_R×K_IMG_H×MTF_P_RH(f) +K_IMG_R×K_IMG_V×MTF_P_RV(f) +K_IMG_G×K_IMG_H×MTF_P_GH(f) +K_IMG_G×K_IMG_V×MTF_P_GV(f) +K_IMG_B×K_IMG_H×MTF_P_BH(f) +K_IMG_B×K_IMG_V×MTF_P_BV(f)...Equation (4)

[0047] K_IMG_R, K_IMG_G, and K_IMG_B are information indicating the magnitude of weighting of the captured image for R, G, and B. K_IMG_H and K_IMG_V are information indicating the magnitude of weighting of the captured image for the contrast direction (horizontal, vertical), and this information is stored in advance in memory inside the lens barrel or the imaging device.

[0048] Furthermore, based on the characteristics calculated for each frequency, a representative value (focus position (P_AF) detected by AF) suited to the characteristics of the BPF 305 is calculated using the following equation 5.

[0049] P_AF = MTF_P_AF(1) × K_AF_FQ(1) +MTF_P_AF(2)×K_AF_FQ(2) +MTF_P_AF(3)×K_AF_FQ(3) +MTF_P_AF(4)×K_AF_FQ(4) +MTF_P_AF(5)×K_AF_FQ(5) +MTF_P_AF(6)×K_AF_FQ(6) +MTF_P_AF(7)×K_AF_FQ(7) +MTF_P_AF(8)×K_AF_FQ(8) +MTF_P_AF(9)×K_AF_FQ(9) +MTF_P_AF(10)×K_AF_FQ(10)...Equation (5)

[0050] Here, MTF_P_AF(1) to MTF_P_AF(10) are MTF_P_AF(f) calculated for each spatial frequency using Equation 3. Also, K_AF_FQ(1) to K_AF_FQ(10) are information indicating the magnitude of weighting for focus detection for each spatial frequency, and are stored in advance in memory inside the lens barrel or the imaging device. Similarly, a representative value of the in-focus position (P_img) of the captured image that matches the characteristics of the signal processing circuit 306 is calculated using Equation 6.

[0051] P_img=MTF_P_IMG(1)×K_IMG_FQ(1) +MTF_P_IMG(2)×K_IMG_FQ(2) +MTF_P_IMG(3)×K_IMG_FQ(3) +MTF_P_IMG(4)×K_IMG_FQ(4) +MTF_P_IMG(5)×K_IMG_FQ(5) +MTF_P_IMG(6)×K_IMG_FQ(6) +MTF_P_IMG(7)×K_IMG_FQ(7) +MTF_P_IMG(8)×K_IMG_FQ(8) +MTF_P_IMG(9)×K_IMG_FQ(9) +MTF_P_IMG(10)×K_IMG_FQ(10)...Equation (6)

[0052] Here, MTF_P_IMG(1) to MTF_P_IMG(10) are MTF_P_IMG(f) calculated for each spatial frequency using Equation 4. Also, K_IMG_FQ(1) to K_IMG_FQ(10) are information indicating the magnitude of weighting for the captured image for each spatial frequency, and are stored in advance in memory within the lens barrel or the imaging device.

[0053] The difference between the representative values ​​is calculated using the following equation 7 to obtain the correction value BP. BP=P_AF-P_img (7) However, with conventional technology, the above calculation cannot be performed until the output of the color information acquisition unit 309 is obtained in step S504. Meanwhile, there is an increasing need to calculate AF information at multiple positions on an image at high speed, and since these multiple positions can sometimes number in the hundreds, the time lag caused by the time it takes to calculate the correction value becomes a serious problem.

[0054] FIG. 6 is an operation timing chart of a conventional imaging device. At t701, exposure is started by opening the shutter, for example, and at t702, reading of the image captured by the exposure from the image sensor 302 is started. Additionally, the color information acquisition unit 309 acquires color information in synchronization with this readout, but the completion of the color information acquisition takes place, for example, near t704 after the completion of readout from the image sensor 302. Even if defocus calculation is started during readout, it takes time until t705 because correlation calculations take time.

[0055] The correction value calculation unit 310 cannot start operation until t704 because it uses the output of the color information acquisition unit 309 to calculate the correction value. The lens can finally start to be driven at timing t706 when both the defocus calculation and the correction value calculation are completed, and the lens is completely driven and in focus at timing t707, so the release time lag is a period of 709.

[0056] In this embodiment, the correction calculation time 708 can be shortened. Fig. 7 is a flowchart of focus detection in Example 1. Note that the steps in the flowcharts of Fig. 7 to Fig. 9 and Fig. 13 are executed by a computer (not shown) in the imaging apparatus executing a computer program stored in a memory (not shown).

[0057] Focus detection is started in step S101, exposure (image capture) and readout processing are performed in step S102, and focus detection calculation is performed in step S103. Meanwhile, in parallel with steps S102 and S103, a first correction value calculation is performed in step S104. Here, step S104 functions as a first correction value calculation means (first correction value calculation step) that calculates a first correction value for correcting the focus position based on aberration information. The first correction value calculation in step S104 will be described later using the flowchart in FIG. 8.

[0058] In step S105, a second correction value is calculated, and in step S106, the lens is driven. Here, step S105 functions as second correction value calculation means (second correction value calculation step) that calculates a second correction value for correcting the focal position based on the first correction value and color information. Also, step S106 functions as correction means (correction step) that corrects the focal position using the second correction value. The calculation of the second correction value in step S105 will be described later using the flowchart in FIG. 9. Thereafter, the above steps S101 to S106 are repeated.

[0059] FIG. 8 is a flowchart showing step S104 in FIG. 7 in detail. In step S107, the process of calculating the first correction value starts, and in step S108, lens aberration information is acquired from the memory in the lens barrel (aberration information acquisition step). The memory in the lens barrel stores the peak position 410 of the contrast curve 408 in FIG. 4, etc.

[0060] In addition, information regarding the peak position 409 of the R contrast curve 405, information regarding the difference between the peak position 411 of the B contrast curve 407 and the peak position 410 of the B contrast curve 408, and weighting data for the above equations 1 to 6 are stored in advance, and these values ​​are also acquired. In step S109, color information of the subject is acquired from the color information acquisition unit 309.

[0061] In step S110, it is determined whether the chromatic aberration of the lens is equal to or greater than a threshold value. That is, if the information on the peak position for each color acquired in step S108 is referenced and the chromatic aberration of the lens is lower than a predetermined threshold (No in step S110), the process proceeds to step S112. In this case, there is little deviation in the focus position due to color, and even if the saturation of the subject is high, the focus position will not be distorted, so in step S112, the color information is not taken into account and the subject is considered to be white and a correction value is calculated.

[0062] With reference to the peak position information for each color acquired in step S108, if the chromatic aberration of the lens is equal to or greater than a predetermined threshold (Yes in step S110), the deviation of the focus position due to color is large, so the process proceeds to step S111. In step S111, correction value information for each color is calculated. In this way, in step S110, whether or not to calculate the first correction value for each color is switched depending on the aberration information acquired by the aberration information acquisition means.

[0063] The calculation in this case is different from the conventional formula 3, and the result is for each color as shown in the following formula 8. MTF_P_AF_R(f)= K_AF_H×MTF_P_RH(f) +K_AF_V×MTF_P_RV(f)...Equation (8)

[0064] Similarly to equation 8, MTF_P_AF_G(f) and MTF_P_AF_B(f) are calculated for G and B, respectively. Similarly, MTF_P_IMG_R(f), MTF_P_IMG_G(f), and MTF_P_IMG_B(f) are calculated. Furthermore, equation 5 in the conventional example becomes equation 9 below.

[0065] P_AF_R =MTF_P_AF_R(1)×K_AF_FQ(1) +MTF_P_AF_R(2)×K_AF_FQ(2) +MTF_P_AF_R(3)×K_AF_FQ(3) +MTF_P_AF_R(4)×K_AF_FQ(4) +MTF_P_AF_R(5)×K_AF_FQ(5) +MTF_P_AF_R(6)×K_AF_FQ(6) +MTF_P_AF_R(7)×K_AF_FQ(7) +MTF_P_AF_R(8)×K_AF_FQ(8) +MTF_P_AF_R(9)×K_AF_FQ(9) +MTF_P_AF_R(10)×K_AF_FQ(10)...Equation (9)

[0066] Here, MTF_P_AF_R(1) to MTF_P_AF_R(10) are MTF_P_AF_R(f) calculated by Equation 8 for each spatial frequency. The same calculation is performed for P_AF_G, P_AF_RB, P_img_R, P_img_G, and P_img_B, and the results become first correction values.

[0067] FIG. 9 is a flowchart showing step S105 in FIG. 7 in detail. In step S113, the process of calculating the second correction value begins, and in step S114, it is determined whether the chromatic aberration is equal to or greater than the threshold value, as in step S110. That is, if the lens does not have large chromatic aberration, the result in step S114 is No, and the final correction value has already been obtained, so no additional calculation of the correction value is required.

[0068] If the answer is Yes in step S114, the process proceeds to step S115, where color information of the subject is acquired from the color information acquisition unit 309 (color information acquisition step). In this way, in step S114, whether or not to calculate the second correction value is switched depending on the aberration information acquired by the aberration information acquisition means. K_AF_R, K_AF_G, and K_AF_B are determined based on the color information acquired here. That is, in step S116, the final correction value is calculated using the color information and the first correction value, for example, according to the following equation 10.

[0069] BP= (P_AF_R×K_AF_R +P_AF_G×K_AF_G +P_AF_B×K_AF_B) -(P_img_R×K_AF_R +P_img_G×K_AF_G +P_img_G×K_AF_G)...Equation (10)

[0070] FIG. 10 is a timing chart of the imaging apparatus of the first embodiment. A description of the same parts as those in the timing chart of the prior art in FIG. 6 will be omitted. 10, 804 is the first correction value calculation process of step S104 in Fig. 7, and 805 is the second correction value calculation process of step S105 in Fig. 7. That is, the correction value calculation process is divided into two, and the first correction value calculation is performed during the exposure period.

[0071] That is, the focus position detection means detects the focus position based on the image of a predetermined frame captured during the period from t701 to t702 at t703 after the end of exposure of the image of the frame (focus position detection step). Meanwhile, the first correction value calculation means calculates the first correction value in advance during the period from t701 to t801 during exposure of the image of the frame. In this way, the time for calculating the correction value in part 806 (the calculation from t705 to t706 in FIG. 6) is shifted to timing 804 and performed in parallel with the exposure operation, so that the release time lag 803 can be made shorter than the release time lag 709 of the prior art.

[0072] Example 2 Fig. 11 is a block diagram of an image capturing apparatus according to embodiment 2. The difference from Fig. 1 is that an object detection unit 1001 has been added. The subject detection unit 1001 analyzes the output image of the signal processing circuit 306 and automatically sets the focus detection area. The focus area setting means may be one that allows the focus detection area to be set manually.

[0073] FIG. 12 is a diagram showing a detection frame 1101 detected by the subject detection unit 1001. The detection of the subject may be performed by normal image recognition or by using deep learning technology, and a detailed description thereof will be omitted.

[0074] FIG. 13 is a flowchart of focus detection in the second embodiment. FIG. 13(A) is a rough flowchart. 7 will not be described. After exposure / readout is performed in step S102, focus detection calculation is performed in step S103 and subject detection / color detection is performed in step S902 in parallel. Then, in parallel with the operations of steps S102, S103, and S902, correction values ​​are calculated in step S901.

[0075] Figure 13(B) is a flowchart showing in detail the correction value calculation process in step S901. The steps up to acquiring lens state information in step S108 and acquiring lens aberration information in step S109 are the same as those in Figure 8. In step S903, color information of the subject is acquired, but the color information acquired in the previous frame is used. Here, step S903 functions as color information acquisition means that acquires color information of the subject within the focus detection area of ​​a frame prior to the predetermined frame for focus detection.

[0076] That is, in Example 1, color information in the image of the frame for which focus detection is to be performed is detected, whereas color information from the previous frame is used in Example 2. This is because, for example, when automatically detecting a subject and detecting the focus of that subject position, there is an extremely high possibility that the color information detected from the subject will be the same even if the focus detection position (subject position) moves significantly.

[0077] In step S904, a correction value is calculated based on color information of the subject, but the difference is that the color information in this case is color information of the area where the subject was detected in the previous frame. Here, step S904 functions as a third correction value calculation means that calculates a third correction value for correcting the focus position based on the aberration information and the color information of the previous frame. Also, step S106 functions as a correction means that corrects the focus position using this third correction value.

[0078] FIG. 14 is a timing chart of the imaging device of the second embodiment. Exposure begins at t1201, the exposure results are read, and subject detection and color detection are performed at t1202. Furthermore, correction value calculation begins in synchronization with the start of the next exposure operation at t701. That is, in this embodiment, the focus position detection means detects the focus position based on images of a predetermined frame from t701 to t702, and the color information acquisition means acquires color information from images of frames from t1201 to t1202 before the predetermined frame. As a result, the second correction calculation in the first embodiment is also unnecessary, and the release time lag t1201 is shorter than 803 in the first embodiment. Note that color information may also be acquired from a frame two or more frames before the predetermined frame.

[0079] Although the preferred embodiment of the present invention has been described above, the focus detection method can also be implemented in a contrast AF system. Furthermore, instead of providing two photoelectric conversion units in each pixel to obtain both A and B signals as shown in Figure 2, one of the photoelectric conversion units in Figure 2 may be shielded from light, and a plurality of pixels that obtain only A signals and a plurality of pixels that obtain only B signals may be arranged within the imaging screen, thereby obtaining parallax images. In addition, although an image sensor combined with RGB color filters is used, the color filters may include complementary color filters. In the second embodiment, color information acquired in the previous frame is used, but color information acquired from information from several frames earlier may also be used as long as the same subject is detected.

[0080] Furthermore, since it is possible to simultaneously detect multiple candidate sub-subjects other than the main subject and switch the sub-subject to the main subject based on the final defocus information, it is desirable to acquire and store color information about the sub-subjects as well. Furthermore, the color information may be obtained by dividing the entire screen into multiple regions in a mesh pattern and acquiring and storing representative color information for each of the multiple regions, and using this color information immediately before parallax image exposure.

[0081] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention. Note that a computer program that realizes part or all of the control in this embodiment and the functions of the above-described embodiment may be supplied to an imaging device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the imaging device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0082] 201 Micro Lens 202, 203 Photoelectric conversion unit 204 Color Filter 301 Lens 302 Image sensor 303 A+B Addition Section 306 Signal Processing Circuit 304 AB separation section 305 BPF 307 Correlation calculation unit 308 Focus detection unit 309 Color information acquisition unit 310 Correction value calculation unit 311 Lens control unit

Claims

1. a focal position detection means for calculating a focal position using an image obtained through an imaging optical system; an aberration information acquisition means for acquiring aberration information of the imaging optical system; a color information acquisition means for acquiring color information of a subject within a focus detection area; a first correction value calculation means for calculating a first correction value for correcting the focal position based on the aberration information; a second correction value calculation means for calculating a second correction value for correcting the focus position based on the first correction value and the color information; a correction means for correcting the focal position using the second correction value; and an imaging device characterized in that the focus position detection means detects the focus position based on the image of a predetermined frame after exposure of the image of the frame is completed, and the first correction value calculation means calculates the first correction value during exposure of the image of the frame.

2. 2. The imaging device according to claim 1, wherein the calculation by the first correction value calculation means is started before the color information is acquired by the color information acquisition means.

3. 2. The imaging apparatus according to claim 1, wherein whether or not to calculate the first correction value for each color is switched depending on the aberration information acquired by the aberration information acquisition unit.

4. 2. The imaging apparatus according to claim 1, wherein whether or not to calculate the second correction value is switched depending on the aberration information acquired by the aberration information acquisition unit.

5. a focus position detection means for calculating a focus position using an image of a predetermined frame taken through an imaging optical system; an aberration information acquisition means for acquiring aberration information of the imaging optical system; a color information acquisition means for acquiring color information of a subject within a focus detection area of ​​a frame preceding the predetermined frame; a third correction value calculation means for calculating a third correction value for correcting the focal position based on the aberration information and the color information; a correction means for correcting the focal position using the third correction value; and The imaging device is characterized in that the color information acquisition means acquires color information of the subject within the focus detection area of ​​a frame prior to the specified frame, in parallel with the focus position detection means calculating the focus position using the image of the specified frame.

6. 6. The imaging apparatus according to claim 5, wherein the focus detection area is set by automatically detecting a subject.

7. 6. The imaging device according to claim 5, wherein the color information acquisition means divides the image into a plurality of regions and acquires the color information for each of the plurality of regions.

8. A computer program for causing a computer to realize each means of the imaging device described in any one of claims 1 to 7.

9. a focal position detection step of calculating a focal position using an image through an imaging optical system; an aberration information acquisition step of acquiring aberration information of the imaging optical system; a color information acquisition step of acquiring color information of a subject within a focus detection area; a first correction value calculation step of calculating a first correction value for correcting the focal position based on the aberration information; a second correction value calculation step of calculating a second correction value for correcting the focus position based on the first correction value and the color information; a correcting step of correcting the focus position using the second correction value; and an imaging method characterized in that the focus position detection step detects the focus position based on the image of a predetermined frame after exposure of the image of the frame is completed, and the first correction value calculation step calculates the first correction value during exposure of the image of the frame.

Images