Imaging device

Abstract

This imaging device comprises: a plurality of first pixels and a plurality of second pixels that are arranged alternately at intervals of one pixel and that output a first signal and a second signal which are used to detect a focal point of an optical system; a plurality of third pixels that are disposed between the plurality of first pixels and the plurality of second pixels and that output a third signal which is used to generate an image; a detection unit that detects the focal point of the optical system using at least two among a first data string which is based on a signal value of the first signal, a second data string which is based on a signal value of the second signal, and a third data string which is based on a signal value of the third signal; and an interpolation unit that interpolates the signal value of the first signal between two adjacent first pixels in the first data string using linear interpolation using the signal value of the first signal which has been output from the two adjacent first pixels, and the signal value of the second signal or the third signal which has been output from a second pixel or a third pixel near the two adjacent first pixels.

Description

Imaging device

[0001] This relates to an imaging device.

[0002] Various methods have been proposed for focusing on a desired subject (for example, see Japanese Patent Application Laid-Open No. 2003-122294). Improvements in focusing accuracy have been desired for some time.

[0003] JP 2014-132321 A

[0004] According to an aspect of the disclosure, an imaging device includes a plurality of first pixels and a plurality of second pixels arranged alternately at every other pixel, each pixel photoelectrically converting light transmitted through an optical system and outputting a first signal and a second signal used for focus detection of the optical system, a plurality of third pixels arranged between the plurality of first pixels and the plurality of second pixels, each pixel photoelectrically converting light transmitted through the optical system and outputting a third signal used for generating an image, a first data string based on signal values ​​of the first signals output from the plurality of first pixels, a second data string based on signal values ​​of the second signals output from the plurality of second pixels, and ... second data string based on signal values ​​of the second signals output from the plurality of second pixels. and a third data string based on the signal values ​​of the third signal output from a third pixel of the plurality of first pixels; and an interpolation unit that, in the first data string, interpolates the signal values ​​of the first signal between two adjacent first pixels of the plurality of first pixels by linear interpolation using the signal values ​​of the first signal output from the two adjacent first pixels, and by interpolating the signal values ​​of the second signal or the third signal output from a second pixel or a third pixel of the plurality of second pixels or the plurality of third pixels that is located near the two adjacent first pixels.

[0005] The configurations of the embodiments described below may be modified as appropriate, and at least a portion of the configuration may be replaced with other components. Furthermore, components that are not particularly limited in terms of their placement may be placed in any position that allows them to achieve their function, not limited to the placement disclosed in the embodiments.

[0006] FIG. 1 is a diagram showing an example of the configuration of an imaging device according to an embodiment. FIG. 2 is a diagram showing an example of the arrangement of pixels in a pixel unit. FIG. 3 is a diagram showing example signal values ​​(pixel values) of signals output from an imaging pixel and first and second focus detection pixels when an image of a pattern having a steep edge is captured. FIG. 4 is a diagram showing data strings obtained by linear interpolation of pixel values ​​at positions where actual pixel values ​​cannot be obtained for an imaging pixel and first and second focus detection pixels. FIGS. 5A and 5B are diagrams for explaining calculation of an interpolated pixel value for a first focus detection pixel in this embodiment. FIG. 6 is a diagram for explaining calculation of an interpolated pixel value for a first focus detection pixel in this embodiment. FIGS. 7A and 7B are diagrams for explaining calculation of an interpolated pixel value for a second focus detection pixel in this embodiment. FIG. 8 is a diagram for explaining calculation of an interpolated pixel value for a second focus detection pixel in this embodiment. FIG. 9 is a diagram for explaining calculation of an interpolated pixel value for an imaging pixel in this embodiment. Fig. 10 is a diagram showing data strings when pixel values ​​at positions where actual pixel values ​​cannot be obtained for an imaging pixel and first and second focus detection pixels are calculated using the method of this embodiment. Fig. 11 is a diagram explaining a method of calculating interpolated pixel values ​​in Modification 1. Fig. 12 is a flowchart showing an example of calculation processing of interpolated pixel values ​​by an interpolation unit (Modification 1). Fig. 13 is a diagram explaining a method of calculating interpolated pixel values ​​in Modification 2. Fig. 14 is a flowchart showing an example of calculation processing of interpolated pixel values ​​by an interpolation unit (Modification 2). Fig. 15 is a diagram explaining a method of calculating interpolated pixel values ​​in Modification 2.

[0007] An electronic camera 1 (hereinafter referred to as camera 1) as an example of an imaging device according to an embodiment will be described below with reference to FIGS. 1 to 10. FIG.

[0008] FIG. 1 is a diagram showing an example of the configuration of a camera 1 according to an embodiment. Note that, in the drawings shown below, an XYZ Cartesian coordinate system is appropriately provided for ease of explanation and understanding. In this coordinate system, the direction from the subject toward the camera body 2 in a camera position (hereinafter referred to as the normal position) when a photographer takes a landscape image with the optical axis OA1 horizontal is defined as the +Z direction. The direction toward the left in the normal position is defined as the +X direction. The direction toward the bottom in the normal position is defined as the +Y direction. Note that the scales of the shapes, lengths, thicknesses, etc. of the various parts shown in the embodiment do not necessarily correspond to the actual objects, and parts not necessary for explanation have been omitted or simplified as appropriate.

[0009] The camera 1 includes a camera body 2 and an interchangeable lens 3. The camera 1 may be configured as an integrated lens camera rather than an interchangeable lens camera.

[0010] (Interchangeable Lens 3) The interchangeable lens 3 includes a lens-side mount unit 301, a photographing optical system (imaging optical system) 31, a lens control unit 32, and a lens memory 33. The lens-side mount unit 301 includes a lens-side connection unit 302. The lens-side connection unit 302 has multiple terminals, such as a terminal for a clock signal, a terminal for a data signal, and a terminal for power supply. The interchangeable lens 3 is detachably attached to the camera body 2 by the lens-side mount unit 301 and a body-side mount unit 201, which will be described later.

[0011] The photographic optical system 31 includes a plurality of lenses, including a zoom lens (variable magnification lens) 31a that changes the focal length and a focus lens (focus adjustment lens) 31b, and an aperture 31c, and forms a subject image on the imaging surface 22a of the image sensor 22. Note that although the zoom lens 31a and the focus lens 31b are shown in Fig. 1 as a schematic illustration, a typical photographic optical system is generally made up of a large number of optical elements.

[0012] The lens control unit 32 is configured with a processor such as a CPU, FPGA, or ASIC, and memories such as ROM and RAM, and controls each part of the interchangeable lens 3 based on a control program. The lens control unit 32 controls the position of the zoom lens 31a, the position of the focus lens 31b, and the drive of the diaphragm 31c based on signals output from the body control unit 21 of the camera body 2. When a signal indicating the movement direction and amount of the focus lens 31b is input from the body control unit 21, the lens control unit 32 moves the focus lens 31b back and forth along the optical axis OA1 based on the signal to adjust the focal position of the photographic optical system 31.

[0013] The lens memory 33 is composed of, for example, a non-volatile storage medium. Information related to the interchangeable lens 3 is stored (recorded) in the lens memory 33 as lens information. The lens information includes data on the optical characteristics of the photographic optical system 31, data on the infinity position and close position of the focus lens 31b, and data on the shortest focal length and longest focal length of the interchangeable lens 3. The optical characteristics of the photographic optical system 31 include the exit pupil distance and F-number (aperture value of the diaphragm 31c). The lens information differs depending on the type of interchangeable lens 3. The lens information may also be stored in an internal memory of the lens control unit 32. The lens information may also be stored in a body memory 23 included in the camera body 2, which will be described later. In this case, the body memory 23 stores lens information for multiple interchangeable lenses 3.

[0014] Writing data to the lens memory 33 and reading data from the lens memory 33 are controlled by the lens control unit 32. When the interchangeable lens 3 is attached to the camera body 2, the lens control unit 32 transmits lens information to the body control unit 21 via the terminals of the lens side connection unit 302 and the body side connection unit 202. The lens control unit 32 also transmits to the body control unit 21 position information (focal length information) of the controlled zoom lens 31a, position information of the controlled focus lens 31b, information on the F-number of the controlled aperture 31c, and the like.

[0015] The lens control unit 32 communicates by sending and receiving information bidirectionally between the camera body 2 and the interchangeable lens 3 via the terminals of the lens side connection unit 302 and the body side connection unit 202 .

[0016] (Camera Body 2) The camera body 2 includes a body-side mount unit 201, an image sensor 22, a body memory 23, a display unit 24, an operation unit 25, and a body control unit 21.

[0017] The body side mount section 201 includes a body side connection section 202. The body side connection section 202 has a plurality of terminals, such as a terminal for a clock signal, a terminal for a data signal, and a terminal for power supply.

[0018] When the interchangeable lens 3 is attached to the camera body 2, a terminal provided on the body-side connector 202 is electrically connected to a terminal provided on the lens-side connector 302. This enables power to be supplied from the camera body 2 to the interchangeable lens 3 and communication between the camera body 2 and the interchangeable lens 3.

[0019] The imaging element 22 is a CMOS image sensor or a CCD image sensor. The imaging element 22 captures an image of a subject formed by the photographing optical system 31. The imaging element 22 includes a pixel unit 221 in which a plurality of pixels each having a photoelectric conversion unit are arranged two-dimensionally (in the row and column directions), and a control unit 222 that controls the pixel unit 221. The photoelectric conversion unit is composed of a photodiode (PD).

[0020] 2 is a diagram showing an example of the arrangement of pixels in the pixel section 221. The pixel section 221 has an imaging pixel 13 that photoelectrically converts received light in a photoelectric conversion section and outputs a signal used to generate an image, and first and second focus detection pixels 11 and 12 that photoelectrically convert received light in a photoelectric conversion section and output a signal used for focus detection of the imaging optical system 31.

[0021] The imaging pixels 13 include pixels (hereinafter referred to as R pixels) having filters with spectral characteristics that separate light in a first wavelength range (red (R) light) from the incident light, pixels (hereinafter referred to as G pixels) having filters with spectral characteristics that separate light in a second wavelength range (green (G) light) from the incident light, and pixels (hereinafter referred to as B pixels) having filters with spectral characteristics that separate light in a third wavelength range (blue (B) light) from the incident light. The R pixels, G pixels, and B pixels are arranged in a Bayer array. The first and second focus detection pixels 11 and 12 are arranged to replace some of the imaging pixels and are distributed across almost the entire imaging surface 22a of the image sensor 22.

[0022] As shown in FIG. 2, in the pixel section 221, a first pixel group 401 in which R pixels 13 and G pixels 13 are alternately arranged in the ±X direction, i.e., in the row direction, and a second pixel group 402 in which G pixels 13 and B pixels 13 are alternately arranged in the row direction, are alternately arranged in the ±Y direction, i.e., in the column direction.

[0023] Some of the second pixel groups 402 among the plurality of second pixel groups 402 are replaced with third pixel groups 403. In the third pixel group 403, the B pixels 13 in the second pixel group 402 are replaced with the first and second focus detection pixels 11, 12. The first and second focus detection pixels 11, 12 each have a light-shielding portion 43. The first and second focus detection pixels 11, 12 are arranged in the same row and alternately arranged every other pixel in the row direction. The third pixel groups 403 are arranged a predetermined number of rows apart from each other.

[0024] The photoelectric conversion unit of the first focus detection pixel 11 receives a light beam that has passed through one of the first and second pupil regions of the exit pupil of the photographing optical system 31. The light-shielding unit 43 of the first focus detection pixel 11 blocks a light beam that has passed through the other of the first and second pupil regions of the exit pupil of the photographing optical system 31. The photoelectric conversion unit of the second focus detection pixel 12 receives a light beam that has passed through a pupil region of the exit pupil different from that of the first focus detection pixel 11. The light-shielding unit 43 of the second focus detection pixel 12 blocks a light beam that has passed through a pupil region of the exit pupil different from that of the first focus detection pixel 11.

[0025] In this embodiment, the first focus detection pixel 11 and the second focus detection pixel 12 each have a filter with spectral characteristics that separates light in the second wavelength range (green (G) light) from the incident light. Note that the filter included in each of the first focus detection pixel 11 and the second focus detection pixel 12 may be a filter with spectral characteristics that separates light in the first wavelength range (red (R) light) or light in the third wavelength range (blue (B) light). Furthermore, the first focus detection pixel 11 and the second focus detection pixel 12 may have a filter with spectral characteristics that separates light in the first, second, and third wavelength ranges from the incident light.

[0026] 1 , the control unit 222 outputs signals from the first and second focus detection pixels 11 and 12 based on an instruction signal from the body control unit 21. The control unit 222 also outputs a signal from the imaging pixel 13 based on an instruction signal from the body control unit 21.

[0027] The body memory 23 is composed of, for example, a non-volatile storage medium, etc. Image data, control programs, etc. are recorded in the body memory 23. Writing data to the body memory 23 and reading data from the body memory 23 are controlled by the body control unit 21. The display unit 24 displays an image based on the image data, an image showing a focus detection area (AF area) such as an AF frame, information related to shooting such as the shutter speed and F-number, a menu screen, etc.

[0028] The operation unit 25 includes various setting switches such as a release button, a power switch, and switches for switching between various modes, and outputs operation signals corresponding to the respective operations to the body control unit 21 .

[0029] The body control unit 21 is configured with a processor such as a CPU, FPGA, or ASIC, and memories such as ROM and RAM, and controls each unit of the camera 1 based on a control program. The body control unit 21 includes a focus detection unit 212, an interpolation unit 211, and a position control unit 213. The interpolation unit 211, the focus detection unit 212, and the position control unit 213 are realized, for example, by the processor executing a program.

[0030] The focus detection unit 212 performs focus detection processing required for autofocus (AF) of the imaging optical system 31. In this embodiment, the focus detection unit 212 calculates the defocus amount by a split-pupil phase difference detection method using an imaging pixel data string based on signal values ​​of signals output from the imaging pixels 13 included in the third pixel group 403 (hereinafter also referred to as actual pixel values ​​of the imaging pixels 13), a first focus detection pixel data string based on signal values ​​of signals output from the first focus detection pixels 11 included in the third pixel group 403 (hereinafter also referred to as actual pixel values ​​of the first focus detection pixels 11), and a second focus detection pixel data string based on signal values ​​of signals output from the second focus detection pixels 12 included in the third pixel group 403 (hereinafter also referred to as actual pixel values ​​of the second focus detection pixels 12).

[0031] Specifically, the focus detection unit 212 calculates a phase difference between the first focus detection pixel data string and the second focus detection pixel data string based on the correlation between the imaging pixel data string and the first focus detection pixel data string and the correlation between the imaging pixel data string and the second focus detection pixel data string, and calculates the amount of deviation between the image plane of the image formed by the photographing optical system 31 and the imaging surface 22a of the image sensor 22. The focus detection unit 212 converts this amount of deviation into a defocus amount based on a predetermined conversion formula. Note that the focus detection unit 212 may also calculate the amount of deviation between the image plane of the image formed by the photographing optical system 31 and the imaging surface 22a of the image sensor 22 by performing a correlation calculation between the first focus detection pixel data string and the second focus detection pixel data string.

[0032] The interpolation unit 211 interpolates pixel values ​​of imaging pixels 13 at positions where no imaging pixels 13 are located in the imaging pixel data string used to calculate the defocus amount. Furthermore, the interpolation unit 211 interpolates pixel values ​​of first focus detection pixels 11 at positions where no first focus detection pixels 11 are located in the first focus detection pixel data string used to calculate the defocus amount. Furthermore, the interpolation unit 211 interpolates pixel values ​​of second focus detection pixels 12 at positions where no second focus detection pixels 12 are located in the second focus detection pixel data string used to calculate the defocus amount. In the following description, pixel values ​​interpolated by the interpolation unit 211 may be referred to as interpolated pixel values.

[0033] The position control unit 213 controls the position of the focus lens 31 b. Specifically, based on the defocus amount calculated by the focus detection unit 212, the position control unit 213 calculates the in-focus position of the focus lens 31 b (the amount of movement of the focus lens 31 b to the in-focus position) for the image formed by the photographing optical system 31 to be focused (imaged) on the imaging surface 22 a of the image sensor 22.

[0034] The position control unit 213 determines whether the defocus amount is within a tolerance, and determines that the lens is in focus if the defocus amount is within the tolerance. On the other hand, the position control unit 213 determines that the lens is not in focus if the defocus amount exceeds the tolerance, and sends a signal to the lens control unit 32 of the interchangeable lens 3 indicating the amount of movement of the focus lens 31b and an instruction to move the lens. The lens control unit 32 moves the focus lens 31b according to the amount of movement, thereby automatically adjusting the focus.

[0035] When the first and second focus detection pixels 11, 12 are arranged in the same row, a larger number of third pixel groups 403 can be arranged compared to when the first and second focus detection pixels 11, 12 are arranged in the same column but in different rows, thereby reducing the area where focus detection is not possible (dead zone).

[0036] However, because the spacing between adjacent first focus detection pixels 11 in the row direction and the spacing between adjacent second focus detection pixels 12 are wider than when the first focus detection pixels 11 and the second focus detection pixels 12 are arranged in the same column but in different rows, there is a risk that the accuracy of calculating the defocus amount may decrease depending on the subject. This point will be explained below.

[0037] FIG. 3 shows an example of signal values ​​(hereinafter referred to as pixel values) output from the imaging pixel 13 and the first and second focus detection pixels 11 and 12 when imaging a pattern PTN1 having portions (edges) where the image brightness changes abruptly, such as from black to white or white to black. Because the first and second focus detection pixels 11 and 12 have light-shielding portions 43, their pixel values ​​are smaller than the pixel values ​​of the neighboring imaging pixels 13. Therefore, the pixel values ​​of the first and second focus detection pixels 11 and 12 are shown as pixel values ​​that have been gain-corrected to align their levels with the pixel value of the imaging pixel 13. This also applies to the subsequent figures. When pixel values ​​vary significantly with image height, it is preferable to change the gain correction coefficient for each image height.

[0038] In Fig. 3, black circles indicate actual pixel values. As shown in Fig. 3, in each data string, actual pixel values ​​are obtained at predetermined intervals, so pixel values ​​must be interpolated for portions where actual pixel values ​​cannot be obtained.

[0039] For example, suppose pixel values ​​at positions where actual pixel values ​​are unavailable are calculated by linear interpolation. Fig. 4 shows data strings for the imaging pixel 13 and the first and second focus detection pixels 11 and 12, where pixel values ​​at positions where actual pixel values ​​are unavailable are calculated by linear interpolation. In Fig. 4, pixel values ​​calculated by linear interpolation are indicated by white circles. Furthermore, dashed lines indicate estimated pixel values ​​based on the actual pixel values ​​of the imaging pixel data string, the first focus detection pixel data string, and the second focus detection pixel data string. The thick dotted lines are straight lines connecting the actual pixel values ​​of adjacent pixels in the row direction.

[0040] 4 shows that the linearly interpolated values ​​may deviate significantly from the estimated values ​​near the edges for the first and second focus detection pixels 11 and 12. Furthermore, the linearly interpolated values ​​deviate from the estimated values ​​near the edges for the imaging pixel 13, which has a closer spacing between adjacent pixels than the first and second focus detection pixels 11 and 12.

[0041] If the defocus amount is calculated using the imaging pixel data string, the first focus detection pixel data string, and the second focus detection pixel data string that include such linearly interpolated values, there is a risk that the defocus amount will not be calculated accurately.

[0042] Therefore, in this embodiment, for the first focus detection pixel 11 and the second focus detection pixel 12, pixel values ​​in columns x where the first focus detection pixel 11 or the second focus detection pixel 12 is not located are interpolated based on the following equation: L'(x) = L(x) + βdG(x) (1) L'(x) represents an interpolated value (interpolated pixel value) of pixel values ​​in columns x where the first focus detection pixel 11 or the second focus detection pixel 12 is not located. L(x) represents a pixel value (linearly interpolated value) obtained by linear interpolation using the actual pixel values ​​of two adjacent first focus detection pixels 11 or the actual pixel values ​​of two adjacent second focus detection pixels 12 in columns x where the first focus detection pixel 11 or the second focus detection pixel 12 is not located.

[0043] dG(x) represents the second derivative of the actual pixel value of a pixel arranged in column x for which an interpolated pixel value is to be calculated, and is expressed as follows: dG(x)=P(x)-(P(x-4)+P(x+4)) / 2 (2) P(x) represents the actual pixel value obtained from a pixel actually arranged in column x, P(x-4) represents the actual pixel value obtained from a pixel actually arranged in column (x-4), and P(x+4) represents the actual pixel value obtained from a pixel actually arranged in column (x+4). β is a parameter that controls the amount of emphasis of the second derivative, and when β is 0, L'(x) becomes L(x).

[0044] That is, in this embodiment, the interpolation unit 211 calculates an interpolated pixel value between two adjacent first focus detection pixels 11 by correcting the pixel value (linearly interpolated value) calculated by linear interpolation using the actual pixel values ​​of two adjacent first focus detection pixels 11 with the second derivative of the actual pixel value of the pixel actually located in column x for which the interpolated pixel value is calculated. The interpolation unit 211 also calculates an interpolated pixel value between two adjacent second focus detection pixels 12 by correcting the pixel value (linearly interpolated value) calculated by linear interpolation using the actual pixel values ​​of two adjacent second focus detection pixels 12 with the second derivative of the actual pixel value of the pixel actually located in column x for which the interpolated pixel value is calculated.

[0045] However, if L'(x) is outside the range of the actual pixel values ​​of the two adjacent first focus detection pixels 11 or outside the range of the actual pixel values ​​of the two adjacent second focus detection pixels 12, the interpolation unit 211 corrects L'(x) to be within the range of the actual pixel values ​​of the two adjacent first focus detection pixels 11 or within the range of the actual pixel values ​​of the two adjacent second focus detection pixels 12.

[0046] Using Figures 5(A) to 8, we will explain how to calculate the interpolated value (interpolated pixel value) of the pixel value of the first focus detection pixel 11 or the second focus detection pixel 12 in a column in which the first focus detection pixel 11 or the second focus detection pixel 12 is not located.

[0047] First, we will explain the first focus detection pixel 11. Figures 5A to 6 are diagrams for explaining the calculation of the interpolated pixel value of the first focus detection pixel 11 in this embodiment. In Figures 5A to 6, black circles indicate the actual pixel values ​​of the imaging pixels 13, black triangles indicate the actual pixel values ​​of the first focus detection pixel 11, and black squares indicate the actual pixel values ​​of the second focus detection pixel 12. Note that the actual pixel values ​​of each pixel indicate the actual pixel values ​​when the pattern PTN1 shown in Figure 3 is imaged.

[0048] The dashed lines indicate estimated pixel values, and the dotted lines are straight lines connecting the actual pixel values ​​of the first focus detection pixel 11 located at position P6 and the actual pixel values ​​of the first focus detection pixel 11 located at position P10. For ease of explanation, the pixel positions (columns) in the X direction are indicated by P1 to P13 in Figures 5A to 6. This is the same for the other figures.

[0049] 5A, we will first explain the calculation of the interpolated pixel value of the first focus detection pixel 11 at position P7, where no first focus detection pixel 11 is located. Position P7 is the position of the line segment closest to the −X side of the three line segments that divide the space between two adjacent first focus detection pixels 11 (the first focus detection pixel 11 located at position P6 and the first focus detection pixel 11 located at position P10) into four equal parts.

[0050] The interpolated pixel value L'(x) of the first focus detection pixel 11 at position P7 where no first focus detection pixel 11 is located can be calculated using the following equations (1) to (3): L'(x) = L(x) + βdG(x) (1) dG(x) = P(x) - (P(x - 4) + P(x + 4)) / 2 (2) L(x) = (3P(x - 1) + P(x + 3)) / 4 (3) where min{P(x - 1), P(x + 3)} ≤ L'(x) ≤ max{P(x - 1), P(x + 3)} (4)

[0051] 5A, P(x-1) and P(x+3) are the actual pixel values ​​of the first focus detection pixel 11. Furthermore, P(x), P(x-4), and P(x+4) are the actual pixel values ​​of the imaging pixel 13.

[0052] In FIG. 5A , the dotted triangle represents the linearly interpolated value L(x) of the pixel value of the first focus detection pixel 11 at position P7. In the example of FIG. 5A , the linearly interpolated value L(x) deviates significantly from the estimated value (dashed line). In this embodiment, the linearly interpolated value is corrected using the second-order derivative dG(x) of the actual pixel value of the imaging pixel 13. This corrects the linearly interpolated value as indicated by the arrows, resulting in the interpolated pixel value L′(x) indicated by the open triangle. It can be seen that correcting the linearly interpolated value with the second-order derivative reduces the difference between the interpolated pixel value L′(x) and the estimated value (dashed line).

[0053] 5B , we will explain how to calculate the interpolated pixel value of the first focus detection pixel 11 at position P8, where no first focus detection pixel 11 is located. Position P8 is the position of the central line segment of three line segments that divide the space between two adjacent first focus detection pixels 11 (the first focus detection pixel 11 located at position P6 and the first focus detection pixel 11 located at position P10) into four equal parts.

[0054] The interpolated pixel value L'(x) of the first focus detection pixel 11 at position P8 where no first focus detection pixel 11 is located can be calculated using the following equations (1), (2), and (5): L'(x) = L(x) + βdG(x) (1) dG(x) = P(x) - (P(x - 4) + P(x + 4)) / 2 (2) L(x) = (2P(x - 2) + P(x + 2)) / 4 (5) where min{P(x - 2), P(x + 2)} ≤ L'(x) ≤ max{P(x - 2), P(x + 2)} (6)

[0055] 5B, P(x-2) and P(x+2) are the actual pixel values ​​of the first focus detection pixel 11. Furthermore, P(x), P(x-4), and P(x+4) are the actual pixel values ​​of the second focus detection pixel 12.

[0056] In FIG. 5B , the dotted triangle represents the linearly interpolated value L(x) of the pixel value of the first focus detection pixel 11. In the example of FIG. 5B , the linearly interpolated value L(x) deviates significantly from the estimated value. In this embodiment, the linearly interpolated value L(x) is corrected using the second derivative dG(x) of the actual pixel value of the second focus detection pixel 12. This corrects the linearly interpolated value, as indicated by the arrows, and obtains the interpolated pixel value L'(x), indicated by the open triangle. It can be seen that correcting the linearly interpolated value with the second derivative reduces the difference between the interpolated pixel value L'(x) and the estimated value (dashed line).

[0057] 6 will now be used to explain the calculation of the interpolated pixel value of the first focus detection pixel 11 at position P9, where no first focus detection pixel 11 is located. Position P9 is the position of the line segment furthest to the +X side of the three line segments that divide the space between two adjacent first focus detection pixels 11 (the first focus detection pixel 11 located at position P6 and the first focus detection pixel 11 located at position P10) into four equal parts.

[0058] The interpolated pixel value L'(x) of the first focus detection pixel 11 at position P9 where no first focus detection pixel 11 is located can be calculated using the following equations (1), (2), and (7): L'(x) = L(x) + βdG(x) (1) dG(x) = P(x) - (P(x - 4) + P(x + 4)) / 2 (2) L(x) = (P(x - 1) + P(x + 3)) / 4 (7) where min{P(x - 1), P(x + 3)} ≤ L'(x) ≤ max{P(x - 1), P(x + 3)} (8)

[0059] 6, P(x-1) and P(x+3) are the actual pixel values ​​of the first focus detection pixel 11. Furthermore, P(x), P(x-4), and P(x+4) are the actual pixel values ​​of the imaging pixel 13.

[0060] In Figure 6, the dotted triangle represents the linearly interpolated value L(x) of the pixel value of the first focus detection pixel 11. In the example of Figure 6, the linearly interpolated value L(x) deviates from the estimated value (dashed line). Here, if the linearly interpolated value L(x) is corrected using the second derivative dG(x) of the actual pixel value of the imaging pixel 13, as indicated by the arrow, the interpolated pixel value will be smaller than the actual pixel value of the first focus detection pixel 11 located at position P10. In this case, equation (8) is applied, and the interpolated pixel value L(x) is corrected to the actual pixel value of the first focus detection pixel 11 located at position P8.

[0061] In this way, the accuracy of the interpolated pixel value can be improved by interpolating the pixel value between two adjacent first focus detection pixels 11 using linear interpolation using the actual pixel values ​​of the two adjacent first focus detection pixels 11 and the second derivative of the pixel value of the second focus detection pixel 12 or the pixel value of the imaging pixel 13. This improves the accuracy of the pixel values ​​included in the first focus detection pixel data string used to calculate the defocus amount, thereby improving the accuracy of calculating the defocus amount. As a result, focusing accuracy can be improved.

[0062] Next, we will explain the second focus detection pixel 12. Figures 7A to 8 are diagrams for explaining the calculation of interpolated pixel values ​​for the second focus detection pixel 12 in this embodiment. In Figures 7A to 8, black circles indicate actual pixel values ​​of the imaging pixels 13, black triangles indicate actual pixel values ​​of the first focus detection pixel 11, and black squares indicate actual pixel values ​​of the second focus detection pixel 12. In addition, dashed lines indicate estimated pixel values, and dotted lines are straight lines connecting the actual pixel values ​​of the second focus detection pixel 12 located at position P4 and the second focus detection pixel 12 located at position P8.

[0063] 7A will be used to explain the calculation of the interpolated pixel value of the second focus detection pixel 12 at position P5, where no second focus detection pixel 12 is located. Position P5 is the position of the line segment furthest to the −X side of the three line segments that divide the space between two adjacent second focus detection pixels 12 (the second focus detection pixel 12 located at position P4 and the second focus detection pixel 12 located at position P8) into four equal parts.

[0064] The interpolated pixel value L'(x) of the second focus detection pixel 12 at position P5 where no second focus detection pixel 12 is located can be calculated using the following equations (1) to (3): L'(x) = L(x) + βdG(x) (1) dG(x) = P(x) - (P(x - 4) + P(x + 4)) / 2 (2) L(x) = (3P(x - 1) + P(x + 3)) / 4 (3) where min{P(x - 1), P(x + 3)} ≤ L'(x) ≤ max{P(x - 1), P(x + 3)} (4)

[0065] 7A, P(x-1) and P(x+3) are the actual pixel values ​​of the second focus detection pixel 12. Furthermore, P(x), P(x-4), and P(x+4) are the actual pixel values ​​of the imaging pixel 13.

[0066] In FIG. 7A , the dotted square represents the linearly interpolated value L(x) of the pixel value of the second focus detection pixel 12 at position P5. In the example of FIG. 7A , the linearly interpolated value L(x) deviates from the estimated value (dashed line). Here, as indicated by the arrow, if the linearly interpolated value L(x) is corrected using the second derivative dG(x) of the actual pixel value of the imaging pixel 13, the interpolated pixel value will be larger than the actual pixel value of the second focus detection pixel 12 located at position P4. In this case, equation (4) is applied to correct the interpolated pixel value L′(x) to the actual pixel value of the second focus detection pixel 12 located at position P4. This results in the interpolated pixel value L′(x) indicated by the white square.

[0067] 7B , we will explain how to calculate the interpolated pixel value of the second focus detection pixel 12 at position P6, where no second focus detection pixel 12 is located. Position P6 is the position of the central line segment of three line segments that divide the space between two adjacent second focus detection pixels 12 (the second focus detection pixel 12 located at position P4 and the second focus detection pixel 12 located at position P8) into four equal parts.

[0068] The interpolated pixel value L'(x) of the second focus detection pixel 12 at position P6 where no second focus detection pixel 12 is located can be calculated using the following equations (1), (2), and (5): L'(x) = L(x) + βdG(x) (1) dG(x) = P(x) - (P(x - 4) + P(x + 4)) / 2 (2) L(x) = (2P(x - 2) + P(x + 2)) / 4 (5) where min{P(x - 2), P(x + 2)} ≤ L'(x) ≤ max{P(x - 2), P(x + 2)} (6)

[0069] 7B, P(x-2) and P(x+2) are the actual pixel values ​​of the second focus detection pixel 12. Furthermore, P(x), P(x-4), and P(x+4) are the actual pixel values ​​of the first focus detection pixel 11.

[0070] In FIG. 7B , the dotted square represents the linearly interpolated value L(x) of the pixel value of the second focus detection pixel 12 at position P6. As can be seen, in the example of FIG. 7B , there is a large discrepancy between the linearly interpolated value L(x) and the estimated value (dashed line). In this embodiment, the linearly interpolated value is corrected using the second derivative dG(x) of the actual pixel value of the first focus detection pixel 11. As a result, the linearly interpolated value is corrected, as indicated by the arrows, to obtain the interpolated pixel value L'(x), indicated by the white square. It can be seen that correcting the linearly interpolated value with the second derivative reduces the difference between the interpolated pixel value L'(x) and the estimated value (dashed line).

[0071] 8 will now be used to explain the calculation of the interpolated pixel value of the second focus detection pixel 12 at position P7, where no second focus detection pixel 12 is located. Position P7 is the position of the line segment furthest to the +X side of the three line segments that divide the space between two adjacent second focus detection pixels 12 (the second focus detection pixel 12 located at position P4 and the second focus detection pixel 12 located at position P8) into four equal parts.

[0072] The interpolated pixel value L'(x) of the second focus detection pixel 12 at position P7 where no second focus detection pixel 12 is located can be calculated using the following equations (1), (2), and (7): L'(x) = L(x) + βdG(x) (1) dG(x) = P(x) - (P(x - 4) + P(x + 4)) / 2 (2) L(x) = (P(x - 3) + P(x + 1)) / 4 (7) where min{P(x - 3), P(x + 1)} ≤ L'(x) ≤ max{P(x - 3), P(x + 1)} (8)

[0073] 8, P(x-3) and P(x+1) are the actual pixel values ​​of the second focus detection pixel 12. Furthermore, P(x), P(x-4), and P(x+4) are the actual pixel values ​​of the imaging pixel 13.

[0074] In Figure 8, the dotted square represents the linearly interpolated value L(x) of the pixel value of the second focus detection pixel 12. In the example of Figure 8, the linearly interpolated value L(x) deviates from the estimated value (dashed line). Here, if the linearly interpolated value L(x) is corrected using the second derivative dG(x) of the actual pixel value of the imaging pixel 13, as indicated by the arrow, the interpolated pixel value will be smaller than the actual pixel value of the second focus detection pixel 12 located at position P8. Therefore, in this case, equation (8) is applied to correct the interpolated pixel value L(x) to the actual pixel value of the second focus detection pixel 12 located at position P8.

[0075] In this way, the accuracy of the interpolated pixel value can be improved by interpolating the pixel value between two adjacent second focus detection pixels 12 using linear interpolation using the actual pixel values ​​of the two adjacent second focus detection pixels 12 and the second derivative of the actual pixel value of the first focus detection pixel 11 or the actual pixel value of the imaging pixel 13. This improves the accuracy of the pixel values ​​included in the second focus detection pixel data string used to calculate the defocus amount, thereby improving the accuracy of calculating the defocus amount.

[0076] Next, we will explain how to calculate the interpolated pixel value of the imaging pixel 13. In this embodiment, the interpolated value (interpolated pixel value) G'(x) of the pixel value between two adjacent imaging pixels 13 is expressed by the following equations (9) to (11). G'(x) = G(x) + αdL(x) (9) G(x) = (P(x-1) + P(x+1)) / 2 (10) dL(x) = P(x) - (P(x-4) + P(x+4)) / 2 (11) where min{P(x-1), P(x+1)} ≦ G'(x) ≦ max{P(x-1), P(x+1)} (12)

[0077] G'(x) represents an interpolated value (interpolated pixel value) of a pixel value in a column x where no imaging pixel 13 is located. G(x) represents a pixel value (linearly interpolated value) in a column x where no imaging pixel 13 is located, calculated by linear interpolation using the actual pixel values ​​of two adjacent imaging pixels 13.

[0078] dL(x) represents the second derivative of the actual pixel value of the pixel actually located in column x for which the interpolated pixel value is to be calculated. α is a parameter that controls the amount of enhancement of the second derivative; if α is 0, G′(x) becomes G(x).

[0079] That is, the interpolated pixel value between two adjacent imaging pixels 13 is obtained by correcting the pixel value (linearly interpolated value) obtained by linear interpolation using the actual pixel values ​​of the two adjacent imaging pixels 13 with the second derivative value of the actual pixel value of the pixel actually located in column x for which the interpolated pixel value is to be obtained.

[0080] 9 is a diagram for explaining the calculation of interpolated pixel values ​​for imaging pixels 13 in this embodiment. In Fig. 9, black circles indicate actual pixel values ​​of imaging pixels 13, black triangles indicate actual pixel values ​​of first focus detection pixels 11, and black squares indicate actual pixel values ​​of second focus detection pixels 12. In addition, dashed lines indicate estimated pixel values, and dotted lines are straight lines connecting the actual pixel values ​​of imaging pixels 13 located at positions P5 and P7.

[0081] As described above, the interpolated pixel value G'(x) between two adjacent imaging pixels 13 can be calculated from the following equations (9) to (11): G'(x) = G(x) + αdL(x) (9) G(x) = (P(x-1) + P(x+1)) / 2 (10) dL(x) = P(x) - (P(x-4) + P(x+4)) / 2 (11) where min{P(x-1), P(x+1)} ≤ G'(x) ≤ max{P(x-1), P(x+1)} (12)

[0082] 9, P(x−1) and P(x+1) are the actual pixel values ​​of the imaging pixel 13. Furthermore, P(x), P(x−4), and P(x+4) are the actual pixel values ​​of the first focus detection pixel 11.

[0083] In Fig. 9 , the dotted circle represents the linearly interpolated value G(x) of the imaging pixel 13. In the example of Fig. 9 , there is a large discrepancy between the linearly interpolated value G(x) and the estimated value (dashed line). In the example of Fig. 9 , the linearly interpolated value G(x) is corrected using the second-order derivative dL(x) of the actual pixel value of the first focus detection pixel 11. As a result, the linearly interpolated value G(x) is corrected, as indicated by the arrows, to obtain the interpolated pixel value G'(x), indicated by the white circle. By correcting the linearly interpolated value using the second-order derivative, the difference between the interpolated pixel value G'(x) and the estimated value (dashed line) is reduced.

[0084] In this way, the accuracy of the interpolated pixel value can also be improved for imaging pixels 13 by interpolating the pixel value between two adjacent imaging pixels 13 using linear interpolation using the actual pixel values ​​of the two adjacent imaging pixels 13 and the second derivative of the actual pixel value of the first focus detection pixel 11 or the second focus detection pixel 12. This improves the accuracy of the pixel values ​​included in the imaging pixel data string used to calculate the defocus amount, thereby improving the accuracy of calculating the defocus amount. As a result, focusing accuracy can be improved.

[0085] 10 is a diagram showing data strings for the imaging pixel 13 and the first and second focus detection pixels 11 and 12, in which pixel values ​​at positions where actual pixel values ​​cannot be obtained are calculated using the method of this embodiment. In Fig. 10, white circles indicate pixel values ​​(linearly interpolated values) calculated using the method of this embodiment. The dashed lines indicate estimated pixel values ​​based on the actual pixel values ​​of the imaging pixel data string, the first focus detection pixel data string, and the second focus detection pixel data string. The thick dotted lines are straight lines connecting the actual pixel values ​​of adjacent pixels in the row direction.

[0086] 10, it can be seen that the linearly interpolated values ​​calculated by the method according to this embodiment are closer to the dashed line indicating the estimated pixel values ​​than in FIG. 4, and the accuracy of the linearly interpolated values ​​is improved.

[0087] As described above in detail, according to this embodiment, the camera 1 includes a plurality of first and second focus detection pixels 11, 12 that are arranged alternately at every other pixel and that photoelectrically convert light that has passed through the photographic optical system 31 and output signals used for focus detection of the photographic optical system 31, and a plurality of imaging pixels 13 that are arranged between the first and second focus detection pixels 11, 12 and that photoelectrically convert light that has passed through the photographic optical system 31 and output signals used for generating an image. The camera 1 also includes a focus detection unit 212 that detects the focus of the photographic optical system 31 using at least two of: a first focus detection pixel data string based on the signal values ​​(pixel values) of the signals output from the first focus detection pixels 11, a second focus detection pixel data string based on the signal values ​​(pixel values) of the signals output from the second focus detection pixels 12, and an imaging pixel data string based on the signal values ​​(pixel values) of the signals output from the imaging pixels 13. Furthermore, the camera 1 is equipped with an interpolation unit 211 that interpolates pixel values ​​(interpolated pixel values) between two adjacent first focus detection pixels 11 in the first focus detection pixel data string using linear interpolation using the pixel values ​​of the two adjacent first focus detection pixels 11 and the pixel values ​​of second focus detection pixels 12 or imaging pixels 13 that are near the two adjacent first focus detection pixels 11.

[0088] This improves the accuracy of the interpolated pixel value compared to when the interpolated pixel value is calculated using only linear interpolation, thereby improving the accuracy of calculating the defocus amount and the focusing accuracy.

[0089] Furthermore, according to this embodiment, the interpolation unit 211 interpolates pixel values ​​between two adjacent first focus detection pixels 11 by correcting the results of linear interpolation using the second derivative of the actual pixel value of the second focus detection pixel 12 or imaging pixel 13 that is near the two adjacent first focus detection pixels 11. The linearly interpolated value is corrected not only using the linearly interpolated value obtained by linear interpolation using the actual pixel values ​​of the two adjacent first focus detection pixels 11, but also using the actual pixel values ​​of other nearby pixels, improving the accuracy of the interpolated pixel value. As a result, the accuracy of calculating the defocus amount improves, enabling improved focusing accuracy.

[0090] Furthermore, in this embodiment, the interpolation unit 211 corrects the interpolated pixel value between two adjacent first focus detection pixels 11 to fall within the range of the pixel values ​​of the two adjacent first focus detection pixels. This prevents the interpolated pixel value from falling outside the range of the pixel values ​​of the two adjacent first focus detection pixels due to correction using the second derivative, thereby improving the accuracy of the interpolated pixel value.

[0091] (Variation 1) If the interpolated pixel values ​​of the first focus detection pixel 11, the second focus detection pixel 12, and the imaging pixel 13 are all calculated using a method in which linearly interpolated values ​​are corrected using second-order derivatives (hereinafter also referred to as the first method), the calculation load is expected to be high. Furthermore, the deviation between the interpolated pixel value and the estimated value is large near edges where the brightness of the subject changes significantly. Therefore, it is possible to calculate the interpolated pixel values ​​near edges using the first method, and to calculate the interpolated pixel values ​​by linear interpolation in other locations. Hereinafter, the method of calculating the interpolated pixel values ​​by linear interpolation may be referred to as the second method.

[0092] 11 is a diagram illustrating a method for calculating interpolated pixel values ​​in Modification 1. In Fig. 11, black circles indicate actual pixel values ​​of imaging pixels 13, black triangles indicate actual pixel values ​​of first focus detection pixels 11, and black squares indicate actual pixel values ​​of second focus detection pixels 12. In addition, dashed lines indicate estimated pixel values, and dotted lines are straight lines connecting the actual pixel values ​​of two adjacent imaging pixels 13.

[0093] The interpolation unit 211 uses the actual pixel values ​​of the imaging pixels 13 where the spacing between adjacent pixels is narrow to determine whether the position where the interpolated pixel value is to be obtained is near an edge, and determines whether to calculate the interpolated pixel value using the first method or the second method.

[0094] 12 is a flowchart showing an example (first modification) of the calculation process of an interpolated pixel value by the interpolation unit 211. Here, a case where an interpolated pixel value between two adjacent imaging pixels 13 is calculated will be described.

[0095] 12 , when calculating an interpolated pixel value between two adjacent imaging pixels 13, the interpolation unit 211 determines whether the absolute value of the amount of change in the actual pixel values ​​of the two adjacent imaging pixels 13 is equal to or greater than a threshold value Th1 (step S11). Near an edge, the actual pixel values ​​change significantly across the edge. Therefore, the threshold value Th1 is set to a value that allows the position where the interpolated pixel value is calculated to be determined to be within an area that includes an edge (for example, section SC4 in FIG. 11 ).

[0096] If the absolute value of the change in the actual pixel values ​​of two adjacent imaging pixels 13 is less than the threshold value Th1 (step S11 / NO), the interpolation unit 211 calculates the interpolated pixel value using the second method (linear interpolation) (step S19) and terminates the processing of Figure 12.

[0097] On the other hand, if the absolute value of the change in the actual pixel values ​​of two adjacent pixels 13 is equal to or greater than the threshold value Th1 (step S11 / YES), the interpolation unit 211 determines whether the absolute value of the change in the actual pixel values ​​on one side of the two adjacent pixels 13 is less than the threshold value Th2 (step S13). The change in actual pixel values ​​is small in the periphery (e.g., sections SC3 and SC5) of an area including an edge (e.g., section SC4 in FIG. 11). Therefore, the threshold value Th2 is set to a value that allows the determination that one side of the two adjacent pixels 13 is a peripheral area of ​​an area including an edge.

[0098] If the absolute value of the change in the actual pixel value on one side of two adjacent imaging pixels 13 is less than the threshold value Th2 (step S13 / YES), the interpolation unit 211 determines whether the absolute value of the change in the actual pixel value on the other side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S15).

[0099] If the absolute value of the change in the actual pixel value on the other side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S15 / YES), the interpolation unit 211 calculates the interpolated pixel value using the first method (step S17) and terminates the processing of Figure 12.

[0100] If the absolute value of the change in the actual pixel values ​​on one side of two adjacent imaging pixels 13 is equal to or greater than the threshold value Th2 (step S13 / NO), or if the absolute value of the change in the actual pixel values ​​on the other side of two adjacent imaging pixels 13 is equal to or greater than the threshold value Th2 (step S15 / NO), the interpolation unit 211 determines that the position at which the interpolated pixel value is to be calculated is not near an edge, calculates the interpolated pixel value using the second method (step S19), and ends the processing of FIG. 12.

[0101] 11 , the absolute value of the change in the actual pixel value between the imaging pixel 13 located at position P8 and the imaging pixel 13 located at position P10 (the change in section SC4) is equal to or greater than the threshold value Th1 (step S11 / YES), and the absolute value of the change in the actual pixel value in section SC3 on the imaging pixel 13 located at position P8 (between the imaging pixel 13 located at position P6 and the imaging pixel 13 located at position P8) and the absolute value of the change in the actual pixel value in section SC5 on the imaging pixel 13 located at position P10 (between the imaging pixel 13 located at position P10 and the imaging pixel 13 located at position P12) are less than the threshold value Th2 (step S13 / YES and step S15 / YES). In this case, when calculating the interpolated pixel value of the imaging pixel 13 located between the imaging pixel 13 located at position P8 and the imaging pixel 13 located at position P10 (position P9), the interpolator 211 calculates the interpolated pixel value using the first method. Furthermore, at other positions (positions included in sections SC1, SC2, SC3, and SC5), when calculating the interpolated pixel value of the imaging pixel 13, the interpolator 211 calculates the interpolated pixel value using the second method.

[0102] When calculating the interpolated pixel values ​​of the first focus detection pixel 11 and the second focus detection pixel 12, the interpolation unit 211 uses the edge determination result using the actual pixel values ​​of the imaging pixel 13. For example, if the position at which the interpolated pixel value of the first focus detection pixel 11 or the second focus detection pixel 12 is calculated is included in any of the sections (sections SC3, SC4, SC5 in FIG. 11 ) in which the position was determined to be near an edge when calculating the interpolated pixel value of the imaging pixel 13, the interpolation unit 211 calculates the interpolated pixel value using the first method, and in other cases the interpolated pixel value is calculated using the second method.

[0103] Note that, for example, if the absolute value of the amount of change in the actual pixel values ​​of two adjacent first focus detection pixels 11 is equal to or greater than threshold value Th1, and the absolute value of the amount of change in the actual pixel values ​​on one side of the two adjacent first focus detection pixels 11 and the absolute value of the amount of change in the actual pixel values ​​on the other side of the two adjacent first focus detection pixels 11 are less than threshold value Th2, the interpolated pixel value of the first focus detection pixel 11 between the two adjacent first focus detection pixels 11 may be calculated using the first method, but in other cases, the second method may be used for calculation. The same applies to the second focus detection pixel 12.

[0104] However, even near an edge, there may be cases where the condition that the absolute value of the change in the actual pixel values ​​of two adjacent imaging pixels 13 is equal to or greater than the threshold value Th1, and the absolute value of the change in the actual pixel values ​​on one side of the two adjacent imaging pixels 13 and the absolute value of the change in the actual pixel values ​​on the other side of the two adjacent imaging pixels 13 are less than the threshold value Th2 is not met.

[0105] Fig. 13 is a diagram illustrating a method for calculating interpolated pixel values ​​in Modification 2. In Fig. 13, black circles indicate actual pixel values ​​of imaging pixels 13, black triangles indicate actual pixel values ​​of first focus detection pixels 11, and black squares indicate actual pixel values ​​of second focus detection pixels 12. In addition, dashed lines indicate estimated pixel values, and dotted lines are straight lines connecting the actual pixel values ​​of two adjacent imaging pixels 13.

[0106] 13, it is considered that an edge exists between positions P9 and P11. The absolute value of the amount of change in the actual pixel value of the imaging pixel 13 in section SC4 is equal to or greater than threshold value Th1, the absolute value of the amount of change in the actual pixel value of the imaging pixel 13 in section SC3 is less than threshold value Th2, but the amount of change in the actual pixel value of the imaging pixel 13 in section SC5 is equal to or greater than threshold value Th2.

[0107] In this case, if the interpolation unit 211 calculates the interpolated pixel value of the captured pixel 13 at position P9 or P11 by linear interpolation, the difference from the estimated value will be large. The method for calculating the interpolated pixel value according to Modification 2 is a method for dealing with such cases.

[0108] 14 is a flowchart showing an example (modification 2) of the calculation process of the interpolation pixel value by the interpolation unit 211. Here, a case where an interpolation pixel value between two adjacent imaging pixels 13 is calculated will be described.

[0109] 14 , the interpolation unit 211 first determines whether or not the amount of change in the actual pixel values ​​of two adjacent imaging pixels 13 is equal to or greater than a threshold value Th1 (step S51). If the amount of change in the actual pixel values ​​of the two adjacent imaging pixels 13 is equal to or greater than the threshold value Th1 (step S51 / YES), the interpolation unit 211 determines whether or not the absolute value of the amount of change in the actual pixel values ​​on one side of the two adjacent imaging pixels 13 is less than a threshold value Th2 (step S53).

[0110] If the absolute value of the change in the actual pixel value on one side of two adjacent imaging pixels 13 is less than the threshold value Th2 (step S53 / YES), the interpolation unit 211 determines whether the absolute value of the change in the actual pixel value on the other side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S55).

[0111] If the absolute value of the change in the actual pixel value on the other side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S55 / YES), the interpolation unit 211 calculates the interpolated pixel value using the first method (step S57) and terminates the processing of Figure 14.

[0112] If the absolute value of the change in actual pixel value on the other side of the two adjacent imaging pixels 13 is greater than or equal to the threshold value Th2 (step S55 / NO), the interpolation unit 211 determines whether the absolute value of the change in actual pixel value of the two adjacent first focus detection pixels 11 or the two adjacent second focus detection pixels 12 on the other side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S63).

[0113] 13 , assume that an interpolated pixel value is being calculated for the imaging pixel 13 at position P9. In this case, the absolute value of the change in the actual pixel value of the imaging pixel 13 in section SC4 that includes position P9 is equal to or greater than threshold value Th1 (YES in step S51), the absolute value of the change in the actual pixel value of the imaging pixel 13 in section SC3 on one side of the two adjacent imaging pixels 13 is less than threshold value Th2 (YES in step S53), but the absolute value of the change in the actual pixel value of the imaging pixel 13 in section SC5 on the other side of the two adjacent imaging pixels 13 is equal to or greater than threshold value Th2 (NO in step S55). In this case, the interpolation unit 211 determines whether the absolute value of the change in the actual pixel value of the two adjacent second focus detection pixels 12 (other pixels) on the other side of the two adjacent imaging pixels 13 (the absolute value of the change in the actual pixel value in section SC6) is less than threshold value Th2 (step S63).

[0114] As shown in Figure 13, if the absolute value of the change in actual pixel value of two adjacent second focus detection pixels 12 (other pixels) on the other side of two adjacent imaging pixels 13 (the absolute value of the change in actual pixel value in section SC6) is less than threshold value Th2, it can be determined that the position where the correction interpolation value is calculated is near the edge.

[0115] Therefore, in FIG. 14, if the absolute value of the change in actual pixel value of two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on the other side of two adjacent imaging pixels 13 is less than threshold value Th2 (step S63 / YES), the interpolation unit 211 calculates the interpolated pixel value using the first method (step S57), and ends the processing of FIG. 14.

[0116] On the other hand, if the absolute value of the change in the actual pixel values ​​of two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on the other side of the two adjacent imaging pixels 13 is greater than or equal to threshold value Th2 (step S63 / NO), the interpolation unit 211 calculates the interpolated pixel value using the second method (step S65) and terminates the processing of Figure 14.

[0117] Meanwhile, if the amount of change in the actual pixel values ​​of two adjacent imaging pixels 13 is greater than or equal to threshold value Th1 (step S51 / YES), but the absolute value of the amount of change in the actual pixel values ​​on one side of the two adjacent imaging pixels 13 is greater than or equal to threshold value Th2 (step S53 / NO), the interpolation unit 211 determines whether the absolute value of the amount of change in the actual pixel values ​​of two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on one side of the two adjacent imaging pixels 13 is less than threshold value Th2 (step S59).

[0118] If the absolute value of the change in actual pixel value between two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on one side of two adjacent imaging pixels 13 is less than the threshold value Th2 (step S59 / YES), the interpolation unit 211 determines whether the absolute value of the change in actual pixel value between the other side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S61).

[0119] If the absolute value of the change in actual pixel value between two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on one side of two adjacent imaging pixels 13 is less than threshold value Th2 (step S59 / YES), and the absolute value of the change in actual pixel value between two adjacent imaging pixels 13 on the other side of two adjacent imaging pixels 13 is less than threshold value Th2 (step S61 / YES), the actual pixel values ​​are considered to be as shown in FIG. 15, so the interpolation unit 211 calculates the interpolated pixel value using the first method (step S57), and the processing of FIG. 14 is terminated.

[0120] On the other hand, if the amount of change in the actual pixel values ​​of two adjacent imaging pixels 13 is less than the threshold value Th1 (step S51 / NO), if the absolute value of the amount of change in the actual pixel values ​​of two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on one side of the two adjacent imaging pixels 13 is equal to or greater than the threshold value Th2 (step S59 / NO), or if the absolute value of the amount of change in the actual pixel values ​​of two adjacent first focus detection pixels 11 or two adjacent second focus detection pixels 12 on one side of the two adjacent imaging pixels 13 is less than the threshold value Th2 (step S59 / YES) but the absolute value of the amount of change in the actual pixel values ​​on the other side of the two adjacent imaging pixels 13 is equal to or greater than the threshold value Th2 (step S61 / NO), the interpolation unit 211 calculates an interpolated pixel value using the second method (step S65), and the processing of FIG. 14 is terminated.

[0121] By doing this, the interpolated pixel values ​​of pixels near edges can be calculated with high accuracy using linear interpolation and second-order differentiation, and the interpolated pixel values ​​of pixels other than those near edges can be calculated using linear interpolation, thereby reducing the time required to calculate the interpolated pixel values ​​in each data string.

[0122] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention.

[0123] For example, in the above-described embodiment, the first and second focus detection pixels 11, 12 are arranged in the same row and alternately arranged every other pixel in the row direction, but this is not limited to this. The first and second focus detection pixels 11, 12 may alternatively be arranged every several pixels in the row direction. Furthermore, the row in which the first focus detection pixels 11 are arranged may be different from the row in which the second focus detection pixels 12 are arranged. Even with this configuration, the accuracy of the interpolated pixel value can be improved compared to simple linear interpolation by interpolating the signal between the discretely arranged first or second focus detection pixels using the signal value of the signal of another pixel located between them.

[0124] REFERENCE SIGNS LIST 1 camera 11 first focus detection pixel 12 second focus detection pixel 13 imaging pixel 21 body control unit 22 imaging element 31 photographing optical system 211 interpolation unit 212 focus detection unit

Claims

1. A plurality of first pixels and a plurality of second pixels arranged alternately at every other pixel, each pixel photoelectrically converting light transmitted through an optical system and outputting a first signal and a second signal used for focus detection of the optical system, a plurality of third pixels respectively arranged between the plurality of first pixels and the plurality of second pixels, each pixel photoelectrically converting light transmitted through the optical system and outputting a third signal used for generating an image, a detection unit detecting the focus of the optical system using at least two of a first data string based on signal values ​​of the first signals output from the plurality of first pixels, a second data string based on signal values ​​of the second signals output from the plurality of second pixels, and a third data string based on signal values ​​of the third signals output from the plurality of third pixels, an interpolation unit that performs linear interpolation using the signal values ​​of the first signal between two adjacent first pixels among the plurality of first pixels in the first data string, and interpolates the signal values ​​of the second signal or the third signal output from a second pixel or a third pixel among the plurality of second pixels or the plurality of third pixels that is located near the two adjacent first pixels.

2. The imaging device according to claim 1, wherein the interpolation unit interpolates the signal value of the first signal between two adjacent first pixels among the plurality of first pixels by correcting the result of the linear interpolation using a second derivative value of the second signal or the third signal output from a second pixel or a third pixel among the plurality of second pixels or the plurality of third pixels that is located near the two adjacent first pixels.

3. The imaging device according to claim 1 or claim 2, wherein the interpolation unit corrects the signal value of the first signal between two adjacent first pixels among the plurality of first pixels to fall within the range of signal values ​​of the first signal output from the two adjacent first pixels.

4. The imaging device according to any one of claims 1 to 3, wherein the interpolation unit performs a first interpolation to interpolate the signal value of the first signal between the two adjacent first pixels using the linear interpolation and the signal value of the second signal or the third signal output from a second pixel or a third pixel output from a second pixel or a third pixel output from the second pixel or the third pixel that is located near the two adjacent first pixels, among the plurality of second pixels or the plurality of third pixels, when an amount of change in the signal value of the third signal between two adjacent third pixels in the vicinity of the two adjacent first pixels is equal to or greater than a first threshold value and the amount of change in the signal value of the third signal on one side of the two adjacent third pixels and the amount of change in the signal value of the third signal on the other side of the two adjacent third pixels are less than a second threshold value; and in other cases, performs a second interpolation to interpolate the signal value of the first signal between two adjacent first pixels among the plurality of first pixels by linear interpolation using the signal values ​​of the first signal output from the two adjacent first pixels.

5. The imaging device of claim 4, wherein the interpolation unit performs the first interpolation when the amount of change in the signal value of the third signal of two adjacent third pixels in the vicinity of the two adjacent first pixels is equal to or greater than the first threshold, the amount of change in the signal value of the third signal on one side of the two adjacent third pixels is less than the second threshold, and the amount of change in the signal value of the third signal on the other side of the two adjacent third pixels is equal to or greater than the second threshold, and when the amount of change in the signal value of the first signal or the signal value of the second signal on the other side of the two adjacent third pixels is less than the second threshold, and performs the second interpolation in other cases.

6. The imaging device described in any one of claims 1 to 5, wherein the interpolation unit interpolates the signal value of the second signal between two adjacent second pixels of the plurality of second pixels in the second data string by linear interpolation using the signal values ​​of the second signal output from the two adjacent second pixels and the signal value of the first signal or the third signal output from a first pixel or a third pixel of the plurality of first pixels or the plurality of third pixels that is located near the two adjacent second pixels.

7. The imaging device described in any one of claims 1 to 6, wherein the interpolation unit interpolates the signal value of the third signal between two adjacent third pixels of the plurality of third pixels in the third data string by linear interpolation using the signal values ​​of the third signal output from the two adjacent third pixels and the signal value of the first signal or the second signal output from a first pixel or a second pixel of the plurality of first pixels or the plurality of second pixels that is located near the two adjacent third pixels.

8. The imaging device according to any one of claims 1 to 7, wherein the detection section performs focus detection of the optical system based on the correlation between the first data string and the second data string.

9. An imaging device according to any one of claims 1 to 7, wherein the detection unit detects the focus of the optical system based on the correlation between the first data string and the third data string and the correlation between the second data string and the third data string.

10. An imaging device comprising: a plurality of first pixels and a plurality of second pixels that photoelectrically convert light that has passed through an optical system and output a first signal and a second signal, respectively, used for focus detection of the optical system; a plurality of third pixels that photoelectrically convert light that has passed through the optical system and output a third signal that is used for generating an image; a detection unit that detects the focus of the optical system using a first data string based on signal values ​​of the first signals output from the plurality of first pixels, a second data string based on signal values ​​of the second signals output from the plurality of second pixels, or a third data string based on signal values ​​of the third signals output from the plurality of third pixels; and an interpolation unit that, in the first data string, interpolates the signal values ​​of the first signals between two adjacent first pixels of the plurality of first pixels using linear interpolation using the signal values ​​of the first signals output from the two adjacent first pixels and the signal value of the third signal output from the third pixel of the plurality of third pixels that is disposed between the two adjacent first pixels.

11. The imaging device according to claim 10, wherein the interpolation unit interpolates the signal value of the first signal between the two adjacent first pixels using the linear interpolation, the signal value of the third signal, and the signal value of the second signal output from the second pixel, among the plurality of second pixels, that is positioned between the two adjacent first pixels.

12. The imaging device described in claim 10 or claim 11, wherein the interpolation unit selectively performs a first interpolation that interpolates using the linear interpolation and a signal value of the third signal output from a third pixel among the plurality of third pixels that is positioned between the two adjacent first pixels, and a second interpolation that interpolates by the linear interpolation without using the signal value of the third signal, and selects and performs either the first interpolation or the second interpolation based on the output of at least one of the first pixel, the second pixel, and the third pixel.

Images