Focus detector and image pick-up device

Abstract

To provide a focusing state detector and a camera that can suitably detect a focus state.SOLUTION: A focus detector comprises: an image pick-up device that has, on an imaging surface picking up an image formed by an optical system, a plurality of areas detecting a focusing state of the image on the imaging surface, and is arranged, on each of the areas, with a plurality of rows of pixels each having a first pixel receiving light passing through a first area of the optical system and outputting a signal and a second pixel receiving light passing through a second area of the optical system and outputting a signal; and a detection unit that detects the focusing state based on the signals output from the rows of pixels, the number of which is smaller than the number of the rows of pixels arranged in one of the plurality of areas.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a focus detection device and an image sensor. [Background technology]

[0002] Conventionally, in a focus detection device having multiple groups of light receiving elements, a technique has been known in which a defocus amount is calculated for each of the multiple groups of light receiving elements, one defocus amount is selected from the multiple calculated defocus amounts, and focus detection is performed based on the selected defocus amount (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-215437 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a focus state detection device and an image sensor that can suitably detect the focus state. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means.

[0006] [1] A focus detection device according to the present invention is characterized in that it comprises an image sensor having a plurality of focus detection areas on an imaging surface that captures an image formed by an optical system and that detects a focus state between the image and the imaging surface, and in each of the plurality of focus detection areas, a plurality of pixel rows are arranged, each including pixels having a first photoelectric conversion unit that receives light that passes through a first region of the optical system and outputs a signal, and a second photoelectric conversion unit that receives light that passes through a second region of the optical system and outputs a signal; and a control unit that detects contrast information of the outputs of the plurality of pixel rows arranged in one of the plurality of focus detection areas, identifies a portion of the pixel rows to be used for focus detection from the plurality of pixel rows arranged in the one focus detection area based on the detection result, and detects the focus state based on the signal output from the identified pixel row. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing a camera according to this embodiment. [Figure 2] FIG. 2 is a front view showing the imaging surface of the imaging element shown in FIG. [Figure 3] FIG. 3 is an enlarged front view of part III in FIG. 2, showing a schematic arrangement of the imaging pixels 221 and the focus detection pixels 222a and 222b. [Figure 4] Figure 4(A) is a front view showing an enlarged view of one of the imaging pixels 221, Figure 4(B) is a front view showing an enlarged view of one of the first focus detection pixels 222a, Figure 4(C) is a front view showing an enlarged view of one of the second focus detection pixels 222b, Figure 4(D) is a cross-sectional view showing an enlarged view of one of the imaging pixels 221, Figure 4(E) is a cross-sectional view showing an enlarged view of one of the first focus detection pixels 222a, and Figure 4(F) is a cross-sectional view showing an enlarged view of one of the second focus detection pixels 222b. [Figure 5] FIG. 5 is a cross-sectional view taken along line VV in FIG. [Figure 6] FIG. 6 is a flowchart showing the operation of the camera according to the first embodiment. [Figure 7] FIG. 7 is a diagram for explaining an example of the operation of the camera 1 according to this embodiment. [Figure 8] FIG. 8 is a diagram for explaining an example of the operation of a conventional camera. [Figure 9] FIG. 9 is a diagram schematically showing the arrangement of the first focus detection pixels 222a and the second focus detection pixels 222b according to this embodiment. [Figure 10] FIG. 10(A) is a graph showing the relationship between the correlation amount and the shift amount in this embodiment, and FIG. 10(B) is a graph showing the relationship between the correlation amount and the shift amount in the prior art. [Figure 11] FIG. 11 is a flowchart showing the operation of the camera according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing an example of frequency components included in the output of a focus detection pixel array. [Figure 13] FIG. 13 is a diagram for explaining a method of calculating contrast information in the third embodiment. [Figure 14] FIG. 14 is a diagram showing the configuration of a camera according to the fourth embodiment. [Figure 15] FIG. 15 is a configuration diagram showing the focus detection module shown in FIG. [Figure 16] FIG. 16 is a front view showing the imaging surface of the imaging element shown in FIG. [Figure 17] FIG. 17 is an enlarged front view of one of the focus detection areas 224 shown in FIG. [Figure 18] FIG. 18(A) is an enlarged front view of one of the imaging pixels 221a, and FIG. 18(B) is a cross-sectional view. [Figure 19] FIG. 19 is a cross-sectional view taken along line XIX-XIX in FIG. [Figure 20] FIG. 20 is an enlarged front view of one of the focus detection areas 224 shown in FIG. [Figure 21] FIG. 21 is a diagram for explaining a method of calculating contrast information in the fourth embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of frequency components in each group. [Figure 23]FIG. 23 is a flowchart showing the operation of the camera according to the fourth embodiment. [Figure 24] FIG. 24 is a diagram for explaining a method of calculating contrast information in the fifth embodiment. [Figure 25] FIG. 25 is a flowchart showing the operation of the camera according to the fifth embodiment. [Figure 26] FIG. 26 is a front view schematically showing an arrangement of focus detection pixels 222a and 222b according to another embodiment. [Figure 27] FIG. 27 is a front view schematically showing an arrangement of focus detection pixels 222a and 222b according to another embodiment. [Figure 28] FIG. 28 is a front view schematically showing an arrangement of focus detection pixels 222a and 222b according to yet another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment will be described with reference to the drawings.

[0009] First Embodiment 1 is a diagram showing the essential components of a digital camera 1 according to an embodiment. Digital camera 1 (hereinafter simply referred to as camera 1) of this embodiment is composed of a camera body 2 and a lens barrel 3, which are detachably coupled to each other via a mount 4.

[0010] Lens barrel 3 is an interchangeable lens that can be attached to and detached from camera body 2. As shown in Figure 1, lens barrel 3 houses a photographic optical system including lenses 31, 32, and 33, and an aperture .

[0011] Lens 32 is a focus lens, and can adjust the focus state of the photographic optical system by moving it in the direction of optical axis L1. Focus lens 32 is provided movably along optical axis L1 of lens barrel 3, and its position is detected by encoder 35, while its position is adjusted by focus lens drive motor 36.

[0012] The diaphragm 34 is configured to have an adjustable aperture diameter centered on the optical axis L1 in order to limit the amount of light beam that passes through the photographing optical system and reaches the image sensor 22, and to adjust the amount of blur. The aperture diameter adjusted by the diaphragm 34 is performed, for example, by sending an appropriate aperture diameter calculated in auto exposure mode from the camera control unit 21 via the lens control unit 37. Alternatively, the set aperture diameter is input from the camera control unit 21 to the lens control unit 37 by manual operation using the operation unit 28 provided on the camera body 2. The aperture diameter of the diaphragm 34 is detected by an aperture aperture sensor (not shown), and the current aperture diameter is recognized by the lens control unit 37.

[0013] Based on commands from the camera control unit 21, the lens control unit 37 controls the entire lens barrel 3, such as driving the focus lens 32 and adjusting the aperture diameter of the diaphragm .

[0014] Meanwhile, in the camera body 2, an image sensor 22 that receives light beams from the photographing optical system is provided at the planned focal plane of the photographing optical system, and a shutter 23 is provided in front of the image sensor 22. The image sensor 22 is composed of a device such as a CCD or CMOS, converts the received optical signal into an electrical signal, and sends it to the camera control unit 21. The photographed image information sent to the camera control unit 21 is sequentially sent to an LCD drive circuit 25 and displayed on an electronic viewfinder (EVF) 26 of the observation optical system. When a release button (not shown) provided on the operation unit 28 is fully pressed, the photographed image information is recorded in a memory 24, which is a recording medium. The memory 24 can be either a removable card-type memory or a built-in memory. The structure of the image sensor 22 will be described in detail later.

[0015] The camera body 2 is provided with an observation optical system for observing the image captured by the image sensor 22. The observation optical system in this embodiment includes an electronic viewfinder (EVF) 26 made of a liquid crystal display element, an LCD drive circuit 25 that drives it, and an eyepiece 27. The LCD drive circuit 25 reads captured image information captured by the image sensor 22 and sent to the camera control unit 21, and drives the electronic viewfinder 26 based on this information. This allows the user to observe the currently captured image through the eyepiece 27. Note that instead of or in addition to the observation optical system with optical axis L2, a liquid crystal display can be provided on the back of the camera body 2, etc., and the captured image can be displayed on this LCD display.

[0016] The camera body 2 is provided with a camera control unit 21. The camera control unit 21 receives various lens information and transmits information such as the defocus amount and aperture diameter to the lens control unit 37. As described above, the camera control unit 21 also reads out pixel output from the image sensor 22, and generates image information by performing predetermined information processing on the read pixel output as necessary, and outputs the generated image information to the LCD drive circuit 25 of the electronic viewfinder 26 and the memory 24. The camera control unit 21 also controls the entire camera 1, such as correcting the image information from the image sensor 22 and detecting the focus adjustment state and aperture adjustment state of the lens barrel 3.

[0017] In addition to the above, the camera control unit 21 also detects the focus state of the photographing optical system by a phase detection method and a contrast detection method based on pixel data read from the image sensor 22. Specific methods for detecting the focus state will be described later.

[0018] The operation unit 28 is an input switch such as a shutter release button that allows the photographer to set various operation modes of the camera 1, and allows switching between autofocus mode and manual focus mode. The various modes set by the operation unit 28 are sent to the camera control unit 21, which controls the overall operation of the camera 1. The shutter release button also includes a first switch SW1 that is turned on when the button is pressed halfway, and a second switch SW2 that is turned on when the button is pressed all the way.

[0019] Next, the imaging element 22 according to this embodiment will be described.

[0020] 2 is a front view showing the imaging surface of the image sensor 22, and FIG. 3 is an enlarged front view of part III in FIG. 2, showing a schematic arrangement of the imaging pixels 221 and the focus detection pixels 222a and 222b.

[0021] As shown in FIG. 3, the image sensor 22 of this embodiment has a plurality of imaging pixels 221 arranged two-dimensionally on the plane of the imaging surface, with green pixels G having color filters that transmit light in the green wavelength range, red pixels R having color filters that transmit light in the red wavelength range, and blue pixels B having color filters that transmit light in the blue wavelength range arranged in a so-called Bayer arrangement. That is, in four adjacent pixel groups 223 (close-packed square lattice arrangement), two green pixels are arranged on one diagonal line, and one red pixel and one blue pixel are arranged on the other diagonal line. The image sensor 22 is configured by repeating this Bayer-arranged pixel group 223 two-dimensionally on the imaging surface of the image sensor 22. In this embodiment, these four pixel groups 223 constitute one pixel.

[0022] The arrangement of the unit pixel groups 223 can be, for example, a close-packed hexagonal lattice arrangement other than the illustrated close-packed square lattice. Furthermore, the configuration and arrangement of the color filters are not limited to this, and an arrangement of complementary color filters (green: G, yellow: Ye, magenta: Mg, cyan: Cy) can also be adopted.

[0023] Fig. 4(A) is an enlarged front view of one of the imaging pixels 221, and Fig. 4(D) is a cross-sectional view. One imaging pixel 221 is composed of a microlens 2211, a photoelectric conversion unit 2212, and a color filter (not shown). As shown in the cross-sectional view of Fig. 4(D), the photoelectric conversion unit 2212 is fabricated on the surface of a semiconductor circuit substrate 2213 of the image sensor 22, and the microlens 2211 is formed on the surface of the photoelectric conversion unit 2212. The photoelectric conversion unit 2212 is shaped to receive an imaging light beam that passes through the exit pupil (e.g., F1.0) of the imaging optical system 31 by the microlens 2211, and receives the imaging light beam.

[0024] 2, focus detection pixel array groups 22a-22i, in which focus detection pixels 222a and 222b are arranged instead of the above-mentioned imaging pixels 221, are provided at nine locations: the center of the imaging surface of the image sensor 22, positions symmetrical to the center, and positions symmetrical above and below the center. As shown in FIG. 3, each focus detection pixel array group is made up of four focus detection pixel arrays L1-L4, and each focus detection pixel array L1-L4 is made up of a plurality of first focus detection pixels 222a and second focus detection pixels 222b arranged alternately and adjacent to one another in a horizontal row. Also, as shown in FIG. 3, in this embodiment, the focus detection pixels 222a and focus detection pixels 222b are arranged in reverse in the X-axis direction in the focus detection pixel arrays L1 and L3 and the focus detection pixel arrays L2 and L4. Furthermore, in this embodiment, as shown in FIG. 3, the first focus detection pixel 222a and the second focus detection pixel 222b are densely arranged without any gaps at the positions of the green pixel G and the blue pixel B of the Bayer-arranged imaging pixel 221.

[0025] 2 are not limited to the positions shown in the figure, and may be arranged in any one location, or in 2 to 8 locations, or may be arranged in 10 or more locations. Furthermore, during actual focus detection, the photographer can manually operate the operation unit 28 to select a desired focus detection pixel group from the multiple arranged focus detection pixel group 22a to 22i as the focus detection area AFP for focus adjustment.

[0026] Fig. 4(B) is an enlarged front view of one of the first focus detection pixels 222a, and Fig. 4(E) is a cross-sectional view of the first focus detection pixel 222a. Fig. 4(C) is an enlarged front view of one of the second focus detection pixels 222b, and Fig. 4(F) is a cross-sectional view of the second focus detection pixel 222b. As shown in Fig. 4(B), the first focus detection pixel 222a is composed of a microlens 2221a and a rectangular photoelectric conversion unit 2222a. As shown in the cross-sectional view of Fig. 4(E), the photoelectric conversion unit 2222a is fabricated on the surface of the semiconductor circuit substrate 2213 of the image sensor 22, and the microlens 2221a is formed on the surface of the photoelectric conversion unit 2222a. As shown in the cross-sectional view of Fig. 4(F), the second focus detection pixel 222b is composed of a microlens 2221b and a photoelectric conversion unit 2222b, and the photoelectric conversion unit 2222b is fabricated on the surface of the semiconductor circuit substrate 2213 of the image sensor 22, with the microlens 2221b formed on that surface.The focus detection pixels 222a and 222b are arranged alternately adjacent to each other in a horizontal row, as shown in Fig. 3, to form each of the focus detection pixel columns L1 to L4.

[0027] The photoelectric conversion units 2222a and 2222b of the first focus detection pixel 222a and the second focus detection pixel 222b are shaped to receive light beams that pass through a predetermined region (e.g., F2.8) of the exit pupil of the imaging optical system via the microlenses 2221a and 2221b. Furthermore, the first focus detection pixel 222a and the second focus detection pixel 222b are not provided with color filters, and their spectral characteristics are a combination of the spectral characteristics of the photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). However, they may also be configured to include one of the same color filters as the imaging pixel 221, such as a green filter.

[0028] Furthermore, although the photoelectric conversion units 2222a and 2222b of the first focus detection pixel 222a and the second focus detection pixel 222b shown in Figures 4(B) and 4(C) are rectangular, the shape of the photoelectric conversion units 2222a and 2222b is not limited to this and can also be other shapes, such as semicircular, elliptical, or polygonal.

[0029] Next, a so-called phase difference detection method will be described, which detects the focus state of the photographing optical system based on the pixel outputs of the focus detection pixels 222a and 222b described above.

[0030] Fig. 5 is a cross-sectional view taken along line VV in Fig. 3, and shows that adjacent focus detection pixels 222a-1, 222b-1, 222a-2, and 222b-2, which are arranged near the imaging optical axis L1, receive light beams AB1-1, AB2-1, AB1-2, and AB2-2 emitted from ranging pupils 351 and 352 of the exit pupil 350. Note that Fig. 5 illustrates only those of the multiple focus detection pixels 222a and 222b that are located near the imaging optical axis L1, but other focus detection pixels besides the focus detection pixels shown in Fig. 5 are similarly configured to receive light beams emitted from the respective pairs of ranging pupils 351 and 352.

[0031] Here, the exit pupil 350 refers to an image set at a position a distance D in front of the microlenses 2221a and 2221b of the focus detection pixels 222a and 222b arranged on the planned focal plane of the imaging optical system. The distance D is a value that is uniquely determined depending on the curvature and refractive index of the microlenses, the distance between the microlenses and the photoelectric conversion units, etc., and this distance D is referred to as the ranging pupil distance. Furthermore, the ranging pupils 351 and 352 refer to the images of the photoelectric conversion units 2222a and 2222b projected by the microlenses 2221a and 2221b of the focus detection pixels 222a and 222b, respectively.

[0032] In FIG. 5, the arrangement direction of the focus detection pixels 222a-1, 222b-1, 222a-2, and 222b-2 coincides with the arrangement direction of the pair of ranging pupils 351 and 352.

[0033] 5, the microlenses 2221a-1, 2221b-1, 2221a-2, and 2221b-2 of the focus detection pixels 222a-1, 222b-1, 222a-2, and 222b-2 are arranged near the planned focal plane of the imaging optical system. The shapes of the photoelectric conversion units 2222a-1, 2222b-1, 2222a-2, and 2222b-2 arranged behind the microlenses 2221a-1, 2221b-1, 2221a-2, and 2221b-2 are projected onto an exit pupil 350 that is distance D away from the microlenses 2221a-1, 2221b-1, 2221a-2, and 2221b-2, respectively, and the projected shapes form distance measurement pupils 351 and 352.

[0034] That is, the relative positional relationship between the microlens and the photoelectric conversion unit in each focus detection pixel is determined so that the projection shape (measuring pupils 351, 352) of the photoelectric conversion unit of each focus detection pixel coincides on the exit pupil 350 at the measuring distance D, thereby determining the projection direction of the photoelectric conversion unit in each focus detection pixel.

[0035] 5, the photoelectric conversion unit 2222a-1 of the first focus detection pixel 222a-1 outputs a signal corresponding to the intensity of an image formed on the microlens 2221a-1 by a light beam AB1-1 that passes through the ranging pupil 351 and heads toward the microlens 2221a-1. Similarly, the photoelectric conversion unit 2222a-2 of the first focus detection pixel 222a-2 outputs a signal corresponding to the intensity of an image formed on the microlens 2221a-2 by a light beam AB1-2 that passes through the ranging pupil 351 and heads toward the microlens 2221a-2.

[0036] The photoelectric conversion unit 2222b-1 of the second focus detection pixel 222b-1 outputs a signal corresponding to the intensity of an image formed on the microlens 2221b-1 by a light beam AB2-1 that passes through the ranging pupil 352 and heads toward the microlens 2221b-1. Similarly, the photoelectric conversion unit 2222b-2 of the second focus detection pixel 222b-2 outputs a signal corresponding to the intensity of an image formed on the microlens 2221b-2 by a light beam AB2-2 that passes through the ranging pupil 352 and heads toward the microlens 2221b-2.

[0037] Then, by grouping the outputs of the photoelectric conversion units 2222a and 2222b of the focus detection pixels 222a and 222b described above into output groups corresponding to the ranging pupil 351 and the ranging pupil 352, respectively, data is obtained regarding the intensity distribution of a pair of images formed on the focus detection pixel row by the focus detection light beams passing through the ranging pupil 351 and the ranging pupil 352, respectively.

[0038] Then, the camera control unit 21 performs a correlation calculation shown in the following equation (1) while relatively shifting the data strings relating to the intensity distribution of the pair of images, i.e., the data string based on the first focus detection pixel 222a and the data string based on the second focus detection pixel 222b in the focus detection pixel string, one-dimensionally. C(k)=Σ|I A(n+k) -I B(n) | ...(1) In the above formula (1), the Σ calculation indicates an accumulation calculation (sum calculation) for n, and I A(n+k) , I B(n)The image shift amount k is an integer and is the shift amount in units of the pixel spacing between the focus detection pixels 222a and 222b. Note that in the calculation results of the above equation (1), the correlation amount C(k) is minimal (the smaller the shift amount, the higher the degree of correlation).

[0039] Then, the correlation amount C(k) is calculated according to the above formula (1), and the defocus amount df is calculated according to the following formula (2) based on the shift amount x at which the correlation amount minimum value C(x) is obtained. Note that in the above formula (2), k is a conversion coefficient (k factor) for converting the shift amount x at which the correlation amount minimum value C(x) is obtained into a defocus amount. df=x·k …(2)

[0040] Furthermore, in this embodiment, the camera control unit 21 detects contrast information from the outputs of multiple focus detection pixel rows L1 to L4, and determines the focus detection pixel row to be used in calculating the defocus amount as the specific focus detection pixel row based on the detected contrast information.

[0041] Specifically, the camera control unit 21 acquires the output of each focus detection pixel array L1-L4 from the image sensor 22 and filters the acquired output of the focus detection pixel arrays L1-L4 using a high-frequency pass filter to extract high-frequency components from the output of the focus detection pixel arrays L1-L4. The camera control unit 21 then compares the high-frequency components of each focus detection pixel array L1-L4 and, based on the comparison results, determines, as the specific focus detection pixel array, the focus detection pixel array of the multiple focus detection pixel arrays L1-L4 that corresponds to the subject with the greatest contrast. Alternatively, the camera control unit 21 may be configured to determine, based on the comparison results, the focus detection pixel array of the multiple focus detection pixel arrays L1-L4 whose output includes the most high-frequency components as the specific focus detection pixel array.

[0042] In this embodiment, an example has been described in which the camera control unit 21 determines a specific focus detection pixel row from among the multiple focus detection pixel rows L1 to L4 by acquiring the outputs of the multiple focus detection pixel rows L1 to L4 from the image sensor 22. However, this is not limited to this configuration. For example, a configuration may be adopted in which a calculation unit provided in the image sensor 22 detects contrast information for each point detection pixel row L1 to L4 based on the output of each focus detection pixel row L1 to L4, determines a specific focus detection pixel row from among the multiple focus detection pixel rows L1 to L4 based on the detected contrast information, and transmits information about the determined specific focus detection pixel row to the camera control unit 21.

[0043] The camera control unit 21 then calculates the defocus amount based on the output of the determined specific focus detection pixel row. In this way, in this embodiment, the defocus amount is calculated based only on the output of the focus detection pixel row determined as the specific focus detection pixel row out of the multiple focus detection pixel rows L1 to L4, so the time required to calculate the defocus amount can be reduced.

[0044] That is, in the past, when there were multiple focus detection pixel rows, the defocus amount was calculated for all of the focus detection pixel rows, and the defocus amount used to drive the focus lens 32 was selected from the multiple calculated defocus amounts. As a result, in the past, not only was the calculation of defocus amounts other than the selected defocus amount was unnecessary, but there were also problems with the time required to calculate the defocus amount because defocus amounts were calculated for all of the multiple focus detection pixel rows L1 to L4. In contrast, in this embodiment, the defocus amount is calculated based only on the output of the focus detection pixel row determined as the specific focusing pixel row, so the time required to calculate the defocus amount can be shortened, and as a result, the time required for focus detection can be shortened.

[0045] In this embodiment, multiple focus detection areas AFP are set within the shooting screen of the shooting optical system corresponding to the focus detection pixel array groups 22a-22i of the image sensor 22, and the photographer can set the focus detection area AFP to be used for focus adjustment via the operation unit 28. For example, if the photographer selects the focus detection area AFP corresponding to the focus detection pixel array group 22a as the focus detection area AFP to be used for focus adjustment, the camera control unit 21 calculates the defocus amount based on the outputs of the focus detection pixel arrays L1-L4 included in the focus detection pixel array group 22a. The method for setting the focus detection area AFP is not limited to the method selected by the photographer. For example, the camera control unit 21 may perform face recognition processing based on image data output from the image sensor 22 to set the focus detection area AFP corresponding to the subject's face as the focus detection area AFP to be used for focus adjustment. Alternatively, the outputs of the focus detection pixel rows L1 to L4 in all focus detection areas AFP set within the shooting screen may be acquired, and the focus detection area AFP to be used for focus adjustment may be set based on the acquired outputs of the focus detection pixel rows L1 to L4.

[0046] Furthermore, in this embodiment, the camera control unit 21 performs focus detection using a contrast detection method in addition to the focus detection using the phase difference detection method described above. Specifically, the camera control unit 21 reads out the output of the imaging pixels 221 of the imaging element 22 and calculates a focus evaluation value based on the read pixel output. This focus evaluation value can be obtained, for example, by extracting high-frequency components of the image output from the imaging pixels 221 of the imaging element 22 using a high-frequency pass filter and integrating these. Alternatively, the focus evaluation value can be obtained by extracting high-frequency components using two high-frequency pass filters with different cutoff frequencies and integrating them.

[0047] Then, camera control unit 21 sends a control signal to lens control unit 37 to drive focus lens 32 at a predetermined sampling interval (distance), calculates a focus evaluation value at each position, and determines the position of focus lens 32 where the focus evaluation value is maximum as the in-focus position. Note that, for example, when the focus evaluation value is calculated while driving focus lens 32, if the focus evaluation value rises twice and then falls twice, this in-focus position can be determined by performing a calculation such as interpolation using these focus evaluation values.

[0048] Next, an example of the operation of the camera 1 in this embodiment will be described with reference to the flowchart shown in FIG.

[0049] First, in step S101, the image sensor 22 acquires output data from the imaging pixel 221 and each of the first focus detection pixels 222a and second focus detection pixels 222b that make up the focus detection pixel arrays L1 to L4.

[0050] In step S102, the camera control unit 21 selects a focus detection area AFP to be used for focus adjustment. For example, if the photographer sets a focus detection area AFP to be used for focus adjustment via the operation unit 28, the camera control unit 21 can select the focus detection area AFP set by the photographer as the focus detection area AFP to be used for focus adjustment. The camera control unit 21 may also be configured to perform face recognition processing on image data output from the image sensor 22, thereby selecting a focus detection area AFP corresponding to the face of the subject as the focus detection area AFP to be used for focus adjustment. Alternatively, the camera control unit 21 may be configured to select the focus detection area AFP to be used for focus adjustment by analyzing the outputs of focus detection pixel rows L1 to L4 of all focus detection areas AFP set within the shooting screen.

[0051] In step S103, the camera control unit 21 detects contrast information for each of the focus detection pixel arrays L1-L4 based on the output of each of the focus detection pixel arrays L1-L4. Specifically, the camera control unit 21 extracts high-frequency components from the pixel output of the focus detection pixel arrays L1-L4 by filtering the outputs of the multiple focus detection pixel arrays L1-L4 corresponding to the focus detection area AFP selected in step S102 using a high-frequency pass filter. The camera control unit 21 then detects information including the amount and intensity of the extracted high-frequency components as contrast information for each of the focus detection pixel arrays L1-L4.

[0052] In step S104, a specific focus detection pixel row is determined by the camera control unit 21. Specifically, based on the contrast information for each of the focus detection pixel rows L1 to L4 detected in step S103, the camera control unit 21 determines, as the specific focus detection pixel row, the focus detection pixel row among the multiple focus detection pixel rows L1 to L4 that corresponds to the subject with the greatest contrast, or the focus detection pixel row whose pixel output contains the most high-frequency components.

[0053] Then, in step S105, the camera control unit 21 calculates the defocus amount based on the output of the specific focus detection pixel row determined in step S104. Then, in step S106, the drive amount of the focus lens 32 is calculated and the focus lens 32 is driven based on the defocus amount calculated in step S105.

[0054] In this manner, focus detection is performed in the optical system according to the first embodiment.

[0055] Next, an example of the operation of the camera 1 according to this embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram for explaining an example of the operation of the camera 1 according to this embodiment. In FIG. 7, the horizontal axis represents time, and the shutter release button is half-pressed at time t5. For example, in the example shown in FIG. 7, at time t1, the focus detection pixels 222a and 222b of the image sensor 22 begin accumulating charge in response to incident light. In this embodiment, the focus detection pixels 222a and 222b are, for example, CMOS image sensors, and in parallel with the charge accumulation, they begin transferring pixel signals in response to the amount of charge accumulated after time t1. Then, at time t3, the transfer of pixel signals that began at time t2 ends, and contrast information is detected and a specific focus detection pixel row is determined (steps S103 and S104). Then, at time t4, calculation of the defocus amount begins based on the output of the determined specific focus detection pixel row (step S105). This causes the lens drive amount to be calculated, and after calculation of the lens drive amount, at time t6, a lens drive instruction is sent to the lens barrel 3 to drive the focus lens 32. Here, in the example shown in Figure 7, the shutter release button is half-pressed at time t5, before the lens drive instruction is issued, and therefore drive of the focus lens 32 begins based on the lens drive instruction at time t6.

[0056] Furthermore, in this embodiment, even after the shutter release button is half-pressed, the defocus amount is repeatedly calculated in accordance with the frame rate of the image sensor 22, and an instruction to drive the focus lens 32 is repeatedly issued based on the calculated defocus amount. For example, in the example shown in Fig. 7, charge accumulation for the second frame begins at time t2, and as a result, the defocus amount for the second frame is calculated at time t7, and an instruction to drive the focus lens 32 is issued at time t8 based on the output result of the focus detection pixel arrays L1 to L4 for the second frame. Similarly, for the third frame and thereafter, the calculation of the defocus amount and the driving of the focus lens 32 based on the defocus amount are repeated for each frame based on the output of the focus detection pixel arrays L1 to L4.

[0057] In this manner, in this embodiment, calculation of the defocus amount and driving of the focus lens 32 based on the defocus amount are repeated for each frame according to the frame rate of the image sensor 22. In this manner, in order to calculate the defocus amount for each frame and instruct the focus lens 32 to be driven, it is necessary to perform the processes from calculating the defocus amount to instructing the focus lens 32 to be driven within the time of one frame. In this regard, in this embodiment, one specific focus detection pixel row is determined based on contrast information, and the defocus amount is calculated only for the output of this specific focus detection pixel row. As shown in FIG. 7, this shortens the time required to calculate the defocus amount, and therefore it is possible to calculate the defocus amount within the time of one frame and drive the focus lens 32 based on the calculated defocus amount.

[0058] On the other hand, FIG. 8 is a diagram illustrating an example of the operation of a conventional camera, and similarly to FIG. 7, the horizontal axis represents time. In the example shown in FIG. 8, the defocus amount calculation, which takes a relatively long time, is performed for all focus detection pixel rows L1 to L4, so the defocus amount calculation time is longer than in the present embodiment shown in FIG. 7. As a result, in the example shown in FIG. 8, the time required from calculating the defocus amount to issuing a lens drive command is longer than the time for one frame at the frame rate, compared to the present embodiment shown in FIG. 7, and there are cases where the lens drive command cannot be issued for each frame. Specifically, in the example shown in FIG. 8, because the process from calculating the defocus amount to issuing a lens drive command cannot be completed within the time for one frame, the time lag T2 from the accumulation of charge to issuing a drive command for the focus lens 32 is twice as long as the time lag T1 in the present embodiment shown in FIG. 7.

[0059] 8, at time t12, an instruction to drive focus lens 32 is issued based on the focus state of the optical system at time t11, which may result in focus lens 32 being driven to a position far beyond the in-focus position or in reduced ability to track the subject. In contrast, in this embodiment, as shown in FIG. 7, an instruction to drive focus lens 32 can be issued at time t10, which has a short time lag, based on the focus state of the optical system at time t9, which makes it possible to appropriately drive focus lens 32 to the in-focus position compared to the conventional example shown in FIG.

[0060] As described above, in the first embodiment, contrast information is detected for each focus detection pixel row L1-L4 based on the output of each focus detection pixel row L1-L4 in the focus detection area AFP. Then, the focus detection pixel row to be used for focus detection is determined based on the detected contrast information. That is, of the multiple focus detection pixel rows L1-L4, the focus detection pixel row corresponding to the subject with the greatest contrast or the focus detection pixel row with the most high-frequency components in its pixel output is determined as the specific focus detection pixel row, and the defocus amount is determined for only this specific focus detection pixel row. This makes it possible to detect the focus state of the optical system for the subject with the greatest contrast or the most high-frequency components, and also reduces the time required to calculate the defocus amount compared to conventional methods of calculating the defocus amount for all of the multiple focus detection pixel rows L1-L4, allowing focus state detection to be repeated at appropriate timing.

[0061] Furthermore, in the first embodiment, the defocus amount is calculated based on the output of one of the multiple focus detection pixel arrays L1-L4, which provides the following advantages over adding or averaging the outputs of multiple focus detection pixel arrays L1-L4. That is, if the subject has contrast in a diagonal direction of the imaging pixels 221, adding or averaging the outputs of the multiple focus detection pixel arrays L1-L4 may actually reduce the contrast, making it impossible to properly detect the subject. In contrast, in this embodiment, the defocus amount is calculated based on the output of one specific focus detection pixel array, which provides the advantage of preventing a reduction in contrast even in such cases and enabling proper detection of the subject.

[0062] Second Embodiment Next, a second embodiment will be described. In the second embodiment, the camera 1 shown in Fig. 1 has the same configuration as the first embodiment described above, except that it operates as described below.

[0063] In the second embodiment, the camera control unit 21 is equipped with multiple filters that transmit frequency components in different frequency bands, extracts frequency components in different frequency bands from the output of each focus detection pixel array L1 to L4, and detects multiple pieces of contrast information for each focus detection pixel array L1 to L4 based on the extracted frequency components. For example, if the camera control unit 21 is equipped with three filters that extract frequency components in three different frequency bands, the camera control unit 21 can detect three pieces of contrast information from the output of one focus detection pixel array. Note that the frequency band of the frequency components extracted in the second embodiment is not particularly limited and can be any frequency band from low frequency band to high frequency band.

[0064] Then, based on the plurality of contrast information, the camera control unit 21 detects one or more focus detection pixel rows L1-L4 to be used for focus detection as specific focus detection pixel rows from among the plurality of focus detection pixel rows L1-L4. For example, the camera control unit 21 can detect, as specific focus detection pixel rows, one or more focus detection pixel rows from among the plurality of focus detection pixel rows L1-L4, where the magnitude of contrast of the corresponding subject is equal to or greater than a predetermined value. Alternatively, the camera control unit 21 can detect, as specific focus detection pixel rows, one or more focus detection pixel rows where the amount of high-frequency components included in the output is equal to or greater than a predetermined value.

[0065] When determining specific focus detection pixel rows, the camera control unit 21 determines the specific focus detection pixel rows so that the number of specific focus detection pixel rows is less than the number of focus detection pixel rows L1 to L4 corresponding to the focus detection area AFP. For example, in this embodiment, four focus detection pixel rows L1 to L4 are arranged in the focus detection area AFP, so the camera control unit 21 determines the specific focus detection pixel rows so that the number of specific focus detection pixel rows is at least three or less.

[0066] Then, when the camera control unit 21 determines that multiple focus detection pixel rows are specific focus detection pixel rows, it adds or averages the outputs of the multiple specific focus detection pixel rows and calculates the defocus amount based on the added or averaged outputs of the multiple specific focus detection pixel rows. Here, Fig. 9 is a diagram that schematically shows only the first focus detection pixel 222a and the second focus detection pixel 222b that make up the four focus detection pixel rows L1 to L4, excluding the imaging pixel 221 from the imaging surface of the image sensor 22 shown in Fig. 3. For example, when adding the outputs of multiple specific focus detection pixel rows, the camera control unit 21 adds or averages the outputs of pixel A1 of the first focus detection pixel row L1 shown in Fig. 9. L1 and the pixel A1 of the second focus detection pixel row L2, which receives the focus detection light beam passing through the same focus detection pupil. L2 and the output of pixel A1 of the third focus detection pixel row L3 L3 and the output of pixel A1 of the fourth focus detection pixel row L4 L4 The output of is added to the pixel sum output I A1Similarly, pixel B1 in the first focus detection pixel row L1 is obtained. L1 and pixel B1 of the second focus detection pixel row L2, which receives the focus detection light beam passing through the same focus detection pupil. L2 and the output of pixel B1 of the third focus detection pixel row L3 L3 and the output of pixel B1 of the fourth focus detection pixel row L4 L4 The output of is added to the pixel sum output I B1 Similarly, A2 L1 and A2 L2 and A2 L3 and A2 L4 and from the output of I A2 , B2 L1 and B2 L2 and B2 L3 and B2 L4 and from the output of I B2 A3 L1 and A3 L2 and A3 L3 and A3 L4 and from the output of I A3 , B3 L1 and B3 L2 and B3 L3 and B3 L4 and from the output of I B3 get.

[0067] Then, the camera control unit 21 uses the obtained pixel addition output to generate a data string based on the first focus detection pixel 222a, i.e., a first image data string I A1 ,I A2 ,I A3 ,...,I An and a data string based on the second focus detection pixel 222b, i.e., a second image data string I B1 ,I B2 ,I B3 ,...,I Bn While relatively shifting the two signals one-dimensionally, the correlation calculation shown in the above formula (1) is performed.

[0068] 3, in the focus detection pixel rows L1 to L4, the first focus detection pixel 222a and the second focus detection pixel 222b are arranged at positions offset by 0.5 pixels from each other. Conventionally, only one of the first focus detection pixel row L1 and the second focus detection pixel row L2 of this embodiment is used as the focus detection pixel row, so the first focus detection pixel 222a and the second focus detection pixel 222b are arranged at positions offset by 0.5 pixels from each other, and the first image data row I obtained using these focus detection pixels is A1 ,I A2 ,I A3 ,...,I An and the second image data sequence I B1 ,I B2 ,I B3 ,...,I Bn The data obtained by the above calculations are shifted by 0.5 pixels from each other. Therefore, when a correlation calculation is performed, the minimum value of the correlation amount C(k) is also shifted by 0.5 pixels, as shown in FIG. 10(B). In such cases, interpolation or other calculations are required to calculate the shift amount and defocus amount at which the correlation amount C(k) reaches its minimum value, which can result in a decrease in focus detection accuracy. This problem tends to be particularly pronounced when the contrast level of the output detected by the focus detection pixel is low.

[0069] In contrast to this, in this embodiment, in the focus detection pixel rows L1, L3 and the focus detection pixel rows L2, L4, the first focus detection pixel 222a and the second focus detection pixel 222b are positioned at positions shifted by 0.5 pixels in the X-axis direction. Therefore, when the pixel addition output obtained by adding the pixel outputs is used, as shown in FIG. 10(A), it is possible to accurately determine the shift amount at which the correlation amount C(k) takes on a minimum value in the in-focus state (a state in which the defocus amount is zero), thereby making it possible to appropriately improve the focus detection accuracy.

[0070] Next, the operation of the camera 1 according to the second embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the operation of the camera 1 according to the second embodiment.

[0071] In step S201, similar to step S101 in the first embodiment, output data from the imaging pixel 221 and each of the first focus detection pixels 222a and second focus detection pixels 222b that make up the multiple focus detection pixel arrays L1 to L4 is acquired by the image sensor 22. Also, in step S202, similar to step S102 in the first embodiment, the camera control unit 21 selects a focus detection area AFP to be used for focus adjustment.

[0072] Then, in step S203, contrast information is detected by the camera control unit 21. In the second embodiment, the camera control unit 21 extracts multiple frequency components from the output of each of the focus detection pixel arrays L1 to L4 by filtering the output of each of the focus detection pixel arrays L1 to L4 using multiple filters that extract frequency components in different frequency bands. The camera control unit 21 then detects information including the amount and intensity of the extracted frequency components as contrast information.

[0073] In step S204, the camera control unit 21 determines a specific focus detection pixel row based on the plurality of contrast information detected in step S203. Specifically, the camera control unit 21 determines, based on the contrast information detected in step S203, one or more focus detection pixel rows from among the plurality of focus detection pixel rows L1 to L4 for which the contrast of the corresponding subject is equal to or greater than a predetermined value, as the specific focus detection pixel row. Alternatively, based on the plurality of contrast information detected in step S203, the camera control unit 21 determines, based on the contrast information detected in step S203, one or more focus detection pixel rows from among the plurality of focus detection pixel rows L1 to L4 for which the amount of high-frequency components included in the output is equal to or greater than a predetermined value, as the specific focus detection pixel row.

[0074] Then, in step S205, the camera control unit 21 calculates the defocus amount to be used in driving the focus lens 32, based on the one or more specific focus detection pixel rows determined in step S204. For example, if the camera control unit 21 has determined multiple focus detection pixel rows as specific focus detection pixel rows, it can calculate a single output by adding or averaging the outputs of the multiple specific focus detection pixel rows, and then calculate the defocus amount to be used in driving the focus lens 32, based on this output.

[0075] In step S206, similarly to step S106 in the first embodiment, the drive amount of the focus lens 32 is calculated and the focus lens 32 is driven based on the defocus amount calculated in step S205.

[0076] In this manner, focus detection is performed in the optical system according to the second embodiment.

[0077] As described above, in the second embodiment, for each of the focus detection pixel rows L1 to L4, frequency components in different frequency bands are extracted based on the output of the focus detection pixel rows L1 to L4, and multiple pieces of contrast information are detected based on the extracted multiple frequency components. One or more specific focus detection pixel rows are then selected from the focus detection pixel rows L1 to L4 based on the multiple pieces of contrast information, and the defocus amount is determined based on the pixel output of the selected specific focus detection pixel row. As described above, in the second embodiment, instead of calculating the defocus amount for all of the focus detection pixel rows L1 to L4, the defocus amount is calculated for a smaller number of specific focus detection pixel rows than the number of focus detection pixel rows L1 to L4, which shortens the time required to calculate the defocus amount and, as a result, enables focus state detection to be repeatedly performed at an appropriate timing.

[0078] Third Embodiment Next, a third embodiment will be described. In the third embodiment, the camera 1 shown in Fig. 1 has the same configuration as the first embodiment described above, except that it operates as described below.

[0079] In the third embodiment, when detecting the focus state of the optical system, the camera control unit 21 extracts frequency components corresponding to the subject from the outputs of the focus detection pixel arrays L1 to L4 by filtering the outputs of the focus detection pixel arrays L1 to L4 obtained from the image sensor 22 using a predetermined band-pass filter. For example, the camera control unit 21 performs a Fourier transform on the outputs of the focus detection pixel arrays L1 to L4 to remove low-frequency components due to the background and high-frequency components corresponding to noise from the frequency components included in the outputs of the focus detection pixel arrays L1 to L4, thereby extracting the frequency components corresponding to the subject. Then, as shown in FIG. 12, the camera control unit 21 detects the data obtained by filtering the outputs of the focus detection pixel arrays L1 to L4 as contrast information representing the frequency components included in the outputs of the focus detection pixel arrays L1 to L4. Note that FIG. 12 is a graph showing an example of frequency components extracted from the outputs of the focus detection pixel arrays L1 to L4.

[0080] Furthermore, based on the contrast information of the detected frequency components, the camera control unit 21 calculates the sum of differences between successive output values ​​of the frequency components to calculate information indicating the amount and intensity of the frequency components contained in the outputs of the focus detection pixel arrays L1 to L4 as the contrast amount of the frequency components. Specifically, the camera control unit 21 calculates the contrast amount of the frequency components based on the following equation (3): Contrast amount = Σ|a m -a m-1 | ...(3) In the above formula (3), a m is the output value of the frequency component corresponding to the output of the focus detection pixel 222a, 222b located at the mth position (mth in the direction of alignment of the focus detection pixel row), and a m-1 is the output value of the frequency component corresponding to the output of the (m-1)th focus detection pixel 222a, 222b.

[0081] That is, the outputs of focus detection pixel columns L1 to L4 are column data made up of the outputs of multiple focus detection pixels 222a, 222b arranged in the horizontal direction (X direction), and the frequency components included in the outputs of focus detection pixel columns L1 to L4 also have output values ​​corresponding to the outputs of the focus detection pixels 222a, 222b that make up focus detection pixel columns L1 to L4, as shown in Fig. 12. Therefore, as shown in the above equation (3), the camera control unit 21 can calculate the difference between the output value of the frequency component corresponding to the output of the second focus detection pixel 222a, 222b in the focus detection pixel columns L1 to L4 and the output value of the frequency component corresponding to the output of the first focus detection pixel 222a, 222b. Similarly, the output value a of the frequency component corresponding to the output of consecutive focus detection pixels 222a, 222b m ,a m-1 (hereinafter, also referred to as successive output values ​​in the frequency component) can be calculated. m ,a m-1 By finding the sum of the differences between these, the contrast amount of the frequency components included in the outputs of the focus detection pixel arrays L1 to L4 can be calculated.

[0082] The contrast amount of a frequency component increases as the intensity (amplitude) of the frequency component increases, and also as the amount (frequency) of the frequency component increases. For example, if the contrast of the subject corresponding to the focus detection pixel row is high and the intensity of the output values ​​near the center of the frequency component increases as shown in FIG. 13A, the difference between consecutive output values ​​near the center of the frequency component also increases as shown in FIG. 13B. As a result, the contrast amount of the frequency component also increases. Furthermore, the greater the amount of frequency component (for example, in the example shown in FIG. 12, the amount of frequency component in focus detection pixel row L3 is greater than that in focus detection pixel row L1), the more frequently the output values ​​of the frequency component fluctuate, and therefore the contrast amount of the frequency component, which is the sum of the differences between consecutive output values ​​of the frequency component, also increases. In this way, by calculating the sum of the differences between consecutive output values ​​of the frequency component as the contrast amount of the frequency component, the contrast amount of the frequency component can be calculated as a value representing the amount and intensity of the frequency component. Note that Figure 13(A) is a diagram showing an example of the output values ​​of frequency components included in the output of a focus detection pixel row, and Figure 13(B) is a graph showing the difference between successive output values ​​in the frequency components shown in Figure 13(A).

[0083] In this way, the camera control unit 21 extracts predetermined frequency components corresponding to the subject from the output of each of the focus detection pixel arrays L1 to L4 and calculates the contrast amount of the extracted frequency components for each of the focus detection pixel arrays L1 to L4. The camera control unit 21 then determines the focus detection pixel array with the highest contrast amount among the focus detection pixel arrays L1 to L4 as the specific focus detection pixel array. For example, in the example shown in FIG. 12, the amount and intensity of frequency components contained in the output of the focus detection pixel array L3 among the focus detection pixel arrays L1 to L4 are the largest, and as a result, the contrast amount of the frequency components contained in the output of the focus detection pixel array L3 is the highest, so the focus detection pixel array L3 is determined to be the specific focus detection pixel array. The camera control unit 21 then calculates the defocus amount based on the output of the determined specific focus detection pixel array.

[0084] As described above, in the third embodiment, as shown in the above formula (3), the sum of the differences in the output values ​​of frequency components corresponding to the outputs of focus detection pixels 222a, 222b that are consecutive in the alignment direction in the focus detection pixel arrays L1 to L4 is calculated as the contrast amount of the frequency components included in the outputs of the focus detection pixel arrays L1 to L4. This makes it possible to appropriately calculate the contrast amount of the frequency components according to the amount and intensity of the frequency components caused by the subject. Then, by calculating the defocus amount based on the output of the focus detection pixel array with the largest contrast amount, focus detection can be performed based on the output of the focus detection pixel array that is most likely to correspond to the subject, and this makes it possible to detect the focusing state of the optical system with respect to the subject with higher accuracy.

[0085] Fourth Embodiment Next, a fourth embodiment will be described. Fig. 14 is a diagram showing the configuration of a camera 1a according to the fourth embodiment. In the fourth embodiment, the camera 1a shown in Fig. 14 has the same configuration as the first embodiment described above, except that it operates as described below.

[0086] First, a camera 1a according to the fourth embodiment will be described. The camera 1a has a camera body 2a and a lens barrel 3. The camera body 2a is a single-lens reflex digital camera and is equipped with a mirror system 250 for guiding a light beam from a subject to an image sensor 220, an optical viewfinder 235, a photometry sensor 237, and a focus detection module 261. The mirror system 250 is equipped with a quick-return mirror 251 that rotates a predetermined angle around a rotation axis 253 between a subject observation position and an image capture position, and a sub-mirror 252 that is journaled by the quick-return mirror 251 and rotates in accordance with the rotation of the quick-return mirror 251. In FIG. 14, the state in which the mirror system 250 is at the subject observation position is indicated by a solid line, and the state in which the mirror system 250 is at the subject image capture position is indicated by a two-dot chain line.

[0087] The mirror system 250 is inserted into the optical path of the optical axis L1 when in the observation position of the subject, and rotates to retract from the optical path of the optical axis L1 when in the imaging position of the subject.

[0088] The quick-return mirror 251 is a half mirror, and when the quick-return mirror 251 is located at the subject observation position, a portion of the light beam (optical axis L1) from the subject is reflected by the quick-return mirror 251 (optical axes L2 and L3) and directed to the optical viewfinder 235 and photometry sensor 237, and a portion of the light beam (optical axis L4) is transmitted and directed to the sub-mirror 252. In contrast, the sub-mirror 252 is a total reflection mirror, and directs the light beam (optical axis L4) that has transmitted through the quick-return mirror 251 to the focus detection module 261.

[0089] Therefore, when mirror system 250 is in the observation position, the light beam (optical axis L1) from the subject is guided to optical viewfinder 235, photometry sensor 237, and focus detection module 261, allowing the photographer to observe the subject, and exposure calculation and detection of the focus adjustment state of focus lens 32 by focus detection module 261 are performed. Then, when the photographer fully presses the release button, mirror system 250 rotates to the shooting position, and the light beam (optical axis L1) from the subject is all guided to image sensor 220, and captured image data is stored in camera memory 24.

[0090] The light beam (optical axis L2) from the subject reflected by the quick-return mirror 251 forms an image on the focusing screen 231, which is arranged on a plane optically equivalent to the image sensor 220, and can be observed through the pentaprism 233 and the eyepiece 27. At this time, the transmissive LCD display 232 displays a focus detection area mark and the like superimposed on the subject image on the focusing screen 231, and also displays information related to shooting, such as the shutter speed, aperture value, and number of shots, in an area outside the subject image. This allows the photographer to observe the subject, its background, and shooting-related information through the optical viewfinder 235 while preparing to shoot.

[0091] The photometry sensor 237 is composed of a two-dimensional color CCD image sensor or the like, and divides the shooting screen into multiple areas and outputs a photometry signal according to the brightness of each area in order to calculate the exposure value when taking a picture. The signal detected by the photometry sensor 237 is output to the camera control unit 21 and used for automatic exposure control, recognition of specific subjects (for example, face recognition), etc.

[0092] The focus detection module 261 is a dedicated focus detection element that performs automatic focusing control using a phase difference detection method, and is fixed at a position optically equivalent to the imaging surface of the image sensor 220 on the light beam (optical axis L4) reflected by the sub-mirror 252.

[0093] FIG. 15 is a diagram showing an example of the configuration of the focus detection module 261 shown in FIG. 14. The focus detection module 261 of this embodiment includes a condenser lens 261a, an aperture mask 261b having a pair of apertures formed therein, a pair of re-imaging lenses 261c, and a pair of line sensors 261d. Although not shown, the line sensor 261d of this embodiment includes a pixel array in which a plurality of pixels are arranged, each having a microlens disposed near the intended focal plane of the imaging optical system and a photoelectric conversion unit disposed relative to the microlens. A pair of light beams passing through a pair of different regions of the exit pupil of the focus lens 32 are received by the pixels arranged in the pair of line sensors 261d, thereby acquiring a pair of image signals. The focus adjustment state can then be detected by calculating the phase shift of the pair of image signals acquired by the pair of line sensors 261d using a correlation calculation, which will be described later.

[0094] For example, as shown in Figure 15, when the subject P is imaged on the equivalent plane (intended imaging plane) 261e of the image sensor 220, the subject is in focus, but if the focus lens 32 moves in the direction of the optical axis L1, the imaging point shifts from the equivalent plane 261e toward the subject (called front focus) or toward the camera body (called back focus), the subject will be out of focus.

[0095] Next, an image sensor 220 according to a fourth embodiment will be described. Fig. 16 is a front view showing the image sensor 220 according to the fourth embodiment, and Fig. 17 is a front view showing an enlarged view of one of the focus detection regions 224 shown in Fig. 16. In the image sensor 220 according to the fourth embodiment, focus detection regions 224a to 224i for focus detection are set at positions corresponding to the focus detection area AFP set on the shooting screen, as shown in Fig. 16, i.e., at a total of nine positions: the center of the imaging surface of the image sensor 220, positions symmetrical to the left and right of the center, and positions symmetrical to those positions above and below.

[0096] In the focus detection areas 224a to 224i, a plurality of imaging pixels 221a are arranged two-dimensionally on the plane of the imaging surface, as shown in Fig. 17. The imaging pixels 221a include green pixels G having color filters that transmit light in the green wavelength range, red pixels R having color filters that transmit light in the red wavelength range, and blue pixels B having color filters that transmit light in the blue wavelength range, and these pixels are arranged in a Bayer array.

[0097] Next, the configuration of the imaging pixel 221a will be described. Fig. 18(A) is an enlarged front view of one of the imaging pixels 221a, and Fig. 18(B) is a cross-sectional view. As shown in Fig. 18(A), the imaging pixel 221a is composed of a microlens 2211, a pair of photoelectric conversion units 2214 and 2215, and a color filter (not shown). As shown in the cross-sectional view of Fig. 18(B), the photoelectric conversion units 2214 and 2215 are fabricated on the surface of a semiconductor circuit substrate 2213 of the imaging element 220, and the microlens 2211 is formed on the surface of the photoelectric conversion units. The pair of photoelectric conversion units 2214 and 2215 are the same size and are arranged symmetrically with respect to the optical axis of the microlens 2211. Furthermore, each of the pair of photoelectric conversion units 2214 and 2215 is shaped to receive an imaging light beam passing through the exit pupil (e.g., F1.0) of the imaging optical system 31 via the microlens 2211. As shown in Figure 18(B), one photoelectric conversion unit 2215 of the imaging pixel 221a receives one light beam AB1, while the other photoelectric conversion unit 2214 of the imaging pixel 221a receives a light beam AB2 that is symmetrical to the light beam AB1 with respect to the optical axis of the microlens 2211.

[0098] The photoelectric conversion units 2214, 2215 of each imaging pixel 221a receive the pair of light beams AB1, AB2 and output a pair of image signals corresponding to the intensities of the received light beams AB1, AB2. In other words, the photoelectric conversion units 2214, 2215 output a pair of image signals corresponding to the intensities of the images formed on the microlenses 2211 by the light beams AB1, AB2. In this embodiment, the pair of image signals output from the photoelectric conversion units 2214, 2215 of the imaging pixels 221a are added together to generate a pixel signal for the imaging pixel 221a, and image data is generated from the pixel signals of the multiple imaging pixels 221a.

[0099] Although the photoelectric conversion units 2214 and 2215 shown in FIG. 18(A) are rectangular, the shape of the photoelectric conversion units 2214 and 2215 is not limited to this and may be other shapes, such as an ellipse, a semicircle, or a polygon.

[0100] Next, a phase difference focus detection method (hereinafter also referred to as image plane phase difference detection) using a pair of image signals output from the photoelectric conversion units 2214, 2215 of the imaging pixels 221a will be described. Note that, hereinafter, a plurality of imaging pixels 221a arranged in the horizontal direction (X direction) as shown in Fig. 17 is referred to as an imaging pixel row 225, and as will be described later, a pair of image data is generated from one imaging pixel row 225 by combining the outputs of the photoelectric conversion units 2214, 2215 of the imaging pixels 221a that make up the imaging pixel row 225, and a defocus amount is calculated based on the generated pair of image data.

[0101] Fig. 19 is a cross-sectional view taken along line XIX-XIX in Fig. 17 and shows that the imaging pixel 221a-1 arranged on the imaging optical axis L and the adjacent imaging pixel 221a-2 receive light beams AB1-1, AB1-2, AB2-1, and AB2-2 emitted from the ranging pupils 341 and 342 of the exit pupil 340. Although not shown, pairs of photoelectric conversion units in the other imaging pixels also receive pairs of light beams emitted from the pairs of ranging pupils 341 and 342. In Fig. 19, the arrangement direction of the photoelectric conversion units 2214-1, 2225-1, 2214-2, and 2215-2 of the imaging pixel 221a coincides with the arrangement direction of the pairs of ranging pupils 351 and 352.

[0102] The photoelectric conversion unit 2214-1 of the imaging pixel 221a-1 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-1 by one light beam AB1-1 that passes through one ranging pupil 341 and heads toward the microlens 2211-1. In response to this, the photoelectric conversion unit 2215-1 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-1 by the other light beam AB2-1 that passes through the other ranging pupil 342 and heads toward the microlens 2211-1.

[0103] Similarly, the photoelectric conversion unit 2214-2 of the imaging pixel 221a-2 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-2 by one light beam AB1-2 that passes through one ranging pupil 341 and heads toward the microlens 2211-2. In response to this, the photoelectric conversion unit 2215-2 outputs a signal corresponding to the intensity of the image formed on the microlens 2211-2 by the other light beam AB2-2 that passes through the other ranging pupil 342 and heads toward the microlens 2211-2.

[0104] Then, by grouping the outputs of the photoelectric conversion units 2214, 2215 of each of the imaging pixels 221a constituting the imaging pixel array 225 into output groups corresponding to the ranging pupil 341 and the ranging pupil 342, it is possible to obtain data relating to the intensity distribution of a pair of images formed on the imaging pixel array 225 by the light beams AB1, AB2 passing through the ranging pupil 341 and the ranging pupil 342, respectively, i.e., a pair of image data. For example, in the example shown in FIG. 17 , eight or more imaging pixel arrays 225 are set, and the camera control unit 21 can obtain a pair of image data from each of the imaging pixel arrays 225.

[0105] The camera control unit 21 can detect the amount of image shift caused by the split-pupil phase difference detection method by performing image shift detection calculation processing such as correlation calculation processing or phase difference detection processing on a pair of image data obtained from each imaging pixel array 225. The camera control unit 21 can then obtain the amount of defocus by performing a conversion calculation on the obtained amount of image shift according to the distance between the centers of gravity of a pair of ranging pupils.

[0106] In this way, the camera control unit 21 can calculate the defocus amount for each imaging pixel row 225 based on a pair of image data obtained from each imaging pixel row 225. However, in the fourth embodiment, in order to reduce the time required for focus detection and perform focus detection at an appropriate timing, the camera control unit 21 specifies the imaging pixel row 225 to be used for focus detection, and calculates the defocus amount based only on the output of the specified imaging pixel row 225.

[0107] Specifically, the camera control unit 21 first extracts frequency components corresponding to the subject from the output of each imaging pixel array 225, and detects information related to the extracted frequency components as contrast information. For example, as in the third embodiment, the camera control unit 21 performs filtering on the output of each imaging pixel array 225 using a predetermined band-pass filter to remove low-frequency components due to the background, etc., and high-frequency components corresponding to noise from the frequency components included in the output of the imaging pixel array 225, and extracts the frequency components corresponding to the subject. Then, the camera control unit 21 can detect, for example, data obtained by filtering the output of the imaging pixel array 225 as contrast information of the frequency components included in the output of the imaging pixel array 225.

[0108] The camera control unit 21 also calculates a contrast amount, which indicates the amount and intensity of the frequency components, based on the contrast information of the extracted frequency components. Similar to FIG. 17, FIG. 20 is a front view showing an enlarged view of one of the focus detection areas 224a-224i shown in FIG. 16, and FIG. 21 is a diagram schematically showing the imaging pixels 221a of the first group LA shown in FIG. 20. In FIGS. 20 and 21, the imaging pixel arrays 225 are represented as imaging pixel arrays A1-A5, B1-B5, C1-C5, D1-D5, and E1-E5, respectively. In the examples shown in FIGS. 21(A) and 21(B), each imaging pixel array will be described as having 12 imaging pixels 221a.

[0109] First, the camera control unit 21 groups the multiple imaging pixel rows 225 set within the focus detection area 224 into groups of a predetermined number of imaging pixel rows 225 (for example, five imaging pixel rows 225). For example, in the example shown in Fig. 20, the camera control unit 21 groups each group of five imaging pixel rows 225 consecutive in the vertical direction (Y direction), such as a first group LA consisting of imaging pixel rows A1 to A5, a second group LB consisting of imaging pixel rows B1 to B5, a third group LC consisting of imaging pixel rows C1 to C5, a fourth group LD consisting of imaging pixel rows D1 to D5, and a fifth group LE consisting of imaging pixel rows E1 to E5. Note that the number of imaging pixel rows 225 to be grouped is not limited to five and can be set as appropriate.

[0110] The camera control unit 21 then calculates the output value of each frequency component included in the output of the imaging pixel arrays 225 belonging to the same group by adding, for each group, the output values ​​of the frequency components included in the output of the imaging pixel arrays 225, and detects this as contrast information of the frequency components for each group. For example, in the example shown in FIG. 21A, the camera control unit 21 calculates an output value a1 by adding the output values ​​of the frequency components corresponding to the output of the first imaging pixel 221a in each of the imaging pixel arrays A1 to A5. Similarly, the camera control unit 21 calculates an output value a2 by adding the output values ​​of the frequency components corresponding to the output of the second imaging pixel 221a in each of the imaging pixel arrays A1 to A5. Similarly, the camera control unit 21 calculates output values ​​a3, a4, ..., a12 for the third and subsequent imaging pixels 221a in each of the imaging pixel arrays A1 to A5, by adding the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction). As a result, the camera control unit 21 can detect the output values ​​a1, a2, a3, . . . , a12 as contrast information of the frequency components in the first group LA.

[0111] Similarly to the first group LA, the camera control unit 21 also detects contrast information of the frequency components in the second group LB, the third group LC, the fourth group LD, and so on. Then, the camera control unit 21 calculates the amount of contrast of the frequency components in each group based on the contrast information of the frequency components of each group. For example, in the example shown in FIG. 21(A), the camera control unit 21 can calculate the amount of contrast of the frequency components in the first group LA by using the output values ​​a1, a2, a3, and so on of the frequency components in the first group LA to calculate the sum of the difference between a1 and a2, the difference between a2 and a3, and the difference between a11 and a12, as shown in the above formula (3).

[0112] In the above example, the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction) are added to calculate the output value of the frequency components in each group, but the present invention is not limited to this configuration, and for example, the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction) may be averaged to calculate the output value of the frequency components in each group. For example, when the brightness or contrast of the subject is relatively low, the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a may be added to calculate the output value of the frequency components in each group, and when the brightness or contrast of the subject is relatively high, the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a may be averaged to calculate the output value of the frequency components in each group.

[0113] 21(B), the output value of the frequency component in each group may be calculated by adding or averaging the output values ​​of the frequency components corresponding to the outputs of the photoelectric conversion units 2214 and 2215 that are arranged at the same position in the horizontal direction (X direction). That is, as shown in FIG. 21(B), the camera control unit 21 calculates the output value b1 by adding or averaging the output values ​​of the frequency components corresponding to the outputs of the photoelectric conversion unit 2214 of the first imaging pixel 221a in each of the imaging pixel arrays A1 to A5. Similarly, the camera control unit 21 calculates the output value c1 by adding or averaging the output values ​​of the frequency components corresponding to the outputs of the photoelectric conversion unit 2215 of the first imaging pixel 221a in each of the pixel arrays A1 to A5. Similarly, for the photoelectric conversion units 2214 and 2215 of the second and subsequent imaging pixels 221a, the camera control unit 21 calculates output values ​​b2, c2, b3, c3, . . . , b12, and c12 by adding or averaging output values ​​of frequency components corresponding to the outputs of the photoelectric conversion units 2214 and 2215 arranged at the same position in the horizontal direction (X direction) in each of pixel columns A1 to A5. This allows the camera control unit 21 to acquire output values ​​consisting of b1, c1, b2, c2, b3, c3, . . . , b12, and c12 as contrast information of the frequency components in the first group LA. Similarly, the camera control unit 21 acquires contrast information of the frequency components in groups LB, LC, LD, LE, . . . as shown in FIG. 22. Note that FIG. 22 illustrates an example of frequency components in each of groups LA, LB, . . . LE shown in FIG. 20.

[0114] Then, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group. For example, in the example shown in FIG. 21(B), the camera control unit 21 can calculate the sum of differences between consecutive output values ​​among the output values ​​b1, c1, b2, c2, b3, c3, . . . , b12, and c12 of the frequency components in the first group LA as the contrast amount of the frequency components, as shown in the above formula (3). Alternatively, the camera control unit 21 may calculate column data by adding the output values ​​b1 and c1, b2 and c2, b3 and c3, . . . , b12 and c12, respectively, and calculate the sum of differences between consecutive values ​​in the calculated column data as the contrast amount of the frequency components. Alternatively, the camera control unit 21 may be configured to separate the output values ​​b1, c1, b2, c2, b3, c3, . . . , b12, c12 into column data consisting of output values ​​b1, b2, b3, . . . , b12 and column data consisting of output values ​​c1, c2, c3, . . . , c12, calculate the contrast amount of the frequency components in each column data, and calculate the larger or smaller contrast amount as the contrast amount of the frequency components in each group.

[0115] Furthermore, in the above example, a configuration has been described in which contrast information of frequency components in each group is detected by adding or averaging all frequency components included in the outputs of all imaging pixel arrays 225 belonging to the same group, but the present invention is not limited to this configuration, and a configuration may be adopted in which the number of frequency components to be added or averaged is changed based on, for example, the brightness of the subject scene, the luminance of the image, the contrast of the image, etc. For example, a configuration may be adopted in which the higher the brightness of the subject scene, the luminance of the image, or the contrast of the image, the fewer the number of frequency components to be added, and the lower the brightness of the subject scene, the luminance of the image, or the contrast of the image, the more the number of frequency components to be added.

[0116] The camera control unit 21 then determines the group with the largest contrast amount among the contrast amounts of the frequency components calculated for each group as a specific group, and calculates the defocus amount based on the output of the imaging pixels 221a included in the specific group. For example, in the example shown in Fig. 22, the third group LC has a large amount and intensity of frequency components, so the contrast amount of the frequency components in the third group LC is the largest among groups LA to LE. In this case, the camera control unit 21 identifies the third group LC as a specific group, and calculates the defocus amount based on the output of the imaging pixels 221a included in the third group.

[0117] For example, as shown in FIG. 21B, the camera control unit 21 calculates a pair of image data (image data consisting of output values ​​b1, b2, b3, and image data consisting of output values ​​c1, c2, c3) by adding or averaging the outputs of the photoelectric conversion units 2214 and 2215 included in a specific group that are arranged at the same position in the horizontal direction (X direction). Then, as shown in the above formula (1), the camera control unit 21 performs a correlation calculation while relatively shifting the calculated pair of image data to calculate a correlation amount C(k) of the pair of image data, and can calculate a defocus amount based on the calculated correlation amount C(k), as shown in the above formula (2). Then, the camera control unit 21 can adjust the focus of the optical system by driving the focus lens 32 based on the calculated defocus amount.

[0118] In the fourth embodiment, the camera control unit 21 can control the driving of the focus lens 32 based on the defocus amount calculated by the focus detection module 261, and can also control the driving of the focus lens 32 based on the defocus amount calculated based on the output of the image sensor 220. For example, when the brightness of light received by the image sensor 220 is low, the camera control unit 21 can control the driving of the focus lens 32 based on the defocus amount calculated by the focus detection module 261. This is because the size of each pixel constituting the line sensor 261d of the focus detection module 261 is designed to be larger than that of the imaging pixels 221a, so that focus detection can be performed with relatively high accuracy even when the light intensity of the light beam is relatively weak. On the other hand, for example, when shooting video, the driving of the focus lens 32 can be controlled based on the defocus amount calculated based on the output of the image sensor 220. This is because image plane phase difference detection based on the output of the image sensor 220 allows focus detection of the optical system to be performed while capturing an image with the image sensor 220, even during video shooting.

[0119] Next, the operation of the camera 1a according to the fourth embodiment will be described. Fig. 23 is a flowchart showing an example of the operation of the camera 1a according to the fourth embodiment. In the example of the operation of the camera 1a shown in Fig. 23, a scene will be described in which the camera control unit 21 performs focus detection based on the output of the image sensor 220.

[0120] First, in step S401, the image sensor 220 acquires output data from the imaging pixels 221a (the photoelectric conversion units 2214 and 2215 of the imaging pixels 221a). Then, in step S402, the camera control unit 21 selects a focus detection area AFP to be used for focus adjustment. For example, the photographer can manually operate the operation unit 28 to select the focus detection area AFP for focus adjustment. As a result, the camera control unit 21 determines the focus detection area 224 corresponding to the selected focus detection area AFP as the focus detection area 224 for focus detection.

[0121] Then, in step S403, the camera control unit 21 sets a plurality of imaging pixels 221a arranged one-dimensionally in the horizontal direction (X direction) as imaging pixel rows 225 in the focus detection area 224 determined in step S402, and detects contrast information of the frequency components included in the output of each imaging pixel row 225.

[0122] Then, in step S404, the camera control unit 21 groups the multiple imaging pixel rows 225 in the focus detection area 224 into groups of a predetermined number of imaging pixel rows 225. For example, as shown in Fig. 20, the camera control unit 21 can group the multiple imaging pixel rows 225 into groups LA, LB, LC, LD, LE, etc., with each group consisting of five imaging pixel rows.

[0123] In step S405, the camera control unit 21 adds up the output values ​​of the frequency components included in the output of the imaging pixel arrays 225 belonging to the same group, thereby detecting contrast information of the frequency components in each group. Specifically, as shown in Fig. 21(A), the camera control unit 21 detects contrast information of the frequency components in each group by adding up the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the horizontal direction (X direction) in multiple imaging pixel arrays 225 belonging to the same group.

[0124] Then, in step S406, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group detected in step S405. For example, the camera control unit 21 can calculate the contrast amount of the frequency components in each group from the output values ​​of the frequency components in each group based on the above formula (3).

[0125] In step S407, the camera control unit 21 compares the contrast amounts of the frequency components calculated for each group in step S406, and determines the group with the highest contrast amount as the specific group for which focus detection will be performed. Then, in step S408, the camera control unit 21 calculates a defocus amount based on the output of the imaging pixels 221a included in the specific group determined in step S407. For example, as shown in FIG. 21B, the camera control unit 21 calculates a pair of image data by adding the outputs of the photoelectric conversion units 2214 and 2215 arranged at the same position in the horizontal direction (X direction) in each imaging pixel array 225 in the specific group. Then, the camera control unit 21 performs a correlation calculation on the calculated pair of image data, and calculates a defocus amount based on the calculation result. Then, in step S409, the camera control unit 21 drives the focus lens 32 based on the defocus amount calculated in step S408.

[0126] In this manner, focus detection is performed in the optical system according to the fourth embodiment.

[0127] 17, in the fourth embodiment, a plurality of imaging pixels 221a, each having a pair of photoelectric conversion units 2214, 2215, are two-dimensionally arranged on the image sensor 220. Of the plurality of imaging pixels 221a, a plurality of imaging pixels 221a arranged in the horizontal direction (X direction) are set as an imaging pixel array 225, and the outputs of the photoelectric conversion units 2214, 2215 of the imaging pixels 221a constituting each imaging pixel array 225 are grouped into output groups corresponding to the ranging pupil 351 and the ranging pupil 352 to obtain a pair of image data, and image plane phase difference detection is performed to calculate a defocus amount based on the obtained pair of image data. As a result, in the fourth embodiment, the focus state of the optical system can be appropriately detected even during video capture, for example.

[0128] In the fourth embodiment, the imaging pixel arrays 225 are grouped into groups of a predetermined number of imaging pixel arrays 225, and the output values ​​of each frequency component included in the output of the imaging pixel arrays 225 belonging to the same group are added together for each group to detect contrast information for the frequency components in each group. The contrast amount for each frequency component in each group is then calculated based on the contrast information for the frequency components in each group. The group with the highest contrast amount is determined as a specific group, and focus detection is performed based on the output of the imaging pixels 221a in the specific group. In this way, in the fourth embodiment, focus detection can be performed on a group that contains many high-frequency components and is likely to contain a subject, thereby more appropriately detecting the focus state of the optical system relative to the subject. In the fourth embodiment, the group with the highest contrast amount is determined as a specific group, and image plane phase difference detection is performed based on the output of the imaging pixels 221a in the specific group. This reduces the time required to calculate the defocus amount compared to calculating the defocus amount based on the output of all imaging pixels 221a, allowing focus detection to be performed at an appropriate timing.

[0129] Fifth Embodiment Next, a fifth embodiment will be described. In the fifth embodiment, a camera 1a shown in Fig. 14 has the same configuration as the fourth embodiment described above, except that it operates as described below.

[0130] As in the fourth embodiment, the image sensor 220 according to the fifth embodiment has imaging pixels 221a, each having a pair of photoelectric conversion units 2214, 2215, arranged two-dimensionally on the image sensor 220. In the fifth embodiment, the camera control unit 21 detects a specific subject such as a person's face, and extracts the output of the imaging pixels 221a corresponding to the specific subject at a ratio according to the color indicating the specific subject, thereby extracting a frequency component according to the specific subject.

[0131] Specifically, the camera control unit 21 first detects a specific subject such as a human face by performing template matching using a template image such as a human face. Then, the camera control unit 21 specifies an area of ​​the image sensor 220 that corresponds to the specific subject as a target area. The camera control unit 21 may be configured to specify the target area within the focus detection areas 224a to 224i, or may be configured to specify the target area in the entire image sensor 220.

[0132] The camera control unit 21 then detects whether or not a target area exists in the focus detection area 224 selected by the user or the camera control unit 21. That is, when a focus detection area AFP for focus adjustment is selected by the user or the camera control unit 21, the camera control unit 21 determines the focus detection area 224 corresponding to the selected focus detection area and determines whether or not a target area exists in the focus detection area 224. If a target area exists in the focus detection area 224, the camera control unit 21 identifies a pixel group 223 in the focus detection area 224 that corresponds to the target area. Furthermore, the camera control unit 21 sets a pixel group array 226 that includes the pixel group 223 that corresponds to the target area. For example, as shown in FIG. 24, the camera control unit 21 can set a column in which a plurality of pixel groups 223, including the pixel group 223 that corresponds to the target area, are arranged one-dimensionally in the horizontal direction (X direction) as the pixel group array 226. Note that FIG. 24 is a diagram schematically illustrating the imaging pixels 221a of the first group LA shown in FIG. 20. In the example shown in FIG. 24, the pixel column 226 will be described as having six pixel groups 223.

[0133] The camera control unit 21 then extracts frequency components corresponding to the pixel group array 226 from the output of the pixel group array 226, and detects contrast information related to the extracted frequency components. Specifically, the camera control unit 21 first extracts output values ​​of the four imaging pixels 221a (including R pixels, G pixels, and B pixels) that make up the pixel group 223 at a ratio that corresponds to a color that indicates a specific subject, such as a person's face (for example, the skin color of the specific subject). For example, the camera control unit 21 can extract output values ​​of the R pixels and G pixels that are higher than the output of the B pixel, so that the ratio corresponds to the skin color of the specific subject. The camera control unit 21 then detects the output value of the pixel group 223 by adding or averaging the output values ​​extracted from the four imaging pixels 221a (R pixels, G pixels, and B pixels) that make up the pixel group 223. Furthermore, the camera control unit 21 detects the output of the pixel group array 226, which is made up of a plurality of pixel groups 223, by detecting the output values ​​of the pixel groups 223 that make up each pixel group array 226 along the arrangement direction of the pixel groups 223. Then, the camera control unit 21 performs filtering processing on the output of the pixel group array 226 using a predetermined band-pass filter, thereby extracting frequency components corresponding to the subject from the output of the pixel group array 226 and detecting contrast information of the extracted frequency components.

[0134] Furthermore, the camera control unit 21 groups the multiple pixel group arrays 226 corresponding to the target area into groups of a predetermined number of pixel group arrays 226, and detects contrast information of the frequency components in each group by adding together the frequency components corresponding to the output of the pixel group arrays 226 that belong to the same group. For example, in the example shown in FIG. 24, the pixel group arrays 226 that belong to the same group are represented by A1' to A3'. In this case, the camera control unit 21 obtains an output value d1 by adding together the output values ​​of the frequency components corresponding to the output of the first pixel group 223 in each of the pixel group arrays A1' to A3'. Similarly, the camera control unit 21 obtains an output value d2 by adding together the output values ​​of the frequency components corresponding to the output of the second pixel group 223 in each of the pixel group arrays A1' to A3'. Similarly, for the third and subsequent pixel groups 223 from the left, the camera control unit 21 acquires output values ​​d3, d4, d5, and d6 by adding the output values ​​of the frequency components corresponding to the output of each pixel group 223, and can detect the acquired output values ​​d1, d2, d3, d4, d5, and d6 as contrast information of the frequency components in the group. Note that the camera control unit 21 may be configured to detect contrast information of the frequency components in each group by averaging the output values ​​of the frequency components corresponding to the output of pixel groups 223 arranged at the same position in the horizontal direction (X direction).

[0135] The camera control unit 21 then calculates the amount of contrast for the frequency components in each group based on the contrast information for the frequency components in each group. Furthermore, if there are multiple groups corresponding to the target area within the focus detection area 224, the camera control unit 21 similarly calculates the amount of contrast for the frequency components in the other groups. The camera control unit 21 then determines the group with the highest contrast amount among the groups for which the contrast amount has been calculated as a specific group, and calculates the amount of defocus based on the output of the imaging pixels 221a in the specific group. Note that the method of calculating the amount of defocus for the specific group can be the same as in the fourth embodiment.

[0136] Next, the operation of the camera 1 according to the fifth embodiment will be described with reference to Fig. 25. Fig. 25 is a flowchart showing an example of the operation of the camera 1 according to the fifth embodiment.

[0137] First, in step S501, detection of a target area corresponding to a specific subject is performed by the camera control unit 21. For example, the camera control unit 21 can detect the specific subject by performing template matching using a template image of the specific subject, such as a person's face, and detect the area of ​​the image sensor 220 that corresponds to the detected specific subject as the target area.

[0138] Then, in step S502, similar to step S402 in the fourth embodiment, a focus detection area AFP to be used for focus adjustment is selected, and the focus detection area 224 corresponding to the selected focus detection area AFP is determined as the focus detection area 224 for performing focus detection.

[0139] In step S503, the camera control unit 21 determines whether the target area detected in step S501 exists within the focus detection area 224 selected in step S502. If the target area exists within the focus detection area 224, the process proceeds to step S504. On the other hand, if the target area does not exist within the focus detection area 224, the process proceeds to step S513. Note that in steps S513 to S515, similar to steps S403 to S405 in the fourth embodiment, contrast information of the imaging pixel arrays 225 set in the focus detection area 224 is detected (step S513), the imaging pixel arrays 225 are grouped into groups of a predetermined number of imaging pixel arrays (step S514), and the output values ​​of frequency components included in the outputs of the imaging pixel arrays 225 belonging to the same group are added up for each group, thereby detecting contrast information of the frequency components in each group (step S515).

[0140] On the other hand, if a target area is detected within the focus detection area 224 in step S503 (step S503 = Yes), the process proceeds to step S504. In step S504, the camera control unit 21 extracts output values ​​of the four imaging pixels 221a that make up each pixel group 223 in the target area at a ratio corresponding to the color indicating the specific subject (for example, the skin color of the specific subject). Then, in step S505, the camera control unit 21 detects the output of each pixel group 223 included in the target area based on the output of the imaging pixels 221a extracted in step S504, and detects the output of each pixel group array 226 made up of multiple pixel groups 223. In other words, the camera control unit 21 adds or averages the output values ​​of the imaging pixels 221a extracted at a ratio corresponding to the color indicating the specific subject for each pixel group 223, thereby detecting the output value of the pixel group 223 corresponding to the skin color of the specific subject. The camera control unit 21 then detects the output of the pixel group array 226 by detecting the output values ​​of the multiple pixel groups 223 that make up the pixel group array 226 along the arrangement direction of the pixel groups 223 in the pixel group array 226 .

[0141] Furthermore, in step S506, the camera control unit 21 detects contrast information of the frequency components included in the output of the pixel group array 226, based on the output of the pixel group array 226 detected in step S505. For example, the camera control unit 21 performs a predetermined filter process on the output of the pixel group array 226, thereby extracting frequency components corresponding to the subject from the output of the pixel group array 226, and detecting contrast information of the extracted frequency components.

[0142] In step S507, the camera control unit 21 groups the pixel group arrays 226 corresponding to the target area into groups of a predetermined number of pixel group arrays 226. Then, in step S508, the camera control unit 21 adds up the output values ​​of the frequency components included in the output of the pixel group arrays 226 that belong to the same group, thereby detecting contrast information of the frequency components in each group.

[0143] In step S509, the camera control unit 21 calculates the contrast amount of the frequency components in each group based on the contrast information of the frequency components in each group detected in step S509 or step S515, for example as shown in the above equation (3).

[0144] Then, in steps S510 to S512, similar to steps S407 to S409 in the fourth embodiment, the group with the highest contrast is determined as the specific group (step S510), and the defocus amount is calculated based on the output of the imaging pixels 221a included in the specific group (step S511).Then, the focus lens 32 is driven based on the calculated defocus amount (step S512).

[0145] In this way, in the fifth embodiment, by extracting the output of each imaging pixel 221a at a ratio according to the color indicating the specific subject, it is possible to extract frequency components attributable to a specific subject, such as a person's face, with higher accuracy. Then, by determining a group for focus detection based on the frequency components extracted in this way and performing focus detection based on the output of the imaging pixels 221a in that group, it is possible to more appropriately detect the focus state of the optical system for the specific subject.

[0146] Although the embodiments have been described above, the above-described embodiments have been described to facilitate understanding of the present invention, and are not intended to limit the present invention. Therefore, each element disclosed in the above-described embodiments is intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0147] For example, in the first to third embodiments described above, a configuration was given in which four focus detection pixel rows L1 to L4 are provided for one focus detection area AFP, but the number of focus detection pixel rows is not limited to this, and the number of focus detection pixel rows in one focus detection area may be two or three, or may be five or more. Also, in the fourth and fifth embodiments described above, a configuration was given in which imaging pixels 221, each having a pair of photoelectric conversion units 2214, 2215, are arranged in the focus detection regions 224a to 224i, but it is also possible to have a configuration in which imaging pixels 221 are arranged over the entire image sensor 220, for example.

[0148] In the first to third embodiments described above, the image sensor 22 includes focus detection pixel arrays L1 to L4, and the defocus amount is calculated based on the outputs of the focus detection pixel arrays L1 to L4. However, the present invention is not limited to this configuration. For example, a phase-difference AF detection module may be provided separately from the image sensor 22, and the defocus amount may be calculated based on a light reception signal received by the phase-difference AF detection module. Specifically, the phase-difference AF detection module may include a plurality of pairs of line sensors that receive light beams passing through the optical system, detect contrast information for each line sensor, and determine a pair of line sensors to be used for focus detection from among the plurality of line sensors based on the detected contrast information. Note that when determining the pair of line sensors to be used for focus detection, a calculation unit included in the phase-difference detection module may determine the pair of line sensors to be used for focus detection. Alternatively, the camera control unit 21 may determine the pair of line sensors to be used for focus detection by acquiring the output of each pair of line sensors.

[0149] In this way, the "photodetector group" may be, for example, focus detection pixel rows L1 to L4 or the pair of line sensors described above. Also, the "focus detection device" may be, for example, image sensor 22, 220 or camera 1 equipped with the image sensor, or a phase-difference AF detection module or camera 1 equipped with the image sensor.

[0150] Furthermore, in the first to third embodiments described above, the first focus detection pixel 222a and the second focus detection pixel 222b are arranged so that they are reversed in the X-axis direction in the focus detection pixel rows L1, L3 and the focus detection pixel rows L2, L4. However, this configuration is not limited to this. For example, as shown in FIG. 26, the focus detection pixels 222a and 222b that make up the focus detection pixel rows L1a to L4a may be arranged so that they are in the same position in the X-axis direction.

[0151] Alternatively, as shown in focus detection pixel arrays L1b and L2b in Fig. 27, each focus detection pixel 222a and 222b may be configured to include a pair of photoelectric conversion units 2222a and 2222b. Specifically, as shown in Fig. 27, in each focus detection pixel 222a and 222b, the pair of photoelectric conversion units 2222a and 2222b is arranged in the X-axis direction, and the light beam emitted from the ranging pupil 351 is received by one of the pair of photoelectric conversion units 2222a and 2222b, and the light beam emitted from the ranging pupil 352 is received by the other photoelectric conversion unit. This allows each focus detection pixel array L1b and L2b to output a pair of image data strings. 27, photoelectric conversion units that receive the light beam emitted from the ranging pupil 351 are shown in gray, and photoelectric conversion units that receive the light beam emitted from the ranging pupil 352 are shown in white. Also, in the example shown in Fig. 27, a configuration having two focus detection pixel rows L1b, L2b is illustrated, but this is not limited to this configuration, and for example, a configuration having only one focus detection pixel row L1b, L2b, or a configuration having three or more rows, may also be used.

[0152] Furthermore, the arrangement of the first focus detection pixels 222a and the second focus detection pixels 222b that make up each focus detection pixel row may be such that they are at least arranged alternately on the same row, and for example, the focus detection pixel row may be configured to include normal imaging pixels 221, as in the focus detection pixel rows L1c to L4c shown in Figure 28.

[0153] In addition to the above-described embodiments, if the contrast of the subject (or the brightness of the subject) is equal to or greater than a predetermined value, as shown in the first embodiment, the focus detection pixel row among the focus detection pixel rows L1 to L4 corresponding to the subject with the greatest contrast, or the focus detection pixel row whose output contains the most high-frequency components, may be determined as the specific focus detection pixel row. On the other hand, if the contrast of the subject (or the brightness of the subject) is less than a predetermined value, as shown in the second embodiment, one or more focus detection pixel rows among the focus detection pixel rows L1 to L4 whose corresponding subject contrast is equal to or greater than a predetermined value, or one or more focus detection pixel rows whose output contains a predetermined amount of high-frequency components, may be determined as the specific focus detection pixel row.

[0154] Furthermore, in the fourth embodiment described above, the contrast amount of the frequency components corresponding to the output of each imaging pixel array 225 is calculated for each group by adding up the frequency components corresponding to the output of each imaging pixel array 225, and the defocus amount is calculated based on the output of the imaging pixel 221a included in the group with the highest contrast amount. However, this configuration is not limited to this, and the following configurations are also possible. That is, the contrast amount of the frequency components corresponding to each imaging pixel array 225 is calculated for each imaging pixel array 225 from the frequency components corresponding to the frequency components, and the imaging pixel array 225 with the highest contrast amount is identified for each group from among the imaging pixel arrays 225 included in the same group. Then, the defocus amount is calculated for each group based on the output of the imaging pixel 221a included in the imaging pixel array 225 with the highest contrast amount, and focus adjustment of the optical system is performed based on these defocus amounts. Alternatively, the contrast amount of the frequency components may be calculated for each imaging pixel array 225, and the defocus amount may be calculated for each imaging pixel array 225, and focus adjustment of the optical system may be performed based on the defocus amount of the imaging pixel array 225 with the highest contrast amount.

[0155] Furthermore, in the fourth and fifth embodiments described above, the defocus amount is calculated based on the output of the imaging pixels 221a included in the group with the highest contrast amount of the frequency component. However, this is not a limitation. For example, the output of the imaging pixels 221a in each group may be weighted based on the magnitude of the contrast amount of the frequency component in each group, and the defocus amount may be calculated based on the weighted output of the imaging pixels 221a. In this case, the camera control unit 21 may weight the output of the imaging pixels 221a in the group with a higher contrast amount of the frequency component in that group. Alternatively, the camera control unit 21 may calculate the defocus amount for each group, and weight the calculated defocus amount according to the contrast amount of the frequency component in each group to calculate the defocus amount used for focus adjustment.

[0156] Furthermore, in the fourth and fifth embodiments described above, a configuration in which a plurality of imaging pixels 221a are arranged two-dimensionally in each of the focus detection regions 224a-224i is illustrated, but this configuration is not limited thereto. For example, each of the focus detection regions 224a-224i may have only a single imaging pixel row 225, or may have two or more discrete or continuous imaging pixel rows 225. Furthermore, in the first to third embodiments described above, a configuration in which the focus detection pixels 222a, 222b are arranged in the horizontal direction (X direction) in the focus detection pixel rows L1-L4 is illustrated, but this configuration is not limited thereto. For example, the focus detection pixels 222a, 222b may be arranged in the vertical direction (Y direction). Furthermore, in the fourth and fifth embodiments described above, a configuration in which a plurality of imaging pixels 221a arranged in the vertical direction (Y direction) constitutes the imaging pixel row 225 may also be configured. In this case, the output value of the frequency component in each group can be calculated by adding the output values ​​of the frequency components corresponding to the output of the imaging pixels 221a arranged at the same position in the vertical direction (Y direction) in each imaging pixel column 225 belonging to the same group.

[0157] Additionally, in the fourth and fifth embodiments described above, a single-lens reflex digital camera 1a equipped with an image sensor 220 has been described as an example, but the present invention is not limited to this configuration, and for example, the digital camera 1 shown in Fig. 1 may be configured to include an image sensor 220. In this case, focus detection can be performed in the same manner as in the fourth and fifth embodiments. [Explanation of symbols]

[0158] 1,1a...Digital camera 2,2a...Camera body 21,220...Camera control unit 22...Image sensor 221, 221a...imaging pixels L1 to L4: Focus detection pixel rows 225...imaging pixel array 226...pixel group array 222a, 222b...Focus detection pixels 3...Lens barrel 32...Focus lens 36...Focus lens drive motor 37...Lens control unit

Claims

[Claim 1] an image sensor having an imaging surface that captures an image formed by an optical system, the imaging surface having a plurality of focus detection areas that detect a focus state between the image and the imaging surface, and a plurality of pixel rows arranged in each of the plurality of focus detection areas, the pixel rows including pixels each having a first photoelectric conversion unit that receives light passing through a first area of ​​the optical system and outputs a signal, and a second photoelectric conversion unit that receives light passing through a second area of ​​the optical system and outputs a signal; a control unit that detects contrast information of outputs from a plurality of pixel rows arranged in one of the plurality of focus detection areas, identifies a portion of pixel rows to be used for focus detection from the plurality of pixel rows arranged in the one focus detection area based on the detection result, and detects a focus state based on signals output from the identified pixel rows; A focus detection device comprising:

Images