Focus control device, imaging apparatus, focus control method, and program

Abstract

To provide a focus control device, an imaging apparatus, a focus control method, and a program capable of improving focusing accuracy in a case where the reliability of a defocus amount is low.SOLUTION: A focus control device determines a defocus amount for driving a focus lens. The focus control device includes a processor and a memory. The processor obtains the number of areas of which reliability of defocus amounts obtained from a plurality of areas set in an imaging region are low, and determines the defocus amount for driving the focus lens on the basis of an index related to a plurality of defocus amounts in a case where the number is equal to or larger than a first threshold.SELECTED DRAWING: Figure 17

Description

[Technical field]

[0001] The technology disclosed herein relates to a focus control device, an imaging device, a focus control method, and a program. [Background technology]

[0002] Patent document 1 discloses a focus detection device that amplifies an accumulated charge signal output by a photoelectric conversion element unit using parameters set in an amplifier unit, reads out the amplified accumulated charge signal as a subject image signal, and has a judgment unit judge whether the subject image signal has an appropriate value for performing focus detection. If it is appropriate, subsequent focus detection calculations etc. are performed, and if it is inappropriate, the amplification parameters of the amplifier unit are changed, the signal is amplified, and read out again as a subject image signal to obtain a subject image signal suitable for focus detection.

[0003] Patent Document 2 discloses an imaging device that captures an image formed by an optical system with an image sensor. The image sensor is equipped with an image synthesis means that receives light beams that have passed through different regions on the exit pupil of the optical system to generate data for a pair of images and adds the data for the pair of images to generate a composite image, and an image processing means detects a defocus amount distribution in the captured image for the composite image and processes the image data based on the defocus amount distribution. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 11-133294 [Patent Document 2] JP 2012-142952 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the techniques disclosed herein is to provide a focus control device, an imaging device, a focus control method, and a program that are capable of improving focusing accuracy when the reliability of the defocus amount is low. [Means for solving the problem]

[0006] In order to achieve the above-mentioned objective, the focus control device disclosed herein is a focus control device that determines a defocus amount for driving a focus lens, and is equipped with a processor and a memory, and the processor calculates the number of areas where the reliability of the defocus amount obtained from a plurality of areas set within an imaging area is low, and if the number is equal to or greater than a first threshold, determines the defocus amount for driving the focus lens based on an index related to the plurality of defocus amounts.

[0007] The index is preferably an index based on the frequency of a plurality of defocus amounts.

[0008] It is preferable that the processor determines a median value of the plurality of defocus amounts as the defocus amount for driving the focus lens.

[0009] If the number is less than the first threshold, the processor preferably determines the defocus amount for driving the focus lens based on the distance to the subject corresponding to the defocus amount.

[0010] When the number is less than the first threshold, the processor preferably determines the closest value of the multiple defocus amounts as the defocus amount for driving the focus lens.

[0011] It is preferable that the processor divides each of the multiple areas into multiple blocks, determines whether or not a block is bad based on the content of signals that affect the reduction in reliability contained in each block, and determines that the reliability is low if the number of bad blocks is equal to or greater than a second threshold value.

[0012] The signal that affects the decrease in reliability is preferably a signal that exceeds the performance limit of the image sensor.

[0013] Preferably, the confidence level is related to the saturation level of a number of pixels contained in the area, and the processor determines the confidence level based on the saturation level.

[0014] It is preferable that the processor divides each of the multiple areas into multiple blocks, determines whether each block is a saturated block based on the percentage of saturated pixels contained in the block, and determines that the reliability is low if the number of saturated blocks is equal to or greater than a second threshold.

[0015] It is preferable that the processor determines an area in which the number of saturated blocks is equal to or greater than a second threshold as a first area, and determines the median of the defocus amounts corresponding to the multiple blocks included in the first area as the defocus amount of the first area.

[0016] It is preferable that the processor determines an area in which the number of saturated blocks is less than a second threshold as a second area, and sets the closest value of the defocus amount corresponding to the multiple blocks included in the second area as the defocus amount of the second area.

[0017] It is preferable that the processor performs a plurality of filtering processes with different frequency characteristics on a plurality of pixels included in the area, and determines the reliability based on evaluation values ​​obtained from the plurality of filtering processes.

[0018] It is preferable that the processor divides each of the multiple areas into multiple blocks, determines whether each block is a high frequency block based on the evaluation value, and determines that the reliability is low if the number of high frequency blocks is equal to or greater than a second threshold value.

[0019] It is preferable that the processor determines an area in which the number of high frequency blocks is equal to or greater than a second threshold as a first area, and determines the median of the defocus amounts corresponding to the multiple blocks included in the first area as the defocus amount of the first area.

[0020] It is preferable that the processor determines an area in which the number of high frequency blocks is less than a second threshold as a second area, and sets the closest value of the defocus amounts corresponding to the multiple blocks included in the second area as the defocus amount of the second area.

[0021] It is preferable that the processor calculates a first defocus amount by performing a correlation calculation based on a first two-dimensional image generated by performing a first filtering process on a two-dimensional image represented by a plurality of pixels included in the block, calculates a second defocus amount by performing a correlation calculation based on a first one-dimensional image generated by performing the first filtering process on a one-dimensional image obtained by vertically adding the two-dimensional images, calculates a third defocus amount by performing a correlation calculation based on a second two-dimensional image generated by performing a second filtering process having frequency characteristics different from those of the first filtering process on the two-dimensional image, and calculates a fourth defocus amount by performing a correlation calculation based on the second one-dimensional image generated by performing the second filtering process on the one-dimensional image, and determines that the block is a high-frequency block if the difference between the first defocus amount and the third defocus amount is greater than both the difference between the first defocus amount and the second defocus amount and the difference between the third defocus amount and the fourth defocus amount.

[0022] Preferably, the confidence level is related to the contrast of a number of pixels contained in the area, and the processor determines the confidence level based on an evaluation value relating to the contrast.

[0023] It is preferable that the processor divides each of the multiple areas into multiple blocks, determines whether each block is a low-contrast block based on the evaluation value, and determines that the reliability is low if the number of low-contrast blocks is greater than or equal to a second threshold.

[0024] It is preferable that the processor determines an area in which the number of low contrast blocks is equal to or greater than a second threshold as a first area, and determines the median of the defocus amounts corresponding to the multiple blocks included in the first area as the defocus amount of the first area.

[0025] It is preferable that the processor determines an area in which the number of low contrast blocks is less than a second threshold as a second area, and sets the closest value of the defocus amounts corresponding to the multiple blocks included in the second area as the defocus amount of the second area.

[0026] It is preferable that the processor determines whether or not each block is a low contrast block based on the luminance distribution of the multiple pixels contained in the block, or the shape of a correlation curve obtained by a correlation calculation based on the multiple pixels contained in the block.

[0027] An imaging device according to the present disclosure includes the above-described focus control device and an imaging element.

[0028] The focus control method disclosed herein is a focus control method for determining a defocus amount for driving a focus lens, and includes a processor determining the number of areas having low reliability of the defocus amount obtained from a plurality of areas set within an imaging area, and if the number is equal to or greater than a first threshold, determining the defocus amount for driving the focus lens based on an index related to the plurality of defocus amounts.

[0029] The program disclosed herein is a program that causes a processor to execute a process for determining a defocus amount for driving a focus lens, and causes the processor to execute a process including determining the number of areas having low reliability of the defocus amount obtained from a plurality of areas set within an imaging area, and if the number is equal to or greater than a first threshold, determining the defocus amount for driving the focus lens based on an index related to the plurality of defocus amounts. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 illustrates an example of a configuration of an imaging device. [Diagram 2] FIG. 2 is a diagram illustrating an example of a configuration of an imaging pixel. [Diagram 3] FIG. 2 is a diagram illustrating an example of a configuration of a phase difference detection pixel. [Figure 4] FIG. 2 is a diagram illustrating an example of a pixel array of an image sensor. [Diagram 5] 2 is a block diagram showing an example of a functional configuration of a processor. FIG. [Figure 6] FIG. 4 is a diagram showing an example of an AF area. [Figure 7] FIG. 13 is a diagram showing an example of a plurality of small areas. [Figure 8] FIG. 2 is a diagram illustrating an example of a plurality of blocks. [Figure 9] FIG. 11 is a diagram illustrating an example of correlation calculation processing. [Figure 10] FIG. 11 is a diagram illustrating an example of correlation calculation processing including filtering processing. [Figure 11] FIG. 13 illustrates an example of a filtering process. [Figure 12] FIG. 11 is a diagram for explaining details of correlation calculation processing. [Figure 13] FIG. 13 is a diagram showing an example of a correlation curve. [Figure 14] FIG. 11 is a diagram illustrating an example of a first selection process. [Figure 15] FIG. 11 is a diagram illustrating an example of a second selection process. [Figure 16] FIG. 11 is a diagram illustrating an example of a second selection process. [Figure 17] 13 is a flowchart showing a flow of a driving defocus amount acquisition process. [Figure 18] 13 is a flowchart illustrating an example of a reliability determination process. [Figure 19] 5 is a flowchart showing an example of the flow of an entire process executed by the imaging device. [Figure 20] 13 is a flowchart showing a reliability determination process according to a first modified example. [Figure 21] FIG. 10 is a diagram illustrating a correlation curve calculated by applying a high-frequency filter and a correlation curve calculated by applying a low-frequency filter. [Figure 22] 13 is a flowchart showing a reliability determination process according to a second modified example. [Diagram 23]11A and 11B are diagrams illustrating a correlation calculation process used in a high-frequency block determination process. [Figure 24] 13 is a flowchart showing the flow of a high-frequency block determination process. [Diagram 25] FIG. 13 is a diagram showing an example of a correlation curve when the contrast is low. [Figure 26] 13 is a flowchart showing a reliability determination process according to a third modified example. [Figure 27] FIG. 2 is a diagram illustrating a defective block. [Figure 28] 13 is a flowchart showing a reliability determination process according to a fourth modified example. [Figure 29] 13 is a flowchart showing a median selection process according to a fifth modified example. [Diagram 30] FIG. 13 is a diagram illustrating a closest value. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0032] First, the terms used in the following description will be explained.

[0033] In the following explanation, "IC" is an abbreviation for "Integrated Circuit". "CPU" is an abbreviation for "Central Processing Unit". "ROM" is an abbreviation for "Read Only Memory". "RAM" is an abbreviation for "Random Access Memory". "CMOS" is an abbreviation for "Complementary Metal Oxide Semiconductor".

[0034] "FPGA" is an abbreviation for "Field Programmable Gate Array". "PLD" is an abbreviation for "Programmable Logic Device". "ASIC" is an abbreviation for "Application Specific Integrated Circuit". "OVF" is an abbreviation for "Optical View Finder". "EVF" is an abbreviation for "Electronic View Finder". "CNN" is an abbreviation for "Convolutional Neural Network". "AF" is an abbreviation for "Auto Focus".

[0035] The technology of the present disclosure will be described by taking a digital camera with interchangeable lenses as an example of an embodiment of an imaging device. Note that the technology of the present disclosure is not limited to the digital camera with interchangeable lenses, and can also be applied to a digital camera with an integrated lens.

[0036] 1 shows an example of the configuration of an imaging device 10. The imaging device 10 is a digital camera with interchangeable lenses. The imaging device 10 is composed of a main body 11 and an imaging lens 12 that is replaceably attached to the main body 11. The imaging lens 12 is attached to the front side of the main body 11 via a camera side mount 11A and a lens side mount 12A.

[0037] The main body 11 is provided with an operation device 13 that includes a dial, a release button, a touch panel, etc. and accepts operations by a user. The operation modes of the imaging device 10 include, for example, a still image capturing mode, a video capturing mode, and an image display mode. The operation device 13 is operated by a user when setting an operation mode. The operation device 13 is also operated by a user when starting to capture a still image or a video. The operation device 13 is also operated by a user when specifying an AF area to be focused on from within the imaging region.

[0038] The main body 11 is also provided with a finder 14. Here, the finder 14 is a hybrid finder (registered trademark). The hybrid finder refers to a finder in which, for example, an optical view finder (hereinafter referred to as "OVF") and an electronic view finder (hereinafter referred to as "EVF") are selectively used. The user can observe an optical image or a live view image of a subject projected by the finder 14 through a finder eyepiece (not shown).

[0039] A display 15 is provided on the rear side of the main body 11. Images based on image signals obtained by imaging, various menu screens, etc. are displayed on the display 15. Instead of the viewfinder 14, the user can also observe a live view image projected by the display 15.

[0040] The main body 11 and the imaging lens 12 are electrically connected by electrical contacts 11B provided on the camera side mount 11A coming into contact with electrical contacts 12B provided on the lens side mount 12A.

[0041] The imaging lens 12 includes an objective lens 30, a focus lens 31, a rear end lens 32, and an aperture 33. The respective components are arranged along the optical axis A of the imaging lens 12 in the following order from the objective side: objective lens 30, aperture 33, focus lens 31, and rear end lens 32. The objective lens 30, focus lens 31, and rear end lens 32 constitute an imaging optical system. The type, number, and arrangement order of the lenses that constitute the imaging optical system are not limited to the example shown in FIG.

[0042] The imaging lens 12 also has a lens drive control unit 34. The lens drive control unit 34 is composed of, for example, a CPU, a RAM, and a ROM. The lens drive control unit 34 is electrically connected to a processor 40 in the main body 11 via an electrical contact 12B and an electrical contact 11B.

[0043] The lens drive control unit 34 drives the focus lens 31 and the aperture 33 based on a control signal transmitted from the processor 40. In order to adjust the focus position of the imaging lens 12, the lens drive control unit 34 controls the drive of the focus lens 31 based on a focus control signal transmitted from the processor 40. The processor 40 detects the focus position using a phase difference method.

[0044] The diaphragm 33 has an aperture whose diameter is variable around the optical axis A. The lens drive control unit 34 controls the drive of the diaphragm 33 based on a control signal for adjusting the diaphragm transmitted from the processor 40 in order to adjust the amount of light incident on the light receiving surface 20A of the image sensor 20.

[0045] Also provided inside the main body 11 are an image sensor 20, a processor 40, and a memory 42. The operations of the image sensor 20, the memory 42, the operation device 13, the viewfinder 14, and the display 15 are controlled by the processor 40.

[0046] The processor 40 is configured by, for example, a CPU. In this case, the processor 40 executes various processes based on a program 43 stored in a memory 42. The processor 40 may be configured by a collection of multiple IC chips. The processor 40 and the memory 42 configure a focus control device.

[0047] The imaging sensor 20 is, for example, a CMOS image sensor. The imaging sensor 20 is disposed such that its optical axis A is perpendicular to the light receiving surface 20A and is located at the center of the light receiving surface 20A. Light that has passed through the imaging lens 12 is incident on the light receiving surface 20A. A plurality of pixels that generate signals by performing photoelectric conversion are formed on the light receiving surface 20A. The imaging sensor 20 photoelectrically converts the light incident on each pixel to generate and output an image signal D. The imaging sensor 20 is an example of an "imaging element" according to the technology of the present disclosure.

[0048] A color filter array in a Bayer pattern is arranged on the light receiving surface 20A of the image sensor 20, and a color filter of any one of R (red), G (green), and B (blue) is arranged to face each pixel. Note that some of the pixels arranged on the light receiving surface 20A of the image sensor 20 may be phase difference detection pixels for detecting a phase difference related to focus control.

[0049] FIG. 2 shows an example of the configuration of an imaging pixel N. FIG. 3 shows an example of the configuration of phase difference detection pixels P1, P2. Each of the phase difference detection pixels P1, P2 receives one of the light beams split in the X direction with the principal ray as the center. Hereinafter, the direction perpendicular to the X direction is referred to as the Y direction. The X direction corresponds to the horizontal direction, and the Y direction corresponds to the vertical direction. The phase difference detection pixels P1, P2 are examples of "pixels" according to the technology of the present disclosure.

[0050] 2, the imaging pixel N includes a photodiode PD as a photoelectric conversion element, a color filter CF, and a microlens ML. The color filter CF is disposed between the photodiode PD and the microlens ML.

[0051] The color filter CF is a filter that transmits light of any one of the colors R, G, and B. The microlens ML collects the light flux LF incident from the exit pupil EP of the imaging lens 12 onto approximately the center of the photodiode PD via the color filter CF.

[0052] 3, each of the phase difference detection pixels P1 and P2 includes a photodiode PD, a light-shielding layer SF, and a microlens ML. Similar to the imaging pixel N, the microlens ML focuses a light beam LF incident from an exit pupil EP of the imaging lens 12 onto approximately the center of the photodiode PD.

[0053] The light-shielding layer SF is formed of a metal film or the like, and is disposed between the photodiode PD and the microlens ML. The light-shielding layer SF blocks a part of the light flux LF that is incident on the photodiode PD via the microlens ML.

[0054] In the phase difference detection pixel P1, the light-shielding layer SF blocks the negative side in the X direction with respect to the center of the photodiode PD. That is, in the phase difference detection pixel P1, the light-shielding layer SF allows the light flux LF from the negative-side exit pupil EP1 to be incident on the photodiode PD, and blocks the light flux LF from the positive-side exit pupil EP2 in the X direction.

[0055] In the phase difference detection pixel P2, the light shielding layer SF shields the positive side in the X direction with respect to the center of the photodiode PD as a reference. That is, in the phase difference detection pixel P2, the light shielding layer SF allows the light flux LF from the positive side exit pupil EP2 to be incident on the photodiode PD, and shields the light flux LF from the negative side exit pupil EP1 in the X direction.

[0056] That is, the phase difference detection pixel P1 and the phase difference detection pixel P2 have different light blocking positions in the X direction. The phase difference detection direction by the phase difference detection pixels P1 and P2 is the X direction (that is, the horizontal direction).

[0057] Fig. 4 shows an example of a pixel array of the image sensor 20. In Fig. 4, "R" represents an imaging pixel N provided with an R color filter CF. "G" represents an imaging pixel N provided with a G color filter CF. "B" represents an imaging pixel N provided with a B color filter CF. Note that the color array of the color filters CF is not limited to the Bayer array, and may be another color array.

[0058] Rows RL including the phase difference detection pixels P1 and P2 are arranged in the Y direction for every 10 pixels. In each row RL, a pair of phase difference detection pixels P1 and P2 and one imaging pixel N are repeatedly arranged in the Y direction. The arrangement pattern of the phase difference detection pixels P1 and P2 is not limited to the example shown in FIG. 4. For example, it may be a pattern in which a plurality of phase difference detection pixels are arranged on one microlens ML as shown in FIG. 5 attached to JP 2018-56703 A.

[0059] Fig. 5 shows an example of the functional configuration of the processor 40. The processor 40 realizes various functional units by executing processes according to a program 43 stored in the memory 42. As shown in Fig. 5, for example, a main control unit 50, an imaging control unit 51, an image processing unit 52, a display control unit 53, an AF area setting unit 54, and a driving defocus amount acquisition unit 55 are realized in the processor 40.

[0060] The main control unit 50 performs overall control of the operation of the imaging device 10 based on output information from the operation device 13. The imaging control unit 51 controls the imaging sensor 20 to execute imaging processing for causing the imaging sensor 20 to perform an imaging operation. The imaging control unit 51 drives the imaging sensor 20 in a still image imaging mode or a video imaging mode.

[0061] The imaging sensor 20 outputs an image signal D including an imaging signal generated by the imaging pixel N and a phase difference pixel signal generated by the phase difference detection pixels P1 and P2.

[0062] The image processing unit 52 acquires the image signal D output from the imaging sensor 20, and performs image processing such as demosaic processing on the acquired image signal D.

[0063] The display control unit 53 causes the display 15 to display an image represented by the image signal D that has been subjected to image processing by the image processing unit 52. Furthermore, the display control unit 53 causes the display 15 to display a live view image based on the image signal D periodically input from the image processing unit 52 during imaging preparation operations before still image capture or video capture. Furthermore, the display control unit 53 causes the display 15 to display an AF area RA, etc., specified by the user using the operation device 13. For example, the operation device 13 is a touch panel provided on the display surface of the display 15, and the user can specify the AF area RA by touching the touch panel with a finger.

[0064] The AF area setting unit 54 sets a rectangular AF area RA within the imaging area based on output information from the operation device 13. In this embodiment, the AF area setting unit 54 divides the AF area RA into multiple small areas SA. Setting information for the AF area RA is input to the driving defocus amount acquisition unit 55. The multiple small areas SA are an example of "multiple areas set within the imaging area" according to the technology of the present disclosure.

[0065] 6 shows an example of an AF area RA. By operating the operation device 13, the user can specify the AF area RA within the imaging region 20B of the image sensor 20. In addition, by operating the operation device 13, the user can specify the size, position, etc. of the AF area RA.

[0066] Fig. 7 shows an example of multiple small areas SA. In the example shown in Fig. 7, the AF area RA is divided into eight small areas SA. Each of the small areas SA is further divided into multiple blocks BL. The number, shape, size, etc. of each of the small areas SA are not limited to the example shown in Fig. 7 and can be changed as appropriate. In addition, the multiple small areas SA may be spaced apart from each other.

[0067] Fig. 8 shows an example of a plurality of blocks BL. In the example shown in Fig. 8, each of the blocks BL is a rectangular area extending in the X direction. Each of the blocks BL includes at least one row RL (see Fig. 4).

[0068] 5, the driving defocus amount acquisition unit 55 is composed of a phase difference pixel signal acquisition unit 56, a correlation calculation unit 57, a reliability determination unit 58, and a selection unit 59. The phase difference pixel signal acquisition unit 56 acquires a phase difference image signal Dp within the AF area RA from the image signal D output from the imaging sensor 20. The phase difference image signal Dp is the pixel value of the phase difference detection pixels P1 and P2.

[0069] The correlation calculation unit 57 calculates the defocus amount for each block BL by performing correlation calculation based on the phase difference detection pixels P1 and P2 for each block BL.

[0070] The reliability determination unit 58 determines the reliability of each defocus amount calculated by the correlation calculation unit 57. When the image signal D includes a signal that affects the decrease in reliability, specifically, a signal that exceeds the limit of the performance (resolution, sensitivity, saturation, S / N ratio, etc.) of the image sensor 20, the reliability of the defocus amount decreases. In this embodiment, the reliability determination unit 58 determines the reliability of the defocus amount based on the saturation of the phase difference detection pixels P1, P2. That is, in this embodiment, the reliability is related to the saturation of the phase difference detection pixels P1, P2, and a signal that exceeds the saturation upper limit is a signal that affects the decrease in reliability.

[0071] A selection unit 59 selects one of the multiple defocus amounts calculated by the correlation calculation unit 57 based on the reliability determined by the reliability determination unit 58, as a driving defocus amount DFd for driving the focus lens 31.

[0072] The driving defocus amount acquisition section 55 outputs the driving defocus amount DFd selected by the selection section 59. The driving defocus amount DFd is an example of a "defocus amount for driving the focus lens" according to the technique of the present disclosure.

[0073] The main control unit 50 adjusts the focus position by driving the focus lens 31 via the lens drive control unit 34 based on the driving defocus amount DFd output from the driving defocus amount acquisition unit 55. As a result, the subject in the AF area RA becomes in focus.

[0074] 9 shows an example of correlation calculation processing. The correlation calculation unit 57 calculates a defocus amount DFb by performing correlation calculation using a first image IP1 composed of signals from a plurality of phase difference detection pixels P1 included in a block BL and a second image IP2 composed of signals from a plurality of phase difference detection pixels P2. The defocus amount DFb is a defocus amount calculated from one block BL.

[0075] 10 shows an example of a correlation calculation process including a filtering process. The correlation calculation unit 57 may calculate the defocus amount DFb by performing a correlation calculation using an image obtained by performing a filtering process on the first image IP1 and an image obtained by performing a filtering process on the second image IP2. The same filtering process is performed on the first image IP1 and the second image IP2.

[0076] The correlation calculation unit 57 may acquire a plurality of defocus amounts DFb by calculating the defocus amount DFb from each block BL without performing a filtering process as shown in Fig. 9, and calculating the defocus amount DFb by performing a filtering process as shown in Fig. 10. Furthermore, the correlation calculation unit 57 may calculate three or more defocus amounts DFb from each block BL by performing a plurality of filtering processes with different frequency characteristics such as cutoff frequencies.

[0077] 11 shows an example of the filtering process. The correlation calculation unit 57 performs the filtering process using a one-dimensional filter FL in which weighting coefficients are arranged in the X direction, which is the phase difference detection direction. Specifically, the pixel value of the converted pixel PA in the input image and the pixel value of the adjacent pixel PB adjacent to the converted pixel PA are weighted and added by the weighting coefficients defined in the filter FL, and the weighted sum is set as the pixel value of the converted pixel PC in the output image. The output image is generated by performing the filtering process with all pixels in the input image as the converted pixel PA.

[0078] The filter FL is also called a kernel. In the example shown in FIG. 11, the filter FL is a kernel of 3×1 pixels, but the filter FL may be a kernel of 5×1 pixels, etc. Kernels such as "121" and "12221" may be used as the filter FL. The filtering process shown in FIG. 11 is a so-called convolution process. The input image is the above-mentioned first image IP1 or second image IP2.

[0079] FIG. 12 explains the details of the correlation calculation process. For the sake of simplicity, FIG. 12 shows the first image IP1 and the second image IP2 as one-dimensional waveforms in the X direction. A known method can be used to calculate the correlation amount between the image IP1 and the second image IP2. For example, an integrated value (hereinafter referred to as an "integrated difference value") of the absolute values ​​of the differences between each point IP1(X,Y) constituting the image IP1 and each point IP2(X+φ,Y) constituting the second image IP2 is calculated.

[0080] Fig. 13 shows an example of a correlation curve representing the relationship between the correlation amount between the image IP1 and the second image IP2 and the phase difference φ. Specifically, Fig. 13 shows a correlation curve representing the relationship between the difference accumulation value and the phase difference φ. The correlation calculation unit 57 calculates the phase difference φ at which the difference accumulation value is minimum (i.e., the correlation amount is maximum) as the defocus amount DFb.

[0081] 14 to 16 explain the selection process by the selection unit 59. The selection process is composed of a first selection process and a second selection process. In the first selection process, the selection unit 59 selects one from the multiple defocus amounts DFb calculated for one small area SA, and sets it as the defocus amount DFa. In the second selection process, the selection unit 59 selects one from the multiple defocus amounts DFa acquired for each small area SA, based on the reliability determined by the reliability determination unit 58, and sets it as the driving defocus amount DFd.

[0082] FIG. 14 shows an example of the first selection process. FIG. 14 is a graph showing a frequency distribution of the defocus amount DFb. A plurality of blocks BL are included in one small area SA, and one or more defocus amounts DFb are calculated from one block BL, so that a plurality of defocus amounts DFb are calculated from one small area SA. For this reason, in the first selection process, the selection unit 59 selects, for example, the closest value among the plurality of defocus amounts DFb calculated from one small area SA as the defocus amount DFa. The closest value is the value closest to the NEAR side (i.e., the value closest to the back focus side). Note that, in the first selection process, the selection unit 59 is not limited to the closest value among the plurality of defocus amounts DFb, and may select a median, an average value, or the like as the defocus amount DFa.

[0083] 15 and 16 show an example of the second selection process. FIG. 15 and FIG. 16 are graphs showing the frequency distribution of the defocus amount DFa. Since the AF area RA includes a plurality of small areas SA, a plurality of defocus amounts DFa are calculated from the AF area RA. In a situation where a landscape is photographed against the light, the AF area RA may include a point light source such as light shining through a gap between tree branches. In this way, when the AF area RA includes a point light source, the correlation curve (see FIG. 13) appears normal at first glance, but the reliability of the defocus amount DFa calculated from the small area SA including the point light source is low. When the reliability of the plurality of defocus amounts DFa is high, it is preferable to select the closest value so as to focus on the subject closest to the camera, but when the reliability is low, it is preferable to select the median value.

[0084] In the second selection process, the selection unit 59 selects the closest value or the median value of the multiple defocus amounts DFa as the driving defocus amount DFd based on the reliability determination result for each small area SA by the reliability determination unit 58. Fig. 15 shows an example in which the closest value is selected as the driving defocus amount DFd when the reliability is high. Fig. 16 shows an example in which the median value is selected as the driving defocus amount DFd when the reliability is low.

[0085] The closest value is a defocus amount that indicates the distance to the nearest subject within the AF area RA. The median is an example of an "index related to multiple defocus amounts," and more specifically, the median is an example of an "index based on the frequency of multiple defocus amounts."

[0086] 17 shows a flow of the driving defocus amount acquisition process by the driving defocus amount acquisition unit 55. In the driving defocus amount acquisition process, first, the phase difference pixel signal acquisition unit 56 acquires the phase difference image signal Dp in the AF area RA (step S10). Next, the correlation calculation unit 57 performs the above-mentioned correlation calculation process to calculate the defocus amount DFa for each small area SA (step S11).

[0087] Next, the reliability determination unit 58 determines the reliability of each small area SA (step S12). As will be described in detail later, the reliability determination unit 58 determines the reliability of the defocus amount DFa calculated for each small area SA based on the saturation of the phase difference detection pixels P1 and P2, and sets the small area SA with low reliability as a low reliability area.

[0088] Next, the selection unit 59 judges whether the number of low reliability areas is equal to or greater than a first threshold (step S13). The first threshold is an integer equal to or greater than 1 and equal to or less than the number of small areas SA (8 in this embodiment). For example, the first threshold is 1. If the number of low reliability areas is less than the first threshold (step S13: NO), the selection unit 59 selects the closest value of the multiple defocus amounts DFa as the driving defocus amount DFd (step S14). On the other hand, if the number of low reliability areas is equal to or greater than the first threshold (step S13: YES), the selection unit 59 selects the median value of the multiple defocus amounts DFa as the driving defocus amount DFd (step S15).

[0089] 18 shows an example of the reliability determination process (step S12) by the reliability determination unit 58. In the reliability determination process, first, one small area SA is selected from the multiple small areas SA included in the AF area RA (step S120).

[0090] Next, the reliability determination unit 58 determines whether each of the blocks BL included in the selected small area SA is a saturated block (step S121). A saturated block is a block BL that includes saturated pixels (phase difference detection pixel P1 or phase difference detection pixel P2 in this embodiment) whose pixel values ​​are equal to or greater than a maximum value or a predetermined threshold value. When the number of saturated pixels included in the block BL is N or more, the reliability determination unit 58 determines that the block BL is a saturated block. N is an integer equal to or greater than 1. That is, the reliability determination unit 58 determines whether the block BL is a saturated block based on the content rate of a signal that affects a decrease in reliability (saturated pixel signal in this embodiment). A saturated block is an example of a "bad block" according to the technology of the present disclosure.

[0091] The reliability determination unit 58 determines whether the number of saturated blocks included in the selected small area SA is equal to or greater than a second threshold (step S122). The second threshold is an integer equal to or greater than 1 and equal to or less than the number of blocks BL included in the small area SA. For example, the second threshold is 1.

[0092] If the number of saturated blocks is equal to or greater than the second threshold (step S122: YES), the reliability determination unit 58 determines that the selected small area SA is a low reliability area (step S123).If the number of saturated blocks is less than the second threshold (step S122: NO), the reliability determination unit 58 shifts the process to step S124.

[0093] After step S122 or step S123, the reliability determination unit 58 determines whether the selected small area SA is the final small area SA (step S124). If the selected small area SA is not the final small area SA (step S124: NO), the reliability determination unit 58 selects an unselected small area SA (step S125) and returns the process to step S121.

[0094] The reliability determination unit 58 repeatedly executes the processes of steps S121 to S125 until it is determined that the selected small area SA is the final small area SA. If the selected small area SA is the final small area SA (step S124: YES), the reliability determination unit 58 ends the process. By the reliability determination process described above, the number of low reliability areas included in the AF area RA is obtained.

[0095] 19 shows an example of the overall process flow executed by the imaging device 10. First, the main control unit 50 determines whether or not an instruction to start imaging preparation has been issued by the user operating the operation device 13 (step S20). If an instruction to start imaging preparation has been issued (step S20: YES), the main control unit 50 controls the imaging control unit 51 to cause the imaging sensor 20 to perform an imaging operation (step S21).

[0096] The image processing unit 52 acquires the image signal D output from the imaging sensor 20 and performs image processing on the image signal D (step S22). The display control unit 53 causes the display 15 to display an image represented by the image signal D after the image processing (step S23).

[0097] Next, the main control unit 50 determines whether or not the user has performed an operation to specify the AF area RA using the operation device 13 (hereinafter, referred to as an "AF area specifying operation") (step S24). If the AF area specifying operation has not been performed (step S24: NO), the main control unit 50 shifts the process to step S28.

[0098] If an AF area designation operation has been performed (step S24: YES), the AF area setting unit 54 sets an AF area RA within the imaging area based on output information from the operation device 13 (step S25).

[0099] Next, the driving defocus amount acquisition unit 55 executes the driving defocus amount acquisition process (see FIGS. 17 and 18) described above to acquire the driving defocus amount DFd (step S26). The main control unit 50 drives the focus lens 31 based on the acquired driving defocus amount DFd (step S27).

[0100] After step S24 or step S27, the main control unit 50 determines whether or not an image capture instruction has been issued by the user operating the operation device 13 (step S28). If no operation instruction has been issued (step S28: NO), the main control unit 50 returns the process to step S21. The processes of steps S21 to S28 are repeatedly executed until the main control unit 50 determines in step S28 that an image capture instruction has been issued.

[0101] If an image capture instruction has been issued (step S28: YES), the main control unit 50 performs still image capture processing in which the image sensor 20 performs an image capture operation and the image signal D that has been image-processed by the image processing unit 52 is recorded as a still image in the memory 42 (step S29). This ends the processing.

[0102] Usually, it is preferable to focus on the nearest subject among the subjects present in the AF area RA, but in a situation where a landscape is photographed against the light, the above-mentioned point light source may be included in the AF area RA. In such a case, the reliability of the defocus amount DFa obtained from the small area SA including at least the point light source is reduced, and the distribution variation of the defocus amount DFa increases as shown in FIG. 16. In addition, when a plurality of subjects with significantly different distances are mixed in the AF area RA, the distribution variation of the defocus amount DFa increases. In such a case where the distribution variation is large, if the closest value is set as the driving defocus amount DFd, it becomes unclear whether the closest subject is in focus, and the focusing accuracy decreases. For example, when the defocus amount DFa with low reliability due to a point light source is set as the closest value, it will result in erroneous focusing.

[0103] In the technique of the present disclosure, the number of low reliability areas with low reliability of the defocus amount DFa is obtained for each of the plurality of small areas SA, and when the number of low reliability areas is equal to or greater than a first threshold, the median value of the plurality of defocus amounts DFa is set as the driving defocus amount DFd. Therefore, according to the technique of the present disclosure, in a situation where the distribution variation of the defocus amount DFa is large as shown in FIG. 16, the median value is set as the driving defocus amount DFd, so that the above-mentioned erroneous focusing is reduced and the focusing accuracy is improved. In addition, in the technique of the present disclosure, when the number of low reliability areas is less than the first threshold, the closest value of the plurality of defocus amounts DFa is set as the driving defocus amount DFd, so that it is possible to suppress the focus accuracy from being reduced by selecting the median value when the reliability is high.

[0104] Various modifications of the above embodiment will be described below.

[0105] [First Modification] In the above embodiment, in the first selection process, the selection unit 59 selects the closest value among the multiple defocus amounts DFb calculated from the small area SA as the defocus amount DFa, regardless of whether the small area SA is a low reliability area or not (see FIG. 14). Alternatively, when the small area SA is not a low reliability area, the selection unit 59 may select the closest value among the multiple defocus amounts DFb calculated from the small area SA as the defocus amount DFa, and when the small area SA is a low reliability area, the selection unit 59 may select the median value among the multiple defocus amounts DFb calculated from the small area SA as the defocus amount DFa.

[0106] Fig. 20 shows a reliability determination process according to Modification 1. The reliability determination process according to this modification is different from the reliability determination process shown in Fig. 18 only in that steps S126 and S127 executed by the selection unit 59 are added.

[0107] If the number of saturated blocks is less than the second threshold (step S122: NO), the selector 59 sets the closest value among the multiple defocus amounts DFb calculated from the selected small area SA as the defocus amount DFa (step S126). On the other hand, if the number of saturated blocks is equal to or greater than the second threshold (step S122: YES), the selector 59 sets the median value among the multiple defocus amounts DFb calculated from the selected small area SA (i.e., the low reliability area) as the defocus amount DFa (step S127). The other processes are the same as those in the above embodiment.

[0108] The small area SA determined to have the number of saturated blocks equal to or greater than the second threshold (i.e., low-reliability area) corresponds to the "first area" according to the technology of the present disclosure. The small area SA determined to have the number of saturated blocks less than the second threshold corresponds to the "second area" according to the technology of the present disclosure.

[0109] In this modification, the closest value or the median value is selected from the multiple defocus amounts DFb calculated from the small areas SA based on the number of saturated blocks, and is set as the defocus amount DFa, so that the reliability of the defocus amount DFa for each small area SA is improved, thereby further improving the focusing accuracy.

[0110] In the example shown in FIG. 20, the selection unit 59 executes steps S126 and S127 during the reliability determination process. However, steps S126 and S127 may be executed after the reliability determination process is completed.

[0111] [Second modified example] In the above embodiment, the reliability is determined based on the saturation of a plurality of phase difference pixels, but the present invention is not limited to this, and the reliability may be determined based on the frequency components contained in the images (first image IP1 and second image IP2) represented by a plurality of pixels. In this modification, the reliability is related to the frequency components contained in the images. In this modification, a plurality of filtering processes with different frequency characteristics are performed on the image, and the reliability is determined based on the evaluation values ​​obtained by the plurality of filtering processes. The confidence level is related to the frequency content contained in the image.

[0112] When the first image IP1 and the second image IP2 contain high-frequency components exceeding the resolution of the image sensor 20, the defocus amount DFb calculated by the correlation calculation process including the filtering process shown in Figure 10 will differ depending on the characteristics of the filter FL applied, and it is difficult to determine which is the true value.

[0113] FIG. 12 shows a schematic diagram of a correlation curve C1 calculated by applying a high-frequency filter as the filter FL, and a correlation curve C2 calculated by applying a low-frequency filter as the filter FL. For example, the high-frequency filter is a kernel with a weighting factor of "121", and the low-frequency filter is a kernel with a weighting factor of "12221". As shown in FIG. 12, a deviation occurs between the correlation curve C1 and the correlation curve C2, so that the defocus amount DFb calculated from each curve differs from each other. In general, it seems that the high-frequency filter calculates a more correct defocus amount DFb, but if a high-frequency component exceeding the resolution of the image sensor 20 is included, the calculated defocus amount DFb will vary randomly due to a slight difference in the angle of view. That is, the reliability of the defocus amount DFa determined from a plurality of defocus amounts DFb will be reduced.

[0114] In this modification, based on the difference in defocus amount DFb between two filters FL with different frequency characteristics, it is determined whether or not the image contains high-frequency components of a certain frequency or more, that is, whether or not the block BL contains high-frequency components of a certain frequency or more. Hereinafter, a block BL containing high-frequency components of a certain frequency or more is referred to as a "high-frequency block." Note that the high-frequency components of a certain frequency or more are, for example, signals that exceed the resolution of the image sensor 20. In this modification, the reliability determination unit 58 determines whether or not the block BL is a high-frequency block based on the content rate of a signal that affects the decrease in reliability (in this embodiment, a high-frequency signal). The high-frequency block is an example of a "bad block" according to the technology of the present disclosure. In addition, in this modification, the reliability determination unit 58 performs multiple filtering processes with different frequency characteristics on the image and determines the reliability based on evaluation values ​​obtained by the multiple filtering processes.

[0115] Fig. 22 shows a reliability determination process according to the second modified example. Of steps S120A to S125A shown in Fig. 22, only steps S121A and S122A differ from steps S120 to S125 shown in Fig. 18. In step S121A, the reliability determination unit 58 determines whether each of the blocks BL included in the selected small area SA is a high-frequency block. In step S122A, the reliability determination unit 58 determines whether the number of high-frequency blocks included in the selected small area SA is equal to or greater than a second threshold. The other processes are the same as those in the above embodiment.

[0116] Note that the reliability determination process according to this modification may be modified in the same manner as the first modification shown in Fig. 20. That is, when the number of high frequency blocks is less than the second threshold, the closest value may be set as the defocus amount DFa, and when the number of high frequency blocks is equal to or greater than the second threshold, the median value may be set as the defocus amount DFa.

[0117] 23 explains the correlation calculation process used in the high frequency block determination process (step S121A). In this modification, as a criterion for determining the difference in defocus amount DFb between two filters FL with different frequency characteristics, correlation calculation process including filtering is performed using two one-dimensional images obtained by performing vertical addition process on the first image IP1 and the second image IP2. Note that the vertical addition process refers to adding pixel values ​​of two-dimensional images in the Y direction to make a one-dimensional image.

[0118] In this modification, first, the reliability determination unit 58 calculates a defocus amount DFb by performing a correlation calculation based on a first two-dimensional image generated by performing a first filtering process on the two-dimensional images (first image IP1 and second image IP2). For example, the first filtering process is a filtering process that applies the above-mentioned high frequency filter as the filter FL. Hereinafter, the defocus amount DFb calculated by the correlation calculation based on the first two-dimensional image is referred to as a "first defocus amount DFb1."

[0119] Furthermore, the reliability determination unit 58 calculates a defocus amount DFb by performing a correlation calculation based on a first one-dimensional image generated by performing a first filtering process on a one-dimensional image (images IP1v and IP2v shown in FIG. 23) obtained by vertically adding two-dimensional images (first image IP1 and second image IP2). Hereinafter, the defocus amount DFb calculated by the correlation calculation based on the first one-dimensional image is referred to as a "second defocus amount DFb2."

[0120] Furthermore, the reliability determination unit 58 calculates a defocus amount DFb by performing a correlation calculation based on a second two-dimensional image generated by performing a second filtering process on the two-dimensional images (the first image IP1 and the second image IP2). The second filtering process has a different frequency characteristic from the first filtering process. For example, the second filtering process is a filtering process that applies the above-mentioned low-frequency filter as the filter FL. Hereinafter, the defocus amount DFb calculated by the correlation calculation based on the second two-dimensional image is referred to as a "third defocus amount DFb3."

[0121] Furthermore, the reliability determination unit 58 calculates a defocus amount DFb by performing a correlation calculation based on a second one-dimensional image generated by performing a second filtering process on a one-dimensional image (images IP1v and IP2v shown in FIG. 23) obtained by vertically adding two-dimensional images (first image IP1 and second image IP2). Hereinafter, the defocus amount DFb calculated by the correlation calculation based on the second one-dimensional image is referred to as a "fourth defocus amount DFb4."

[0122] The reliability determination unit 58 determines that the block BL to be determined is a high-frequency block when the difference ΔFL between the first defocus amount DFb1 and the third defocus amount DFb3 is greater than both the difference ΔV1 between the first defocus amount DFb1 and the second defocus amount DFb2 and the difference ΔV2 between the third defocus amount DFb3 and the fourth defocus amount DFb4. Note that the differences ΔFL, ΔV1, and ΔV2 are all absolute values ​​of the differences. Also, the differences ΔFL, ΔV1, and ΔV2 are examples of evaluation values ​​obtained by multiple filtering processes.

[0123] This determination method is based on the property that, for a normal subject, the first to fourth defocus amounts DFb1 to DFb4 are distributed to converge onto a normal distribution, but for a subject containing high frequencies, the difference ΔFL is larger than both the difference ΔV1 and the difference ΔV2.

[0124] 24 shows the flow of the high-frequency block determination process (step S121A). First, the reliability determination unit 58 selects one block BL from the multiple blocks BL included in the small area SA (step S1210).

[0125] Next, the reliability determination unit 58 performs each of the above processes based on the two-dimensional image of the selected block BL to calculate a first defocus amount DFb1 (step S1211), a second defocus amount DFb2 (step S1212), a third defocus amount DFb3 (step S1213), and a fourth defocus amount DFb4 (step S1214).

[0126] Next, the reliability determination unit 58 calculates the difference ΔFL (step S1215), calculates the difference ΔV1 (step S1216), and calculates the difference ΔV2 (step S1217).

[0127] Next, the reliability determination unit 58 determines whether or not ΔFL>ΔV1 and ΔFL>ΔV2 are satisfied (step S1218). If ΔFL>ΔV1 and ΔFL>ΔV2 are satisfied (step S1218: YES), the reliability determination unit 58 determines that the selected block BL is a high-frequency block (step S1219). If ΔFL>ΔV1 and ΔFL>ΔV2 are not satisfied (step S1218: NO), the reliability determination unit 58 shifts the process to step S1220.

[0128] After step S1218 or step S1219, the reliability determination unit 58 determines whether the selected block BL is the final block BL (step S1220). If the selected block BL is not the final block BL (step S1220: NO), the reliability determination unit 58 selects an unselected block BL (step S1221) and returns the process to step S1211.

[0129] The reliability determination unit 58 repeatedly executes the processes of steps S1211 to S1221 until it is determined that the selected block BL is the final block BL. If the selected block BL is the final block BL (step S1220: YES), the reliability determination unit 58 ends the process. The number of high frequency blocks is obtained by the above process.

[0130] [Third Modification] In this modification, the reliability is determined based on the contrast of the images (the first image IP1 and the second image IP2), that is, in this modification, the reliability is related to the contrast of a plurality of pixels.

[0131] When the contrast is equal to or greater than a certain value, one minimum value appears in the correlation curve as shown in FIG. 13, and the reliability of the calculated defocus amount DFb is high. On the other hand, when the contrast is less than a certain value, multiple minimum values ​​may appear in the correlation curve as shown in FIG. 25, and the reliability of the calculated defocus amount DFb is low. In this modification, a block BL whose contrast is less than a certain value is called a "low contrast block." In this modification, the reliability determination unit 58 determines whether or not the block BL is a low contrast block based on the content rate of a signal (in this embodiment, a low contrast signal) that affects the decrease in reliability. A low contrast block is an example of a "bad block" according to the technology of the present disclosure. In addition, in this modification, the reliability determination unit 58 determines the reliability based on an evaluation value related to the contrast.

[0132] Fig. 26 shows a reliability determination process according to the third modified example. Of steps S120B to S125B shown in Fig. 26, only steps S121B and S122B differ from steps S120 to S125 shown in Fig. 18. In step S121B, the reliability determination unit 58 determines whether each of the blocks BL included in the selected small area SA is a low-contrast block. In step S122B, the reliability determination unit 58 determines whether the number of low-contrast blocks included in the selected small area SA is equal to or greater than a second threshold. The other processes are the same as those in the above embodiment.

[0133] In this modification, in step S121B, the reliability determination unit 58 determines the contrast based on the luminance distribution of the image in the block BL. For example, the reliability determination unit 58 determines the difference between the maximum and minimum luminance values ​​of the image as the evaluation value of the contrast. Note that the reliability determination unit 58 may also determine the standard deviation of the luminance distribution of the image as the evaluation value of the contrast. When the luminance distribution of the image is used to determine the contrast in this way, it is preferable that the image has already undergone sensitivity correction.

[0134] The reliability determination unit 58 may also determine the contrast based on the shape of the correlation curve obtained by the correlation calculation process. In this case, for example, the reliability determination unit 58 may use the difference between the maximum and minimum values ​​of the correlation curve, the standard deviation, the number of minimum values, or the number of maximum values, or a combination of these, as the evaluation value of the contrast. Note that even when the S / N ratio is high, such as when an image contains a repeated pattern of vertical lines, multiple minimum values ​​may appear in the correlation curve, so it is preferable to use the difference between the maximum and minimum values ​​of the correlation curve in addition to the number of minimum values ​​of the correlation curve as the evaluation value of the contrast. When the contrast is low, the number of minimum values ​​increases and the difference between the maximum and minimum values ​​decreases.

[0135] Note that the reliability determination process according to this modification may be modified in the same manner as the first modification shown in Fig. 20. That is, when the number of low contrast blocks is less than the second threshold, the closest value may be set as the defocus amount DFa, and when the number of low contrast blocks is equal to or greater than the second threshold, the median value may be set as the defocus amount DFa.

[0136] [Fourth Variation] In the above embodiment, the factor related to the reliability is saturation, in the second modification, the factor related to the reliability is frequency component, and in the third modification, the factor related to the reliability is contrast. In this modification, the factors related to the reliability are saturation, frequency component, and contrast. Specifically, as shown in FIG. 27, when the block BL corresponds to at least one of the saturated block described in the above embodiment, the high-frequency block described in the second modification, and the low-contrast block described in the third modification, the reliability determination unit 58 determines the block BL as a bad block.

[0137] Fig. 28 shows a reliability judgment process according to the fourth modified example. Of steps S120C to S125C shown in Fig. 28, only steps S121C and S122C differ from steps S120 to S125 shown in Fig. 18. In step S121C, the reliability judgment unit 58 judges whether or not each of the blocks BL included in the selected small area SA is a bad block. In step S122C, the reliability judgment unit 58 judges whether or not the number of bad blocks included in the selected small area SA is equal to or greater than a second threshold value. The other processes are the same as those in the above embodiment.

[0138] In this modification, the selection unit 59 selects the closest value or the median value of the multiple defocus amounts DFa as the driving defocus amount DFd based on the number of saturated blocks, the number of high frequency blocks, the number of low contrast blocks, and the number of bad blocks. For example, when at least one of the number of saturated blocks, the number of high frequency blocks, the number of low contrast blocks, and the number of bad blocks is equal to or greater than the first threshold value, the selection unit 59 selects the median value as the driving defocus amount DFd. When all of the number of saturated blocks, the number of high frequency blocks, the number of low contrast blocks, and the number of bad blocks are less than the first threshold value, the selection unit 59 selects the closest value as the driving defocus amount DFd.

[0139] Note that the reliability determination process according to this modification may be modified in the same manner as the first modification shown in Fig. 20. That is, when the number of bad blocks is less than the second threshold, the closest value may be set as the defocus amount DFa, and when the number of bad blocks is equal to or greater than the second threshold, the median value may be set as the defocus amount DFa.

[0140] In this modification, the reliability is determined using a plurality of factors, so that the focusing accuracy is further improved.

[0141] [Fifth Modification] When the exposure conditions during focus control are conditions that degrade the S / N ratio, such as low brightness, it is preferable to select the median value from multiple defocus amounts DFb obtained for each block BL (hereinafter referred to as the "inter-block median") rather than selecting the median value from multiple defocus amounts DFa obtained for each small area SA as in the above embodiment (hereinafter referred to as the "inter-area median").

[0142] 29 shows the median selection process (step S15) according to the fifth modified example. In step S15, the selection unit 59 first acquires the exposure conditions (aperture value, shutter speed, ISO sensitivity, etc.) during focus control (step S150). Next, the selection unit 59 calculates the variation σ of the expected focus position based on the acquired exposure conditions (step S151). The variation σ is a standard deviation and is expressed in units where the depth of field is 1.

[0143] Next, the selection unit 59 determines whether the number of samples for obtaining the median is sufficient based on the variation σ (step S152). The number of samples is the number of defocus amounts DFb obtained from one small area SA. In other words, the number of samples is the product of the number of blocks BL included in one small area SA and the number of filters FL used in the above-mentioned filtering process.

[0144] If σ>3, the selection unit 59 determines that the number of samples is sufficient (step S152: YES), and executes the above-mentioned inter-area median (step S153). On the other hand, if σ≦3, the selection unit 59 determines that the number of samples is insufficient (step S152: NO), and executes the above-mentioned inter-block median (step S154).

[0145] The relationship between the variation (standard deviation) σ and the number of samples n will be described below. First, the error range E of the focus position is expressed by the following formula (1) when the reliability coefficient in the interval estimation is 95% and the standard error is SE. E = 1.96 × SE (1)

[0146] Since the standard error SE is σ / sqrt(n), the error range E is expressed by the following equation (2) using the variation σ and the number of samples n. E = 1.96 × σ / sqrt(n) (2)

[0147] On the condition that the error range E is less than the depth of field, the relationship between the variation σ and the number of samples n is expressed by the following formula (3). n>(1.96×σ) 2 (3)

[0148] According to the above formula (3), when the variation σ is three times the depth of field (i.e., σ=3), it is necessary to set n>34.6 to guarantee the reliability of the median value selected from the samples. When the criterion is set to σ=3 as above, it is preferable to set the number of samples n to about 35.

[0149] [About the median] In this disclosure, the median is not limited to the value that is strictly in the middle of the frequency distribution, but also includes values ​​that are near the middle.

[0150] [About the closest value] In the present disclosure, the closest value is not limited to the value on the NEAR side in the frequency distribution. For example, since an isolated value located on the NEAR side in the frequency distribution may be an abnormal value, the closest value may be obtained from a distribution other than such an isolated value.

[0151] For example, as shown in FIG. 30, when an outlier exists on the NEAR side in the frequency distribution and the distribution excluding the outlier is divided into a plurality of groups G, one value selected from the NEAR side group G may be determined as the closest value. In this case, for example, it is preferable to select the mode or average value of the distribution in the NEAR side group G as the closest value. It is preferable to determine a set of values ​​that exist at intervals of Y% or less of the depth of field as one group G. It is also preferable to exclude values ​​that are not Z% of the depth of field from the NEAR side of one group G in order to prevent one group G from becoming a large set. And when the groups G are defined as above, it is preferable to determine values ​​that do not belong to any group G as outliers.

[0152] [Other variations] In the above embodiment, the display control unit 53 causes the display 15 to display an image, but the display control unit 53 may cause the viewfinder 14 to display an image instead of the display 15 or in addition to the display 15. In this case, the focus control device may be configured to enable the user to specify the AF area RA via the eye-gaze input device.

[0153] The technology disclosed herein is not limited to digital cameras, but can also be applied to electronic devices such as smartphones and tablet terminals that have an imaging function.

[0154] In the above embodiment, the hardware structure of the control unit, with processor 40 being an example, may be any of the following various processors. The above various processors include a CPU, which is a general-purpose processor that functions by executing software (programs), as well as a processor such as an FPGA, whose circuit configuration can be changed after manufacture. FPGAs include dedicated electric circuits, which are processors having a circuit configuration designed specifically for executing specific processing, such as PLDs or ASICs.

[0155] The control unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple control units may be configured with one processor.

[0156] There are several possible examples of configuring multiple control units with one processor. A first example is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as clients and servers, and this processor functions as multiple control units. A second example is a form in which a processor is used that realizes the functions of the entire system including multiple control units with one IC chip, as typified by system on chip (SOC). In this way, the control unit can be configured as a hardware structure using one or more of the various processors described above.

[0157] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0158] The program may also be stored in a non-transitory computer-readable storage medium.

[0159] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0160] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0161] The above explanation makes it possible to understand the following techniques. [Additional note 1] A focus control device that determines a defocus amount for driving a focus lens, A processor and a memory, The processor, Calculating the number of areas having low reliability of the defocus amount obtained from a plurality of areas set in the imaging region; When the number is equal to or greater than a first threshold value, the defocus amount for driving the focus lens is determined based on a plurality of indices related to the defocus amounts. Focus control device. [Additional note 2] The index is an index based on a frequency of a plurality of the defocus amounts. Item 1. A focus control device according to claim 1. [Additional note 3] The processor, determining a median value of the plurality of defocus amounts as the defocus amount for driving the focus lens; 3. The focus control device according to claim 1 or 2. [Additional note 4] The processor, If the number is less than a first threshold, the defocus amount for driving the focus lens is determined based on a distance to a subject corresponding to the defocus amount. Item 3. A focus control device according to any one of items 1 to 3. [Additional note 5] The processor, When the number is less than a first threshold value, the closest value of the plurality of defocus amounts is determined as the defocus amount for driving the focus lens. 5. A focus control device according to claim 1, wherein the focus control device is a focus control device having a focus control mechanism. [Additional note 6] The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether or not each of the blocks is a bad block based on a content rate of a signal that affects a decrease in the reliability, the signal being included in each of the blocks; If the number of the bad blocks is equal to or greater than a second threshold, the reliability is determined to be low. 6. A focus control device according to claim 1, wherein the focus control device is a focus control device having a focus control mechanism. [Additional note 7] The signal that affects the decrease in reliability is a signal that exceeds the performance limit of the image sensor. 7. The focus control device according to claim 6. [Additional note 8] the confidence level is related to a saturation level of a plurality of pixels included in the area; The processor determines the confidence level based on the saturation level. 6. A focus control device according to claim 1, wherein the focus control device is a focus control device having a focus control mechanism. [Additional note 9] The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether each of the blocks is a saturated block based on a content rate of the saturated pixels; If the number of saturated blocks is equal to or greater than a second threshold, the reliability is determined to be low. 9. The focus control device according to claim 8. [Additional Note 10] The processor, determining the area in which the number of saturated blocks is equal to or greater than the second threshold as a first area; a median value of defocus amounts corresponding to the plurality of blocks included in the first area is set as a defocus amount of the first area. 10. The focus control device according to claim 9. [Additional Note 11] The processor, determining the area in which the number of saturated blocks is less than the second threshold as a second area; a closest value of the defocus amounts corresponding to the plurality of blocks included in the second area is set as the defocus amount of the second area; Item 11. The focus control device according to item 10. [Additional Note 12] The processor, performing a plurality of filtering processes with different frequency characteristics on a plurality of pixels included in the area; determining the reliability based on evaluation values ​​obtained by a plurality of filtering processes; 6. A focus control device according to claim 1, wherein the focus control device is a focus control device having a focus control mechanism. [Additional Note 13] The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether or not each of the blocks is a high frequency block based on the evaluation value; When the number of the high-frequency blocks is equal to or greater than a second threshold, the reliability is determined to be low. Item 13. The focus control device according to item 12. [Additional Note 14] The processor, The area in which the number of high-frequency blocks is equal to or greater than the second threshold is determined to be a first area; a median value of defocus amounts corresponding to the plurality of blocks included in the first area is set as a defocus amount of the first area. Item 14. The focus control device according to item 13. [Additional Note 15] The processor, The area in which the number of high-frequency blocks is less than the second threshold is determined to be a second area; a closest value of the defocus amounts corresponding to the plurality of blocks included in the second area is set as the defocus amount of the second area; Item 15. The focus control device according to item 14. [Additional Note 16] The processor, calculating a first defocus amount by performing a correlation calculation based on a first two-dimensional image generated by performing a first filtering process on a two-dimensional image represented by a plurality of pixels included in the block; calculating a second defocus amount by performing a correlation calculation based on a first one-dimensional image generated by performing the first filtering process on a one-dimensional image obtained by vertically adding the two-dimensional images; calculating a third defocus amount by performing a correlation calculation based on a second two-dimensional image generated by performing a second filtering process on the two-dimensional image, the second filtering process having a frequency characteristic different from that of the first filtering process; calculating a fourth defocus amount by performing a correlation calculation based on a second one-dimensional image generated by performing the second filtering process on the one-dimensional image; determining that the block is the high frequency block when the difference between the first defocus amount and the third defocus amount is larger than both the difference between the first defocus amount and the second defocus amount and the difference between the third defocus amount and the fourth defocus amount; 16. A focus control device according to any one of claims 13 to 15. [Additional Note 17] The confidence level is related to a contrast of a plurality of pixels included in the area; The processor determines the reliability based on an evaluation value related to the contrast. 6. A focus control device according to claim 1, wherein the focus control device is a focus control device having a focus control mechanism. [Additional Note 18] The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether or not each of the blocks is a low contrast block based on the evaluation value; If the number of the low contrast blocks is equal to or greater than a second threshold, the reliability is determined to be low. Item 18. The focus control device according to item 17. [Additional Note 19] The processor, determining the area in which the number of the low contrast blocks is equal to or greater than the second threshold as a first area; a median value of defocus amounts corresponding to the plurality of blocks included in the first area is set as a defocus amount of the first area. Item 19. The focus control device according to item 18. [Additional Note 20] The processor, determining the area in which the number of the low contrast blocks is less than the second threshold as a second area; a closest value of the defocus amounts corresponding to the plurality of blocks included in the second area is set as the defocus amount of the second area; 20. The focus control device according to claim 19. [Additional Note 21] The processor, determining whether each of the blocks is a low contrast block based on a luminance distribution of the plurality of pixels included in the block or a shape of a correlation curve obtained by a correlation calculation based on the plurality of pixels included in the block; 20. The focus control device according to claim 18 or 19. [Additional Note 22] A focus control device according to any one of claims 1 to 21, An imaging element; An imaging device comprising: [Explanation of symbols]

[0162] 10. Imaging device 11 Main unit 11A Camera side mount 11B Electrical contacts 12 Imaging lens 12A Lens side mount 12B Electrical contact 13 Operating device 14 Finder 15 Display 20 Image sensor 20A light receiving surface 20B Imaging area 30 Objective Lens 31 Focus Lens 32 Rear lens 34 Lens drive control unit 40 processors 42 Memory 43 Programs 50 Main control unit 51 Imaging control unit 52 Image processing section 53 Display control unit 54 Area setting section 55 Drive defocus amount acquisition unit 56 Phase difference pixel signal acquisition section 57 Correlation calculation section 58 Reliability judgment unit 59 Selection section A optical axis BL Block C1,C2 correlation curve CF Color Filter D Image signal DFa, DFb Defocus amount DFb1 1st defocus amount DFb2 Second defocus amount DFb3 3rd defocus amount DFb4 4th defocus amount DFd Defocus amount for driving Dp phase difference image signal EP,EP1,EP2 Exit pupil FL Filter G Group IP1 1st image IP2 2nd image LF Luminous Flux ML Micro Lens N Imaging pixels P1, P2 Phase difference detection pixels PA conversion pixel PB adjacent pixels PC converted pixels PD Photodiode RA AF Area RL line SA Small Area SF light shielding layer

Claims

1. A focus control device that determines a defocus amount for driving a focus lens, A processor and a memory, The processor, Calculating the number of areas having low reliability of the defocus amount obtained from a plurality of areas set in the imaging region; When the number is equal to or greater than a first threshold value, the defocus amount for driving the focus lens is determined based on a plurality of indices related to the defocus amounts. Focus control device.

2. The index is an index based on a frequency of a plurality of the defocus amounts. The focus control device according to claim 1 .

3. The processor, determining a median value of the plurality of defocus amounts as the defocus amount for driving the focus lens; The focus control device according to claim 1 .

4. The processor, If the number is less than a first threshold, the defocus amount for driving the focus lens is determined based on a distance to a subject corresponding to the defocus amount. The focus control device according to claim 1 .

5. The processor, When the number is less than a first threshold value, the closest value of the plurality of defocus amounts is determined as the defocus amount for driving the focus lens. The focus control device according to claim 1 .

6. The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether or not each of the blocks is a bad block based on a content rate of a signal that affects a decrease in the reliability, the signal being included in each of the blocks; If the number of the bad blocks is equal to or greater than a second threshold, the reliability is determined to be low. The focus control device according to claim 1 .

7. The signal that affects the decrease in reliability is a signal that exceeds the performance limit of the image sensor. The focus control device according to claim 6.

8. the confidence level is related to a saturation level of a plurality of pixels included in the area; The processor determines the confidence level based on the saturation level. The focus control device according to claim 1 .

9. The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether each of the blocks is a saturated block based on a content rate of the saturated pixels; If the number of saturated blocks is equal to or greater than a second threshold, the reliability is determined to be low. The focus control device according to claim 8 .

10. The processor, determining the area in which the number of saturated blocks is equal to or greater than the second threshold as a first area; a median value of defocus amounts corresponding to the plurality of blocks included in the first area is set as a defocus amount of the first area. The focus control device according to claim 9 .

11. The processor, determining the area in which the number of saturated blocks is less than the second threshold as a second area; a closest value of the defocus amounts corresponding to the plurality of blocks included in the second area is set as the defocus amount of the second area; The focus control device according to claim 10.

12. The processor, performing a plurality of filtering processes with different frequency characteristics on a plurality of pixels included in the area; determining the reliability based on evaluation values ​​obtained by a plurality of filtering processes; The focus control device according to claim 1 .

13. The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether or not each of the blocks is a high frequency block based on the evaluation value; When the number of the high-frequency blocks is equal to or greater than a second threshold, the reliability is determined to be low. The focus control device according to claim 12.

14. The processor, The area in which the number of the high-frequency blocks is equal to or greater than the second threshold is determined to be a first area; a median value of defocus amounts corresponding to the plurality of blocks included in the first area is set as a defocus amount of the first area. The focus control device according to claim 13.

15. The processor, The area in which the number of the high frequency blocks is less than the second threshold is determined to be a second area; a closest value of the defocus amounts corresponding to the plurality of blocks included in the second area is set as the defocus amount of the second area; The focus control device according to claim 14.

16. The processor, calculating a first defocus amount by performing a correlation calculation based on a first two-dimensional image generated by performing a first filtering process on a two-dimensional image represented by a plurality of pixels included in the block; calculating a second defocus amount by performing a correlation calculation based on a first one-dimensional image generated by performing the first filtering process on a one-dimensional image obtained by vertically adding the two-dimensional images; calculating a third defocus amount by performing a correlation calculation based on a second two-dimensional image generated by performing a second filtering process on the two-dimensional image, the second filtering process having a frequency characteristic different from that of the first filtering process; calculating a fourth defocus amount by performing a correlation calculation based on a second one-dimensional image generated by performing the second filtering process on the one-dimensional image; When a difference between the first defocus amount and the third defocus amount is larger than both a difference between the first defocus amount and the second defocus amount and a difference between the third defocus amount and the fourth defocus amount, the block is determined to be the high frequency block. The focus control device according to claim 13.

17. The confidence level is related to a contrast of a plurality of pixels included in the area; The processor determines the reliability based on an evaluation value related to the contrast. The focus control device according to claim 1 .

18. The processor, Dividing each of the plurality of areas into a plurality of blocks, and determining whether or not each of the blocks is a low contrast block based on the evaluation value; If the number of the low contrast blocks is equal to or greater than a second threshold, the reliability is determined to be low. The focus control device according to claim 17.

19. The processor, determining the area in which the number of the low contrast blocks is equal to or greater than the second threshold as a first area; a median value of defocus amounts corresponding to the plurality of blocks included in the first area is set as a defocus amount of the first area. The focus control device according to claim 18.

20. The processor, determining the area in which the number of the low contrast blocks is less than the second threshold as a second area; a closest value of the defocus amounts corresponding to the plurality of blocks included in the second area is set as the defocus amount of the second area; The focus control device according to claim 19.

21. The processor, determining whether each of the blocks is a low contrast block based on a luminance distribution of the plurality of pixels included in the block or a shape of a correlation curve obtained by a correlation calculation based on the plurality of pixels included in the block; The focus control device according to claim 18.

22. A focus control device according to any one of claims 1 to 21, An imaging element; An imaging device comprising:

23. A focus control method for determining a defocus amount for driving a focus lens, comprising: The processor: determining the number of areas having low reliability of the defocus amount obtained from a plurality of areas set in the imaging region; determining the defocus amount for driving the focus lens based on a plurality of indices related to the defocus amounts when the number is equal to or greater than a first threshold value; A focus control method comprising:

24. A program for causing a processor to execute a process for determining a defocus amount for driving a focus lens, determining the number of areas having low reliability of the defocus amount obtained from a plurality of areas set in the imaging region; determining the defocus amount for driving the focus lens based on a plurality of indices related to the defocus amounts when the number is equal to or greater than a first threshold value; A program that causes a processor to execute a process including the steps of:

Images