Automatic focus detection method and device, photographing device, and storage medium

Abstract

The present disclosure relates to an automatic focus detection method and device, an imaging device, and a storage medium. The automatic focus detection method includes a step of dividing an image of a pair of subject image signals obtained by pupil division into regions, a step of performing a correlation calculation in parallel between each divided region of one of the images of the pair of subject image signals and a corresponding divided region of the other image, a step of calculating a defocus amount for each divided region in parallel based on a result of the correlation calculation, a step of grouping divided regions corresponding to subjects at the same shooting distance into the same group based on the defocus amount for each divided region, a step of calculating a reliability of the result of the correlation calculation in parallel, and a step of excluding a divided region that satisfies a predetermined condition from the group based on the calculated reliability for each group. [Effect] Even when there are a plurality of subjects with different shooting distances, it is possible to selectively focus on any one of the plurality of subjects with different shooting distances with high accuracy and high speed.

Description

[Technical field]

[0001] The present disclosure relates to the field of digital imaging, and in particular to an auto focus detection method and apparatus, an imaging device, and a non-volatile computer readable storage medium. [Background technology]

[0002] In a split-pupil phase difference autofocus detection device mounted on a camera, the defocus amount is calculated from the phase difference (relative position change) between a pair of subject images split into pupils.When there are multiple subjects with different shooting distances, the split-pupil phase difference autofocus detection device outputs a value that contains a mixture of multiple phase shift amounts, which causes a problem that it may not be possible to calculate the defocus amount of the subject with high accuracy.

[0003] To solve this problem, for example, a pupil-splitting phase difference autofocus detection device described in Patent Document 1 (JP 2021-148841 A) performs a correlation calculation on an image signal sequence output by a sensor, and if the correlation calculation result determines that it is reliable, calculates a defocus amount based on the correlation calculation result, and if the correlation calculation result determines that it is unreliable, compares the reliability of the image signal at the start side of the image signal sequence on which the correlation calculation was performed with the reliability of the image signal at the end side, and narrows down the image signal sequence by excluding at least one of the image signals at the start and end sides from the image signal sequence based on the comparison result, and repeats the narrowing down of the image signal sequence and the correlation calculation for the range of the narrowed down image signal sequence until it is determined to be reliable.

[0004] In Patent Document 1, the defocus amount of the subject is calculated with high accuracy by repeating narrowing down and correlation calculation for the range of the image signal sequence after narrowing down until it is determined to be reliable, and excluding signals of subjects with different shooting distances. Therefore, it is necessary to repeatedly execute the correlation calculation, and it takes time to focus on the subject. Summary of the Invention

[0005] In consideration of the above-described situation, the present disclosure proposes an automatic focus detection method and device, an imaging device, and a non-volatile computer-readable storage medium that can focus on one of multiple subjects with different shooting distances with high accuracy and high speed, even when multiple subjects with different shooting distances are present.

[0006] According to one aspect of the present disclosure, there is provided an autofocus detection method including: an area division step of dividing an image of a pair of subject image signals obtained by pupil division into areas; a correlation calculation step of performing a correlation calculation between each divided area of ​​one of the images of the pair of subject image signals and a corresponding divided area of ​​the other image; a defocus amount calculation step of calculating a defocus amount for each divided area based on the correlation calculation result obtained in the correlation calculation step; a grouping processing step of grouping the divided areas corresponding to subjects at the same shooting distance into the same group based on the defocus amount for each divided area; a reliability calculation step of calculating a reliability of the correlation calculation result obtained in the correlation calculation step; and an exclusion processing step of excluding, for each group, divided areas that satisfy a predetermined condition from the group based on the reliability calculated in the reliability calculation step.

[0007] According to another aspect of the present disclosure, there is provided an autofocus detection device including: a region division unit that divides an image of a pair of subject image signals into regions after pupil division; a correlation calculation unit that performs correlation calculation between each divided region of one of the images of the pair of subject image signals and a corresponding divided region of the other image; a defocus amount calculation unit that calculates a defocus amount for each divided region based on a correlation calculation result by the correlation calculation unit; a grouping processing unit that groups the divided regions corresponding to subjects at the same shooting distance into the same group based on the defocus amount for each divided region; a reliability calculation unit that calculates a reliability of the correlation calculation result by the correlation calculation unit; and an exclusion processing unit that excludes, for each group, divided regions that satisfy a predetermined condition from the group based on the reliability calculated by the reliability calculation unit.

[0008] According to another aspect of the present disclosure, there is provided an imaging device including a processor and a memory unit for storing instructions executable by the processor, the processor being configured to perform the above-mentioned auto focus detection method when it executes the instructions stored in the memory unit.

[0009] According to another aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a processor, causing the automatic focus detection method described above to be implemented.

[0010] According to another aspect of the present disclosure, a computer program product is provided that includes computer readable code or a non-volatile computer readable storage medium storing computer readable code, which, when executed in a processor of an electronic device, causes the processor of the electronic device to perform the above-mentioned automatic focus detection method.

[0011] According to the autofocus detection method and device, the photographing device, and the non-volatile computer-readable storage medium of each aspect of the present disclosure, in a device capable of splitting a light beam from a subject and projecting it onto different sensors or different sensor regions, and calculating a defocus amount from a phase difference between a pair of subject images, a pair of pupil-split subject images output from a sensor are grouped based on a defocus amount calculated by a correlation calculation result for each divided region, and image information of each divided region is compiled for each group as image information of subjects at the same shooting distance, and disturbance factors are eliminated, so that autofocus detection of the subject can be performed with high accuracy and high speed even when there are multiple subjects with different shooting distances. In addition, by compiling image information of each divided region for each group as image information of subjects at the same shooting distance, it is possible to selectively focus on any subject, such as a subject with a large area at the center of the angle of view, a subject with the closest shooting distance, or a subject with the highest reliability, among multiple subjects with different shooting distances.

[0012] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0013] The drawings, which form a part of the specification, illustrate exemplary embodiments, features and aspects of the present disclosure and, together with the specification, are used to interpret the principles of the present disclosure. [Figure 1] FIG. 1 shows a flow chart of an auto focus detection method according to one embodiment of the present disclosure. [Diagram 2] FIG. 2 shows a flowchart of an auto focus detection method according to another embodiment of the present disclosure. [Diagram 3] FIG. 3 shows a state in which a pair of pupil-divided subject images output from an image sensor are divided into regions. [Figure 4] FIG. 4 shows an example of a similarity insufficiency determination related to the reliability. [Diagram 5] FIG. 5 shows an example of a contrast deficiency determination related to the confidence level. [Figure 6] FIGS. 6(a) and 6(b) show examples of the correspondence between the index and the reliability. [Figure 7] FIG. 7 shows an example of identifying outliers based on the defocus amount. [Figure 8] FIG. 8 shows an example of parallel processing of correlation calculation, reliability calculation, defocus amount calculation, and calculation of average reliability and average defocus amount. [Figure 9] FIG. 9 is a block diagram of an auto focus detection apparatus according to an exemplary embodiment. [Figure 10] FIG. 10 shows an example of the configuration of an image capturing apparatus to which an automatic focus detection device according to the embodiment is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Various exemplary embodiments, features, and aspects of the present disclosure will now be described in detail with reference to the drawings, in which like reference numerals represent elements of the same or similar function. While illustrating various aspects of the embodiments in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0015] The word "exemplary" herein means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or superior over other embodiments.

[0016] In addition, in order to better explain the present disclosure, many specific details are shown in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can be similarly implemented without any specific details. In some embodiments, in order to emphasize the gist of the present disclosure, detailed descriptions are not provided for methods, means, elements and circuits that are well known to those skilled in the art.

[0017] In related technology, a method has been proposed as an automatic focus detection method in which signals from subjects with different shooting distances are excluded by repeatedly narrowing down the aperture and performing correlation calculations on the range of the image signal string after narrowing down the aperture until it is determined to be reliable, but this method has the problem that it takes a long time to focus on the subject. Therefore, in the present disclosure, the above problem is solved by grouping a pair of subject images obtained by pupil division based on the defocus amount calculated by the correlation calculation results for each divided area, consolidating information for multiple subjects with different shooting distances, and eliminating disturbance factors.

[0018] In this disclosure, as shown in FIG. 1, an automatic focus detection method is proposed which is executed in equipment products that perform camera focus adjustment, such as mobile phones, industrial cameras, security cameras, automotive cameras, camera modules, and the like.

[0019] In the embodiment shown in FIG. 1, the auto focus detection method may include the following steps.

[0020] Step S110 (area division step): The images of the pair of pupil-divided object image signals are each divided into areas.

[0021] In this embodiment, the light beam from the subject is pupil-split and projected onto different sensors or different sensor regions, and a pair of subject image signals output from the different sensors or different sensor regions are each divided into regions. The number of divided regions can be set arbitrarily. It is desirable to make the divided regions as small as possible so that signals containing a mixture of phase shift amounts due to a plurality of subjects with different shooting distances can be easily eliminated.

[0022] Step S120 (correlation calculation step): A correlation calculation is performed between each divided area of ​​one of the images of the pair of subject image signals and the corresponding divided area of ​​the other image.

[0023] In this embodiment, for each divided region of the images of a pair of subject image signals, a correlation value is obtained by performing a correlation calculation between one divided region of the images of the pair of subject image signals and the corresponding divided region of the other image.

[0024] For each divided region of a pair of pupil-divided subject image signals, it is preferable to execute a calculation to obtain the difference for each pixel data and the sum of their absolute values ​​while shifting by a predetermined amount within a certain shift range, and execute a correlation calculation from the sum of the absolute values ​​of the differences obtained by shifting by the predetermined amount to obtain a phase difference. In other words, it is preferable to fix a pixel data group of one divided region of the pair of subject image signals, and shift a pixel data group of the other corresponding divided region by pixel to obtain a phase difference. Then, it is preferable to calculate a movement direction and a movement amount of the optical system of the focus adjustment device based on the obtained phase difference.

[0025] In one possible implementation, the correlation calculation is performed in parallel for each divided region of the image of the pair of object image signals, which can further reduce the calculation time.

[0026] Step S130 (defocus amount calculation step): The defocus amount for each divided region is calculated based on the correlation calculation result obtained in the correlation calculation step.

[0027] In this embodiment, the defocus amount for each divided region is calculated based on the correlation calculation result for each divided region.

[0028] It is preferable to estimate sub-pixels obtained by further dividing a pixel, identify the point at which the correlation value is maximum on a sub-pixel basis, and calculate the defocus amount.

[0029] In one possible implementation, the calculation of the defocus amount based on the correlation calculation result is performed in parallel for each divided region of the image of the pair of object image signals, which can further reduce the calculation time.

[0030] Step S140 (grouping process step): Based on the defocus amount for each divided region, the divided regions corresponding to subjects at the same shooting distance are grouped into the same group.

[0031] In this embodiment, based on the calculated defocus amount for each divided region, divided regions corresponding to subjects at the same shooting distance are grouped together, i.e., each of the image information of the divided regions is grouped into image information of subjects having a large area at the center of the angle of view or the closest subject.

[0032] Step S150 (reliability calculation step): The reliability of the correlation calculation result obtained in the correlation calculation step is calculated.

[0033] In one possible implementation, the reliability of the correlation calculation result is calculated in parallel for each divided region of the image of a pair of object image signals, which can further reduce the calculation time.

[0034] It is preferable to calculate the reliability from a plurality of elements such as contrast and similarity of the divided regions of the image of a pair of pupil-divided subject image signals. In one possible implementation form, the reliability is calculated based on at least one of a saturated pixel threshold, a deficient pixel threshold, contrast, and a degree of matching. It is also preferable to set the highest reliability to 1.0 and the lowest reliability to 0.0.

[0035] Step S160 (exclusion processing step): For each group, a divided area that satisfies a predetermined condition is excluded from the group based on the reliability calculated in the reliability calculation step.

[0036] In one possible implementation, a threshold is set for each group based on the reliability calculated in the reliability calculation step, and divided regions with a reliability equal to or lower than the threshold are excluded from the group. Also, divided regions with an outlier in the defocus amount calculated in the defocus amount calculation step may be excluded from the group.

[0037] Thus, according to the present embodiment, in an apparatus capable of splitting a light beam from a subject and projecting it onto different sensors or different sensor regions, and calculating a defocus amount from a phase difference between a pair of subject images, a pair of subject images split into pupils and output from a sensor are grouped based on a defocus amount calculated by a correlation calculation result for each divided region, and image information of each divided region is compiled for each group as image information of subjects at the same shooting distance, and disturbance factors are eliminated, so that even when there are multiple subjects with different shooting distances, it is possible to perform automatic focus detection of the subject with high accuracy and high speed. In addition, by compiling image information of each divided region for each group as image information of subjects at the same shooting distance, it is possible to selectively focus on any subject, such as a subject with a large area at the center of the angle of view, a subject with the closest shooting distance, or a subject with the highest reliability, among multiple subjects with different shooting distances.

[0038] 2 shows a flowchart of an auto focus detection method according to another embodiment of the present disclosure. As shown in FIG. 2, the auto focus detection method may include the following steps.

[0039] Step S210: The images of the pair of pupil-divided object image signals are each divided into regions.

[0040] Each of the images of the pair of pupil-divided subject image signals is divided into regions in the same manner. That is, the divided regions of the images of the pair of pupil-divided subject image signals have the same size and shape. FIG. 3 shows a state in which the images of the pair of pupil-divided subject images output from the image sensor are divided into regions. In FIG. 3, the images are divided into a plurality of rectangular regions, but this is not limited thereto. Regions of other shapes, such as square regions, may also be used. Correlation calculation, reliability calculation, and defocus amount calculation are performed for each region of the divided subject images. In addition, it is desirable to make the divided regions as small as possible so that image signals in which phase shift amounts due to a plurality of subjects with different shooting distances are mixed are easily eliminated.

[0041] Step S220: A correlation calculation is performed for the divided regions of the image of the pair of object image signals.

[0042] The calculation means for the correlation calculation can calculate the correlation value by a known method. For example, there is a method of performing a calculation to obtain the difference between each pixel data of a pair of pupil-divided subject images and the sum of their absolute values ​​while shifting by a predetermined amount in pixel units, and then performing a correlation calculation from the sum of the absolute values ​​of the differences obtained by shifting by the predetermined amount to obtain the phase difference. As an example, when the pupil-divided right-eye image is R, the pupil-divided left-eye image is L, the pixel position is n, and the shift amount is i, the formula for obtaining the correlation value is as follows.

number

[0043] In addition, because correlation calculations can be performed for each divided region, parallel calculations (using multiple CPU cores) are possible, as shown in Figure 8. This makes it possible to further reduce calculation time.

[0044] Step S230: The defocus amount for each divided region is calculated based on the correlation calculation result.

[0045] The defocus amount calculation means performs sub-pixel estimation using a spatial structure on the correlation calculation result (phase value) for each divided region using a known method to calculate the defocus amount. Sub-pixel estimation is performed using a broken line approximation or parabolic approximation method using gradient information based on correlation values ​​before and after the shift amount of the minimum correlation value calculated by the correlation calculation. As an example, if the shift amount of the minimum correlation value is min, the formula for the parabolic approximation method is as follows:

number

[0046] Since the defocus amount can be calculated for each divided region, parallel calculation (using multiple CPU cores) is possible as shown in Figure 8. This makes it possible to further reduce the calculation time.

[0047] Step S240: Based on the defocus amount for each divided region, divided regions corresponding to subjects at the same shooting distance are grouped into the same group.

[0048] The calculation means for the grouping process can perform grouping by clustering using the k-means method, for example, for the defocus amount for each divided region, using a known method.

[0049] Step S250: The reliability of the correlation calculation result is calculated.

[0050] The reliability calculation means may, for example, perform saturation and insufficiency judgments, similarity insufficiency judgments, and contrast insufficiency judgments for each pixel of a pair of pupil-split subject images to calculate the reliability.

[0051] As an index related to reliability, a saturated pixel threshold and a deficient pixel threshold are used for determining whether the pixel value of each pixel in each divided area is saturated or deficient. If even one of the pixels used in the correlation calculation exceeds the saturated pixel threshold or falls below the deficient pixel threshold, the reliability is set to 0.

[0052] As one index related to reliability, in the similarity insufficiency judgment, the minimum value of the correlation value is used as an index of reliability, as shown in Figure 4. If the minimum value is large, it is considered that the similarity is low, which indicates a low degree of match and low reliability.

[0053] As an index related to reliability, in one example of contrast insufficiency determination, the difference between the maximum and minimum correlation values ​​is used as an index of reliability, as shown in FIG. 5. The difference represents the range of the correlation value map. If the difference is small, it is considered that the contrast is low, indicating low reliability.

[0054] In another example of a method for determining insufficient contrast, as an index related to reliability, the sum of the absolute values ​​of the differences between adjacent pixels in the right-eye image or the left-eye image of a pair of pupil-split subject images is calculated, and if this value is small, it is considered that the contrast is low, indicating low reliability. If the pupil-split right-eye image is R and the pixel position is n, the formula for calculating the right-eye image contrast is as follows.

number

number

[0055] The result of the reliability calculation method described above can be used as the reliability as it is, but it is preferable to associate the index with the reliability, for example as shown in Fig. 6, with the highest reliability being 1.0 and the lowest being 0.0. The association may be linear as shown in Fig. 6(a) or nonlinear as shown in Fig. 6(b). The index in Fig. 6 refers to, for example, a value for determining insufficient similarity or insufficient contrast.

[0056] In addition, although the reliability of one of the reliability calculation methods described above can be used as the reliability, it is preferable to combine these multiple reliability calculation methods to calculate the final reliability. When the highest reliability of each of the reliability calculation methods described above is 1.0 and the lowest reliability is 0.0, the similarity insufficiency judgment and contrast insufficiency judgment may be weighted. An example of a formula for calculating the final reliability is shown below.

number

[0057] Since the reliability calculation can be performed for each divided region, parallel calculation (using multiple CPU cores) is possible as shown in Figure 8. This makes it possible to further reduce the calculation time. In addition, it is possible to calculate the reliability value after grouping based on the reliability calculated from the results of correlation calculation for each divided region.

[0058] Step S260: For each group, outliers and divided regions with low signal reliability are excluded from the group.

[0059] A threshold is set based on the reliability calculated in step S250, and if the reliability of a divided region is equal to or lower than the threshold, the divided region is excluded from the group. For example, the average reliability of the divided regions for each group may be calculated as a threshold, and if the reliability of a divided region is equal to or lower than the threshold of the group to which it belongs, the divided region may be excluded from the group.

[0060] Also, based on the defocus amount of each divided region for each group formed by clustering the divided regions, an average may be set as a threshold as shown in Fig. 7, and divided regions that are outliers with respect to the threshold of the group may be excluded from the group. This allows for more accurate auto focus detection of the subject.

[0061] Step S270: Selectively focus on the subject.

[0062] For each group, the reliability average and the defocus amount average of the divided regions excluding outliers and divided regions with low signal reliability may be calculated, and the subject may be selectively focused on based on the calculated reliability average and defocus amount average. Since the reliability average and the defocus amount average can be calculated for each group, parallel calculation (using multiple CPU cores) is possible as shown in FIG. 8. This allows the calculation time to be further reduced.

[0063] When the defocus amount calculated for each group is of interest, it is possible to focus on the subject at the closest shooting distance. When the reliability of each group is of interest, it is also possible to focus on the subject with the highest reliability. When the number and positions of elements (divided regions) of each group are of interest, it is also possible to focus on the subject with the largest area at the center of the angle of view.

[0064] Although the above description of the automatic focus detection method is based on an example in which the reliability calculation is performed after the defocus amount calculation, a person skilled in the art would understand that the present disclosure is not limited to this example. For example, the reliability calculation may be performed before the defocus amount calculation, or the reliability calculation and the defocus amount calculation may be performed simultaneously.

[0065] Thus, according to this embodiment, in an apparatus capable of splitting a light beam from a subject and projecting it onto different sensors or different sensor regions, and calculating a defocus amount from a phase difference between a pair of subject images, a pair of subject images split into pupils and output from a sensor are grouped based on a defocus amount calculated by a correlation calculation result for each divided region, and image information of each divided region is compiled for each group as image information of subjects at the same shooting distance, and outliers and low-reliability signals that do not belong to the group are eliminated. As a result, when there are multiple subjects with different shooting distances, signals containing a mixture of phase shift amounts due to multiple subjects are eliminated or divided into different groups, and regions with similar defocus amounts or high reliability are divided into the same group and averaged, thereby enabling high-precision autofocus detection. In addition, since calculations can be performed for each divided region in parallel, autofocus detection can be performed at high speed.

[0066] Fig. 9 is a block diagram of an automatic focus detection device according to an exemplary embodiment. As shown in Fig. 9, the automatic focus detection device may include an area division unit 901 that divides the image of a pair of subject image signals obtained by pupil division into areas, a correlation calculation unit 902 that performs correlation calculation between each divided area of ​​one of the images of the pair of subject image signals and the corresponding divided area of ​​the other image, a defocus amount calculation unit 903 that calculates a defocus amount for each divided area based on a correlation calculation result by the correlation calculation unit 902, a grouping processing unit 904 that groups the divided areas corresponding to subjects at the same shooting distance into the same group based on the defocus amount for each divided area, a reliability calculation unit 905 that calculates the reliability of the correlation calculation result by the correlation calculation unit 902, and an exclusion processing unit 906 that excludes divided areas that satisfy a predetermined condition from the group based on the reliability calculated by the reliability calculation unit 905 for each group.

[0067] In one possible implementation, the correlation calculation unit 902 performs correlation calculations in parallel for each divided region of the image of the pair of object image signals.

[0068] In one possible implementation form, the defocus amount calculation unit 903 calculates the defocus amount based on the correlation calculation result in parallel for each divided region of the image of the pair of object image signals.

[0069] In one possible implementation form, the reliability calculation unit 905 calculates the reliability of the correlation calculation result in parallel for each divided region of the image of the pair of object image signals.

[0070] In one possible implementation form, the exclusion processing unit 906 excludes, for each group, a divided area whose reliability is below a threshold value from the group, the threshold value being set based on the reliability calculated by the reliability calculation unit 905.

[0071] In one possible implementation form, the exclusion processor 906 excludes, for each group, a divided region in which the defocus amount is an outlier from the group.

[0072] In one possible implementation, the confidence calculation 905 calculates the confidence based on at least one of a saturated pixel threshold, a deficient pixel threshold, contrast, and a degree of matching.

[0073] Regarding the apparatus in the above-mentioned embodiment, the specific method by which each unit performs an operation is described in detail in the embodiment related to the method, and therefore a detailed description thereof will be omitted here.

[0074] 10 shows an example of the configuration of an image capturing apparatus to which an automatic focus detection device according to an embodiment of the present invention is applied. This image capturing apparatus includes an optical lens unit 1001, an image sensor unit 1002, an image data generating unit 1003, a lens driving unit 1004, and the above-mentioned automatic focus detection device 1005.

[0075] The optical lens unit 1001 focuses a subject image on the image sensor unit 1002. The optical lens unit 1001 also moves the focus based on a focus control signal from a lens driving unit 1004.

[0076] The lens driving unit 1004 generates a focus control signal from information output from the automatic focus detection device 1005 and outputs the signal to the optical lens unit 1001 .

[0077] The image sensor unit 1002 generates RAW data for each pixel according to the optical image of the subject focused by the optical lens unit 1001, and outputs the RAW data to an image data generation unit 1003. The image sensor unit 1002 also outputs a pair of pupil-split subject images to an autofocus detection device 1005.

[0078] An image data generating unit 1003 performs predetermined signal processing on the RAW data of each pixel to generate image data and output it to the outside.

[0079] As described above, the autofocus detection device 1005 includes the area division unit 901, the correlation calculation unit 902, the defocus amount calculation unit 903, the grouping processing unit 904, the reliability calculation unit 905, and the exclusion processing unit 906. The autofocus detection device 1005 outputs information for focusing on a subject selected based on a pair of pupil-divided subject image signals to the lens driving unit 1004. The autofocus detection device 1005 may be realized by hardware, software, or a combination of hardware and software.

[0080] According to this embodiment, in an apparatus capable of splitting a light beam from a subject and projecting it onto different sensors or different sensor regions, and calculating a defocus amount from a phase difference between a pair of subject images, a pair of subject images split into pupils and output from a sensor are grouped based on a defocus amount calculated by a correlation calculation result for each divided region, and image information of each divided region is compiled for each group as image information of subjects at the same shooting distance, and disturbance factors are eliminated, so that even when there are multiple subjects with different shooting distances, it is possible to perform automatic focus detection of the subject with high accuracy and high speed. In addition, by compiling image information of each divided region for each group as image information of subjects at the same shooting distance, it is possible to selectively focus on any subject, such as a subject with a large area at the center of the angle of view, a subject with the closest shooting distance, or a subject with the highest reliability, among multiple subjects with different shooting distances.

[0081] An embodiment of the present disclosure further provides a computer readable storage medium having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the above-mentioned automatic focus detection method. The computer readable storage medium may be a volatile or non-volatile computer readable storage medium.

[0082] An embodiment of the present disclosure further provides an electronic device including a processor and a memory unit for storing instructions executable by the processor, the processor being configured to perform the above-mentioned auto focus detection method when executing the instructions stored in the memory unit.

[0083] An embodiment of the present disclosure further provides a computer program product that includes computer readable code or a non-volatile computer readable storage medium storing computer readable code, which, when executed in a processor of an electronic device, causes the processor of the electronic device to perform the above-mentioned automatic focus detection method.

[0084] Although the embodiments of the present disclosure have been described above, the above description is merely illustrative and not exhaustive, and is not limited to the embodiments disclosed. Various modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms selected in this specification are intended to appropriately interpret the principles, practical applications, or improvements to existing technology of the embodiments, or to allow other skilled in the art to understand the embodiments disclosed in the text.

Claims

1. a region dividing step of dividing an image of a pair of pupil-divided object image signals into regions; a correlation calculation step of performing a correlation calculation between each divided area of ​​one of the images of the pair of subject image signals and a corresponding divided area of ​​the other image; a defocus amount calculation step of calculating a defocus amount for each divided region based on a correlation calculation result obtained in the correlation calculation step; a grouping process step of grouping the divided regions corresponding to subjects at the same shooting distance into the same group based on the defocus amount for each divided region; a reliability calculation step of calculating a reliability of a correlation calculation result obtained in the correlation calculation step; an exclusion processing step of excluding, for each group, divided areas that satisfy a predetermined condition based on the reliability calculated in the reliability calculation step, from the group.

2. 2. The autofocus detection method according to claim 1, wherein in said correlation calculation step, correlation calculations are performed in parallel for each divided region of the image of the pair of subject image signals.

3. 2. The autofocus detection method according to claim 1, wherein in the defocus amount calculation step, the calculation of the defocus amount based on the correlation calculation result is performed in parallel for each divided area of ​​the image of the pair of subject image signals.

4. 2. The autofocus detection method according to claim 1, wherein in the reliability calculation step, the reliability of the correlation calculation result is calculated in parallel for each divided region of the image of the pair of subject image signals.

5. In the exclusion process, for each group, a divided region having a reliability equal to or less than a threshold is excluded from the group; 2. The autofocus detection method according to claim 1, wherein the threshold value is set based on the reliability calculated in the reliability calculation step.

6. 2. The auto focus detection method according to claim 1, wherein the exclusion processing step further comprises excluding, for each group, a divided area in which the defocus amount is an outlier, from the group.

7. 2. The autofocus detection method according to claim 1, wherein in the reliability calculation step, the reliability is calculated based on at least one of a saturated pixel threshold, a deficient pixel threshold, a contrast, and a degree of coincidence.

8. an area division unit that divides an image of a pair of subject image signals obtained by pupil division into areas; a correlation calculation unit that performs correlation calculation between each divided area of ​​one of the images of the pair of subject image signals and a corresponding divided area of ​​the other image; a defocus amount calculation unit that calculates a defocus amount for each divided region based on a correlation calculation result by the correlation calculation unit; a grouping processing unit that groups the divided regions corresponding to subjects at the same shooting distance into the same group based on a defocus amount for each of the divided regions; a reliability calculation unit that calculates the reliability of a correlation calculation result by the correlation calculation unit; and an exclusion processing unit that excludes, for each group, divided areas that satisfy a predetermined condition based on the reliability calculated by the reliability calculation unit.

9. A processor; a memory unit for storing instructions executable by a processor; 8. An imaging apparatus, comprising: a processor configured to execute an automatic focus detection method according to claim 1 when the processor executes instructions stored in the memory unit.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, the computer program instructions, when executed by a processor, causing an automatic focus detection method according to any one of claims 1 to 7 to be realized.

Images