Focus adjustment device control method, focus adjustment device, and imaging device

Abstract

The present invention provides a focus adjustment device, a control method thereof, an imaging device, and a computer program. [Solution] A control method for a focus adjustment device includes dividing one of a first image and a second image into a plurality of regions and searching for feature regions in the plurality of regions, performing a correlation calculation for each of the searched feature regions to obtain a correlation value, separately calculating a first defocus amount for each of the plurality of feature regions, sorting the first defocus amounts in ascending or descending order to group the plurality of feature regions by their respective first defocus amounts to obtain one or more feature groups, calculating a second defocus amount for each feature group based on the first defocus amount for each feature region in each feature group, calculating a reliability for each feature group, and outputting the second defocus amount and the reliability for each feature group.

Description

[Technical Field]

[0001] The present invention relates to the field of digital imaging technology, and more particularly to a control method for a focusing device, a focusing device, and an imaging device. [Background technology]

[0002] Conventional focus adjustment devices calculate a defocus amount from the phase difference between a pair of pupil-split object images and then focus on a certain object, but a pupil-splitting phase-difference automatic focus adjustment device may output a mixture of phase differences when there are multiple objects at different shooting distances, making it impossible to calculate the defocus amount of the object with high accuracy. Furthermore, conventional technology attempts to calculate the defocus amount of the object with high accuracy by excluding signals from objects at different shooting distances, but this requires repeated correlation calculations, which takes a long time to achieve a result where the object is in focus. Summary of the Invention [Problem to be solved by the invention]

[0003] In order to solve the above-mentioned technical problems, the present invention provides a control method for a focus adjustment device, a focus adjustment device, and an imaging device. [Means for solving the problem]

[0004] In a first aspect of the present invention, a control method for a focus adjustment device is provided, which is applied to a focus adjustment device. The control method for a focus adjustment device includes: dividing one of a first image and a second image obtained by pupil-splitting light beams of a plurality of subjects into a plurality of regions, and searching for a feature region in the plurality of regions; performing a correlation calculation for each of the searched feature regions to obtain a correlation value; separately calculating a first defocus amount for each of the plurality of feature regions based on the correlation value for each of the plurality of feature regions; sorting the first defocus amount for each of the plurality of feature regions in ascending or descending order, and grouping the plurality of feature regions by their respective first defocus amounts to obtain one or more feature groups; The method includes calculating a second defocus amount for each feature group in the one or more feature groups based on a first defocus amount for each feature region in each feature group in the one or more feature groups, calculating a reliability for each feature group in the one or more feature groups, and outputting the second defocus amount and reliability for each feature group in the one or more feature groups, wherein the second defocus amount for each feature group in the one or more feature groups is distance information between an imaging element of a focus adjustment device and a focal point formed from a subject at one shooting distance corresponding to each feature group.

[0005] A second aspect of the present invention provides a focus adjustment device. The focus adjustment device includes a characteristic region search unit, a correlation value calculation unit, a defocus amount calculation unit, a grouping processing unit, a reliability calculation unit, and an operation output unit. The characteristic region search unit is used to divide one of a first image and a second image obtained by pupil-splitting light beams of a plurality of subjects into a plurality of regions and search for characteristic regions in the plurality of regions, based on the first and second images, the correlation value calculation unit is used to perform a correlation calculation for each of the plurality of searched characteristic regions to obtain a correlation value, the defocus amount calculation unit is used to calculate a first defocus amount for each of the plurality of characteristic regions separately based on the correlation value of each of the plurality of characteristic regions, and the grouping processing unit sorts the first defocus amount for each of the plurality of characteristic regions in ascending or descending order and groups the plurality of characteristic regions by their respective first defocus amounts to create one or more feature groups. The reliability calculation unit is used to calculate the reliability of each feature group in one or more feature groups, and the calculation output unit is used to calculate a second defocus amount for each feature group in one or more feature groups based on the first defocus amount of each feature region for each feature group in one or more feature groups, and to output the second defocus amount and reliability for each feature group in one or more feature groups, and the second defocus amount for each feature group in one or more feature groups is distance information between the imaging element of the focus adjustment device and a focus formed from a subject at one shooting distance corresponding to each feature group.

[0006] A third aspect of the present invention provides a focus adjustment device for use in an imaging device, the focus adjustment device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the above-described focus adjustment device control method.

[0007] In a fourth aspect of the present invention, there is provided an imaging device, the imaging device comprising the above-described focus adjustment device.

[0008] In a fifth aspect of the present invention, there is provided a computer-readable storage medium having executable instructions stored thereon, the executable instructions, when executed by a processor, performing the above-described method for controlling a focusing device.

[0009] The present invention provides a control method for a focus adjustment device, a focus adjustment device, and an imaging device. The control method for a focus adjustment device includes: dividing one of a first image and a second image obtained by pupil-splitting light beams of a plurality of subjects into a plurality of regions and searching for a feature region in the plurality of regions; performing a correlation operation for each of the searched feature regions to obtain a correlation value; calculating a first defocus amount for each of the plurality of feature regions separately based on the correlation value for each of the plurality of feature regions; sorting the first defocus amounts for each of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their respective first defocus amounts to obtain one or more feature groups; The present invention includes calculating a second defocus amount for each of the one or more feature groups based on a first defocus amount for each feature region for each of the one or more feature groups, calculating a reliability for each of the one or more feature groups, and outputting the second defocus amount and the reliability for each of the one or more feature groups, wherein the second defocus amount for each of the one or more feature groups is distance information between an imaging element of a focus adjustment device and a focal point formed by an object at one shooting distance corresponding to each of the feature groups. When objects at multiple different shooting distances are present, the present invention enables the focus adjustment device to focus on an object at one shooting distance with high precision, high speed, and selectively, thereby realizing autofocus in the focus adjustment device. [Brief explanation of the drawings]

[0010] In order to more clearly describe the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings described are some embodiments of the present application, and those skilled in the art can obtain other drawings from these drawings without creative efforts. [Figure 1] 10 is a flowchart of a control method for a focus adjustment device according to an embodiment of the present invention. [Figure 2] 1. FIG. 4 is a further sub-flowchart of step S11 in FIG. [Figure 3] 2A and 2B are schematic diagrams of a first image and a second image according to an embodiment of the present invention; [Figure 4] FIG. 1 is a schematic diagram illustrating sorting and grouping according to an embodiment of the present invention. [Figure 5] 1. FIG. 4 is a further sub-flowchart of step S14 in FIG. [Figure 6] FIG. 6 is a schematic diagram illustrating the sorting grouping of FIG. 5. [Figure 7] 1. FIG. 4 is a further sub-flowchart of step S14 in FIG. [Figure 8] FIG. 8 is a schematic diagram illustrating the sorting grouping of FIG. 7. [Figure 9] FIG. 1 is a schematic diagram illustrating the calculation of reliability according to an embodiment of the present invention. [Figure 10] FIG. 10 is a schematic diagram illustrating calculating a confidence score according to another embodiment of the present invention. [Figure 11] 1 is a block diagram showing an imaging apparatus according to an embodiment of the present invention; [Figure 12] FIG. 10 is a schematic diagram showing the structure of an imaging device according to another embodiment of the present invention. [Figure 13] 1 is a block diagram showing a focus adjustment device according to an embodiment of the present invention; [Figure 14] FIG. 10 is a block diagram showing a focus adjustment device according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments that can be obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0012] In the description of the present invention, the direction or positional relationship indicated by terms such as "upper" and "lower" is the direction or positional relationship shown in the drawings. In the description of the present invention, the term "connection" primarily refers to a physical structural connection unless otherwise specified, and may also include meanings such as direct connection and indirect connection when specified. Terms such as "first" and "second" used in the specification, claims, and drawings of the present invention are used merely to distinguish the objects being described and do not have any order or technical meaning. Furthermore, the term "comprises" and its variations are intended to cover a non-exclusive inclusion.

[0013] Referring to Fig. 1, Fig. 1 is a flowchart of a control method for a focus adjustment device according to an embodiment of the present invention. The control method for a focus adjustment device is applied to a focus adjustment device. The control method for a focus adjustment device includes the following steps S11 to S14.

[0014] S11: Based on a first image and a second image obtained by pupil-splitting light beams from a plurality of subjects, one of the first image and the second image is divided into a plurality of regions, and characteristic regions are searched for in the plurality of regions.

[0015] S12: A correlation calculation is performed for each of the plurality of retrieved feature regions to obtain a correlation value.

[0016] S13: Calculating a first defocus amount for each of the plurality of feature regions separately based on the correlation value for each of the plurality of feature regions.

[0017] S14: sorting the first defocus amounts of the plurality of feature regions in ascending or descending order, and grouping the plurality of feature regions by their respective first defocus amounts to obtain one or more feature groups;

[0018] In step S15, a second defocus amount is calculated for each of the one or more feature groups based on the first defocus amount of each feature region for each of the one or more feature groups.

[0019] S16, calculating the reliability of each feature group in one or more feature groups.

[0020] S17: Outputting a second defocus amount and reliability for each feature group in one or more feature groups, where the second defocus amount for each feature group in one or more feature groups is distance information between the imaging element of the focus adjustment device and a focal point formed from a subject at one shooting distance corresponding to each feature group.

[0021] The control method for a focus adjustment device provided by the present invention causes the focus adjustment device to output a second defocus amount and reliability for an object at each of a plurality of different shooting distances. When objects at a plurality of different shooting distances are present, the focus adjustment device can be made to focus selectively on an object at one shooting distance with high precision, high speed, and realize automatic focusing in the focus adjustment device, improving the focusing precision and focusing speed of the focus adjustment device. Furthermore, by calculating reliability to represent the reliability of each feature group, the focus adjustment device can be made to selectively focus on an object with a high reliability.

[0022] The specific process is described below. Light beams from multiple subjects are pupil-split to obtain a pair of subject images, i.e., a first image and a second image. Based on the first and second images, the first or second image is divided into multiple regions, and feature regions are searched for in the multiple regions based on certain conditions. Multiple feature regions can be searched for in the multiple regions, and a correlation calculation is performed for each of the searched feature regions to obtain a correlation value for each feature region. A first defocus amount for each feature region is calculated separately based on the correlation values ​​for each feature region. The first defocus amount for each feature region is sorted in ascending or descending order, and the multiple feature regions are grouped based on the first defocus amount for each feature region to obtain one or more feature groups. Each feature group in the one or more feature groups corresponds to a subject at one of multiple different shooting distances. Furthermore, information for each feature group can be summarized as information for a corresponding subject at one shooting distance. A second defocus amount for each feature group is calculated based on the first defocus amount for each feature region in each feature group, and a reliability for each feature group is calculated. The focus adjustment device is caused to output a second defocus amount and reliability for each feature group in one or more feature groups. Furthermore, distance information between the imaging element of the focus adjustment device and a focal point corresponding to an object at each shooting distance is confirmed based on the second defocus amount. The reliability of each feature group is evaluated based on the level of reliability. The first defocus amount is a distance value between the imaging element of the focus adjustment device and a focal point formed from a certain feature region, and the second defocus amount is a distance value between the imaging element of the focus adjustment device and a focal point formed from an object at one shooting distance corresponding to each feature group.

[0023] In some embodiments, when there are multiple objects at different shooting distances, the user can operate and control the focus adjustment device to select and focus on one of the objects at the selected shooting distance, and the focus adjustment device finds a corresponding second defocus amount based on the object at the selected shooting distance, and controls the lens in the focus adjustment device to move based on the second defocus amount, so that the imaging element of the focus adjustment device is positioned exactly at the focus formed by the object at that shooting distance, and the object at the selected shooting distance is brought into focus.

[0024] The subjects at a plurality of different shooting distances include a subject at the closest shooting distance, a subject at an intermediate distance among the subjects at a plurality of different shooting distances, and the like.

[0025] As can be appreciated, the imaging element can be, but is not limited to, an optical sensor or the like.

[0026] In some embodiments, the second defocus amount is an average value of the first defocus amounts of the plurality of feature regions in each feature group. When there are a plurality of objects at different shooting distances, the average value of the first defocus amounts of the plurality of feature regions in each feature group is used as the second defocus amount to indicate distance information between the imaging element of the focus adjustment device and the focal points formed by the objects at each shooting distance, thereby enabling high-precision focusing on the objects at each shooting distance.

[0027] 2 and 3, Fig. 2 is a further sub-flowchart of step S11 in Fig. 1, and Fig. 3 is a schematic diagram of a first image and a second image according to an embodiment of the present invention. Step S11, dividing one of the first image and the second image into a plurality of regions based on the first image and the second image obtained by pupil-splitting a plurality of object light beams, and searching for a feature region in the plurality of regions, includes the following steps S111 to S113.

[0028] S111: based on a first image and a second image obtained by pupil-splitting a plurality of light beams of a subject, one of the first image and the second image is divided into a plurality of regions;

[0029] In step S112, the feature amount for each local region within the divided regions is compared with a preset feature amount threshold value.

[0030] In step S113, if the feature amount of one local region is equal to or greater than the preset feature amount threshold, the local region is determined to be a feature region.

[0031] In step S114, if the feature amount of one local region is less than the preset feature amount threshold, the local region is not determined as a feature region.

[0032] Therefore, by selecting a local area whose feature amount is greater than a preset feature amount threshold and setting it as a feature area, it is possible to avoid the influence of noise on subsequent steps to some extent and improve the accuracy of subsequent focusing.

[0033] Specifically, the feature values ​​for each local region in the divided regions are calculated, and the feature values ​​for each local region are separately compared with a preset feature threshold value to search for zero or more feature regions within each region.

[0034] In some embodiments, the plurality of feature regions are found by searching for feature regions directly in the entire region of one of the first image and the second image without dividing one of the first image and the second image into a plurality of regions.

[0035] The preset feature threshold is a feature value that is not affected by noise.

[0036] In some embodiments, the feature is contrast, and by selecting a region where the contrast is equal to or greater than a preset feature threshold as a feature region, focusing accuracy can be improved.

[0037] As shown in Fig. 3, the first and second images in Fig. 3 are a pair of subject images obtained by pupil division. In Fig. 3, the star-shaped areas (only a portion is shown) are multiple feature regions searched for in the first image. Each feature region is made as small as possible, because making the feature region as small as possible can minimize the possibility of image signals from subjects at multiple different shooting distances being mixed in that feature region.

[0038] Searching for feature regions in a plurality of regions includes searching for feature regions in a plurality of regions in parallel.

[0039] By searching for feature regions in multiple regions in parallel, it is possible to save search time, improve search efficiency, and operate the focus adjustment device at high speed.

[0040] As described above, searching for feature regions in multiple regions in parallel is applicable only when one of the first image and the second image is divided into multiple regions.

[0041] Performing a correlation calculation for each of the plurality of searched feature regions to obtain a correlation value includes performing a correlation calculation in parallel for each of the plurality of searched feature regions to obtain a correlation value.

[0042] By performing correlation calculations in parallel for each of the searched feature areas to obtain correlation values, it is possible to save time in correlation calculations, improve the efficiency of correlation calculations, and operate the focus adjustment device at high speed.

[0043] In some embodiments, a correlation operation is performed separately for each of the plurality of feature regions in the retrieved first image with a corresponding region in the second image to obtain a plurality of correlation values, or a correlation operation is performed separately for each of the plurality of feature regions in the retrieved second image with a corresponding region in the first image to obtain a plurality of correlation values.

[0044] Therefore, a correlation calculation is performed between the first image and the second image to obtain a correlation value representing the correlation between the first image and the second image, allowing the focus adjustment device to more accurately distinguish between objects at different shooting distances, and when there are multiple objects at different shooting distances, to achieve focus on an object at one selected shooting distance, thereby improving focusing accuracy.

[0045] Specifically, in some embodiments, a plurality of pixels for each of a plurality of characteristic regions in a first image are fixed, and the positions of a plurality of pixels in the corresponding regions in the second image are separately shifted by a predetermined amount to obtain a plurality of pixel values ​​at the new positions after the shift. Then, the differences between the plurality of pixel values ​​for each of the characteristic regions in the first image and the plurality of pixel values ​​at the new positions obtained by the shift in the second image are separately calculated, a plurality of sums of absolute differences are calculated, and a plurality of correlation values ​​for each of the characteristic regions are further calculated.

[0046] Specifically, in some embodiments, a plurality of pixels for each of a plurality of characteristic regions in the second image are fixed, and the positions of a plurality of pixels in the corresponding regions in the first image are separately shifted by a predetermined amount to obtain a plurality of pixel values ​​at the new positions after the shift. Then, differences between the plurality of pixel values ​​for each of the characteristic regions in the second image and the plurality of pixel values ​​at the new positions obtained by the shift in the first image are separately calculated, a plurality of sums of absolute differences are calculated, and a plurality of correlation values ​​for each of the characteristic regions are further calculated.

[0047] As a method for calculating the correlation value, in addition to the above-mentioned Sum of Absolute Differences (SAD) calculation method, a Sum of Squared Differences (SSD) calculation method, a Normalized Cross-Correlation (NCC) calculation method, etc. may also be used.

[0048] A parallax exists between the first image and the second image, which causes a position difference between some object images in the first image and the second image. Therefore, as an example, a pixel in a certain region of one of the first image and the second image is fixed, and the position is shifted by a predetermined amount based on the position of a pixel in a corresponding region in the other image, to obtain the pixel value of the pixel at the new position, and a correlation operation is performed to calculate a correlation value.

[0049] The calculated correlation value can represent the correlation between the first image and the second image. For example, the shift amount is any value within a preset shift range, and each value within the preset shift range is substituted into the correlation calculation process to calculate multiple correlation values. The correlation value with the highest correlation among the multiple correlation values ​​is found, and this correlation value is used as the correlation value of the corresponding feature region.

[0050] Specifically, as an example, the mathematical formula for performing a correlation calculation for each of the plurality of searched feature regions to obtain a correlation value is as follows:

number

[0051] As described above, as a specific example, if R(n) is the pixel value of a pixel located at position n in the first image, and the pixel located at position n in the second image is used as a reference point, L(n+i) is the pixel value of a pixel whose shift amount from the pixel located at position n is i, so the above shift does not shift the pixel located at position n, but rather uses the pixel located at position n as a reference point and shifts n by the shift amount i, resulting in the pixel value of the pixel located at position n+i. n is the position coordinate (x, y), x is the abscissa value of the pixel on the horizontal axis, y is the ordinate value of the pixel on the vertical axis, and i is the shift amount of the abscissa value of n on the horizontal axis.

[0052] Furthermore, as one example, the correlation value with the highest correlation is selected from the multiple correlation values ​​for each feature region, the shift amount used in the process of calculating the correlation value with the highest correlation is set as the integer part of the phase difference for each feature region, two correlation values ​​are calculated from two shift amounts before and after the shift amount of the correlation value with the highest correlation, a subpixel estimate for each feature region is further calculated based on the calculated two correlation values ​​and the correlation value with the highest correlation, the subpixel estimate for each feature region is set as the decimal part of the phase difference for each feature region, and a phase difference for each feature region is obtained based on the integer part of the phase difference for each feature region and the decimal part of the phase difference for each feature region. Then, a first defocus amount for each feature region for the multiple feature regions is calculated based on the phase differences for each feature region for the multiple feature regions.

[0053] Therefore, by calculating the decimal part of the phase difference for each characteristic region, the accuracy of the calculated phase difference for each characteristic region can be improved, and the accuracy of multiple first defocus amounts for each characteristic region can be further improved, making it possible to more accurately distinguish between subjects at different shooting distances when multiple subjects at different shooting distances are present, and to more accurately focus on a selected subject at one shooting distance.

[0054] Specifically, as an example, the formula for calculating the sub-pixel estimation value for each feature region is as follows: If S(min-1)?S(min+1), then j=[S(min-1)-S(min+1)] / [2×S(min-1)-2×S(min)], and S(min-1) <S(min+1)であると、j=[S(min+1)-S(min-1)] / [2×S(min+1)-2×S(min)]である。 where j is the subpixel estimate, min is the shift amount of the most highly correlated correlation value, S(min) is the most highly correlated correlation value, S(min-1) is the correlation value of the shift amount before the shift amount of the most highly correlated correlation value, and S(min+1) is the correlation value of the shift amount after the shift amount of the most highly correlated correlation value.

[0055] As an example, if the first image and the second image are completely identical, the shift amount of the correlation value with the highest correlation is zero, and if the multiple pixel values ​​of one feature region of the first image are each completely identical to the multiple pixel values ​​obtained after shifting the corresponding region of the second image, the correlation value with the highest correlation is zero.

[0056] Calculating the first defocus amount for each of the plurality of feature regions separately based on the correlation value for each of the plurality of feature regions includes calculating the first defocus amount for each of the plurality of feature regions in parallel based on the correlation value for each of the plurality of feature regions.

[0057] Therefore, by calculating the first defocus amount for each of a plurality of feature regions in parallel, it is possible to save time in calculating the first defocus amount, improve calculation efficiency, and operate the focus adjustment device at high speed.

[0058] 4, which is a schematic diagram illustrating sorting and grouping according to an embodiment of the present invention, sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their respective first defocus amounts to obtain one or more feature groups includes sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order and integrating, among the plurality of feature regions after sorting, a plurality of feature regions having similar first defocus amounts into one feature group.

[0059] Therefore, among the plurality of feature regions, a plurality of feature regions having similar first defocus amounts are combined into one feature group to obtain one or more feature groups. Each feature group corresponds to an object at one of the plurality of objects at different shooting distances. By performing grouping, the focus adjustment device can better distinguish between objects at different shooting distances according to the grouping result, and the focus adjustment device can selectively focus on an object at one shooting distance with high accuracy, high speed, and in a single operation.

[0060] FIG. 4 illustrates a case where two subjects are located at different shooting distances. In FIG. 4, the first defocus amounts are sorted in ascending order, and then the feature regions are consecutively numbered based on the sorting result. In FIG. 4, the numbers on the horizontal axis represent the feature region numbers, and the numbers on the vertical axis represent the first defocus amounts. After consecutive numbering, the graph result shown in FIG. 4 can be obtained, and two feature groups (i.e., the dotted line portions in FIG. 4) can be obtained based on the grouping rules described above. The gentle curve indicates multiple feature regions with similar first defocus amounts, and these feature regions can be combined into one feature group. The steeply sloped curve indicates a portion where the shooting distance changes between two subjects located at different shooting distances, and the feature regions in this portion cannot be combined into one feature group.

[0061] In some embodiments, the first defocus amount of each of the plurality of feature regions can be sorted in descending order and grouped by the above-described method.

[0062] By sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order and then integrating the plurality of feature regions having similar first defocus amounts into one feature group, the effects of noise and the like can be reduced. Therefore, by grouping the plurality of feature regions, it is possible to more accurately distinguish between subjects at a plurality of different shooting distances.

[0063] 4, if the first defocus amount is a positive number, the optical system in the focus adjustment device moves in a positive direction toward the multiple subjects, and if the first defocus amount is a negative number, the optical system in the focus adjustment device moves in a reverse direction away from the multiple subjects. The absolute value of the first defocus amount represents the amount of movement of the optical system in the focus adjustment device, and the smaller the absolute value of the first defocus amount, the smaller the amount of movement of the optical system. Different values ​​of the second defocus amounts for multiple feature groups indicate the presence of multiple subjects at shooting distances.

[0064] 5 and 6, Fig. 5 is a further sub-flowchart of step S14 in Fig. 1, and Fig. 6 is a schematic diagram showing the sorting and grouping in Fig. 5. Sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their first defocus amounts to obtain one or more feature groups includes the following steps S21 to S24.

[0065] S21: sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order, and calculating the difference between the first defocus amounts of two adjacent feature regions after sorting.

[0066] In step S22, the calculated difference or differences are compared with a preset defocus amount boundary difference threshold.

[0067] In step S23, if some of the calculated one or more differences are equal to or greater than the preset defocus amount boundary difference threshold, the feature regions are grouped based on the calculated one or more differences to obtain one or more feature groups.

[0068] In step S24, if the calculated one or more differences are all less than the preset defocus amount boundary difference threshold, the plurality of feature regions are not grouped based on the calculated one or more differences.

[0069] Therefore, by grouping multiple feature areas based on the difference in the first defocus amount between two adjacent feature areas, the focus adjustment device can better distinguish between objects at different shooting distances based on the grouping results, and the focus adjustment device can focus on an object at one shooting distance with high precision, high speed, and selectively.

[0070] As shown in Fig. 6, the curve in Fig. 6 represents the graphing result after sorting the first defocus amounts of each of the plurality of feature regions in ascending order and consecutively numbering the plurality of feature regions based on the sorting result. The bar graph in Fig. 6 represents the graphing result of the difference in the first defocus amount between two adjacent feature regions. If one or more of the calculated differences are equal to or greater than a preset defocus amount boundary difference threshold, the plurality of feature regions are grouped using the difference greater than the preset defocus amount boundary difference threshold as a boundary, and the plurality of feature regions between the two boundaries are integrated into one feature group. By grouping the plurality of feature regions in this manner, one or more feature groups are obtained. In Fig. 6, the left vertical axis represents the first defocus amount, the right vertical axis represents the difference in the first defocus amount between the two adjacent feature regions, and the horizontal axis represents the feature region number.

[0071] As described above, specifically, a portion where the difference in the first defocus amount between two adjacent feature regions after rearrangement is equal to or greater than the preset defocus amount boundary difference threshold represents a portion where the shooting distance between two subjects at different shooting distances changes; in other words, this image contains at least two subjects at different shooting distances, and therefore the difference equal to or greater than the preset defocus amount boundary difference threshold is used as the boundary for grouping.

[0072] 7 and 8, Fig. 7 is a further sub-flowchart of step S14 in Fig. 1, and Fig. 8 is a schematic diagram showing the sorting and grouping in Fig. 7. Sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their first defocus amounts to obtain one or more feature groups includes the following steps S31 to S33.

[0073] S31: sorting the first defocus amounts of each of the plurality of feature regions in ascending or descending order and dividing the first defocus amounts into a plurality of first defocus amount sections;

[0074] In step S32, an average difference between the first defocus amounts of a plurality of feature regions included in each of the plurality of first defocus amount sections is calculated.

[0075] S33: Based on the number of feature regions included in each first defocus amount section among the plurality of first defocus amount sections and the average difference value between the first defocus amounts of the plurality of feature regions included in each first defocus amount section among the plurality of first defocus amount sections, the plurality of feature regions are grouped to obtain one or more feature groups.

[0076] Therefore, by using a method of grouping based on the average difference between the first defocus amounts of the plurality of feature areas included in each first defocus amount section and the number of feature areas included in each first defocus amount section, the plurality of feature areas can be grouped more accurately, allowing the focus adjustment device to better distinguish between objects at different shooting distances based on the grouping results, and enabling the focus adjustment device to focus on an object at one shooting distance with high precision, high speed, and selectively.

[0077] Specifically, as an example, as shown in Fig. 8, a plurality of first defocus amounts are divided into a plurality of first defocus amount sections using 0.05 as the section division range value, and the average value of differences between the first defocus amounts of a plurality of feature regions included in each first defocus amount section is calculated, i.e., Fig. 8 is a graph formed by this grouping method. In Fig. 8, the right vertical axis indicates the number of feature regions included in each first defocus amount section, the left vertical axis indicates the average value of differences between the first defocus amounts of a plurality of feature regions included in each first defocus amount section, and the horizontal axis indicates a plurality of first defocus amount sections formed by sorting and dividing the plurality of first defocus amounts in ascending order.

[0078] Specifically, in FIG. 8, the number of feature regions included in each of the plurality of first defocus amount sections and the peak of the average value of the differences between the first defocus amounts of the plurality of feature regions included in each of the plurality of first defocus amount sections are set as the boundary, and the plurality of feature regions included in the plurality of first defocus amount sections between the two boundaries are integrated into one feature group.

[0079] Referring to Fig. 9, Fig. 9 is a schematic diagram showing calculation of reliability according to an embodiment of the present invention, and is a conceptual diagram showing the angle of the hypotenuse. Fig. 10 is a schematic diagram showing calculation of reliability according to another embodiment of the present invention, and shows that the reliability changes depending on the angle of the hypotenuse of the preceding and following feature groups.

[0080] As an example, as shown in FIG. 9, the first defocus amounts of each of the multiple feature regions are sorted in ascending order and graphed, and then, for each feature group, one or more right-angled triangles are formed, with the difference between the smallest and largest first defocus amounts among the multiple feature regions as the height. The smaller the hypotenuse angle of the one or more formed right-angled triangles, the higher the reliability of the feature group corresponding to the right-angled triangle. In FIG. 9, the numbers on the horizontal axis represent the feature region numbers, and the numbers on the vertical axis represent the first defocus amounts. Since there are two feature groups in FIG. 9, two right-angled triangles appear, representing the reliability of the two feature groups.

[0081] In some embodiments, the more feature regions each feature group contains, the higher the confidence level for each feature group.

[0082] In some other embodiments, even if the hypotenuse angles of the right triangles formed from the feature group before and the feature group after a feature group are both larger than the hypotenuse angle of the right triangle formed from the feature group, the reliability of the feature group is high and the part with the larger hypotenuse angle represents the part where the shooting distance of subjects at two different shooting distances changes.

[0083] As shown in FIG. 10 , the first defocus amounts of each of the multiple feature regions are sorted in ascending order and graphed. A right-angled triangle is then formed with the number of feature regions included in each group as the base and the difference between the smallest and largest first defocus amounts among the multiple feature regions in each group as the height. As shown in FIG. 10 , vertical lines represent group boundaries. When calculating the reliability of a group, the group is defined as a highly reliable group because both the preceding and succeeding groups have large hypotenuse angles. Alternatively, the group is defined as a less reliable group because the preceding group has a large hypotenuse angle and the succeeding group has a small hypotenuse angle. If the preceding group has a large hypotenuse angle and the succeeding group has a small hypotenuse angle, the group may be the beginning of a valid group. In FIG. 10 , the horizontal axis represents the feature region number, and the vertical axis represents the first defocus amount.

[0084] Furthermore, when calculating the reliability of a group, if the hypotenuse angle of both the preceding and succeeding groups is small, the group is defined as a group with low reliability. In this case, the group may be the junction between two valid groups, that is, the part where the shooting distance of two objects with different shooting distances changes. If the hypotenuse angle of the group preceding the group is small and the hypotenuse angle of the group following the group is large, the group is defined as a group with low reliability and may be the end of a valid group.

[0085] The valid group is a group with a high degree of reliability, and the valid group is the above-mentioned characteristic group, which indicates a subject at one shooting distance among a plurality of subjects at different shooting distances.

[0086] As can be seen from the above, reliability is not calculated using a formula, but is graphed in a diagram, the reliability of a certain group is determined using the above method, and a result indicating whether the reliability is high or low is obtained. It is desirable to weight each reliability described above according to each condition to determine the final reliability. For example, even if the reliability is low due to the angle of the hypotenuse of the preceding and following groups, it is desirable to weight the reliability so that it is high if the number of feature regions is relatively large and the angle of the hypotenuse is relatively small.

[0087] Calculating the confidence for each feature group in one or more feature groups includes calculating the confidence for each feature group in multiple feature groups in parallel.

[0088] Therefore, when grouping to obtain multiple feature groups, calculating the reliability of each feature group in the multiple feature groups in parallel saves time in calculating the reliability, improves calculation efficiency, and allows the focus adjustment device to operate at high speed.

[0089] 11, which is a block diagram showing an image capturing apparatus 100 according to an embodiment of the present invention. The image capturing apparatus 100 includes a focus adjustment apparatus 1.

[0090] When capturing an image with the imaging device 100, if there are a plurality of subjects at different shooting distances, the focus adjustment device 1 can be made to focus on the subject at one of the subject distances with high precision, high speed and selectively.

[0091] In some embodiments, the imaging device 100 is an adjustable focus appliance such as a mobile phone, an industrial camera, a security camera, an automotive camera, a camera module, or the like.

[0092] Referring to FIG. 12, FIG. 12 is a schematic diagram showing the structure of an imaging device 100 according to an embodiment of the present invention. The imaging device 100 further includes an image sensor unit 2, an image data generation unit 3, an optical lens unit 4, and a lens driver 5. The image sensor unit 2 is used to pupil-split light beams of multiple objects to obtain first and second images and output the first and second images to the focus adjustment device 1. The image sensor unit 2 is also used to generate RAW data of multiple pixels included in the optical images of the multiple objects and output the RAW data to the image data generator 3. The image data generator 3 is used to generate image data based on the RAW data and output the image data. The lens driver 5 is connected to the optical lens unit 4. The lens driver 5 is used to generate a focus adjustment control signal based on the second defocus amount output from the focus adjustment device 1 and output the focus adjustment control signal to the optical lens unit 4. The optical lens unit 4 is used to move based on the received focus adjustment control signal to focus on one of multiple objects at different shooting distances. The optical lens unit 4 is further used to condense the light beams of a plurality of subjects onto the image sensor unit 2 and cause the image sensor unit 2 to perform pupil division of the light beams of a plurality of subjects.

[0093] The optical lens unit 4 moves based on the received focus adjustment control signal, and when there are multiple subjects at different shooting distances, the imaging device 100 can focus on one of the subjects at one shooting distance with high precision, high speed, and selectively.

[0094] In the present invention, the imaging device 100 has a function of determining a defocus amount (the above-mentioned first defocus amount) based on the phase difference between a pair of object images obtained by pupil-splitting a light beam from an object. That is, the present invention can be applied to an imaging device 100 that determines a defocus amount (the above-mentioned first defocus amount) based on the phase difference between a pair of object images obtained by pupil-splitting a light beam from an object. A plurality of object light beams are collected on different image sensors or different image sensor areas of the image sensor unit 2, and the image sensor unit 2 performs pupil-splitting on the plurality of object light beams to obtain a pair of object images, i.e., the first image and the second image.

[0095] Referring to FIG. 13, FIG. 13 is a block diagram showing a focus adjustment device 1 according to an embodiment of the present invention. The focus adjustment device 1 is applied to an imaging device 100. The focus adjustment device 1 includes a feature region search unit 11, a correlation value calculation unit 12, a defocus amount calculation unit 13, a grouping processing unit 14, a reliability calculation unit 15, and an arithmetic output unit 16. The feature region search unit 11 is used to divide one of a first image and a second image obtained by pupil-splitting light beams from multiple subjects into multiple regions and search for feature regions in the multiple regions. The correlation value calculation unit 12 is used to perform a correlation calculation for each of the searched feature regions to obtain a correlation value. The defocus amount calculation unit 13 is used to calculate a first defocus amount for each of the multiple feature regions separately based on the correlation value of each of the multiple feature regions. The grouping processing unit 14 is used to sort the first defocus amounts for each of the multiple feature regions in ascending or descending order and group the multiple feature regions by their respective first defocus amounts to obtain one or more feature groups. The reliability calculation unit 15 is used to calculate the reliability of each feature group in one or more feature groups. The calculation output unit 16 is used to calculate a second defocus amount for each feature group in one or more feature groups based on the first defocus amount of each feature region for each feature group in one or more feature groups, and to output the second defocus amount and reliability for each feature group in one or more feature groups. The second defocus amount for each feature group in one or more feature groups is distance information between the imaging element of the focus adjustment device and a focal point formed from a subject at one shooting distance corresponding to each feature group.

[0096] When there are a plurality of subjects at different shooting distances, the focus adjustment device 1 can quickly and highly precisely focus on a subject at one of the plurality of subjects at different shooting distances.

[0097] The focus adjustment device 1 itself has a structure that calculates the defocus amount based on the phase difference between the first image and the second image obtained by pupil division.

[0098] 14, which is a block diagram showing a focus adjustment device 1 according to another embodiment of the present invention. The focus adjustment device 1 is applied to an imaging device. The focus adjustment device 1 includes a processor 17 and a memory 18. The memory 18 is connected to the processor 17 and stores a computer program. The processor 17 executes the computer program to perform the focus adjustment device control method described in any of the above-described embodiments.

[0099] When there are a plurality of subjects at different shooting distances, the focus adjustment device 1 can quickly and highly precisely focus on a subject at one of the plurality of subjects at different shooting distances.

[0100] The present invention further provides a computer-readable storage medium having a computer program stored therein, which, when executed by the processor 17, performs the focus adjustment device control method described in any of the above-described embodiments.

[0101] Those skilled in the art should understand that all or part of the steps of the various methods in the above-mentioned embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable medium. The memory 17 can include a flash memory, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0102] The above are only specific embodiments of the present application, and the scope of protection of the present application is not limited thereto. Those skilled in the art may easily think of modifications or substitutions within the technical scope disclosed in the present application, and all such modifications or substitutions shall be included in the scope of the present application. The embodiments and features of the embodiments of the present application may be combined unless they conflict. Therefore, the scope of protection of the present application shall be determined by the claims.

Claims

1. A control method for a focus adjustment device, which is applied to a focus adjustment device, the control method for the focus adjustment device comprising: Dividing one of the first image and the second image into a plurality of regions based on a first image and a second image obtained by pupil-splitting a plurality of light beams of a subject, and searching for a feature region in the plurality of regions; performing a correlation calculation for each of the retrieved feature regions to obtain a correlation value; calculating a first defocus amount for each of the plurality of feature regions separately based on the correlation value for each of the plurality of feature regions; sorting the first defocus amounts of the plurality of feature regions in ascending or descending order, and grouping the plurality of feature regions by their first defocus amounts to obtain one or more feature groups; calculating a second defocus amount for each feature group in the one or more feature groups based on a first defocus amount for each feature region in the one or more feature groups; calculating a reliability for each feature group in the one or more feature groups; outputting a second defocus amount and a reliability for each of the one or more feature groups, wherein the second defocus amount for each of the one or more feature groups is distance information between an imaging element of the focus adjustment device and a focal point formed from an object at one shooting distance corresponding to each of the feature groups; Including, A control method for a focus adjustment device.

2. Dividing one of the first image and the second image into a plurality of regions based on a first image and a second image obtained by pupil-splitting a light beam of a plurality of objects, and searching for a feature region in the plurality of regions, Dividing one of the first image and the second image into a plurality of regions based on a first image and a second image obtained by pupil-division of a plurality of object light beams; separately comparing the feature values ​​for each of the local regions within the divided regions with a preset feature value threshold; When the feature amount of one of the local regions is equal to or greater than the preset feature amount threshold, the local region is determined to be the feature region; Including, 2. The control method for a focus adjustment device according to claim 1.

3. Sorting the first defocus amounts of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their first defocus amounts to obtain one or more feature groups includes: sorting the first defocus amounts of the plurality of feature regions in ascending or descending order, and integrating, among the sorted plurality of feature regions, a plurality of feature regions having similar first defocus amounts into one feature group; 2. The control method for a focus adjustment device according to claim 1.

4. Sorting the first defocus amounts of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their first defocus amounts to obtain one or more feature groups includes: sorting the first defocus amounts of the plurality of feature regions in ascending or descending order, and calculating a difference between the first defocus amounts of two adjacent feature regions after the sorting; comparing the calculated difference or differences with a preset defocus amount boundary difference threshold; If a part of the one or more calculated differences is equal to or greater than a preset defocus amount boundary difference threshold, grouping the plurality of feature regions based on the one or more calculated differences to obtain one or more feature groups; Including, 2. The control method for a focus adjustment device according to claim 1.

5. Sorting the first defocus amounts of the plurality of feature regions in ascending or descending order and grouping the plurality of feature regions by their first defocus amounts to obtain one or more feature groups includes: sorting the first defocus amounts of the plurality of feature regions in ascending or descending order and dividing the first defocus amounts into a plurality of first defocus amount sections; calculating an average difference between the first defocus amounts of a plurality of feature regions included in each of the plurality of first defocus amount sections; acquiring one or more feature groups by grouping the plurality of feature regions based on the number of feature regions included in each first defocus amount section among the plurality of first defocus amount sections and an average value of differences between the first defocus amounts of the plurality of feature regions included in each first defocus amount section among the plurality of first defocus amount sections; Including, 2. The control method for a focus adjustment device according to claim 1.

6. The searching for a feature region in the plurality of regions includes: searching the plurality of regions in parallel for feature regions; 2. The control method for a focus adjustment device according to claim 1.

7. The correlation calculation is performed for each of the plurality of searched feature regions to obtain a correlation value. performing a correlation calculation in parallel for each of the plurality of searched feature regions to obtain a correlation value; 2. The control method for a focus adjustment device according to claim 1.

8. Calculating a first defocus amount for each of the plurality of feature regions separately based on the correlation value for each of the plurality of feature regions includes: calculating a first defocus amount for each of the plurality of feature regions in parallel based on the correlation value for each of the plurality of feature regions; 2. The control method for a focus adjustment device according to claim 1.

9. Calculating the reliability of each feature group in the one or more feature groups includes: calculating a reliability for each of the plurality of feature groups in parallel; 2. The control method for a focus adjustment device according to claim 1.

10. A focus adjustment device applied to an imaging device, The image processing apparatus includes a feature region search unit, a correlation value calculation unit, a defocus amount calculation unit, a grouping processing unit, a reliability calculation unit, and a calculation output unit, the characteristic region search unit is used to divide one of the first image and the second image into a plurality of regions based on a first image and a second image obtained by pupil-splitting a plurality of light beams of a subject, and to search for a characteristic region in the plurality of regions; the correlation value calculation unit is used to perform a correlation calculation for each of the plurality of searched feature regions to obtain a correlation value; the defocus amount calculation unit is used to calculate a first defocus amount for each of the plurality of feature regions separately based on a correlation value for each of the plurality of feature regions; the grouping processing unit is used to sort the first defocus amounts of the plurality of feature regions in ascending order or descending order, and group the plurality of feature regions by their first defocus amounts to obtain one or more feature groups; the reliability calculation unit is used to calculate a reliability for each feature group in one or more feature groups; the calculation output unit is used to calculate a second defocus amount for each feature group in the one or more feature groups based on a first defocus amount for each feature region in each feature group in the one or more feature groups, and to output the second defocus amount and reliability for each feature group in the one or more feature groups, and the second defocus amount for each feature group in the one or more feature groups is distance information between an imaging element of a focus adjustment device and a focal point formed from an object at one shooting distance corresponding to each feature group. A focusing device characterized by:

11. A focus adjustment device applied to an imaging device, A focus adjustment device comprising: a processor; and a memory connected to the processor, wherein a computer program is stored in the memory; and the processor executes the computer program to execute the focus adjustment device control method according to any one of claims 1 to 9.

12. An imaging device comprising the focus adjustment device according to claim 10.

13. A computer program causing a processor to execute the control method for a focus adjustment device according to any one of claims 1 to 9.

Images