Focus adjustment device and control method thereof, imaging device, and computer-readable storage medium

Abstract

The present invention provides a focus adjustment device and a control method thereof, an imaging device, and a computer-readable storage medium. [Solution] A control method for a focus adjustment device includes: searching for one or more feature regions from a first image of a pair of subject images obtained by pupil-splitting light beams from multiple subjects, calculating a correlation value for each feature region based on the pair of subject images; calculating a first defocus amount for each feature region based on the correlation value calculation results for each feature region; dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region; and calculating a reliability for each group based on at least the feature amount of each feature region and the number or area of ​​the feature regions in each group.

Description

[Technical Field]

[0001] The present invention relates to the field of imaging technology, and more particularly to a focus adjustment device and a control method thereof, an imaging device, and a computer-readable storage medium. [Background technology]

[0002] Currently, the split-pupil phase difference focusing method is a common autofocusing method. The principle of this focusing method is to split the light beam from the subject on the exit pupil plane, focus the split light beam on a pair of pixel rows on a two-dimensional image sensor, and perform focus detection based on the phase difference between the image signals (i.e., the left-eye image signal and the right-eye image signal) after photoelectric conversion of the pair of pixel rows.

[0003] When there are multiple subjects with different shooting distances, the image sensor may output a mixed value of multiple phase differences, which may make it impossible to selectively focus on the subject with the closest shooting distance or the subject with an intermediate shooting distance, among the multiple subjects. Summary of the Invention [Problem to be solved by the invention]

[0004] In order to solve the above-mentioned technical problems or to at least partially solve the above-mentioned technical problems, the present invention provides a focusing device and a control method thereof, an imaging device, and a computer-readable storage medium. [Means for solving the problem]

[0005] To achieve the above-mentioned object, a first aspect of the present invention provides a control method for a focus adjustment device, which is applied to a focus adjustment device. The control method for a focus adjustment device includes: searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting light beams of multiple objects, the pair of object images including a left-eye image and a right-eye image, the first image being either the left-eye image or the right-eye image; calculating a correlation value for each feature region based on the pair of object images; calculating a first defocus amount for each feature region based on the correlation value calculation result for each feature region; dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region; and calculating a reliability for each group based on at least the feature amount of each feature region and the number or area of ​​the feature regions in each group, wherein the average first defocus amount of each feature region in each group and the reliability for each group are both used to determine the defocus amount of an object at one shooting distance corresponding to the group.

[0006] The control method for a focus adjustment device provided by the present application searches for one or more feature regions from a first image, performs a correlation value calculation for each feature region, calculates a first defocus amount for each feature region, and divides the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region, thereby associating the feature regions of different groups with subjects at different shooting distances.Accordingly, the defocus amount for a subject at one shooting distance corresponding to each group can be determined based on the first defocus amount of each feature region in each group, and it is possible to selectively focus on one of the multiple subjects, subjects at different shooting distances such as a subject at the closest shooting distance, a subject at an intermediate distance, or a subject at the farthest shooting distance, or a subject with a high reliability.

[0007] A second aspect of the present invention provides a focus adjustment device. The focus adjustment device includes a feature region search unit, a correlation calculation unit, a defocus amount calculation unit, a grouping processing unit, and a reliability calculation unit. The feature region search unit is used to search for one or more feature regions from a pair of object images based on a first image in the pair of object images obtained by pupil-splitting light beams from multiple objects, the pair of object images including a left-eye image and a right-eye image, and the first image is either the left-eye image or the right-eye image. The correlation calculation unit is used to calculate a correlation value for each feature region based on the pair of object images. The defocus amount calculation unit is used to calculate a first defocus amount for each feature region based on the correlation value calculation result for each feature region. The grouping processing unit is used to divide one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region. The reliability calculation unit is used to calculate the reliability for each group based on at least the feature amount of each feature area in each group and the number or area of ​​the feature areas, and the first defocus amount average of each feature area in each group and the reliability for each group are both used to determine the defocus amount of the subject at one shooting distance corresponding to the group.

[0008] A third aspect of the present invention provides a focus adjustment device, the focus adjustment device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the focus adjustment device control method of the first aspect.

[0009] In a fourth aspect of the present invention, there is provided an imaging device, the imaging device comprising the focus adjustment device of the second aspect.

[0010] In a fifth aspect of the present invention, there is provided a computer-readable storage medium having executable instructions stored thereon, the executable instructions, when executed by a processor, performing the method for controlling a focus adjustment device of the first aspect. [Brief explanation of the drawings]

[0011] [Figure 1]4 is a first flowchart of a control method for a focusing device provided by an embodiment of the present invention; [Figure 2] 10 is a second flowchart of a control method for a focusing device provided by an embodiment of the present invention. [Figure 3] 2 is a schematic diagram illustrating a pair of object images provided by an embodiment of the present invention; [Figure 4] FIG. 2 is a first schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 5] FIG. 4 is a second schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 6] FIG. 10 is a third schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 7] FIG. 10 is a fourth schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 8] FIG. 5 is a fifth schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention. [Figure 9] FIG. 6 is a sixth schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 10] 1 is a schematic diagram showing the structure of a focusing device provided by an embodiment of the present invention; [Figure 11] 1 is a schematic diagram illustrating the structure of an imaging device provided by an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, the technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments that can be obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0013] The terms "first," "second," etc. described in the present specification are used merely to distinguish the objects being described and do not have any ordering or technical significance. Furthermore, the terms "comprise," "include," or variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units may include not only the listed steps or units, but also steps or units not listed, or may include other steps or units inherent to such a process, method, product, or apparatus.

[0014] It should be noted that features in the embodiments of the present invention may be combined with each other as long as they do not conflict.

[0015] Referring to Figures 1 and 2, Figure 1 is a first flowchart of a control method for a focus adjustment device provided by an embodiment of the present invention, and Figure 2 is a second flowchart of a control method for a focus adjustment device provided by an embodiment of the present invention.

[0016] As shown in Fig. 1, the present invention provides a control method for a focus adjustment device. The control method for a focus adjustment device is applied to a focus adjustment device. The control method for a focus adjustment device includes the following steps S1 to S4.

[0017] In step S1, one or more feature regions are searched for in a first image of a pair of object images obtained by pupil-splitting a light beam from a plurality of objects. The pair of object images includes a left-eye image and a right-eye image, and the first image is the left-eye image or the right-eye image.

[0018] In step S2, a correlation value is calculated for each feature region based on the pair of subject images.

[0019] In step S3, a first defocus amount is calculated for each feature region based on the correlation value calculation result for each feature region.

[0020] In step S4, one or more feature regions are divided into one or more groups based on the first defocus amount and coordinate position information of each feature region.

[0021] In step S5, a reliability for each group is calculated based on at least the feature amount of each feature region in each group. The average first defocus amount of each feature region in each group and the reliability for each group are used to determine the defocus amount of the object at one shooting distance corresponding to the group.

[0022] By calculating the reliability of each group, it is possible to select multiple groups based on the reliability, and therefore it is possible to selectively focus on one of multiple subjects, subjects at different shooting distances such as the subject at the closest shooting distance or the subject at an intermediate distance, or the subject at the farthest shooting distance, or subjects with high reliability.

[0023] In the present invention, when calculating reliability, it is necessary to weight each condition (for example, the number and / or area of ​​feature regions in each group) to obtain the reliability. In this way, even if the average value of the magnitude of the feature amount for each feature region of a certain group is small, if the number or area of ​​the feature regions of that group is larger than those of other groups, the reliability obtained by weighting will be high. Using groups as units, the larger the feature amount of each feature region in each group, the higher the reliability; the larger the number of feature regions included in each group, the higher the reliability; and the larger the area of ​​the feature regions included in each group, the higher the reliability.

[0024] The control method for a focus adjustment device provided by the present invention searches for one or more feature regions from a first image, performs a correlation value calculation for each feature region, calculates a first defocus amount for each feature region, and divides the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region, thereby associating the feature regions of different groups with subjects at different shooting distances.Accordingly, the defocus amount for a subject at one shooting distance corresponding to each group can be determined based on the first defocus amount of each feature region in each group, and it is possible to selectively focus on one of the multiple subjects, subjects at different shooting distances such as a subject at the closest shooting distance, a subject at an intermediate distance, or a subject at the farthest shooting distance, or a subject with a high reliability.

[0025] A pair of subject images obtained by pupil-splitting light beams from multiple subjects refers to a pair of subject images (including a left-eye image and a right-eye image) obtained by pupil-splitting light beams from multiple subjects and then projecting them onto different image sensors or different image sensor areas. When multiple subjects are at different shooting distances, for example, as shown in Figure 3, one scene has four subjects (including subjects 101 to 104) and the shooting distance increases sequentially from subject 101 to subject 104, and a conventional pupil-splitting phase-difference autofocus method cannot selectively focus on the subject (subject 101) that is closest to the subject 101 to subject 104 or the subject (subject 102 or subject 103) that is located at an intermediate shooting distance.

[0026] The term "defocus amount" refers to the distance between the imaging position of the subject and the imaging plane of the optical lens unit. When the imaging position of the subject is exactly located on the imaging plane, the optical lens unit is in a focused state, when the imaging position of the subject is on the side of the imaging plane closer to the subject, the optical lens unit is in a front defocused state, and when the imaging position of the subject is on the side of the imaging plane farther from the subject, the optical lens unit is in a rear defocused state.

[0027] The term "correlation value" refers to a metric of the similarity (correlation) between two signals. Specifically, when the optical lens unit is in focus with respect to a certain feature region, the left-eye image and right-eye image of that feature region are the same, the phase difference (also called image shift amount) between them is zero, and the correlation is highest. When the optical lens unit is in a pre-defocused or post-defocused state with respect to that feature region, the phase difference between the left-eye image and right-eye image of that feature region is not zero, and there is a proportional relationship between the phase difference between the left-eye image and right-eye image for each feature region and the defocus amount of that feature region. In other words, the defocus amount increases as the phase difference between the left-eye image and right-eye image increases. The direction of the phase difference corresponds to the direction of the defocus amount (the side of the imaging plane closer to the subject / the side of the imaging plane farther from the subject).

[0028] By calculating a correlation value for each feature region, a phase difference between the left-eye image and the right-eye image for each feature region is obtained, and therefore a first defocus amount for each feature region is obtained.One or more feature regions can be divided into one or more groups based on the first defocus amount and coordinate position information of each feature region, and the feature regions of different groups can be associated with subjects at different shooting distances.By selecting a feature region of one group from the plurality of groups of feature regions, determining the movement direction and movement amount of the optical lens unit based on the first defocus amount of the feature region of that group, and controlling the movement of the optical lens unit based on the movement direction and movement amount, it is possible to selectively focus on one of the plurality of subjects, subjects at different shooting distances such as a subject at the closest shooting distance, a subject at an intermediate distance, or a subject at the farthest shooting distance, or a subject with a high reliability.

[0029] In an embodiment of the present invention, a feature region refers to a local region whose feature quantity extracted from a first image satisfies certain requirements. For example, if the feature quantity of a certain local region is greater than a preset feature quantity threshold, the local region is determined to be a feature region. The feature quantity is determined by the specifications of the image sensor, and illustratively, the feature quantity includes contrast, color, etc.

[0030] In some embodiments, searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting a plurality of object light beams includes: Dividing a first image into a plurality of sub-regions based on a first image in a pair of object images obtained by pupil-splitting a plurality of object light beams; performing parallel searches on the plurality of sub-regions in units of sub-regions to search for feature regions separately from the plurality of sub-regions; Includes.

[0031] As shown in FIG. 2, multiple central processing unit (CPU) cores can be used to perform parallel searches on multiple sub-regions.

[0032] In this way, parallel searches can be performed on multiple sub-regions to improve search speed.

[0033] In some embodiments, performing parallel searches on a plurality of sub-regions in units of sub-regions and separately searching for feature regions from the plurality of sub-regions includes: Comparing a feature amount for each local region in the plurality of sub-regions with a preset feature amount threshold, with each sub-region being a unit; If the feature amount of one local region is greater than the preset feature amount threshold, the local region is determined to be a feature region; Including,

[0034] The preset feature threshold is preferably a threshold that is not affected by noise. If the feature region is large, there is a high possibility that objects at different shooting distances will be mixed, so the smaller the feature region, the more likely it is that image signals containing a mixture of phase shift amounts due to multiple objects at different shooting distances will be eliminated.

[0035] In the present invention, by determining as feature regions only local regions whose feature amounts are greater than the preset feature amount threshold, it is possible to eliminate the influence of noise and the like and improve focusing accuracy.

[0036] In some embodiments, the correlation value calculation result includes an integer part min of the phase difference of the feature region, where min is also called the maximum correlation shift amount.

[0037] Calculating correlation values ​​for each feature region is calculating a correlation value when the feature region in the second image is shifted by a different preset shift amount by a preset correlation value calculation method based on a pixel value of each pixel in the feature region in the first image and a pixel value when each pixel in the feature region in the second image is shifted by a different preset shift amount, wherein the first image is one of a left-eye image and a right-eye image, the second image is the other of the left-eye image and the right-eye image, and the preset shift amount is an integer; comparing correlation values ​​when the feature region in the second image is shifted by different preset shift amounts, and determining the preset shift amount corresponding to the highest correlation value as an integer part of the phase difference of the feature region; Includes.

[0038] In the embodiments of the present invention, unless otherwise stated, the term "phase difference of a feature region" refers to the amount of shift between the left-eye image and the right-eye image when there is the highest correlation between the pixel values ​​of the feature region in the left-eye image and the pixel values ​​of the feature region in the right-eye image.

[0039] In some embodiments, the preset correlation value calculation method includes, but is not limited to, a sum of absolute differences (SAD) calculation method, a sum of squared differences (SSD) calculation method, and a normalized cross-correlation (NCC) calculation method.

[0040] For example, assuming that the preset correlation value calculation method is a sum of absolute difference calculation method, calculating a correlation value when the feature region in the second image is shifted by different preset shift amounts by the preset correlation value calculation method based on the pixel value of each pixel in the feature region in the first image and the pixel value when each pixel in the feature region in the second image is shifted by different preset shift amounts is as follows: Fixing a pixel value for each pixel in the feature region in the first image; Shifting each pixel of the characteristic region in the second image by a different preset shift amount separately, and obtaining pixel values ​​when each pixel of the characteristic region in the second image is shifted by a different preset shift amount; Separately calculating differences between pixel values ​​of each pixel in the characteristic region in the first image and pixel values ​​of each pixel in the characteristic region in the second image when the pixel is shifted by different preset shift amounts; determining the sum of absolute values ​​of the differences between the pixel values ​​of each pixel in the characteristic region in the first image and the pixel values ​​of each pixel in the characteristic region in the second image when the pixel is shifted by an arbitrary preset shift amount as a correlation value when the characteristic region in the second image is shifted by an arbitrary preset shift amount, thereby obtaining a correlation value when the characteristic region in the second image is shifted by a different preset shift amount; Includes.

[0041] The different preset shift amounts are all within the preset shift range.

[0042] Specifically, if the preset correlation value calculation method is the sum of absolute difference calculation method, the correlation value of each feature region can be calculated by the following formula.

number

[0043] In some embodiments, the correlation calculation result further includes the decimal part of the phase difference for each feature region.

[0044] Calculating the correlation value for each feature region further includes calculating the decimal part of the phase difference of the feature region by a preset curve approximation method based on the correlation value corresponding to the integer part of the phase difference of the feature region and the correlation values ​​corresponding to two shift amounts adjacent to the integer part of the phase difference.

[0045] The integer part of the phase difference is represented by min, and the two shift amounts adjacent to the integer part of the phase difference include a shift amount min-1 before the integer part of the phase difference and a shift amount min+1 after the integer part of the phase difference.

[0046] When calculating the correlation value, the value of the preset shift amount i takes only an integer without considering the decimal part, so there may be an error between min and the actual phase difference. In this embodiment, after calculating the integer part min of the phase difference of the feature region, the decimal part j of the phase difference of the feature region is calculated by a preset curve approximation method, and the first defocus amount of the feature region is calculated based on min+j, so that a more accurate first defocus amount can be calculated.

[0047] In some embodiments, the curve fitting method is a piecewise linear fitting method or a parabolic fitting method.

[0048] Exemplarily, taking the case where the curve approximation method is the equidistant straight line approximation method, based on the correlation value corresponding to the integer part of the phase difference of the feature region and the correlation values corresponding to two shift amounts adjacent to the integer part of the phase difference, calculating the fractional part of the phase difference of the feature region by the preset curve approximation method is when the correlation value corresponding to the shift amount before the integer part of the phase difference is greater than or equal to the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) ≥ S(min + 1), calculating the fractional part of the phase difference by the first formula, and when the correlation value corresponding to the shift amount before the integer part of the phase difference is less than the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) < S(min + 1), calculating the fractional part of the phase difference by the second formula, and is included.

[0049] The first formula is j = [S(min - 1) - S(min + 1)] / [2×S(min - 1) - 2×S(min)].

[0050] The second formula is j = [S(min + 1) - S(min - 1)] / [2×S(min + 1) - 2×S(min)].

[0051] j is the fractional part of the phase difference, S(min) is the correlation value corresponding to the integer part of the phase difference, S(min - 1) is the correlation value corresponding to the shift amount before the integer part of the phase difference, and S(min + 1) is the correlation value corresponding to the shift amount after the integer part of the phase difference.

[0052] In some embodiments, performing the correlation value calculation for each feature region includes performing parallel calculations on one or more feature regions in units of feature regions to separately calculate the correlation value for each feature region.

[0053] As shown in FIG. 2, parallel calculations can be performed on one or more feature regions by a plurality of CPU cores.

[0054] By performing parallel calculations on one or more feature regions, the calculation speed can be improved.

[0055] In some embodiments, calculating the first defocus amount for each feature region includes performing parallel calculations on one or more feature regions on a feature region-by-feature region basis to separately calculate the first defocus amount for each feature region.

[0056] As shown in FIG. 2, multiple CPU cores can be used to perform parallel calculations on one or more feature regions.

[0057] In this way, parallel calculations can be performed on one or more feature regions, thereby improving the calculation speed.

[0058] Calculating the first defocus amount of a feature region based on the phase difference of the feature region is a conventional technique, and therefore the present invention will not be described in detail.

[0059] The present invention provides two grouping methods for grouping feature regions. The two grouping methods are a dividing line grouping method and a block grouping method. The two grouping methods will be introduced in detail below with reference to FIGS. 4 to 9.

[0060] Referring to Figure 4, in some embodiments, the feature regions can be grouped using a dividing line grouping method, where one or more feature regions are divided into one or more groups based on the first defocus amount and coordinate position information of each feature region, such as: determining a region where a difference in first defocus amounts between adjacent feature regions exceeds a preset first defocus amount difference threshold based on coordinate position information of the feature regions as a dividing line between the adjacent feature regions; Integrating feature regions surrounded by the same dividing line into the same group; Includes.

[0061] For example, a region where the difference in the first defocus amount between adjacent feature regions, such as the top, bottom, left, right, upper left diagonal, upper right diagonal, lower left diagonal, or lower right diagonal, exceeds a preset first defocus amount difference threshold, can be set as a dividing line between adjacent feature regions. As shown in Fig. 4, one or more feature regions can be divided into two groups by the dividing line, and the feature regions in the two groups correspond to two objects at different shooting distances.

[0062] In some embodiments, merging feature regions bounded by the same dividing line into the same group can be performed by: determining an isolated dividing line that cannot divide the feature region as an invalid dividing line; Integrating feature regions surrounded by the same effective dividing line into the same group; Includes.

[0063] For example, as shown in FIG. 5, in some scenes, when the dividing line grouping method is adopted, isolated dividing lines may be obtained. Therefore, these isolated dividing lines can be determined as invalid dividing lines, and only valid dividing lines can be selected and grouped, thereby improving the grouping efficiency.

[0064] In some embodiments, dividing one or more feature regions into one or more groups based on the first defocus amount and the coordinate position information of each feature region includes: If the dividing line grouping method cannot be adopted to generate the dividing line, the method further includes adopting a chunk grouping method to divide the one or more feature regions into one or more groups.

[0065] In some embodiments, after integrating the feature regions surrounded by the same effective dividing line into the same group, the control method of the focus adjustment device includes: The method further includes calculating an average value of the first defocus amount for each feature region for each group, and merging groups whose difference in the average values ​​of the first defocus amounts is smaller than a preset defocus amount average value difference threshold and which are adjacent to each other into the same group.

[0066] For example, as shown in FIG. 6, depending on the shooting conditions, the type of subject, etc., a single subject may be divided into one or more groups. Therefore, by merging groups whose difference in the average values ​​of the first defocus amounts is smaller than the preset defocus amount average value difference threshold and whose positions are adjacent into the same group, the accuracy of the grouping can be improved, and it is possible to prevent a single subject from being divided into multiple groups as much as possible.

[0067] 7 and 8, in some embodiments, a cluster grouping method can be adopted to group the feature regions, where one or more feature regions are divided into one or more groups according to the first defocus amount and coordinate position information of each feature region, such as: Dividing one or more feature regions into a predetermined number of blocks based on coordinate position information of each feature region; Calculating an average value of the first defocus amount of the feature region included in each block; Integrating clusters of feature regions whose difference in the average values ​​of the first defocus amounts is smaller than a preset threshold value for the difference in the average values ​​of the first defocus amounts and whose positions are adjacent into the same group; Includes.

[0068] For example, when the block grouping method is used to group the feature regions, the average shrink / enlarge method can be used to process the first defocus amount of the feature regions to achieve the grouping of the feature regions.

[0069] Specifically, as shown in FIG. 8, the average reduction method is as follows: Dividing one or more feature regions into a predetermined number of blocks (e.g., 6 x 8 blocks); Calculating an average value of the first defocus amount of the feature region included in each block; shrinking each block according to an average value of the first defocus amount; Integrating clusters of feature regions whose difference in the average values ​​of the first defocus amounts is smaller than a preset threshold value for the difference in the average values ​​of the first defocus amounts and whose positions are adjacent into the same group; Includes.

[0070] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes calculating the reliability of each group based on the feature values ​​of each feature region in each group and the number of feature regions included in each group.

[0071] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes calculating the reliability of each group based on the feature values ​​of each feature region in each group and the area of ​​the feature regions included in each group.

[0072] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes calculating the reliability of each group based on the feature values ​​of each feature region in each group, the number of feature regions included in each group, and the area of ​​the feature regions included in each group.

[0073] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes performing parallel calculations on one or more groups on a group-by-group basis, based on at least the feature values ​​of each feature region in each group, to calculate the reliability of each group separately.

[0074] As shown in FIG. 2, multiple CPU cores can perform parallel operations on one or more groups.

[0075] In this way, parallel calculations can be performed on one or more groups, improving the calculation speed.

[0076] As shown in Fig. 9, in some scenes, a single object may be divided into one or more groups depending on the size of each block, the shooting conditions, the type of object, and other factors, so in this case, the reliability of the divided groups should be reduced.

[0077] In some embodiments, after calculating a reliability for each group based on the feature amount of each feature region in at least each group, the control method for the focus adjustment device includes: calculating a second defocus amount for each of the one or more groups based on a first defocus amount for each feature region for each of the one or more groups; outputting a second defocus amount and a reliability for each of the one or more groups; Further includes:

[0078] The second defocus amount and the reliability for each group are used together to determine the defocus amount of the object at the corresponding shooting distance of the group.

[0079] In this way, one feature region of one group corresponds to one object, and the second defocus amount of the group is the defocus amount of the corresponding object. By determining the target movement direction and target movement amount of the optical lens unit based on the second defocus amount of the group and controlling the movement of the optical lens unit based on the target movement direction and target movement amount, it is possible to achieve focusing on the object corresponding to the group.

[0080] In some embodiments, calculating the second defocus amount for each of the one or more groups based on the first defocus amount for each of the feature regions for each of the one or more groups includes determining an average value of the first defocus amounts for each of the feature regions for each of the groups as the second defocus amount for the group.

[0081] For example, if a group includes three feature regions and the first defocus amounts of the three feature regions are 0.4, 0.5, and 0.6, respectively, then the second defocus amount of the group is (0.4+0.5+0.6) / 3=0.5.

[0082] 10, which is a schematic diagram showing the structure of a focus adjustment device provided by an embodiment of the present invention. Based on the same inventive concept, the present invention further provides a focus adjustment device 40 including a feature region search unit 41, a correlation calculation unit 42, a defocus amount calculation unit 43, a grouping processing unit 44, and a reliability calculation unit 45.

[0083] The characteristic region search unit 41 is used to search for one or more characteristic regions from a first image of a pair of subject images obtained by pupil-splitting light beams from multiple subjects. The pair of subject images includes a left-eye image and a right-eye image, and the first image is either the left-eye image or the right-eye image.

[0084] The correlation calculation unit 42 is used to calculate a correlation value for each feature region based on a pair of subject images.

[0085] The defocus amount calculation unit 43 is used to calculate a first defocus amount for each feature region based on the correlation value calculation result for each feature region.

[0086] The grouping processing unit 44 is used to divide one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region.

[0087] The reliability calculation section 45 is used to calculate the reliability of each group based on at least the feature amount of each feature region in each group and the number or area of ​​the feature regions.

[0088] The average first defocus amount of each feature region in each group and the reliability of each group are both used to determine the defocus amount of the subject at one shooting distance corresponding to that group.

[0089] In some embodiments, the characteristic region search unit 41 is specifically used to divide a first image of a pair of subject images obtained by pupil-splitting light beams from a plurality of subjects into a plurality of sub-regions, and to perform parallel searches on the plurality of sub-regions on a sub-region basis to separately search for characteristic regions from the plurality of sub-regions.

[0090] In some embodiments, the characteristic region search unit 41 is specifically used to compare the feature values ​​of each local region in a plurality of subregions with a preset feature value threshold value, with the subregion as a unit, and if the feature value of one local region is greater than the preset feature value threshold value, to determine that local region as a characteristic region.

[0091] In some embodiments, the correlation calculation unit 42 is specifically used to calculate correlation values ​​when the feature region in the second image is shifted by different preset shift amounts using a preset correlation value calculation method, based on the pixel values ​​of each pixel in the feature region in the first image and the pixel values ​​of each pixel in the feature region in the second image when the pixel is shifted by different preset shift amounts, compare the correlation values ​​when the feature region in the second image is shifted by different preset shift amounts, and determine the preset shift amount corresponding to the highest correlation value as the integer part of the phase difference of the feature region.

[0092] The first image is one of a left-eye image and a right-eye image, the second image is the other of a left-eye image and a right-eye image, and the preset shift amount is an integer.

[0093] In some embodiments, specifically, the correlation calculation unit 42 fixes the pixel value of each pixel in the feature region in the first image, separately shifts each pixel in the feature region in the second image by different preset shift amounts, obtains the pixel values when each pixel in the feature region in the second image is shifted by different preset shift amounts, separately calculates the difference between the pixel value of each pixel in the feature region in the first image and the pixel value when each pixel in the feature region in the second image is shifted by different preset shift amounts, and determines the sum of absolute values of the differences between the pixel value of each pixel in the feature region in the first image and the pixel value when each pixel in the feature region in the second image is shifted by an arbitrary preset shift amount as the correlation value when the feature region in the second image is shifted by an arbitrary preset shift amount. Therefore, it is used to obtain the correlation values when the feature region in the second image is shifted by different preset shift amounts.

[0094] In some embodiments, the correlation calculation unit 42 is further used to calculate the fractional part of the phase difference of the feature region by the preset curve approximation method based on the correlation value corresponding to the integer part of the phase difference of the feature region and the correlation values corresponding to two shift amounts adjacent to the integer part of the phase difference.

[0095] In some embodiments, when the correlation value corresponding to the shift amount before the integer part of the phase difference is greater than or equal to the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) ≥ S(min + 1), the correlation calculation unit 42 calculates the fractional part of the phase difference according to the first formula. When the correlation value corresponding to the shift amount before the integer part of the phase difference is less than the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) < S(min + 1), the correlation calculation unit 42 is used to calculate the fractional part of the phase difference according to the second formula.

[0096] The first formula is j = [S(min - 1) - S(min + 1)] / [2×S(min - 1) - 2×S(min)].

[0097] The second formula is j=[S(min+1)-S(min-1)] / [2×S(min+1)-2×S(min)].

[0098] j is the decimal part of the phase difference, S(min) is the correlation value corresponding to the integer part of the phase difference, S(min-1) is the correlation value corresponding to the shift amount before the integer part of the phase difference, and S(min+1) is the correlation value corresponding to the shift amount after the integer part of the phase difference.

[0099] In some embodiments, the correlation calculation unit 42 is specifically used to perform parallel calculations on one or more feature regions, with each feature region being a unit, to calculate a correlation value for each feature region separately.

[0100] In some embodiments, the defocus amount calculation unit 43 is specifically used to perform parallel calculations on one or more feature regions, with each feature region as a unit, to separately calculate the first defocus amount for each feature region.

[0101] In some embodiments, the grouping processing unit 44 is specifically used to, based on the coordinate position information of the feature regions, set an area where the difference in the first defocus amount between adjacent feature regions exceeds a preset first defocus amount difference threshold as a dividing line between the adjacent feature regions, and to integrate the feature regions surrounded by the same dividing line into the same group.

[0102] In some embodiments, the grouping processing unit 44 is specifically used to determine an isolated dividing line that cannot divide a feature region as an invalid dividing line, and to combine feature regions surrounded by the same valid dividing line into the same group.

[0103] In some embodiments, the grouping processor 44 is further adapted to employ a chunk grouping method to divide one or more feature regions into one or more groups if a dividing line cannot be generated using the dividing line grouping method.

[0104] In some embodiments, the grouping processing unit 44 further combines feature regions surrounded by the same effective dividing line into the same group, calculates the average value of the first defocus amount of each feature region for each group, and combines groups whose difference in the average values ​​of the first defocus amounts is smaller than the preset defocus amount average difference threshold and which are adjacent to each other into the same group.

[0105] In some embodiments, the grouping processing unit 44 is specifically used to divide one or more feature regions into a preset number of blocks based on the coordinate position information of each feature region, calculate the average value of the first defocus amount of the feature regions included in each block, and merge the blocks of feature regions whose difference in the average value of the first defocus amount is smaller than the preset defocus amount average value difference threshold and whose positions are adjacent into the same group.

[0106] The first defocus amount of each feature region in each group and the reliability of each group are both used to determine the defocus amount of the object at one shooting distance corresponding to that group.

[0107] In some embodiments, the reliability calculation unit 45 is specifically used to calculate the reliability of each group based on the feature amount of each feature region in each group and the number of feature regions included in each group.

[0108] In some embodiments, the reliability calculation unit 45 is specifically used to calculate the reliability of each group based on the feature amount of each feature region in each group and the area of ​​the feature regions included in each group.

[0109] In some embodiments, the reliability calculation unit 45 is specifically used to calculate the reliability of each group based on the feature amount of each feature region in each group, the number of feature regions included in each group, and the area of ​​the feature regions included in each group.

[0110] In some embodiments, the reliability calculation unit 45 specifically performs parallel calculations on one or more groups, with each group as a unit, based on the feature amount of each feature region in at least each group, and calculates the reliability of each group separately.

[0111] In some embodiments, the focus adjustment device 40 further includes a calculation output unit, which is used to calculate a second defocus amount for each of the one or more groups based on the first defocus amount of each feature region for each of the one or more groups, and output the second defocus amount and a reliability for each of the one or more groups.

[0112] In some embodiments, the calculation output unit is specifically used to determine the average value of the first defocus amounts of the feature regions for each group as the second defocus amount of the group.

[0113] The focus adjustment device 40 provided by the present invention corresponds to the above-mentioned focus adjustment device control method, and for a more detailed description, please refer to the contents of each embodiment of the above-mentioned focus adjustment device control method.

[0114] Based on the same inventive concept, the present invention provides yet another focus adjustment device 40. The focus adjustment device 40 includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to perform the focus adjustment device control method described in any of the above-described embodiments.

[0115] 11, which is a schematic diagram showing the structure of an imaging device provided by an embodiment of the present invention. Based on the same inventive concept, the present invention further provides an imaging device 1. The imaging device 1 includes a focusing device 40 described in any of the above-mentioned embodiments.

[0116] In some embodiments, the imaging device 1 further includes an optical lens unit 10, a lens driving unit 20, an image sensor unit 30, and an image data generating unit 50.

[0117] The image sensor unit 30 outputs a pair of pupil-divided subject images to the focus adjustment device 40, and outputs RAW data to the image data generation unit 50.

[0118] The optical lens unit 10 performs pupil division of the subject light beam and focuses the subject image on the image sensor unit 30. The optical lens unit 10 further moves based on a focus control signal from the lens driving unit 20 to bring the subject into focus.

[0119] The lens driving unit 20 generates a focus control signal based on the information output from the focus adjustment device 40 (for example, the second defocus amount and reliability for each group), and outputs the focus control signal to the optical lens unit 10.

[0120] The image sensor unit 30 generates RAW data for each pixel based on the optical image of the subject focused by the optical lens unit 10, and outputs the RAW data for each pixel to the image data generation unit 50. The image sensor unit 30 also generates a pair of pupil-divided subject images, and outputs the pair of subject images to the focus adjustment device 40.

[0121] The image data generating unit 50 generates image data by performing predetermined signal processing on the RAW data of each pixel, and outputs the image data to the outside.

[0122] Based on the same inventive concept, the present invention further provides a computer-readable storage medium having executable instructions stored therein, which, when executed by a processor, performs the method for controlling a focusing device described in any of the above-described embodiments.

[0123] The computer storage medium of an embodiment of the present invention may employ any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-limiting list) of computer-readable storage media include an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium that contains or stores a program. The program may be used by or in combination with an instruction execution system, apparatus, or device.

[0124] A computer-readable signal medium may include a data signal, either transmitted in baseband or propagated as part of a carrier wave, carrying computer-readable program code. The data signal may include, but is not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium. A computer-readable medium may transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0125] The program code contained in the computer readable medium may be transmitted by any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, or the like, or any suitable combination of the above.

[0126] Computer program code for carrying out operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and the like, as well as conventional procedural programming languages ​​such as C. The program code can run entirely on the user's computer, partially on the user's computer, as a separate package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0127] Although embodiments of the present invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited only by the claims.

Claims

1. A control method for a focus adjustment device, which is applied to a focus adjustment device, the control method for the focus adjustment device comprising: searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting light beams of a plurality of objects, the first image including a left-eye image and a right-eye image, the first image being the left-eye image or the right-eye image; calculating a correlation value for each feature region based on the pair of subject images; calculating a first defocus amount for each feature region based on a correlation value calculation result for each feature region; Dividing the one or more feature regions into one or more groups based on a first defocus amount and coordinate position information of each feature region, each group including at least one feature region, and each feature region being included in one group; calculating a reliability for each group based on at least the feature amount of each feature region in each group and the number or area of ​​the feature regions, and the first defocus amount of each feature region in each group and the reliability for each group are both used to determine a defocus amount of an object at one shooting distance corresponding to the group; 10. A control method for a focus adjustment device, comprising:

2. Searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting a light beam of a plurality of objects, based on the first image, Dividing a first image into a plurality of sub-regions based on a first image in a pair of object images obtained by pupil-splitting a plurality of object light beams; performing parallel searches on the plurality of sub-regions in units of sub-regions to search for feature regions separately from the plurality of sub-regions; Including, 2. The control method for a focus adjustment device according to claim 1.

3. performing parallel searches on the plurality of sub-regions in units of sub-regions and separately searching for feature regions from the plurality of sub-regions; comparing a feature amount for each local region in the plurality of sub-regions with a preset feature amount threshold value, with each sub-region being a unit; When the feature amount of one of the local regions is greater than the preset feature amount threshold, the local region is determined to be the feature region; Including, 3. The control method for a focus adjustment device according to claim 2.

4. Calculating correlation values ​​for each feature region is performing parallel calculations on the one or more feature regions in units of feature regions to calculate correlation values ​​for each feature region separately; 2. The control method for a focus adjustment device according to claim 1.

5. Calculating the first defocus amount for each feature region includes: performing parallel calculations on the one or more feature regions in units of feature regions to calculate the first defocus amount for each feature region separately; 2. The control method for a focus adjustment device according to claim 1.

6. Dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region includes: determining a region where a difference in first defocus amounts between adjacent feature regions exceeds a preset first defocus amount difference threshold based on coordinate position information of each feature region as a dividing line between the adjacent feature regions; Integrating feature regions surrounded by the same dividing line into the same group; Including, 2. The control method for a focus adjustment device according to claim 1.

7. Dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region includes: Dividing the one or more feature regions into a predetermined number of blocks based on coordinate position information of each feature region; Calculating an average value of the first defocus amount of the feature region included in each block; Integrating clusters of feature regions whose difference in the average values ​​of the first defocus amounts is smaller than a preset threshold value for the difference in the average values ​​of the first defocus amounts and whose positions are adjacent into the same group; Including, 2. The control method for a focus adjustment device according to claim 1.

8. Calculating the reliability of each group based on at least the feature amount of each feature region in each group and performing parallel calculations on one or more groups on a group-by-group basis based on the feature amounts of each feature region in at least each group, to separately calculate the reliability of each group.

2. The control method for a focus adjustment device according to claim 1.

9. A focusing device, The apparatus includes a feature region search unit, a correlation calculation unit, a defocus amount calculation unit, a grouping processing unit, and a reliability calculation unit, the characteristic region search unit is used to search for one or more characteristic regions from a first image of a pair of subject images obtained by pupil-splitting a light beam of a plurality of subjects, the pair of subject images including a left-eye image and a right-eye image, and the first image is the left-eye image or the right-eye image; the correlation calculation unit is used to calculate a correlation value for each feature region based on the pair of subject images, the defocus amount calculation unit is used to calculate a first defocus amount for each feature region based on a correlation value calculation result for each feature region, the grouping processing unit is used to divide the one or more feature regions into one or more groups based on a first defocus amount and coordinate position information of each feature region, each group including at least one feature region, and each feature region is included in one group; the reliability calculation unit is used to calculate the reliability of each group based on at least the feature amount of each feature region in each group and the number or area of ​​the feature regions, and the first defocus amount of each feature region in each group and the reliability of each group are both used to determine the defocus amount of an object at one shooting distance corresponding to the group. A focusing device characterized by:

10. A focusing device, A focus adjustment device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to execute the focus adjustment device control method according to any one of claims 1 to 8.

11. An imaging device, An imaging device comprising the focus adjustment device according to claim 9.

12. A computer program causing a processor to execute the control method for a focus adjustment device according to any one of claims 1 to 8.

Images