Focusing device and its control method, imaging device and computer-readable storage medium

Abstract

The present invention provides a focus adjustment device and a control method thereof, an imaging device, and a computer-readable storage medium. [Solution] A control method for a focus adjustment device includes: searching for one or more characteristic regions from a first image in a pair of subject images obtained by pupil-splitting light beams from multiple subjects, calculating a correlation value for each characteristic region based on the pair of subject images; calculating a first defocus amount for each characteristic region based on the correlation value calculation results for each characteristic region; dividing the one or more characteristic regions into one or more groups based on the first defocus amount and coordinate position information of each characteristic region; and calculating a reliability for each group based on at least the feature amount of each characteristic region and the number or area of ​​the characteristic regions in each group.

Description

[Technical Field]

[0001] The present invention relates to the field of imaging technology, and more particularly to a focus adjustment device and a control method thereof, an imaging device, and a computer-readable storage medium. [Background technology]

[0002] Currently, the split-pupil phase difference focusing method is a common autofocusing method. The principle of this focusing method is to split the light beam from the subject on the exit pupil plane, focus the split light beam on a pair of pixel rows on a two-dimensional image sensor, and perform focus detection based on the phase difference between the image signals (i.e., the left-eye image signal and the right-eye image signal) after photoelectric conversion of the pair of pixel rows.

[0003] When there are multiple subjects with different shooting distances, the image sensor may output a mixed value of multiple phase differences, which may make it impossible to selectively focus on the subject with the closest shooting distance or the subject with an intermediate shooting distance, among the multiple subjects. Summary of the Invention [Problem to be solved by the invention]

[0004] In order to solve the above-mentioned technical problems, or to at least partially solve the above-mentioned technical problems, the present invention provides a focusing device and a control method thereof, an imaging device, and a computer-readable storage medium. [Means for solving the problem]

[0005] To achieve the above-mentioned object, a first aspect of the present invention provides a control method for a focus adjustment device, which is applied to a focus adjustment device. The control method for a focus adjustment device includes: searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting light beams of multiple objects, the pair of object images including a left-eye image and a right-eye image, the first image being either the left-eye image or the right-eye image; calculating a correlation value for each feature region based on the pair of object images; calculating a first defocus amount for each feature region based on the correlation value calculation result for each feature region; dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region; and calculating a reliability for each group based on at least the feature amount of each feature region and the number or area of ​​the feature regions in each group, wherein the average first defocus amount of each feature region in each group and the reliability for each group are both used to determine the defocus amount of an object at one shooting distance corresponding to the group.

[0006] The control method for a focus adjustment device provided by the present application searches for one or more feature regions from a first image, performs a correlation value calculation for each feature region, calculates a first defocus amount for each feature region, and divides the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region, thereby associating the feature regions of different groups with subjects at different shooting distances.Accordingly, the defocus amount for a subject at one shooting distance corresponding to each group can be determined based on the first defocus amount of each feature region in each group, and it is possible to selectively focus on one of the multiple subjects, subjects at different shooting distances such as a subject at the closest shooting distance, a subject at an intermediate distance, or a subject at the farthest shooting distance, or a subject with a high reliability.

[0007] A second aspect of the present invention provides a focus adjustment device. The focus adjustment device includes a feature region search unit, a correlation calculation unit, a defocus amount calculation unit, a grouping processing unit, and a reliability calculation unit. The feature region search unit is used to search for one or more feature regions from a pair of object images based on a first image in the pair of object images obtained by pupil-splitting light beams from multiple objects, the pair of object images including a left-eye image and a right-eye image, and the first image is either the left-eye image or the right-eye image. The correlation calculation unit is used to calculate a correlation value for each feature region based on the pair of object images. The defocus amount calculation unit is used to calculate a first defocus amount for each feature region based on the correlation value calculation result for each feature region. The grouping processing unit is used to divide one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region. The reliability calculation unit is used to calculate the reliability for each group based on at least the feature amount of each feature area in each group and the number or area of ​​the feature areas, and the first defocus amount average of each feature area in each group and the reliability for each group are both used to determine the defocus amount of the subject at one shooting distance corresponding to the group.

[0008] A third aspect of the present invention provides a focus adjustment device, the focus adjustment device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the focus adjustment device control method of the first aspect.

[0009] In a fourth aspect of the present invention, there is provided an imaging device, the imaging device comprising the focus adjustment device of the second aspect.

[0010] In a fifth aspect of the present invention, there is provided a computer-readable storage medium having executable instructions stored thereon, the executable instructions, when executed by a processor, performing the method for controlling a focus adjustment device of the first aspect. [Brief explanation of the drawings]

[0011] [Figure 1]4 is a first flowchart of a control method for a focusing device provided by an embodiment of the present invention; [Figure 2] 10 is a second flowchart of a control method for a focusing device provided by an embodiment of the present invention. [Figure 3] 2 is a schematic diagram illustrating a pair of object images provided by an embodiment of the present invention; [Figure 4] FIG. 2 is a first schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 5] FIG. 4 is a second schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 6] FIG. 10 is a third schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 7] This is a fourth schematic diagram of a method for grouping feature regions provided by embodiments of the present invention. [Figure 8] FIG. 5 is a fifth schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention. [Figure 9] FIG. 6 is a sixth schematic diagram of a method for grouping feature regions provided by an embodiment of the present invention; [Figure 10] This is a schematic diagram showing the structure of a focus adjustment device provided by an embodiment of the present invention. [Figure 11] 1 is a schematic diagram illustrating the structure of an imaging device provided by an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, the technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments that can be obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0013] The terms "first," "second," etc. described in the present specification are used merely to distinguish the objects being described and do not have any ordering or technical significance. Furthermore, the terms "comprise," "include," or variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units may include not only the listed steps or units, but also steps or units not listed, or may include other steps or units inherent to such a process, method, product, or apparatus.

[0014] It should be noted that features in the embodiments of the present invention may be combined with each other as long as they do not conflict.

[0015] Referring to Figures 1 and 2, Figure 1 is a first flowchart of a control method for a focus adjustment device provided by an embodiment of the present invention, and Figure 2 is a second flowchart of a control method for a focus adjustment device provided by an embodiment of the present invention.

[0016] As shown in Fig. 1, the present invention provides a control method for a focus adjustment device. The control method for a focus adjustment device is applied to a focus adjustment device. The control method for a focus adjustment device includes the following steps S1 to S4.

[0017] In step S1, one or more feature regions are searched for in a first image of a pair of object images obtained by pupil-splitting a light beam from a plurality of objects. The pair of object images includes a left-eye image and a right-eye image, and the first image is the left-eye image or the right-eye image.

[0018] In step S2, a correlation value is calculated for each feature region based on the pair of subject images.

[0019] In step S3, a first defocus amount is calculated for each feature region based on the correlation value calculation result for each feature region.

[0020] In step S4, one or more feature regions are divided into one or more groups based on the first defocus amount and coordinate position information of each feature region.

[0021] In step S5, a reliability for each group is calculated based on at least the feature amount of each feature region in each group. The average first defocus amount of each feature region in each group and the reliability for each group are used to determine the defocus amount of the object at one shooting distance corresponding to the group.

[0022] By calculating the reliability of each group, it is possible to select multiple groups based on the reliability, and therefore it is possible to selectively focus on one of multiple subjects, subjects at different shooting distances such as the subject at the closest shooting distance or the subject at an intermediate distance, or the subject at the farthest shooting distance, or subjects with high reliability.

[0023] In the present invention, when calculating reliability, it is necessary to weight each condition (for example, the number and / or area of ​​feature regions in each group) to obtain the reliability. In this way, even if the average value of the magnitude of the feature amount for each feature region of a certain group is small, if the number or area of ​​the feature regions of that group is larger than those of other groups, the reliability obtained by weighting will be high. Using groups as units, the larger the feature amount of each feature region in each group, the higher the reliability; the larger the number of feature regions included in each group, the higher the reliability; and the larger the area of ​​the feature regions included in each group, the higher the reliability.

[0024] The control method for a focus adjustment device provided by the present invention searches for one or more feature regions from a first image, performs a correlation value calculation for each feature region, calculates a first defocus amount for each feature region, and divides the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region, thereby associating the feature regions of different groups with subjects at different shooting distances.Accordingly, the defocus amount for a subject at one shooting distance corresponding to each group can be determined based on the first defocus amount of each feature region in each group, and it is possible to selectively focus on one of the multiple subjects, subjects at different shooting distances such as a subject at the closest shooting distance, a subject at an intermediate distance, or a subject at the farthest shooting distance, or a subject with a high reliability.

[0025] A pair of subject images obtained by pupil-splitting light beams from multiple subjects refers to a pair of subject images (including a left-eye image and a right-eye image) obtained by pupil-splitting light beams from multiple subjects and then projecting them onto different image sensors or different image sensor areas. When multiple subjects are at different shooting distances, for example, as shown in Figure 3, one scene has four subjects (including subjects 101 to 104) and the shooting distance increases sequentially from subject 101 to subject 104, and a conventional pupil-splitting phase-difference autofocus method cannot selectively focus on the subject (subject 101) that is closest to the subject 101 to subject 104 or the subject (subject 102 or subject 103) that is located at an intermediate shooting distance.

[0026] The term "defocus amount" refers to the distance between the imaging position of the subject and the imaging plane of the optical lens unit. When the imaging position of the subject is exactly located on the imaging plane, the optical lens unit is in a focused state, when the imaging position of the subject is on the side of the imaging plane closer to the subject, the optical lens unit is in a front defocused state, and when the imaging position of the subject is on the side of the imaging plane farther from the subject, the optical lens unit is in a rear defocused state.

[0027] The term "correlation value" refers to a metric of the similarity (correlation) between two signals. Specifically, when the optical lens unit is in focus with respect to a certain feature region, the left-eye image and right-eye image of that feature region are the same, the phase difference (also called image shift amount) between them is zero, and the correlation is highest. When the optical lens unit is in a pre-defocused or post-defocused state with respect to that feature region, the phase difference between the left-eye image and right-eye image of that feature region is not zero, and there is a proportional relationship between the phase difference between the left-eye image and right-eye image for each feature region and the defocus amount of that feature region. In other words, the defocus amount increases as the phase difference between the left-eye image and right-eye image increases. The direction of the phase difference corresponds to the direction of the defocus amount (the side of the imaging plane closer to the subject / the side of the imaging plane farther from the subject).

[0028] By calculating a correlation value for each feature region, a phase difference between the left-eye image and the right-eye image for each feature region is obtained, and therefore a first defocus amount for each feature region is obtained.One or more feature regions can be divided into one or more groups based on the first defocus amount and coordinate position information of each feature region, and the feature regions of different groups can be associated with subjects at different shooting distances.By selecting a feature region of one group from the plurality of groups of feature regions, determining the movement direction and movement amount of the optical lens unit based on the first defocus amount of the feature region of that group, and controlling the movement of the optical lens unit based on the movement direction and movement amount, it is possible to selectively focus on one of the plurality of subjects, subjects at different shooting distances such as a subject at the closest shooting distance, a subject at an intermediate distance, or a subject at the farthest shooting distance, or a subject with a high reliability.

[0029] In an embodiment of the present invention, a feature region refers to a local region whose feature quantity extracted from a first image satisfies certain requirements. For example, if the feature quantity of a certain local region is greater than a preset feature quantity threshold, the local region is determined to be a feature region. The feature quantity is determined by the specifications of the image sensor, and illustratively, the feature quantity includes contrast, color, etc.

[0030] In some embodiments, searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting a plurality of object light beams includes: Based on the first image of a pair of subject images obtained by pupil splitting of the light beams of multiple subjects, the first image is divided into multiple sub-regions, This involves performing parallel searches on multiple subdomains, using subdomains as the unit, and searching for feature regions separately from multiple subdomains. Includes:

[0031] As shown in Figure 2, multiple CPU (Central Processing Unit) cores can perform parallel searches across multiple sub-regions.

[0032] In this way, parallel searches can be performed on multiple subdomains to improve search speed.

[0033] In some embodiments, performing parallel searches on a plurality of sub-regions in units of sub-regions and separately searching for feature regions from the plurality of sub-regions includes: This involves comparing local region-specific features and preset feature thresholds across multiple sub-regions, using sub-regions as the unit. If the feature quantity of a local region is greater than the preset feature threshold, that local region is determined to be a feature region. Including,

[0034] The preset feature threshold is preferably a threshold that is not affected by noise. When the feature region is large, there is a high possibility that subjects from different shooting distances are mixed together, so a smaller feature region is preferable, and therefore image signals in which phase shift amounts from multiple subjects from different shooting distances are mixed together are more easily excluded.

[0035] In the present invention, by determining as feature regions only local regions whose feature amounts are greater than the preset feature amount threshold, it is possible to eliminate the influence of noise and the like and improve focusing accuracy.

[0036] In some embodiments, the correlation value calculation result includes an integer part min of the phase difference of the feature region, where min is also called the maximum correlation shift amount.

[0037] Calculating correlation values ​​for each feature region is calculating a correlation value when the feature region in the second image is shifted by a different preset shift amount by a preset correlation value calculation method based on a pixel value of each pixel in the feature region in the first image and a pixel value when each pixel in the feature region in the second image is shifted by a different preset shift amount, wherein the first image is one of a left-eye image and a right-eye image, the second image is the other of the left-eye image and the right-eye image, and the preset shift amount is an integer; comparing correlation values ​​when the feature region in the second image is shifted by different preset shift amounts, and determining the preset shift amount corresponding to the highest correlation value as an integer part of the phase difference of the feature region; Includes:

[0038] In the embodiments of the present invention, unless otherwise stated, the term "phase difference of a feature region" refers to the amount of shift between the left-eye image and the right-eye image when there is the highest correlation between the pixel values ​​of the feature region in the left-eye image and the pixel values ​​of the feature region in the right-eye image.

[0039] In some embodiments, the preset correlation value calculation method includes, but is not limited to, a sum of absolute differences (SAD) calculation method, a sum of squared differences (SSD) calculation method, and a normalized cross-correlation (NCC) calculation method.

[0040] For example, assuming that the preset correlation value calculation method is a sum of absolute difference calculation method, calculating a correlation value when the feature region in the second image is shifted by different preset shift amounts by the preset correlation value calculation method based on the pixel value of each pixel in the feature region in the first image and the pixel value when each pixel in the feature region in the second image is shifted by different preset shift amounts is as follows: The steps include fixing the pixel values ​​of each pixel in the feature region in the first image, Shifting each pixel of the characteristic region in the second image by a different preset shift amount separately, and obtaining pixel values ​​when each pixel of the characteristic region in the second image is shifted by a different preset shift amount; Separately calculating differences between pixel values ​​of each pixel in the characteristic region in the first image and pixel values ​​of each pixel in the characteristic region in the second image when the pixel is shifted by different preset shift amounts; determining the sum of absolute values ​​of the differences between the pixel values ​​of each pixel in the characteristic region in the first image and the pixel values ​​of each pixel in the characteristic region in the second image when the pixel is shifted by an arbitrary preset shift amount as a correlation value when the characteristic region in the second image is shifted by an arbitrary preset shift amount, thereby obtaining a correlation value when the characteristic region in the second image is shifted by a different preset shift amount; Includes:

[0041] All different preset shift amounts are within the preset shift range.

[0042] Specifically, if the preset correlation value calculation method is the sum of absolute difference calculation method, the correlation value of each feature region can be calculated by the following formula.

number

[0043] In some embodiments, the correlation calculation result further includes the decimal part of the phase difference for each feature region.

[0044] Calculating the correlation value for each feature region further includes calculating the decimal part of the phase difference of the feature region by a preset curve approximation method based on the correlation value corresponding to the integer part of the phase difference of the feature region and the correlation values ​​corresponding to two shift amounts adjacent to the integer part of the phase difference.

[0045] The integer part of the phase difference is represented by min, and the two shift amounts adjacent to the integer part of the phase difference include a shift amount min-1 before the integer part of the phase difference and a shift amount min+1 after the integer part of the phase difference.

[0046] When calculating the correlation value, the value of the preset shift amount i takes only an integer without considering the decimal part, so there may be an error between min and the actual phase difference. In this embodiment, after calculating the integer part min of the phase difference of the feature region, the decimal part j of the phase difference of the feature region is calculated by a preset curve approximation method, and the first defocus amount of the feature region is calculated based on min+j, so that a more accurate first defocus amount can be calculated.

[0047] In some embodiments, the curve fitting method is a piecewise linear fitting method or a parabolic fitting method.

[0048] Exemplarily, taking the case where the curve approximation method is the equidistant straight line approximation method, based on the correlation value corresponding to the integer part of the phase difference of the feature region and the correlation values corresponding to two shift amounts adjacent to the integer part of the phase difference, calculating the fractional part of the phase difference of the feature region by the preset curve approximation method is when the correlation value corresponding to the shift amount before the integer part of the phase difference is greater than or equal to the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) ≥ S(min + 1), calculating the fractional part of the phase difference by the first formula, and when the correlation value corresponding to the shift amount before the integer part of the phase difference is less than the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) < S(min + 1), calculating the fractional part of the phase difference by the second formula, and are included.

[0049] The first formula is j = [S(min - 1) - S(min + 1)] / [2×S(min - 1) - 2×S(min)].

[0050] The second formula is j = [S(min + 1) - S(min - 1)] / [2×S(min + 1) - 2×S(min)].

[0051] j is the fractional part of the phase difference, S(min) is the correlation value corresponding to the integer part of the phase difference, S(min - 1) is the correlation value corresponding to the shift amount before the integer part of the phase difference, and S(min + 1) is the correlation value corresponding to the shift amount after the integer part of the phase difference.

[0052] In some embodiments, performing the correlation value calculation for each feature region includes performing parallel calculations on one or more feature regions in units of feature regions to separately calculate the correlation value for each feature region.

[0053] As shown in FIG. 2, parallel calculations can be performed on one or more feature regions by a plurality of CPU cores.

[0054] Performing parallel operations on one or more feature regions can improve computation speed.

[0055] In some embodiments, calculating the first defocus amount for each feature region includes performing parallel calculations on one or more feature regions on a feature region-by-feature region basis to separately calculate the first defocus amount for each feature region.

[0056] As shown in FIG. 2, multiple CPU cores can be used to perform parallel calculations on one or more feature regions.

[0057] In this way, performing parallel operations on one or more feature regions can improve computation speed.

[0058] Since calculating the first defocus amount of a feature region based on the phase difference of the feature region is a conventional technique, the present invention will not be described in detail.

[0059] The present invention provides two grouping methods for grouping feature regions. The two grouping methods are a dividing line grouping method and a block grouping method. The two grouping methods will be introduced in detail below with reference to FIGS. 4 to 9.

[0060] Referring to FIG. 4, in some embodiments, the feature regions can be grouped using a dividing line grouping method, where one or more feature regions are divided into one or more groups based on the first defocus amount and coordinate position information of each feature region, such as: determining a region where a difference in first defocus amounts between adjacent feature regions exceeds a preset first defocus amount difference threshold based on coordinate position information of the feature regions as a dividing line between the adjacent feature regions; Integrating feature regions enclosed by the same dividing line into the same group, Includes:

[0061] For example, a region where the difference in the first defocus amount between adjacent feature regions, such as the top, bottom, left, right, upper left diagonal, upper right diagonal, lower left diagonal, or lower right diagonal, exceeds a preset first defocus amount difference threshold, can be set as a dividing line between adjacent feature regions. As shown in Fig. 4, one or more feature regions can be divided into two groups by the dividing line, and the feature regions in the two groups correspond to two objects at different shooting distances.

[0062] In some examples, integrating feature regions enclosed by the same dividing line into the same group is possible. determining an isolated dividing line that cannot divide the feature region as an invalid dividing line; Integrating feature regions surrounded by the same effective dividing line into the same group; Includes:

[0063] As illustrated in Figure 5, in some scenes, when using the segmented line grouping method, it is possible to obtain isolated segmented lines. Therefore, by identifying these isolated segmented lines as invalid segmented lines and selecting and grouping only the valid segmented lines, the grouping efficiency can be improved.

[0064] In some embodiments, dividing one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region is: If a disconnected line cannot be generated using the disconnected line grouping method, the method further includes dividing one or more feature regions into one or more groups using the block grouping method.

[0065] In some embodiments, after integrating the feature regions surrounded by the same effective dividing line into the same group, the control method of the focus adjustment device includes: This further includes calculating the average first defocus amount for each feature region in each group, and merging groups into the same group if the difference between the average first defocus amounts is less than a preset average defocus amount difference threshold and the locations are adjacent.

[0066] For example, as shown in FIG. 6, depending on the shooting conditions, the type of subject, etc., a single subject may be divided into one or more groups. Therefore, by merging groups whose difference in the average values ​​of the first defocus amounts is smaller than the preset defocus amount average value difference threshold and whose positions are adjacent into the same group, the accuracy of the grouping can be improved, and it is possible to prevent a single subject from being divided into multiple groups as much as possible.

[0067] Referring to Figures 7 and 8, in some embodiments, feature regions can be grouped using a block grouping method. In this case, one or more feature regions can be divided into one or more groups based on the first defocus amount and coordinate position information of each feature region. Dividing one or more feature regions into a predetermined number of blocks based on coordinate position information of each feature region; Calculating an average value of the first defocus amount of the feature region included in each block; The first group is formed when the difference in the average values ​​of the defocus amounts is smaller than the preset threshold for the difference in average values ​​of the defocus amounts, and when clusters of feature regions that are adjacent in location are merged into the same group. Includes:

[0068] For example, when grouping feature regions using a block grouping method, the first defocus amount of the feature region can be processed using an average reduction / expansion method to achieve feature region grouping.

[0069] Specifically, as shown in FIG. 8, the average reduction method is as follows: Dividing one or more feature regions into a predetermined number of blocks (e.g., 6 x 8 blocks); Calculating an average value of the first defocus amount of the feature region included in each block; shrinking each block according to an average value of the first defocus amount; The first group is formed when the difference in the average values ​​of the defocus amounts is smaller than the preset threshold for the difference in average values ​​of the defocus amounts, and when clusters of feature regions that are adjacent in location are merged into the same group. Includes:

[0070] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes calculating the reliability of each group based on the feature values ​​of each feature region in each group and the number of feature regions included in each group.

[0071] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes calculating the reliability of each group based on the feature values ​​of each feature region in each group and the area of ​​the feature regions included in each group.

[0072] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes calculating the reliability of each group based on the feature values ​​of each feature region in each group, the number of feature regions included in each group, and the area of ​​the feature regions included in each group.

[0073] In some embodiments, calculating the reliability of each group based on at least the feature values ​​of each feature region in each group includes performing parallel calculations on one or more groups on a group-by-group basis, based on at least the feature values ​​of each feature region in each group, to calculate the reliability of each group separately.

[0074] As shown in FIG. 2, multiple CPU cores can perform parallel operations on one or more groups.

[0075] In this way, parallel calculations can be performed on one or more groups, improving the calculation speed.

[0076] As shown in Figure 9, in some scenes, a single subject may be divided into one or more groups due to the size of each block, the shooting conditions, the type of subject, and other factors. Therefore, in these cases, the reliability of the divided groups should be reduced.

[0077] In some embodiments, after calculating the confidence level for each group based on the feature quantities of each feature region in at least each group, the control method for the focus adjustment device is as follows: Based on the first defocus amount for each feature region in one or more groups, the second defocus amount for each group in one or more groups is calculated. outputting a second defocus amount and a reliability for each of the one or more groups; Further includes:

[0078] The second defocus amount and confidence level for each group are used together to determine the defocus amount of the subject at the corresponding shooting distance for that group.

[0079] In this way, one feature region of one group corresponds to one object, and the second defocus amount of the group is the defocus amount of the corresponding object. By determining the target movement direction and target movement amount of the optical lens unit based on the second defocus amount of the group and controlling the movement of the optical lens unit based on the target movement direction and target movement amount, it is possible to achieve focusing on the object corresponding to the group.

[0080] In some embodiments, calculating the second defocus amount for each of the one or more groups based on the first defocus amount for each of the feature regions for each of the one or more groups includes determining an average value of the first defocus amounts for each of the feature regions for each of the groups as the second defocus amount for the group.

[0081] For example, suppose a group contains three feature regions, and the first defocus amounts for the three feature regions are 0.4, 0.5, and 0.6, respectively. Then the second defocus amount for that group is (0.4 + 0.5 + 0.6) / 3 = 0.5.

[0082] Referring to Figure 10, Figure 10 is a schematic diagram showing the structure of a focus adjustment device provided by an embodiment of the present invention. Based on the same inventive concept, the present invention further provides a focus adjustment device 40 comprising a feature area search unit 41, a correlation calculation unit 42, a defocus amount calculation unit 43, a grouping processing unit 44, and a reliability calculation unit 45.

[0083] The feature region search unit 41 is used to search for one or more feature regions from the first image of a pair of subject images obtained by pupil splitting of the light beams of multiple subjects. The pair of subject images includes a left eye image and a right eye image, and the first image is either the left eye image or the right eye image.

[0084] The correlation calculation unit 42 is used to calculate a correlation value for each feature region based on a pair of subject images.

[0085] The defocus amount calculation unit 43 is used to calculate a first defocus amount for each feature region based on the correlation value calculation result for each feature region.

[0086] The grouping processing unit 44 is used to divide one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region.

[0087] The confidence calculation unit 45 is used to calculate the confidence level for each group based on the feature quantities and the number or area of ​​each feature region in each group.

[0088] The first average defocus amount for each feature region within each group, as well as the confidence level for each group, are both used to determine the defocus amount of a subject at a single shooting distance corresponding to that group.

[0089] In some embodiments, the characteristic region search unit 41 is specifically used to divide a first image of a pair of subject images obtained by pupil-splitting light beams from a plurality of subjects into a plurality of sub-regions, and to perform parallel searches on the plurality of sub-regions on a sub-region basis to separately search for characteristic regions from the plurality of sub-regions.

[0090] In some embodiments, the characteristic region search unit 41 is specifically used to compare the feature values ​​of each local region in a plurality of subregions with a preset feature value threshold value, with the subregion as a unit, and if the feature value of one local region is greater than the preset feature value threshold value, to determine that local region as a characteristic region.

[0091] In some embodiments, the correlation calculation unit 42 is specifically used to calculate correlation values ​​when the feature region in the second image is shifted by different preset shift amounts using a preset correlation value calculation method, based on the pixel values ​​of each pixel in the feature region in the first image and the pixel values ​​of each pixel in the feature region in the second image when the pixel is shifted by different preset shift amounts, compare the correlation values ​​when the feature region in the second image is shifted by different preset shift amounts, and determine the preset shift amount corresponding to the highest correlation value as the integer part of the phase difference of the feature region.

[0092] The first image is one of the left-eye and right-eye images, the second image is the other of the left-eye and right-eye images, and the preset shift amount is an integer.

[0093] In some embodiments, specifically, the correlation calculation unit 42 fixes the pixel value of each pixel in the feature region in the first image, separately shifts each pixel in the feature region in the second image by different preset shift amounts, obtains the pixel values when each pixel in the feature region in the second image is shifted by different preset shift amounts, separately calculates the difference between the pixel value of each pixel in the feature region in the first image and the pixel values when each pixel in the feature region in the second image is shifted by different preset shift amounts, determines the sum of absolute values of the differences between the pixel value of each pixel in the feature region in the first image and the pixel values when each pixel in the feature region in the second image is shifted by an arbitrary preset shift amount as the correlation value when the feature region in the second image is shifted by an arbitrary preset shift amount, and thus is used to obtain the correlation values when the feature region in the second image is shifted by different preset shift amounts.

[0094] In some embodiments, the correlation calculation unit 42 is further used to calculate the fractional part of the phase difference of the feature region by the preset curve approximation method based on the correlation value corresponding to the integer part of the phase difference of the feature region and the correlation values corresponding to two shift amounts adjacent to the integer part of the phase difference.

[0095] In some embodiments, the correlation calculation unit 42 is further used to calculate the fractional part of the phase difference by the first formula when the correlation value corresponding to the shift amount before the integer part of the phase difference is greater than or equal to the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) ≥ S(min + 1), and calculate the fractional part of the phase difference by the second formula when the correlation value corresponding to the shift amount before the integer part of the phase difference is less than the correlation value corresponding to the shift amount after the integer part of the phase difference, that is, when S(min - 1) < S(min + 1).

[0096] The first formula is j = [S(min - 1) - S(min + 1)] / [2×S(min - 1) - 2×S(min)].

[0097] The second formula is j=[S(min+1)-S(min-1)] / [2×S(min+1)-2×S(min)].

[0098] j is the decimal part of the phase difference, S(min) is the correlation value corresponding to the integer part of the phase difference, S(min-1) is the correlation value corresponding to the shift amount before the integer part of the phase difference, and S(min+1) is the correlation value corresponding to the shift amount after the integer part of the phase difference.

[0099] In some embodiments, the correlation calculation unit 42 is specifically used to perform parallel calculations on one or more feature regions, with each feature region being a unit, to separately calculate a correlation value for each feature region.

[0100] In some embodiments, the defocus amount calculation unit 43 is specifically used to perform parallel calculations on one or more feature regions, with each feature region being a unit, to separately calculate the first defocus amount for each feature region.

[0101] In some embodiments, the grouping processing unit 44 is used to integrate feature regions into the same group by defining a dividing line between adjacent feature regions where the difference in the first defocus amount of adjacent feature regions exceeds a preset first defocus amount difference threshold, based on the coordinate position information of the feature region.

[0102] In some embodiments, the grouping processing unit 44 is specifically used to determine an isolated dividing line that cannot divide a feature region as an invalid dividing line, and to combine feature regions surrounded by the same valid dividing line into the same group.

[0103] In some embodiments, the grouping processor 44 is further adapted to employ a chunk grouping method to divide one or more feature regions into one or more groups if a dividing line cannot be generated using the dividing line grouping method.

[0104] In some embodiments, the grouping processing unit 44 further combines feature regions surrounded by the same effective dividing line into the same group, calculates the average value of the first defocus amount of each feature region for each group, and combines groups whose difference in the average values ​​of the first defocus amounts is smaller than the preset defocus amount average difference threshold and which are adjacent to each other into the same group.

[0105] In some embodiments, the grouping processing unit 44 is specifically used to divide one or more feature regions into a preset number of blocks based on the coordinate position information of each feature region, calculate the average value of the first defocus amount of the feature regions included in each block, and merge the blocks of feature regions whose difference in the average value of the first defocus amount is smaller than the preset defocus amount average value difference threshold and whose positions are adjacent into the same group.

[0106] The first defocus amount of each feature region in each group and the reliability of each group are both used to determine the defocus amount of the object at one shooting distance corresponding to that group.

[0107] In some embodiments, the reliability calculation unit 45 is specifically used to calculate the reliability of each group based on the feature amount of each feature region in each group and the number of feature regions included in each group.

[0108] In some embodiments, the reliability calculation unit 45 is specifically used to calculate the reliability of each group based on the feature amount of each feature region in each group and the area of ​​the feature regions included in each group.

[0109] In some embodiments, the reliability calculation unit 45 is specifically used to calculate the reliability of each group based on the feature amount of each feature region in each group, the number of feature regions included in each group, and the area of ​​the feature regions included in each group.

[0110] In some embodiments, the reliability calculation unit 45 specifically performs parallel calculations on one or more groups, with each group as a unit, based on the feature amount of each feature region in at least each group, and calculates the reliability of each group separately.

[0111] In some embodiments, the focus adjustment device 40 further includes a calculation output unit, which is used to calculate a second defocus amount for each of the one or more groups based on the first defocus amount of each feature region for each of the one or more groups, and output the second defocus amount and a reliability for each of the one or more groups.

[0112] In some embodiments, the calculation output unit is specifically used to determine the average value of the first defocus amounts of the feature regions for each group as the second defocus amount of the group.

[0113] The focus adjustment device 40 provided by the present invention corresponds to the above-mentioned focus adjustment device control method, and for a more detailed description, please refer to the contents of each embodiment of the above-mentioned focus adjustment device control method.

[0114] Based on the same inventive concept, the present invention provides yet another focus adjustment device 40. The focus adjustment device 40 includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to perform the focus adjustment device control method described in any of the above-described embodiments.

[0115] 11, which is a schematic diagram showing the structure of an imaging device provided by an embodiment of the present invention. Based on the same inventive concept, the present invention further provides an imaging device 1. The imaging device 1 includes a focusing device 40 described in any of the above-mentioned embodiments.

[0116] In some embodiments, the imaging device 1 further includes an optical lens unit 10, a lens driving unit 20, an image sensor unit 30, and an image data generating unit 50.

[0117] The image sensor unit 30 outputs a pair of pupil-divided subject images to the focus adjustment device 40, and outputs RAW data to the image data generation unit 50.

[0118] The optical lens unit 10 performs pupil division of the subject light beam and focuses the subject image on the image sensor unit 30. The optical lens unit 10 further moves based on a focus control signal from the lens driving unit 20 to bring the subject into focus.

[0119] The lens driving unit 20 generates a focus control signal based on the information output from the focus adjustment device 40 (for example, the second defocus amount and reliability for each group), and outputs the focus control signal to the optical lens unit 10.

[0120] The image sensor unit 30 generates RAW data for each pixel based on the optical image of the subject focused by the optical lens unit 10, and outputs the RAW data for each pixel to the image data generation unit 50. Furthermore, the image sensor unit 30 generates a pair of pupil-divided subject images and outputs the pair of subject images to the focus adjustment device 40.

[0121] The image data generating unit 50 generates image data by performing predetermined signal processing on the RAW data of each pixel, and outputs the image data to the outside.

[0122] Based on the same inventive concept, the present invention further provides a computer-readable storage medium having executable instructions stored therein, which, when executed by a processor, performs the method for controlling a focusing device described in any of the above-described embodiments.

[0123] The computer storage medium in the embodiments of the present invention may employ any combination of one or more computer-readable media. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of more than these. More specific examples (non-limiting list) of computer-readable storage media include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium that contains or stores a program. The program may be used by or in combination with an instruction execution system, apparatus, or device.

[0124] A computer-readable signaling medium may include data signals transmitted in the baseband or propagated as part of a carrier wave, the data signals carrying computer-readable program code. The data signals may include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signaling medium may be any computer-readable medium other than a computer-readable storage medium. The computer-readable medium may transmit, propagate, or transmit programs used by or in combination with instruction execution systems, apparatus, or devices.

[0125] The program code contained in the computer readable medium may be transmitted by any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, or the like, or any suitable combination of the above.

[0126] The computer program code for performing the operations of the present invention can be written in one or more programming languages ​​or a combination thereof, and the programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or it can be connected to an external computer (for example, via the Internet using an Internet service provider).

[0127] Although embodiments of the present invention are illustrated and described, those skilled in the art will understand that these embodiments can be modified, altered, substituted and transformed in various ways without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited by the claims.

Claims

1. A control method for a focus adjustment device, which is applied to a focus adjustment device, the control method for the focus adjustment device comprising: searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting light beams of a plurality of objects, the first image including a left-eye image and a right-eye image, the first image being the left-eye image or the right-eye image; calculating a correlation value for each feature region based on the pair of subject images; calculating a first defocus amount for each feature region based on a correlation value calculation result for each feature region; Dividing the one or more feature regions into one or more groups based on a first defocus amount and coordinate position information of each feature region; calculating a reliability for each group based on at least the feature amount of each feature region in each group and the number or area of ​​the feature regions, and the first defocus amount of each feature region in each group and the reliability for each group are both used to determine a defocus amount of an object at one shooting distance corresponding to the group; 10. A control method for a focus adjustment device, comprising:

2. Searching for one or more feature regions from a first image of a pair of object images obtained by pupil-splitting a light beam of a plurality of objects, based on the first image, Dividing a first image into a plurality of sub-regions based on a first image in a pair of object images obtained by pupil-splitting a plurality of object light beams; performing parallel searches on the plurality of sub-regions in units of sub-regions to search for feature regions separately from the plurality of sub-regions; Including, 2. The control method for a focus adjustment device according to claim 1.

3. performing parallel searches on the plurality of sub-regions in units of sub-regions and separately searching for feature regions from the plurality of sub-regions; comparing a feature amount for each local region in the plurality of sub-regions with a preset feature amount threshold value, with each sub-region being a unit; When the feature amount of one of the local regions is greater than the preset feature amount threshold, the local region is determined to be the feature region; Including, 3. The control method for a focus adjustment device according to claim 2.

4. Calculating correlation values ​​for each feature region is performing parallel calculations on the one or more feature regions in units of feature regions to calculate correlation values ​​for each feature region separately; 2. The control method for a focus adjustment device according to claim 1.

5. Calculating the first defocus amount for each feature region includes: performing parallel calculations on the one or more feature regions in units of feature regions to calculate the first defocus amount for each feature region separately; 2. The control method for a focus adjustment device according to claim 1.

6. Dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region includes: determining a region where a difference in first defocus amounts between adjacent feature regions exceeds a preset first defocus amount difference threshold based on coordinate position information of each feature region as a dividing line between the adjacent feature regions; Integrating feature regions surrounded by the same dividing line into the same group; Including, 2. The control method for a focus adjustment device according to claim 1.

7. Dividing the one or more feature regions into one or more groups based on the first defocus amount and coordinate position information of each feature region includes: Dividing the one or more feature regions into a predetermined number of blocks based on coordinate position information of each feature region; Calculating an average value of the first defocus amount of the feature region included in each block; Integrating clusters of feature regions whose difference in the average values ​​of the first defocus amounts is smaller than a preset threshold value for the difference in the average values ​​of the first defocus amounts and whose positions are adjacent into the same group; Including, 2. The control method for a focus adjustment device according to claim 1.

8. Calculating the reliability of each group based on at least the feature amount of each feature region in each group and performing parallel calculations on one or more groups on a group-by-group basis based on the feature amounts of each feature region in at least each group, to separately calculate the reliability of each group.

2. The control method for a focus adjustment device according to claim 1.

9. A focusing device, The apparatus includes a feature region search unit, a correlation calculation unit, a defocus amount calculation unit, a grouping processing unit, and a reliability calculation unit, the characteristic region search unit is used to search for one or more characteristic regions from a first image of a pair of subject images obtained by pupil-splitting a light beam of a plurality of subjects, the pair of subject images including a left-eye image and a right-eye image, and the first image is the left-eye image or the right-eye image; the correlation calculation unit is used to calculate a correlation value for each feature region based on the pair of subject images, the defocus amount calculation unit is used to calculate a first defocus amount for each feature region based on a correlation value calculation result for each feature region, the grouping processing unit is used to divide the one or more feature regions into one or more groups based on a first defocus amount and coordinate position information of each feature region; the reliability calculation unit is used to calculate the reliability of each group based on at least the feature amount of each feature region in each group and the number or area of ​​the feature regions, and the first defocus amount of each feature region in each group and the reliability of each group are both used to determine the defocus amount of an object at one shooting distance corresponding to the group. A focusing device characterized by:

10. A focusing device, A focus adjustment device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to execute the focus adjustment device control method according to any one of claims 1 to 8.

11. An imaging device, An imaging device comprising the focus adjustment device according to claim 9.

12. A computer program causing a processor to execute the control method for a focus adjustment device according to any one of claims 1 to 8.

Images