An auto-focusing method, apparatus, electronic device, and storage medium

Abstract

The application discloses an automatic focusing method and device, electronic equipment and storage medium, and belongs to the technical field of camera shooting. The automatic focusing method comprises the following steps: S100, determining an initial search interval; S200, determining a first detection point and a second detection point according to the search interval; S300, acquiring an aperture value of a camera and images collected by the camera when a lens is located at the first detection point and the second detection point; S400, determining a focusing window in the images; S500, calculating a definition value of the focusing window in the images based on the aperture value; S600, adjusting the search interval according to the size of the definition value of the focusing window in the two images; and S700, judging whether the length of the adjusted search interval is less than or equal to a preset value; if yes, the adjusted search interval is determined as an optimal focusing interval; and if no, the step S200 is executed. The application improves the focusing accuracy.

Description

Technical Field

[0001] This invention belongs to the field of camera technology, and in particular relates to an automatic focusing method, device, electronic device and storage medium. Background Technology

[0002] Autofocus (AF) is a crucial component of image signal processing (ISP), automatically adjusting the lens focal length to capture the sharpest frame. It's a common feature in devices such as camcorders, digital cameras, and mobile phones. Compared to phase-difference detection, contrast-detection autofocus eliminates the need for additional photoelectric sensors. Contrast-detection autofocus typically relies on real-time video frame image processing and lens adjustment to achieve autofocus, offering advantages such as simple structure, low power consumption, wide applicability, and high cost-effectiveness.

[0003] Autofocus functionality includes focus window selection, sharpness evaluation function, zoom tracking, and focus search. Among these, the proper selection of the focus window is particularly important, as it is the prerequisite and foundation for zooming and focusing. Conventional focus window selection methods are divided into static and dynamic windowing methods. Due to the computationally intensive and time-consuming nature of dynamic windowing methods, which involve Gaussian non-uniform sampling, first-order moments, and the saliency mechanism of human vision, they are difficult to apply to 8K ultra-high-definition real-time shooting. Therefore, static windowing methods such as the center area, golden ratio, and inverted T-shape are commonly used in practice. For ultra-high-definition cameras with variable apertures, the autofocus effect under traditional static windowing is less than ideal, or even results in defocusing. Figure 1 In scenes with large apertures, the depth of field is shallow, making the center-area method more applicable. However, traditional inverted T-shaped or golden ratio windowing methods include both the subject and background in the sharpness evaluation, resulting in a multi-peaked sharpness function curve and leading to errors in the autofocus result. Figure 2 The small aperture scene shown exhibits a large depth of field, with the inverted T-shape and golden ratio windowing methods producing good focusing results. However, the center area method, which does not take the background into account, results in less sharpness. Therefore, in the context of variable aperture ultra-high-definition cameras, traditional autofocus methods suffer from limitations such as focus jumps and low focusing accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an automatic focusing method, apparatus, electronic device and storage medium.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] According to a first aspect of the present invention, an autofocus method is applied to a variable aperture camera, the autofocus method comprising:

[0007] Step S100. Determine the initial search interval;

[0008] Step S200. Determine the first detection point and the second detection point according to the search interval;

[0009] Step S300. Obtain the aperture value of the camera and the image captured by the camera when the lens is located at the first detection point and the second detection point;

[0010] Step S400. Determine the focus window in the image;

[0011] Step S500. Calculate the sharpness value of the focus window in the image based on the aperture value;

[0012] Step S600. Adjust the search range according to the sharpness value of the focus window in the two frames;

[0013] Step S700. Determine whether the length of the adjusted search interval is less than or equal to the preset value: if yes, then determine that the adjusted search interval is the optimal focus interval; if no, then proceed to step S200.

[0014] Furthermore, step S300 further includes: processing the image into a grayscale image.

[0015] Further, the image is processed into a grayscale image, including:

[0016] If the image is YUV encoded, then the value of the Y channel of each pixel in the image is taken as the gray value of that pixel, and then the corresponding grayscale image is obtained.

[0017] If the image is RGB encoded, the grayscale value of each pixel in the image is obtained by the formula Gray = 0.299R + 0.587G + 0.114B, and then the corresponding grayscale image is obtained.

[0018] Where Gray represents the grayscale value, R represents the pixel value of a pixel in the R channel, G represents the pixel value of a pixel in the G channel, and B represents the pixel value of a pixel in the B channel.

[0019] Furthermore, the calculation formulas for the first detection point and the second detection point are as follows:

[0020] b1=a+μL

[0021] a1=b-μL

[0022] In the formula, a1 represents the first probe point; b1 represents the second probe point; μ∈(0,1); [a,b] represents the current search interval; L represents the length of the search interval, L=ba.

[0023] Furthermore, the focus window includes a central portion, an outer portion, and corner portions of the image, and each of the central portion, outer portion, and corner portions includes multiple image blocks located at preset positions.

[0024] Further, step S500 includes:

[0025] Step S510. Using the template The gradient value G is obtained by convolving the gray values ​​of each pixel in image block I. I (i,j), G I (i,j) represents the gradient value at coordinates (i,j) in image patch I;

[0026] Step S520. Calculate the sharpness value for each image patch:

[0027]

[0028] In the formula, f(I) represents the sharpness value of image block I;

[0029] Step S530. Calculate the sharpness value of the focus window based on the sharpness value of the image patch:

[0030] f img =∑f center +h(F)[α∑f outer +β∑f corner ]

[0031]

[0032] In the formula, f img Let f(F) represent the sharpness value of the focus window, α be the weighted hyperparameter for the outer portion, β be the weighted hyperparameter for the corner portion, and h(F) be the aperture normalization function. Max It is the maximum value of the variable aperture, F. Min It is the minimum value of the variable aperture, where F represents the aperture value when acquiring the image. center ∑f represents the sharpness value of the image patch in the center of the image. outer ∑f represents the sharpness value of the image patch at the outer edge of the image. corner This represents the sharpness value of image blocks at the edges and corners of the image.

[0033] Further, step S600 includes:

[0034] The sharpness value of the focused window in the image captured by the camera at the first detection point is denoted as f(a1);

[0035] The sharpness value of the focused window in the image acquired by the camera at the second detection point is denoted as f(b1);

[0036] If f(a1) > f(b1), then adjust the search interval to [a, b1];

[0037] If f(a1) < f(b1), then adjust the search interval to [a1, b];

[0038] If f(a1) = f(b1), then adjust the search interval to [a1, b1];

[0039] Where a1 represents the first detection point, b1 represents the second detection point, and [a, b] represents the current search interval.

[0040] According to the second aspect of the present invention, an automatic focusing device is applied to a camera with a variable aperture. The automatic focusing device includes:

[0041] An interval determination module, configured to determine an initial search interval and adjust the search interval according to the magnitudes of the clarity values of the focusing windows in two frames of images;

[0042] A detection point determination module, configured to determine a first detection point and a second detection point according to the search interval;

[0043] An image acquisition module, configured to obtain the aperture value of the camera and the images acquired by the camera when the lens is located at the first detection point and the second detection point;

[0044] A clarity calculation module, configured to determine the focusing window in the image and calculate the clarity value of the focusing window in the image based on the aperture value;

[0045] A judgment module, configured to judge whether the length of the adjusted search interval is less than or equal to a preset value;

[0046] An optimal interval determination module, configured to determine the search interval as the optimal focusing interval when the length of the search interval is less than or equal to the preset value.

[0047] According to the third aspect of the present invention, an electronic device includes:

[0048] A memory that stores execution instructions; and

[0049] A processor that executes the execution instructions stored in the memory, such that the processor executes the automatic focusing method according to the first aspect of the present invention.

[0050] According to the fourth aspect of the present invention, a computer-readable storage medium stores execution instructions, and when the execution instructions are executed by a processor, they are used to implement the automatic focusing method according to the first aspect of the present invention.

[0051] The beneficial effects of the present invention are:

[0052] (1) When the aperture value is large, the depth of field of the optical system is small, and the difference in sharpness between the subject and the background is large. The traditional multi-window selection strategy does not distinguish between the subject and the background and includes them in the sharpness function evaluation. This leads to a multi-peak phenomenon in the sharpness curve, and it is easy to focus on the background during zoom search, resulting in errors. The aperture normalization function of the present invention can adaptively allocate the subject and background with a corresponding weight ratio according to the current aperture value, ensuring that the sharpness curve has a single peak and improving the focusing accuracy.

[0053] (2) When the aperture value is small, the depth of field of the optical system is large, and the difference in sharpness between the subject and the background is small. The traditional single-window selection strategy only focuses on the subject, resulting in less sharp background areas at the edges and corners, and poor real-time frame viewing effect. The sharpness evaluation function of this invention performs weighted calculation on the center, periphery and corner areas, so that both the subject and the background are in the sharper areas, thus improving the focusing effect;

[0054] (3) The method of the present invention is simple, effective and versatile. Compared with the traditional method, under the same O(n) time complexity, the present invention can adaptively adjust the proportion of each part of the region of interest according to the variable aperture value of the camera, improve the autofocus effect, and can be adapted to resolutions such as 8K / 4K / 2K / 1080P. Attached Figure Description

[0055] Figure 1 The central area under a large aperture and the traditional inverted T-shaped selection window method;

[0056] Figure 2 The central area under a small aperture and the traditional inverted T-shaped selection window method;

[0057] Figure 3 This is a flowchart of one embodiment of the autofocusing method in this invention;

[0058] Figure 4 A schematic diagram of an image block that is the focus window in an image;

[0059] Figure 5 The sharpness function curves of the method of this invention and the traditional methods of inverted T-shape and golden section are shown under large aperture.

[0060] Figure 6 The sharpness function curves of the method of this invention and the traditional method in the center area are shown under small aperture.

[0061] Figure 7 The images show the focus frames and magnified views of the center region method (left) and the method of this invention (right) under small aperture.

[0062] Figure 8 This is a block diagram illustrating one embodiment of the autofocus device in this invention. Detailed Implementation

[0063] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] See Figures 3 to 8 The present invention provides an autofocusing method, apparatus, electronic device, and storage medium:

[0065] The first aspect of this embodiment discloses an autofocus method applied to a variable aperture camera. For example... Figure 3 As shown, the autofocus method includes steps S100 to S600, which are described in detail below.

[0066] Step S100. Determine the initial search interval.

[0067] Generally, the maximum range of camera lens movement can be set as the initial search range. The movement of the lens can be driven by a stepper motor.

[0068] Step S200. Determine the first detection point and the second detection point according to the search interval.

[0069] In some embodiments, the calculation formulas for the first detection point and the second detection point are as follows:

[0070] b1=a+μL

[0071] a1=b-μL

[0072] In the formula, a1 represents the first detection point; b1 represents the second detection point; μ∈(0,1), generally taken as 0.618, but can also be other values ​​according to actual needs; [a,b] represents the current search interval; L represents the length of the search interval, L=ba.

[0073] Step S300. Obtain the aperture value of the camera and the image captured by the camera when the lens is located at the first detection point and the second detection point.

[0074] Specifically, step S300 includes: acquiring the camera's aperture value; acquiring an image captured by the camera when the lens is at the first detection point; and acquiring an image captured by the camera when the lens is at the second detection point. During the focusing process, the camera's aperture value remains constant.

[0075] In some embodiments, step S300 further includes: processing the image into a grayscale image. If the image is an 8K image, then the resolution of its grayscale image is 7680(W)×4320(H). Specifically, processing the image into a grayscale image includes: if the image is YUV encoded, then taking the value of the Y channel of each pixel in the image as the grayscale value of that pixel, and then obtaining the corresponding grayscale image; if the image is RGB encoded, then the grayscale value of each pixel in the image is obtained by the formula Gray = 0.299R + 0.587G + 0.114B, and then obtaining the corresponding grayscale image; where Gray represents the grayscale value, R represents the pixel value of the pixel in the R channel, G represents the pixel value of the pixel in the G channel, and B represents the pixel value of the pixel in the B channel.

[0076] Step S400. Determine the focus window in the image.

[0077] In some embodiments, the focus window includes a central portion, an outer portion, and corner portions of an image, and each of the central portion, outer portion, and corner portions includes multiple image blocks located at preset positions.

[0078] The image block can be set as a rectangular block.

[0079] For example, such as Figure 4As shown, the region of the image is (-0.5W~0.5W, -0.5H~0.5H). Therefore: the central region comprises 5 image blocks, with center coordinates of (0,0), (-0.1W,0), (0.1W,0), (0,-0.1H), and (0,0.1H); the outer region comprises 20 image blocks, with center coordinates of (-0.3W,0), (-0.2W,0), (-0.1H,0), (-0.1W,0), (-0.1H,0), (-0.1H,0), (-0.1H,0), (-0.2W,0), (-0.1H ... ,0.1H), (-0.2W,0), (-0.2W,-0.1H), (-0.1W,0.2H), (-0.1W,0.1H), (-0.1W,-0.1H), (- 0.1W,-0.2H), (0,0.3H), (0,0.2H), (0,-0.2H), (0,-0.3H), (0.1W,0.2H), (0.1W,0.1H), (0.1W, -0.1H), (0.1W, -0.2H), (0.2W, 0.1H), (0.2W, 0), (0.2W, -0.1H), (0.3W, 0); the corner portion includes 12 image blocks, with the center coordinates of these 12 image blocks being (-0.4W, -0.2H), (-0.4W, -0.3H), (-0.4W, -0.4H), (-0.3W, -0.3H). H), (-0.3W,-0.4H), (-0.2W,-0.4H), (0.4W,-0.2H), (0.4W,-0.3H), (0.4W,-0.4H), (0.3W,-0.3H), (0.3W,-0.4H), (0.2W,-0.4H); where the resolution of each image block is 0.1W×0.1H, W represents the width of the image, and H represents the height of the image.

[0080] Step S500. Calculate the sharpness value of the focus window in the image based on the aperture value.

[0081] In some embodiments, since the focus window includes a central portion, a peripheral portion, and corner portions of the image, and each of the central portion, peripheral portion, and corner portions includes multiple image blocks located at preset positions, step S500 includes:

[0082] Step S510. Using the template The gradient value G is obtained by convolving the gray values ​​of each pixel in image block I. I (i,j), G I (i,j) represents the gradient value at coordinates (i,j) in image patch I.

[0083] Step S520. Calculate the sharpness value for each image patch:

[0084]

[0085] In the formula, f(I) represents the clarity value of the image block I.

[0086] Step S530. Calculate the clarity value of the focus window based on the clarity value of the image block:

[0087] f img = ∑f center + h(F)[α∑f outer + β∑f corner

[0088]

[0089] In the formula, f img represents the clarity value of the focus window, α is the weighted hyperparameter of the peripheral part, β is the weighted hyperparameter of the corner part, h(F) is the aperture normalization function, F Max is the maximum value of the variable aperture, F Min is the minimum value of the variable aperture, F represents the aperture value when collecting the image, f center represents the clarity value of the image block in the central part of the image, ∑f outer represents the clarity value of the image blocks in the peripheral part of the image, ∑f corner represents the clarity value of the image blocks in the corner part of the image.

[0090] In this embodiment, the proportion of each part of the region of interest is adaptively adjusted according to the variable aperture value of the camera. When the aperture value F is large, the function h(F) is large, and the proportion of the clarity values calculated for the peripheral part (Outer region) and the corner part (Corner region) of the image is large; when the aperture value F is small, the function h(F) is small, and the main part of the clarity value is the central part (Center region).

[0091] Step S600. Adjust the search range according to the magnitude of the clarity values of the focus windows in two frames of images.

[0092] In some embodiments, the step S600 includes: recording the clarity value of the focus window in the image collected by the camera when the focal length is at the first detection point as f(a1); recording the clarity value of the focus window in the image collected by the camera when the focal length is at the second detection point as f(b1); if f(a1) > f(b1), then adjust the search range to [a, b1]; if f(a1) < f(b1), then adjust the search range to [a1, b]; if f(a1) = f(b1), then adjust the search range to [a1, b1]; where, a1 represents the first detection point, b1 represents the second detection point, and [a, b] represents the current search range.

[0093] ​Step S700. Determine whether the length of the adjusted search interval is less than or equal to the preset value: if yes, then determine that the adjusted search interval is the optimal focus interval; if no, then proceed to step S200.

[0094] In this embodiment, it is assumed that the focusing requirement is met when the lens is located at any position within the optimal focusing range. In some embodiments, the midpoint of the optimal focusing range can be identified as the optimal focusing position.

[0095] A camera with a large aperture has a small depth of field. Figure 5 It can be concluded that, compared to the inverted T-shape and golden section methods, the sharpness evaluation curve of the method in this embodiment has a single peak and a wide, steep width, which can accurately determine the position where the subject is sharpest. In contrast, the inverted T-shape and golden section methods, because they do not distinguish between the subject and background areas in the region of interest, result in multi-peak evaluation curves, which can lead to focus failure.

[0096] A smaller camera aperture results in a larger depth of field. Figure 6 It can be seen that the curves of the method in this embodiment and the central region method have a lateral offset.

[0097] Figure 7 The image shows a focus frame and a magnified view of the center region method (left) and the method of this embodiment (right) under small aperture. Figure 7 The comparison of the two methods of focusing frames shows that, compared with the central region method which focuses on the subject and ignores the background, the method in this embodiment can take both the subject and the background into account, so that both are within a relatively clear range.

[0098] The second aspect of this embodiment discloses an autofocus device applied to a variable aperture camera. For example... Figure 8 As shown, the autofocus device includes an interval determination module, a detection point determination module, an image acquisition module, a sharpness calculation module, a judgment module, and an optimal interval determination module.

[0099] The interval determination module is used to determine the initial search interval and adjust the search interval based on the sharpness values ​​of the focus windows in the two frames. For a description of the interval determination module, please refer to the descriptions of steps S100 and S600.

[0100] The detection point determination module is used to determine a first detection point and a second detection point based on the search interval. A description of the detection point determination module can be found in the description of step S200.

[0101] The image acquisition module is used to acquire the aperture value of the camera, as well as the image captured by the camera when the lens is located at the first detection point and the second detection point. For a description of the image acquisition module, please refer to the description of step S300.

[0102] A sharpness calculation module is used to determine the focus window in the image and calculate the sharpness value of the focus window in the image based on the aperture value. For a description of the sharpness calculation module, please refer to the descriptions of steps S400 and S500.

[0103] The judgment module is used to determine whether the length of the adjusted search interval is less than or equal to a preset value. For a description of the judgment module, please refer to the description of step S700.

[0104] The optimal range determination module is used to determine the search range as the optimal focusing range when the length of the search range is less than or equal to a preset value. For a description of the optimal range determination module, please refer to the description of step S700.

[0105] A third aspect of this embodiment discloses an electronic device, which includes a memory and a processor. The memory stores execution instructions; the processor executes the execution instructions stored in the memory, causing the processor to perform the autofocus method described in the first aspect of this embodiment.

[0106] The fourth aspect of this embodiment discloses a computer-readable storage medium storing executable instructions, which, when executed by a processor, are used to implement the autofocus method described in the first aspect of this embodiment.

[0107] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. An auto focus method applied to a camera with a variable aperture, characterized by, The automatic focusing method comprises: Step S100. determining an initial search interval; Step S200. determining a first detection point and a second detection point according to the search interval; Step S300. acquiring an aperture value of a camera and images collected by the camera when a lens is located at the first detection point and the second detection point; Step S400. determining a focusing window in the images; Step S500. calculating a sharpness value of the focusing window in the images based on the aperture value; Step S600. adjusting the search interval according to the size of the sharpness value of the focusing window in the two images; Step S700. judging whether the length of the adjusted search interval is less than or equal to a preset value: if yes, determining the adjusted search interval as an optimal focusing interval; if no, executing Step S200; In Step S500: The focusing window comprises a center part, a peripheral part and a corner part of the image, and the center part, the peripheral part and the corner part each comprise a plurality of image blocks located at preset positions; The sharpness value of the focusing window is calculated based on the sharpness values of the image blocks wherein, denotes the sharpness value of the focused window, is a weighted hyper parameter of the peripheral portion, is a weighted hyper parameter of the corner portion, is an aperture normalization function, is a maximum value of the variable aperture, is a minimum value of the variable aperture, F denotes an aperture value at the time of capturing an image, denotes the sharpness value of the image center portion image block, denotes the sharpness value of the image peripheral portion image block, denotes the sharpness value of the image corner portion image block.

2. The method of claim 1, wherein, The step S300 further comprises processing the images into grayscale images.

3. A method of auto-focusing according to claim 2, wherein, Processing the images into grayscale images comprises: If the images are YUV encoded, taking the value of the Y channel of each pixel point in the images as the grayscale value of the pixel point, and then obtaining the grayscale images corresponding to the images; If the image is RGB encoded, the gray value of each pixel in the image is obtained by the formula and then a gray image corresponding to the image is obtained. wherein, wherein, R denotes a pixel value of the pixel point in the R channel, G denotes a pixel value of the pixel point in the G channel, and B denotes a pixel value of the pixel point in the B channel.

4. The method of claim 1, wherein, The calculation formula of the first detection point and the second detection point is: In the formula, represents a first detection point; represents a second detection point; ; represents a current search interval; L represents a length of the search interval, .

5. The method of claim 1, wherein, Step S500 comprises: Step S510. Using the template The gradient value is obtained by convoluting each pixel gray value in the image block I , represents the gradient value of the coordinate (i, j) in the image block I Step S520. calculating the sharpness value of each image block.

6. The method of claim 1, wherein, The step S600 comprises: The definition of the sharpness value of the focus window in the image collected by the camera at the first detection point is denoted as ; The definition of the sharpness value of the focus window in the image collected by the camera at the second detection point is denoted as ; If , then the search interval is adjusted to ; If , then the search interval is adjusted to ; If , then the search interval is adjusted to ; wherein denotes a first probing point, denotes a second probing point, denotes a current search interval.

7. An auto-focusing apparatus applied to a variable aperture video camera and performing the auto-focusing method of any one of claims 1 to 6, characterized by, The automatic focusing device comprises: An interval determination module configured to determine an initial search interval and adjust the search interval according to the size of the sharpness value of the focusing window in the two images; A detection point determination module configured to determine a first detection point and a second detection point according to the search interval; An image acquisition module configured to acquire an aperture value of a camera and images collected by the camera when a lens is located at the first detection point and the second detection point; A sharpness calculation module configured to determine a focusing window in the images and calculate a sharpness value of the focusing window in the images based on the aperture value; A judgment module configured to judge whether the length of the adjusted search interval is less than or equal to a preset value; An optimal interval determination module configured to determine the search interval as an optimal focusing interval when the length of the search interval is less than or equal to the preset value.

8. An electronic device, comprising: comprise: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the automatic focusing method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores execution instructions, and the execution instructions are executed by the processor to implement the automatic focusing method in any one of claims 1 to 6.

Images