Focusing method and device, electronic equipment, chip and medium

Abstract

The invention provides a focusing method and device, electronic equipment, a chip and a medium, and relates to the technical field of camera automatic focusing. The focusing method comprises the steps of collecting a first preview image based on a first focusing frame; based on the first preview image, determining an object area in a focusing block through a semantic segmentation model, the focusing block being obtained by dividing a first focusing frame; determining a foreground object area in the focusing block according to the object area and the focusing block; if the foreground object area is located in the edge area of the first focusing frame, the first focusing frame is updated to be a second focusing frame, and focusing is conducted according to the second focusing frame. Through the technical scheme provided by the invention, the problem of virtual focus blurring of the picture center main body caused when the foreground object is located at the edge of the focusing frame is solved, and the user experience of automatic focusing is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of camera autofocus, and in particular, to a focusing method, device, electronic device, chip and medium. Background Art

[0002] Autofocus is an important module in camera algorithms, and users hope that the scenes they shoot can obtain clear focused images. In related phase focusing technologies, following the foreground-first logic, the region of interest (ROI) of the focusing frame is generally divided into multiple small windows to focus on the foreground as much as possible. When the foreground object is at the edge of the focusing frame and there is only a small part of foreground details in the focusing window, the phase focusing algorithm will also give priority to focusing on the edge foreground object. As a result, when the ROI of the focusing frame is large, it will focus on the edge foreground object, causing the central subject to be out of focus, which greatly affects the user's shooting experience. Summary of the Invention

[0003] The present disclosure provides a focusing method, device, electronic device, chip and medium to solve the problem that the central subject in the picture is out of focus and blurred when the foreground object is located at the edge of the focusing frame. By determining the in-focus block of the focusing frame, performing semantic segmentation on the picture in the in-focus block, and determining whether the foreground object is located at the edge of the focusing frame, if so, adjusting the focusing frame to focus on the main body of the picture, thereby improving the user experience of autofocus.

[0004] A first aspect embodiment of the present disclosure proposes a focusing method, which includes:

[0005] Collect a first preview image based on a first focusing frame;

[0006] Based on the first preview image, determine the object region in the in-focus block through a semantic segmentation model, and the in-focus block is obtained by dividing the first focusing frame;

[0007] Determine the foreground object region in the in-focus block according to the object region and the in-focus block;

[0008] If the foreground object region is located in the edge region of the first focusing frame, update the first focusing frame to a second focusing frame and perform focusing according to the second focusing frame.

[0009] In an embodiment of the present disclosure, determining the object region in the in-focus block based on the first preview image through a semantic segmentation model includes:

[0010] Obtain the phase difference attribute of the first preview image, and the phase difference attribute includes a phase difference convergence value and a phase difference confidence level;

[0011] Determine a focused block based on the phase difference attributes of all sub - focused blocks of the first preview image, where the sub - focused blocks are obtained by dividing based on the first focus frame;

[0012] Based on the first part of the first preview image, determine the object area in the focused block through a semantic segmentation model, and the first part includes the focused block.

[0013] In an embodiment of the present disclosure, determining a focused block based on the phase difference attributes of all sub - focused blocks of the first preview image includes:

[0014] For all sub - focused blocks, determine multiple sub - focused blocks with a phase difference confidence greater than or equal to the first confidence threshold as the first focused block sequence;

[0015] Re - order the first focused block sequence in descending order of the phase difference convergence value to obtain the second focused block sequence;

[0016] Use the sub - focused blocks in the second focused block sequence with a phase difference convergence value less than or equal to the first convergence threshold as the focused block.

[0017] In an embodiment of the present disclosure, determining the foreground object area in the focused block according to the object area and the focused block includes:

[0018] Determine the first object area sequence with a phase difference convergence value less than or equal to the second convergence threshold among all object areas, and use the ratio of the sum of the areas of all object areas in the first object area sequence to the area of the focused block as the focused area ratio;

[0019] Determine the object area with the highest phase difference confidence in the first object area sequence as the first object area;

[0020] If the focused area ratio of the first object area is greater than or equal to the first ratio threshold, confirm focus, otherwise, use the first object area as the foreground object area.

[0021] In an embodiment of the present disclosure, before collecting the first preview image based on the first focus frame, it further includes:

[0022] Compare the height of the first focus frame with the first focus frame height threshold and compare the width of the first focus frame with the first focus frame width threshold;

[0023] If the height of the first focus frame is greater than or equal to the first focus frame height threshold and / or the width of the first focus frame is greater than or equal to the first focus frame width threshold, collect the first preview image based on the first focus frame, otherwise, confirm focus.

[0024] In one embodiment of the present disclosure, before updating the first focus frame to the second focus frame when the foreground object region is located in the edge region of the first focus frame, the method further includes:

[0025] Determine the object center of the foreground object region, where the object center includes the centroid, geometric center, and center of gravity;

[0026] Determine the shortest boundary distance according to the object center and the first focus frame;

[0027] Determine whether the foreground object region is located in the edge region of the first focus frame according to the shortest boundary distance.

[0028] In one embodiment of the present disclosure, determining whether the foreground object region is located in the edge region of the first focus frame according to the shortest boundary distance includes:

[0029] Compare the shortest boundary distance with the first distance threshold;

[0030] If the shortest boundary distance is less than or equal to the first distance threshold, it is determined that the foreground object region is located in the edge region of the first focus frame.

[0031] In one embodiment of the present disclosure, updating the first focus frame to the second focus frame includes:

[0032] Determine the width of the second focus frame according to the preset width ratio and the width of the first focus frame;

[0033] Determine the height of the second focus frame according to the preset height ratio and the height of the first focus frame.

[0034] In a second aspect embodiment of the present disclosure, a focusing device is provided, and the device includes:

[0035] An acquisition module, configured to acquire a first preview image based on the first focus frame;

[0036] A semantic segmentation module, configured to determine the object region in the in-focus block based on the first preview image through a semantic segmentation model, where the in-focus block is obtained by dividing the first focus frame;

[0037] A foreground determination module, configured to determine the foreground object region in the in-focus block according to the object region and the in-focus block;

[0038] A focusing module, configured to update the first focus frame to the second focus frame if the foreground object region is located in the edge region of the first focus frame, and perform focusing according to the second focus frame.

[0039] A third aspect embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of the first aspect embodiments of the present disclosure.

[0040] A fourth aspect embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method of the first aspect embodiments of the present disclosure.

[0041] A fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the method of any one of the first aspect embodiments of the present disclosure.

[0042] A sixth aspect embodiment of the present disclosure provides a chip, including at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method of any one of the first aspect embodiments of the present disclosure through logic circuits or by executing code instructions.

[0043] In summary, according to the focusing method proposed by the present disclosure, based on the first focusing frame, a first preview image is collected, providing a data source for automatic focusing; based on the first preview image, the object area in the in-focus block is determined through a semantic segmentation model, and the in-focus block is obtained by dividing the first focusing frame, determining the objects contained in the in-focus block of the focusing frame; according to the object area and the in-focus block, the foreground object area in the in-focus block is determined, and the foreground object is segmented and located from the in-focus block, providing a foreground object for reference in automatic focusing; if the foreground object area is located in the edge area of the first focusing frame, the first focusing frame is updated to a second focusing frame, and focusing is performed according to the second focusing frame. Based on the fact that the foreground object is located at the edge of the focusing frame, the size of the focusing frame is further adjusted to ensure that the focusing frame can focus on the main subject in the center of the picture, improving the user experience of automatic focusing.

[0044] It should be understood that the above general description and subsequent detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation to the present disclosure.

[0046] Figure 1 Schematic diagram of misfocusing on the foreground in scenario 1 in the related art;

[0047] Figure 2 Schematic diagram of the misfocused foreground in scenario 2 in the related art;

[0048] Figure 3 Schematic diagram of the misfocused foreground in scenario 3 in the related art;

[0049] Figure 4 Flowchart of a focusing method according to an embodiment of the present disclosure;

[0050] Figure 5 Flowchart of determining an object region in a focused block based on a first preview image through a semantic segmentation model according to an embodiment of the present disclosure;

[0051] Figure 6 Schematic diagram of a first focusing frame in a first preview image in this embodiment;

[0052] Figure 7 In one implementation manner of this embodiment Figure 1 Schematic diagram after image segmentation;

[0053] Figure 8 In one implementation manner of this embodiment Figure 2 Schematic diagram after image segmentation;

[0054] Figure 9 In one implementation manner of this embodiment Figure 3 Schematic diagram after image segmentation;

[0055] Figure 10 Flowchart of determining a focused block based on the phase difference attributes of all sub - focused blocks of a first preview image according to an embodiment of the present disclosure;

[0056] Figure 11 Flowchart of determining a focused area in this embodiment;

[0057] Figure 12 Flowchart of determining a foreground object region in a focused block according to an object region and a focused block according to an embodiment of the present disclosure;

[0058] Figure 13 Schematic diagram of an object region and a foreground object region in this embodiment;

[0059] Figure 14 Flowchart of determining whether a first focusing frame is too large before collecting a first preview image based on the first focusing frame according to an embodiment of the present disclosure;

[0060] Figure 15A flowchart for determining whether a foreground object is located in the edge area of a first focus frame before updating the first focus frame to a second focus frame in an embodiment of the present disclosure;

[0061] Figure 16 A schematic diagram of the shortest boundary distance between the object center and the first focus frame in this embodiment;

[0062] Figure 17 A flowchart for determining whether a foreground object area is located in the edge area of a first focus frame according to the shortest boundary distance in an embodiment of the present disclosure;

[0063] Figure 18 A flowchart for updating a first focus frame to a second focus frame in an embodiment of the present disclosure;

[0064] Figure 19 A flowchart for an out-of-focus foreground optimization method in an embodiment of the present disclosure;

[0065] Figure 20 A schematic diagram of focus frame update in an embodiment of the present disclosure;

[0066] Figure 21 A schematic structural diagram of a focus device in an embodiment of the present disclosure;

[0067] Figure 22 A block diagram of an electronic device for implementing the focus method of the present disclosure shown according to an exemplary embodiment;

[0068] Figure 23 A schematic structural diagram of a chip in an embodiment of the present disclosure. Detailed implementation manners

[0069] The embodiments of the present disclosure will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, but should not be construed as limiting the present disclosure.

[0070] Auto focus is a technology that adjusts the lens or focal length in a camera or other device to obtain a clear image, which fully automates the manual focusing process. Contrast autofocus and phase autofocus are two common implementations of autofocus. The principle of contrast autofocus is to find the position with the maximum contrast according to the change in the contrast of the image at the focus, that is, the position of accurate focus. Phase autofocus arranges phase difference pixel points on the photosensitive element, which are specifically used for phase detection. By the distance between pixels and its changes, etc., the offset value of focusing is determined to achieve accurate focus. Common camera focusing algorithms include Phase Detection Auto Focus (PDAF), Continuous AutoFocus (CAF), and Time of Flight (TOF) focusing. Among them, TOF focusing relies on a laser sensor and is not available in many devices in scenarios with long focal lengths and bright environments. The Auto Focus (AF) algorithm gives priority to PDAF focusing. When PDAF is not available, CAF focusing can be used. For the PDAF focusing algorithm, it follows the foreground-first logic. Generally, the focusing ROI is divided into multiple small windows and focuses on the foreground as much as possible.

[0071] Phase autofocus uses multi-window focusing. When the phase difference (PD) of the window is credible, the PD is considered to be in focus. Generally, multi-window focusing follows the foreground-first principle and focuses on the foreground object. On the other hand, some focusing algorithms incorporate a saliency focusing algorithm. Through this algorithm, the main body that the user is interested in is estimated, and then the focus is adjusted to the main body object. However, the focus points that the user is interested in vary, and the main body recognition algorithm is more likely to regard the foreground object as the main body. Therefore, it is also very easy to focus on the edge foreground. So based on the current focusing algorithms, when the foreground object appears at the edge of the focusing frame, there is a probability of mis-focusing on the foreground, giving a feeling that the central main body is out of focus.

[0072] Figure 1 It is a schematic diagram of mis-focusing on the foreground in Scenario 1 in the related technology. As Figure 1 shown, Scenario 1 is a scenario where a duck model is placed on the upper end of the display. The duck model is located in the center of the current image and is the main body of the current captured image. However, since the display under the duck model is closer to the camera and is the foreground object, the focus of the camera's autofocus is on the display at the lower edge of the focusing frame, resulting in the foreground display at the edge of the focusing frame being in focus, while the central main body model is out of focus and blurred.

[0073] Figure 2 It is a schematic diagram of mis-focusing on the foreground in Scenario 2 in the related technology. As Figure 2As shown, Scene 2 is an office scene. The monitor is located at the bottom of the screen and is a foreground object. In this scene, the workbench in the front is located in the center of the screen and is the main body of the current captured image. However, since the monitor is a foreground object and is located at the lower edge part of the focus frame, it causes foreground focusing, while the main body in the center of the screen is out of focus and blurred.

[0074] Figure 3 It is a schematic diagram of misfocusing on the foreground in Scene 3 in the related art. As Figure 3 shown, Scene 3 is a sunshade scene. The trees and flowers behind the sunshade are located in the center of the screen, and the support columns of the sunshade are foreground objects, resulting in the focus of the camera's automatic focus being on the support columns. Since the column is located at the left edge part of the focus frame, the column in the three in-focus blocks on the left side of the focus frame is in a focused state, while the main bodies such as the trees and flowers in the center of the screen are out of focus and blurred.

[0075] In the above three scenes, in the related automatic focus technology, the foreground object is used as the focus point, resulting in the main body in the center of the screen being out of focus and blurred when the foreground object is at the edge of the focus frame. And it is impossible to focus on the main body that the user is really interested in, which greatly affects the user experience of automatic focus.

[0076] The method proposed in this disclosure is applied to the automatic focus task, and its application has rich scenarios. For mobile phones, cameras, monitoring devices, etc., the focus efficiency of the picture target can be improved through this focus method. It can be widely used in machine vision, monitoring all-in-ones, and other fields that use image data. In the embodiments of this disclosure, the application scenarios are not limited.

[0077] Next, the focus method provided in this application will be introduced in detail with reference to the accompanying drawings.

[0078] Figure 4 It is a flowchart of a focus method according to an embodiment of this disclosure. As Figure 4 shown in the embodiment, the focus method includes:

[0079] Step 401, based on the first focus frame, collect the first preview image.

[0080] In this embodiment, the first focus frame is a tool to help the user place the shooting object in a clear focus. Through the first focus frame, the objects in the picture can be made clearer. Its shape can be rectangular or circular. Preferably, the shape is mostly rectangular. The first preview image is an image collected by the camera according to the focus parameters set by the current first focus frame. Based on the first focus frame of the camera, the first preview image is collected.

[0081] Step 402, based on the first preview image, determine the object area in the in-focus block through a semantic segmentation model. The in-focus block is obtained by dividing the first focus frame.

[0082] In this embodiment, the semantic segmentation model is a computer vision technology that can assign each pixel in the input image to a specific semantic category to obtain dense classification at the pixel level. There are many types of semantic segmentation models, including U-Net, Fully Convolutional Networks (FCN), Segmentation Network (SegNet), etc. Preferably, the present disclosure uses U-Net as the semantic segmentation model. The in-focus block refers to the block where the in-focus point is located among multiple identically sized rectangular blocks obtained by dividing the first focus frame into 3x3 or 4x4 blocks. Since the PD value can reflect the in-focus or clarity of the image in phase autofocus, the in-focus block has the smallest PD value among all blocks. The object area refers to the area of each object obtained by semantically segmenting the image in the in-focus block into different types of objects. Based on the first preview image, the image containing the in-focus block in the first preview image is used as the input to the semantic segmentation model, and the object areas in the in-focus block are obtained through semantic segmentation.

[0083] Step 403: Determine the foreground object area in the in-focus block according to the object area and the in-focus block.

[0084] In this embodiment, the foreground object area refers to the pixel group of the object closest to the object in the captured image. The foreground object area is determined from multiple areas of the object area according to the object area and the in-focus block in the in-focus block.

[0085] Step 404: If the foreground object area is located in the edge area of the first focus frame, update the first focus frame to the second focus frame and perform autofocus according to the second focus frame.

[0086] In this embodiment, the edge area of the first focus frame refers to the area close to the boundary of the first focus frame. The second focus frame refers to the focus frame obtained by adjusting the width and / or height of the first focus frame.

[0087] In summary, according to the autofocus method proposed by the present disclosure, based on the first focus frame, the first preview image is captured, providing a data source for autofocus; based on the first preview image, the object areas in the in-focus block are determined through the semantic segmentation model. The in-focus block is obtained by dividing the first focus frame, and the objects contained in the in-focus block of the focus frame are determined; according to the object area and the in-focus block, the foreground object area in the in-focus block is determined, and the foreground object is segmented and located from the in-focus block, providing a reference foreground object for autofocus; if the foreground object area is located in the edge area of the first focus frame, the first focus frame is updated to the second focus frame and autofocus is performed according to the second focus frame. Based on the fact that the foreground object is located at the edge of the focus frame, the size of the focus frame is adjusted to ensure that the focus frame can focus on the main subject in the center of the picture, improving the user experience of autofocus.

[0088] Figure 5 A flowchart for determining an object region in a focused block based on a first preview image through a semantic segmentation model according to an embodiment of the present disclosure. Figure 5 is a further description of Figure 4 step 402 of, based on Figure 5 the embodiment shown in

[0089] Step 501, obtain the phase difference attribute of the first preview image, where the phase difference attribute includes a phase difference convergence value and a phase difference confidence level.

[0090] In this embodiment, the phase difference attribute can be a value of a class or a structure type containing phase difference data. The phase difference attribute includes a phase difference convergence value and a phase difference confidence level. Among them, the phase difference convergence value represents the magnitude of the phase difference, and the phase difference confidence level represents the credibility corresponding to the current phase difference convergence value. The phase difference attribute of the first preview image can be obtained by calculating the difference in phase between the pixels of the first preview image, or provided by the acquisition device of the first preview image.

[0091] Step 502, determine a focused block based on the phase difference attributes of all sub-focused blocks of the first preview image, where the sub-focused blocks are obtained by dividing based on a first focus frame.

[0092] In this embodiment, the sub-focused block is a first focus frame divided into multiple rectangular blocks of the same size, such as blocks of specifications such as 3x3 and 4x4. The focused block is the block with the clearest picture among the sub-focused blocks.

[0093] Figure 6 A schematic diagram of a first focus frame in a first preview image in this embodiment. In this embodiment, as Figure 6 shown, the first preview image is the camera shooting interface of a mobile phone, and the central rectangular frame formed by the intersection of two horizontal lines and two vertical lines in the center of the interface is the first focus frame. When the height of the first focus frame is greater than or equal to a preset height threshold CenterRoiHeightThr, and the width of the first focus frame is greater than or equal to a preset width threshold CenterRoiWidthThr, it means that the focus frame is relatively large and is more likely to contain foreground objects at the edge, thereby triggering the recognition of whether there are foreground objects at the edge of the focus frame during autofocus.

[0094] Step 503, determine the object region in the focused block based on the first part of the first preview image, where the first part includes the focused block.

[0095] In this embodiment, the first part of the first preview image is the part of the first preview image that includes the in-focus block, which can be the entire first preview image, or the image part corresponding to the first focus frame in the first preview image, or the image part corresponding to the in-focus block in the first preview image.

[0096] Figure 7 In one implementation manner of this embodiment Figure 1 is a schematic diagram of image segmentation. In this embodiment, as Figure 7 shown, the focus frame is composed of 3x3 blocks, the focus frame is located in the center of the picture, and the shooting subject is a duckling. Figure 1 After semantic segmentation processing, the focus frame also includes the monitor under the duckling, the wall behind the duckling, and the telephoto area in the lower right corner of the focus frame.

[0097] Figure 8 In one implementation manner of this embodiment Figure 2 is a schematic diagram of image segmentation. In this embodiment, as Figure 8 shown, the focus frame is divided into 3x3 blocks, the focus frame is located in the center of the picture, and the shooting subjects are the desktop and monitor of the remote workbench. The foreground object monitor, the remote wall, the upper ceiling, etc. are also segmented in the focus frame.

[0098] Figure 9 In one implementation manner of this embodiment Figure 3 is a schematic diagram of image segmentation. In this embodiment, as Figure 9 shown, the focus frame is divided into 3x3 blocks, the focus frame is located in the center of the picture, and the shooting subjects are the trees and flowers behind the sunshade. The focus frame also includes the segmented building, structure, support columns of the sunshade, etc.

[0099] In this embodiment, according to the first preview image, the object area in the in-focus block is obtained through the semantic segmentation model, and the objects included in the in-focus block of the focus frame are determined, providing a basis for determining the foreground object.

[0100] Figure 10 is a flowchart for determining the in-focus block based on the phase difference attributes of all sub-focus blocks of the first preview image in an embodiment of the present disclosure. Figure 10 is a further description of Figure 5 step 502 of Figure 10 shown in the embodiment, and includes the following steps:

[0101] Step 1001, for all sub-focus blocks, determine multiple sub-focus blocks with a phase difference confidence greater than or equal to the first confidence threshold as the first focus block sequence.

[0102] In this embodiment, the first confidence threshold is a threshold used to measure the magnitude of the phase difference confidence. Optionally, the first confidence threshold is 20. The first focus block sequence refers to the focus blocks with high phase difference confidence among all sub-focus blocks. For all sub-focus blocks, using the phase difference confidence of each sub-focus block, one or more sub-focus blocks with a phase difference confidence greater than or equal to the first confidence threshold are determined as the first focus block sequence.

[0103] Step 1002: Reorder the first focus block sequence in descending order of the phase difference convergence value to obtain a second focus block sequence.

[0104] In this embodiment, the second focus block sequence is the focus block sequence obtained by arranging the first focus blocks in descending order of the phase difference convergence value. For the first focus block sequence, use the phase difference convergence value of each sub-focus block therein to perform a descending order arrangement to obtain the second focus block sequence.

[0105] Step 1003: Use the sub-focus blocks in the second focus block sequence with a phase difference convergence value less than or equal to the first convergence threshold as the in-focus blocks.

[0106] In this embodiment, the first convergence threshold is a threshold used to measure the magnitude of the phase difference convergence value. Optionally, the first convergence threshold is 0.1. In the second focus block sequence, use the sub-focus blocks with a phase difference convergence value less than or equal to the first convergence threshold of each sub-focus block as the in-focus blocks.

[0107] In this embodiment, based on the phase difference confidence and the phase difference convergence value of all sub-focus blocks of the first preview image, the in-focus blocks are determined by screening, providing a search range for the positioning of the foreground object.

[0108] Figure 11 This is a flowchart of a method for determining the in-focus area in this embodiment. As Figure 11 shown, in this embodiment, based on the phase difference convergence threshold PdConvThr and the confidence threshold ConfThr of the sub-focus blocks, the in-focus blocks are screened from the sub-focus blocks. Among them, the screening condition is that the phase difference convergence value BlockPd_i of the sub-focus block is less than PdConvThr, and the phase difference confidence BlockConf_i of the sub-focus block is greater than ConfThr. If there is a block Block_i that meets this condition among all sub-focus blocks, then this block Block_i is used as the in-focus area and recorded as the in-focus block FocusedBlock_i.

[0109] Figure 12 This is a flowchart of a method for determining the foreground object area in the in-focus block according to the object area and the in-focus block in an embodiment of the present disclosure. Figure 12 It is for Figure 5and Figure 4 Specific description of step 403 in Figure 12 The illustrated embodiment includes the following steps:

[0110] Step 1201: Determine the first object area sequence in all object areas where the phase difference convergence value is less than or equal to the second convergence threshold, and use the ratio of the sum of the areas of all object areas in the first object area sequence to the area of the in-focus block as the in-focus area ratio.

[0111] In this embodiment, the second convergence threshold refers to the threshold for measuring the magnitude of the phase difference convergence value. The first object area sequence is the object areas in the in-focus block sorted according to the phase difference convergence value. The in-focus area ratio is the ratio of the sum of the areas of the in-focus object areas obtained after segmentation in the in-focus block to the area of the in-focus block, indicating the proportion of the in-focus object areas in the in-focus block.

[0112] Step 1202: Determine the object area with the highest phase difference confidence in the first object area sequence as the first object area.

[0113] In this embodiment, the first object area is the object area with the highest phase difference confidence in the first object area sequence. The first object area is determined through the phase difference confidence of each object area.

[0114] Step 1203: If the in-focus area ratio of the first object area is greater than or equal to the first ratio threshold, confirm focusing; otherwise, use the first object area as the foreground object area.

[0115] In this embodiment, the first ratio threshold is the ratio threshold for measuring the magnitude of the in-focus area ratio. In the in-focus block, if there is a first object area whose in-focus area ratio is greater than or equal to the first ratio threshold, it means that the in-focus block contains an object that is in-focus and most likely the main body, that is, the current focusing is accurate, and then confirm focusing. Otherwise, it means that there is no object area with a large and clear area ratio in the in-focus block, and the clearest first object area in the in-focus block is used as the foreground object area. That is, it indicates that the object area is not the main body of the picture and has a small area.

[0116] In this embodiment, the phase difference of the object area and the area ratio of the object area to the in-focus block are used to determine the foreground object area, which determines the foreground object for autofocus.

[0117] Figure 13 It is a schematic diagram of an object area and a foreground object area in this embodiment. In this embodiment, as Figure 13As shown, the current first focus frame contains a total of 9 sub-focus blocks numbered from 0 to 8. By using the phase difference attribute of each sub-focus block for screening, the sub-focus block numbered 5 is determined as the in-focus block. This in-focus block is semantically segmented into three parts. The first part is the telephoto area of the duck's body above the in-focus block, the second part is the area of the display in the lower left of the in-focus block, and the third part is the telephoto area in the lower right of the in-focus block. Among them, the convergence value of the phase difference in the second part is the smallest. If the first ratio threshold is 50%, and the area ratio of the second part to the current in-focus block is less than the first ratio threshold, it indicates that the clearest object is not the main subject of the picture, then the second part is used as the foreground object area.

[0118] Figure 14 This is a flowchart for determining whether the first focus frame is too large before capturing the first preview image based on the first focus frame in an embodiment of the present disclosure. Figure 14 It is an explanation before step 401 of Figure 4 Based on the embodiment shown in Figure 14 it includes the following steps:

[0119] Step 1401, compare the height of the first focus frame with the first focus frame height threshold and compare the width of the first focus frame with the first focus frame width threshold.

[0120] In this embodiment, the first focus frame height threshold refers to the threshold used to represent the height of the first focus frame. The first focus frame width threshold refers to the threshold used to represent the width of the first focus frame. Compare the height of the first focus frame with the first focus frame height threshold, and at the same time, compare the width of the first focus frame with the first focus frame width threshold.

[0121] Step 1402, if the height of the first focus frame is greater than or equal to the first focus frame height threshold and / or the width of the first focus frame is greater than or equal to the first focus frame width threshold, then capture the first preview image based on the first focus frame, otherwise, confirm focus.

[0122] In this embodiment, if at least one of the two judgment conditions that the height of the first focus frame is greater than or equal to the first focus frame height threshold and the width of the first focus frame is greater than or equal to the first focus frame width threshold is true, then the camera captures the first preview image based on the first focus frame. If both of these two judgment conditions are false or not satisfied, it means that the current first focus frame is smaller than the predetermined size, and focus can be confirmed.

[0123] In this embodiment, before collecting the first preview image based on the first focus frame, it is determined whether the first focus frame is too large. If so, the foreground object located at the edge of the focus frame is detected by the technical solution of the present disclosure, so as to solve the problem that the focus frame is too large, the foreground object at the edge is in focus while the central subject is out of focus and blurred. If not, it means that there is no such problem, and the focus can be confirmed.

[0124] Figure 15 It is a flowchart for determining whether the foreground object is located in the edge area of the first focus frame before updating the first focus frame to the second focus frame in an embodiment of the present disclosure. Figure 15 Yes, for Figure 4 Specific description of step 404, based on Figure 15 The embodiment shown includes the following steps:

[0125] Step 1501, determine the object center of the foreground object area, and the object center includes the centroid, geometric center, and center of gravity.

[0126] In this embodiment, the object center of the foreground object area refers to the central coordinates representing the foreground object area, and the object center includes the centroid, geometric center, center of gravity, etc. Optionally, the average value of the pixel point coordinates of the foreground object area is used as the object center.

[0127] Step 1502, determine the shortest boundary distance according to the object center and the first focus frame.

[0128] In this embodiment, the shortest boundary distance refers to the straight-line distance between the object center and the nearest side of the first focus frame. This distance is used to represent whether the foreground object area is located at the edge part of the first focus frame.

[0129] Step 1503, determine whether the foreground object area is located in the edge area of the first focus frame according to the shortest boundary distance.

[0130] In this embodiment, according to the shortest boundary distance determined in the above steps, it can be determined whether the foreground object area is located in the edge area of the first focus frame. Thus, it provides a trigger condition for automatic focusing to eliminate the interference of the foreground object at the edge and focus on the central main body area.

[0131] Figure 16 It is a schematic diagram of the shortest boundary distance between the object center and the first focus frame in this embodiment. In this embodiment, as shown in combination with Figure 15 and Figure 13 , the center position of the second part in the focused area, that is, the foreground object area, the nearest distance from the boundary of the first focus frame is the shortest boundary distance FocusedPointDist.

[0132] Figure 17A flowchart for determining whether a foreground object region is located in an edge region of a first focus frame according to the shortest boundary distance in an embodiment of the present disclosure. In this embodiment, Figure 17 is a further description of Figure 15 step 1503, based on the Figure 17 embodiment shown, including the following steps:

[0133] Step 1701, compare the shortest boundary distance with a first distance threshold.

[0134] In this embodiment, the first distance threshold is a threshold for measuring the length of the shortest distance of the foreground object region from the boundary. Optionally, the first distance threshold is 25% of the width or height of the sub-focus frame. After obtaining the shortest boundary distance, compare the shortest boundary distance with the first distance threshold.

[0135] Step 1702, if the shortest boundary distance is less than or equal to the first distance threshold, determine that the foreground object region is located in the edge region of the first focus frame.

[0136] In this embodiment, if the shortest boundary distance is less than or equal to the first distance threshold, it indicates that the foreground object region is located in the edge region of the first focus frame. Confirm that the foreground object region is at the edge of the first focus frame.

[0137] In this embodiment, through the shortest boundary distance, it is determined whether the foreground object region is located at the edge of the first focus frame. If the foreground object region is located at the edge of the first focus frame, it provides a trigger condition for the camera to update the first focus frame.

[0138] Figure 18 A flowchart for updating the first focus frame to a second focus frame in an embodiment of the present disclosure. In this embodiment, Figure 18 is a further description of Figure 17 based on the Figure 18 embodiment shown, including the following steps:

[0139] Step 1801, determine the width of the second focus frame according to a preset width ratio and the width of the first focus frame.

[0140] In this embodiment, after confirming that the foreground object region is located in the edge region of the first focus frame, update the first focus frame. Multiply the width of the first focus frame by the preset width ratio as the width of the second focus frame, that is, the width value of the updated first focus frame. Among them, the value range of the preset width ratio is (0, 1).

[0141] Step 1802, determine the height of the second focus frame according to a preset height ratio and the height of the first focus frame.

[0142] In this embodiment, the height of the second focus frame is obtained by multiplying the height of the first focus frame by a preset height ratio, that is, the height value of the updated first focus frame. Here, the value range of the preset height ratio is (0, 1).

[0143] In this embodiment, the width ratio BoundaryWidthRatio refers to the ratio value for adjusting the width of the first focus frame, and this value is between 0 and 1. The width Width of the second focus frame is equal to the width OriWidth of the first focus frame multiplied by the width ratio BoundaryWidthRatio, that is, Width = BoundaryWidthRatio * OriWidth. The height ratio BoundaryHeightRatio refers to the ratio value for adjusting the height of the first focus frame, and this value is between 0 and 1. The height Height of the second focus frame is equal to the height OriHeight of the first focus frame multiplied by the height ratio BoundaryHeightRatio, that is, Height = BoundaryHeightRatio * OriHeight. Optionally, BoundaryHeightRatio and BoundaryWidthRatio can be set to half of the height and width of the sub-focus block, that is, the second focus frame is the result of the inward shrinkage of the first focus frame.

[0144] In this embodiment, by updating the first focus frame to the second focus frame, the inward shrinkage of the first focus frame is achieved, thereby avoiding the interference caused by the foreground object at the edge of the first focus frame to the camera shooting, and then performing focusing based on the second focus frame, that is, focusing on the central subject, which improves the user experience of focusing.

[0145] Figure 19 It is a flowchart of a method for optimizing foreground of misfocus in an embodiment of the present disclosure. In this embodiment, as Figure 19 shown.

[0146] First step, determine the size of the ROI (denoted as CenterRoi) corresponding to the first focus frame in the first preview image. Compare the width CenterRoiWidth of the first focus frame with the first focus frame width threshold CenterRoiWidthThr, and the height CenterRoiHeight of the first focus frame with the first focus frame height threshold CenterRoiHeightThr. If CenterRoiWidth >= CenterRoiWidthThr and / or CenterRoiHeight >= CenterRoiHeightThr, then perform the following edge misfocus optimization logic processing; otherwise, perform normal focusing with the original CenterRoi.

[0147] In the second step, perform image semantic segmentation on the first preview image.

[0148] In the third step, determine the in-focus block. For the sub-focus blocks subROI obtained by dividing the first focus frame CenterRoi, each subROI is screened by the phase difference convergence threshold PdConvThr and the phase difference confidence threshold ConfThr. Compare each sub-focus block BlockPd_i in the subROI with PdConvThr and ConfThr, and retain the block with a small phase difference convergence value and a high confidence as the in-focus block Block_i, and record it as FocusedBlock_i.

[0149] In the fourth step, determine whether the in-focus block is too small. In the in-focus block, if the sum of the areas AreaFocusedBlock_i of the object regions that have been determined by semantic segmentation is less than the in-focus area threshold FocusedAreaThr, it indicates that the objects in the object region are foreground objects with small areas and in focus, then it is possible to determine whether the object region is located on the boundary of the focus frame. Otherwise, it indicates that the original CenterRoi is in normal focus.

[0150] In the fifth step, determine whether the object region is close to the boundary. Mark the center point of the largest in-focus object region MaxAreaFocusedBlock_i with a high phase difference confidence and a small phase difference convergence value, calculate the shortest boundary distance from this center point to the nearest boundary of the first focus frame, and record it as FocusedPointDist. If the shortest boundary distance is less than the boundary distance threshold BoundaryDistThr, it indicates that the foreground object is located on the edge of the focus frame. Then adjust the focus frame for autofocus. Otherwise, it indicates that the foreground object is not located on the edge part of the focus frame, and there is no need to adjust the focus frame, and continue to use the original CenterRoi for normal focus.

[0151] In the sixth step, adjust the focus frame. According to the width ratio BoundaryWidthRatio and the height ratio BoundaryHeightRatio, adjust the first focus frame to the second focus frame. The width Width of the second focus frame is equal to the width OriWidth of the first focus frame multiplied by the width ratio BoundaryWidthRatio, that is, Width = BoundaryWidthRatio * OriWidth. The height Height of the second focus frame is equal to the height OriHeight of the first focus frame multiplied by the height ratio BoundaryHeightRatio, that is, Height = BoundaryHeightRatio * OriHeight. Thus, the update of the focus frame is completed, and then normal focus is performed using the new second focus frame.

[0152] Figure 20 A schematic diagram of focus frame update according to an embodiment of the present disclosure. In this embodiment, as Figure 20 shown, the left figure is a schematic diagram of the first focus frame, and the right figure is a schematic diagram of the second focus frame. Before the focus method of the present disclosure is executed, the focus frame of the first preview image is the first focus frame, where the main body of the duck in the picture is blurred and displayed, while the edge foreground object display is clearly imaged. After being processed by the focus method of the present disclosure, the focus frame of the first preview image is the second focus frame. Obviously, the second focus frame is the focus frame obtained by shrinking and adjusting the first focus frame, which focuses on the duck main body itself, avoiding the interference of foreground objects, and thus presenting a clear picture main body.

[0153] The embodiment of the present disclosure provides a focus method, which acquires the first preview image collected through the first focus frame, providing a data source for automatic focus; based on the first preview image, determines the object area in the in-focus block through a semantic segmentation model, determining the object included in the in-focus block of the focus frame; according to the object area and the in-focus block, determines the foreground object area in the in-focus block, segmenting and locating the foreground object from the in-focus block, providing a reference foreground object for automatic focus; if the foreground object area is located in the edge area of the first focus frame, updates the first focus frame to the second focus frame, and based on the foreground object being located at the edge of the focus frame, further adjusts the size of the focus frame to ensure that the focus frame can focus on the main body in the center of the picture, improving the user experience of automatic focus.

[0154] Corresponding to the methods provided in the above several embodiments, the present disclosure also provides a focus device. Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above several embodiments, the implementation manners of the methods are also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.

[0155] Figure 21 A schematic structural diagram of a focus device 2100 according to an embodiment of the present disclosure. As Figure 21 shown, the focus device includes:

[0156] An acquisition module 2110, configured to acquire a first preview image based on the first focus frame;

[0157] A semantic segmentation module 2120, configured to determine the object area in the in-focus block through a semantic segmentation model based on the first preview image, where the in-focus block is obtained by dividing the first focus frame;

[0158] A foreground determination module 2130, configured to determine the foreground object area in the in-focus block according to the object area and the in-focus block;

[0159] The focusing module 2140 is configured to update the first focusing frame to a second focusing frame and perform focusing according to the second focusing frame if the foreground object area is located in the edge area of the first focusing frame.

[0160] In some embodiments, the semantic segmentation module 2120 is configured to:

[0161] Obtain the phase difference attributes of the first preview image, where the phase difference attributes include a phase difference convergence value and a phase difference confidence level;

[0162] Determine a focused block based on the phase difference attributes of all sub-focusing blocks of the first preview image, where the sub-focusing blocks are obtained by dividing based on the first focusing frame;

[0163] Based on the first part of the first preview image, determine the object area in the focused block through a semantic segmentation model, where the first part includes the focused block.

[0164] In some embodiments, the semantic segmentation module 2120 determines the focused block based on the phase difference attributes of all sub-focusing blocks of the first preview image in the following manner:

[0165] For all sub-focusing blocks, determine multiple sub-focusing blocks with a phase difference confidence level greater than or equal to a first confidence threshold as a first focusing block sequence;

[0166] Reorder the first focusing block sequence in descending order of the phase difference convergence value to obtain a second focusing block sequence;

[0167] Use the sub-focusing blocks in the second focusing block sequence with a phase difference convergence value less than or equal to a first convergence threshold as the focused block.

[0168] In some embodiments, the foreground determination module 2120 determines the foreground object area in the focused block based on the object area and the focused block in the following manner:

[0169] Determine a first object area sequence in which the phase difference convergence value of all object areas is less than or equal to a second convergence threshold, and use the ratio of the sum of the areas of all object areas in the first object area sequence to the area of the focused block as the focused area ratio;

[0170] Determine the object area with the highest phase difference confidence level in the first object area sequence as the first object area;

[0171] If the focused area ratio of the first object area is greater than or equal to a first ratio threshold, exit the focusing frame adjustment logic; otherwise, use the first object area as the foreground object area.

[0172] In some embodiments, before collecting the first preview image based on the first focusing frame, the collection module 2110 is further configured to:

[0173] Compare the height of the first focus frame with the first focus frame height threshold and compare the width of the first focus frame with the first focus frame width threshold;

[0174] If the height of the first focus frame is greater than or equal to the first focus frame height threshold and / or the width of the first focus frame is greater than or equal to the first focus frame width threshold, collect a first preview image based on the first focus frame; otherwise, confirm focus.

[0175] In some embodiments, before the focusing module 2140 updates the first focus frame to the second focus frame if the foreground object area is located in the edge area of the first focus frame, it is further configured to:

[0176] Determine the object center of the foreground object area, where the object center includes the centroid, geometric center, and center of gravity;

[0177] Determine the shortest boundary distance according to the object center and the first focus frame;

[0178] Determine whether the foreground object area is located in the edge area of the first focus frame according to the shortest boundary distance.

[0179] In some embodiments, the focusing module 2140 determines whether the foreground object area is located in the edge area of the first focus frame according to the shortest boundary distance in the following manner:

[0180] Compare the shortest boundary distance with the first distance threshold;

[0181] If the shortest boundary distance is less than or equal to the first distance threshold, determine that the foreground object area is located in the edge area of the first focus frame.

[0182] In some embodiments, the focusing module 2140 updates the first focus frame to the second focus frame in the following manner:

[0183] Determine the width of the second focus frame according to the preset width ratio and the width of the first focus frame;

[0184] Determine the height of the second focus frame according to the preset height ratio and the height of the first focus frame.

[0185] In summary, through the focusing device, a first preview image is collected based on the first focus frame; based on the first preview image, the object area in the in-focus block is determined through a semantic segmentation model, and the in-focus block is obtained by dividing the first focus frame; according to the object area and the in-focus block, the foreground object area in the in-focus block is determined; if the foreground object area is located in the edge area of the first focus frame, the first focus frame is updated to the second focus frame, and focusing is performed according to the second focus frame. This device solves the problem of the main subject in the center of the picture being out of focus and blurred when the foreground object is located at the edge of the focus frame, and improves the user experience of automatic focusing.

[0186] In the above embodiments provided by the present application, the methods and apparatuses provided by the embodiments of the present application are introduced. To implement the various functions in the methods provided by the embodiments of the present application, an electronic device may include a hardware structure, software modules, and implement the above various functions in the form of a hardware structure, software modules, or a combination of a hardware structure and software modules. A certain function among the above various functions may be executed in the manner of a hardware structure, software modules, or a combination of a hardware structure and software modules.

[0187] Figure 22 It is a block diagram of an electronic device 2200 for implementing the above focus method shown according to an exemplary embodiment.

[0188] For example, the electronic device 2200 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0189] Referring to Figure 22 , the electronic device 2200 may include one or more of the following components: a processing component 2202, a memory 2204, a power supply component 2206, a multimedia component 2208, an audio component 2210, an input / output (I / O) interface 2212, a sensor component 2214, and a communication component 2216.

[0190] The processing component 2202 generally controls the overall operation of the electronic device 2200, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 2202 may include one or more processors 2220 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 2202 may include one or more modules to facilitate the interaction between the processing component 2202 and other components. For example, the processing component 2202 may include a multimedia module to facilitate the interaction between the multimedia component 2208 and the processing component 2202.

[0191] The memory 2204 is configured to store various types of data to support the operation of the electronic device 600. Examples of these data include instructions for any application or method operating on the electronic device 2200, contact data, phone book data, messages, pictures, videos, etc. The memory 2204 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0192] The power supply component 2206 provides power for various components of the electronic device 2200. The power supply component 2206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 2200.

[0193] The multimedia component 2208 includes a screen that provides an output interface between the electronic device 2200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 2208 includes a front camera and / or a rear camera. When the electronic device 2200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0194] The audio component 2210 is configured to output and / or input audio signals. For example, the audio component 2210 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 2200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 2204 or transmitted via the communication component 2216. In some embodiments, the audio component 2210 further includes a speaker for outputting audio signals.

[0195] The I / O interface 2212 provides an interface between the processing component 2202 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power-on button, and a lock button.

[0196] The sensor assembly 2214 includes one or more sensors for providing a status assessment of various aspects of the electronic device 2200. For example, the sensor assembly 2214 can detect the on / off state of the electronic device 2200, the relative positioning of components, such as components for the display and keypad of the electronic device 2200. The sensor assembly 2214 can also detect a change in the position of the electronic device 2200 or a component of the electronic device 2200, the presence or absence of user contact with the electronic device 2200, the orientation or acceleration / deceleration of the electronic device 2200, and the temperature change of the electronic device 2200. The sensor assembly 2214 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 2214 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 2214 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0197] The communication component 2216 is configured to facilitate communication between the electronic device 2200 and other devices in a wired or wireless manner. The electronic device 2200 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or a combination thereof. In an exemplary embodiment, the communication component 2216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 2216 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0198] In an exemplary embodiment, the electronic device 2200 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0199] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 2204 including instructions, and the above instructions can be executed by the processor 2220 of the electronic device 2200 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0200] Embodiments of the present disclosure also propose a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute the focusing method described in the above embodiments of the present disclosure.

[0201] Embodiments of the present disclosure also propose a computer program product, including a computer program, where the computer program, when executed by a processor, implements the focusing method described in the above embodiments of the present disclosure.

[0202] Figure 23 FIG. 2300 is a schematic structural diagram of a chip 2300 for implementing the above focusing method according to an exemplary embodiment.

[0203] Referring to Figure 23 , the chip 2300 includes at least one communication interface 2301 and a processor 2302; the communication interface 2301 is configured to receive signals input to the chip 2300 or signals output from the chip 2300, and the processor 2302 communicates with the communication interface 2301 and implements the focusing method described in the above embodiments through logic circuits or by executing code instructions.

[0204] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order different from those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0205] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0206] Any process or method description represented in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process. Moreover, the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be executed not in the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, as would be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0207] Logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing a logical function and can be embodied specifically in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processing module, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. As used in this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (control method), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable medium upon which a program can be printed, as the program can be electronically obtained, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0208] It should be understood that various parts of the embodiments of the present disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits having logic gates for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0209] Those of ordinary skill in the art can understand that all or part of the steps carried out in the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0210] In addition, in each of the various embodiments of the present disclosure, the functional units can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc.

[0211] Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A focusing method, characterized in that, The method includes: Collecting a first preview image based on a first focus frame; Based on the first preview image, determining an object region in a focused block through a semantic segmentation model, where the focused block is obtained by dividing the first focus frame; Determining a foreground object region in the focused block according to the object region and the focused block; If the foreground object region is located in an edge region of the first focus frame, updating the first focus frame to a second focus frame and performing focusing according to the second focus frame.

2. The method according to claim 1, wherein The determining, based on the first preview image, an object region in the focused block through a semantic segmentation model includes: Obtaining a phase difference attribute of the first preview image, where the phase difference attribute includes a phase difference convergence value and a phase difference confidence level; Based on the phase difference attributes of all sub-focused blocks of the first preview image, determining the focused block, where the sub-focused blocks are obtained by dividing the first focus frame; Based on a first part of the first preview image, determining the object region in the focused block through the semantic segmentation model, where the first part includes the focused block.

3. The method according to claim 2, wherein The determining the focused block based on the phase difference attributes of all sub-focused blocks of the first preview image includes: For all sub-focused blocks, determining a plurality of sub-focused blocks with the phase difference confidence level greater than or equal to a first confidence threshold as a first focused block sequence; Reordering the first focused block sequence in descending order of the phase difference convergence value to obtain a second focused block sequence; Taking the sub-focused blocks in the second focused block sequence with the phase difference convergence value less than or equal to a first convergence threshold as the focused block.

4. The method according to claim 2, wherein The determining the foreground object region in the focused block according to the object region and the focused block includes: Determining a first object region sequence in which the phase difference convergence value of all the object regions is less than or equal to a second convergence threshold, and using the ratio of the sum of the areas of all the object regions in the first object region sequence to the area of the focused block as a focused area ratio; Determining the object region with the highest phase difference confidence level in the first object region sequence as a first object region; If the focused area ratio of the first object region is greater than or equal to a first ratio threshold, confirming focusing; otherwise, using the first object region as the foreground object region.

5. The method according to claim 1, characterized in that, Before collecting the first preview image based on the first focus frame, it further includes: Comparing the height of the first focus frame with a first focus frame height threshold and comparing the width of the first focus frame with a first focus frame width threshold; If the height of the first focus frame is greater than or equal to the first focus frame height threshold and / or the width of the first focus frame is greater than or equal to the first focus frame width threshold, collecting a first preview image based on the first focus frame; otherwise, confirming focusing.

6. The method according to claim 1, wherein Before updating the first focus frame to a second focus frame if the foreground object region is located in an edge region of the first focus frame, the method further includes: Determine the object center of the foreground object area, where the object center includes the centroid, geometric center, and center of gravity; Determine the shortest boundary distance according to the object center and the first focusing frame; Determine whether the foreground object area is located in the edge area of the first focusing frame according to the shortest boundary distance.

7. The method according to claim 6, wherein The determining whether the foreground object area is located in the edge area of the first focusing frame according to the shortest boundary distance includes: Compare the shortest boundary distance with a first distance threshold; If the shortest boundary distance is less than or equal to the first distance threshold, determine that the foreground object area is located in the edge area of the first focusing frame.

8. The method according to claim 7, characterized in that The updating the first focusing frame to a second focusing frame includes: Determine the width of the second focusing frame according to a preset width ratio and the width of the first focusing frame; Determine the height of the second focusing frame according to a preset height ratio and the height of the first focusing frame.

9. A focusing device, characterized in that, The device includes: An acquisition module, configured to acquire a first preview image based on a first focusing frame; A semantic segmentation module, configured to determine an object area in a focused block based on the first preview image through a semantic segmentation model, where the focused block is obtained by dividing the first focusing frame; A foreground determination module, configured to determine a foreground object area in the focused block according to the object area and the focused block; A focusing module, configured to update the first focusing frame to a second focusing frame if the foreground object area is located in the edge area of the first focusing frame, and perform focusing according to the second focusing frame.

10. An electronic device, characterized in that, Includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-8.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-8.

13. A chip, characterized in that, Includes at least one processor and a communication interface; the communication interface is configured to receive a signal input to the chip or a signal output from the chip, and the processor communicates with the communication interface and implements the method according to any one of claims 1-8 through a logic circuit or by executing code instructions.

Images