A method and apparatus for determining the focus area, electronic equipment, and storage medium.

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for determining a focus area. The method includes: determining an initial focus area containing a subject in an image to be focused; determining at least one target pixel value based on the pixel values ​​of pixels contained in the initial focus area, and clustering the pixels in the image to be focused according to the determined target pixel values; and determining the area covered by the clustered pixel clusters as the target focus area for focusing the image to be focused.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to a method and apparatus for determining a focus area, an electronic device, and a storage medium. Background Technology

[0002] With the advent of the era of universal photography, users have increasingly higher demands for the image quality of their photos. A prerequisite for obtaining high-quality images is that the image itself possesses high sharpness. Therefore, how to accurately focus on the subject is a problem that urgently needs to be solved in the image shooting process.

[0003] In related technologies, a fixed area in the image is typically designated as the focus area to achieve focusing on the subject. For example, the center area of ​​the image can be defaulted to the focus area for metering, focusing, and other operations. However, in practical applications, the subject may not be located within the fixed focus area, leading to inaccurate focusing and affecting the sharpness of the final image. Summary of the Invention

[0004] This disclosure provides a method and apparatus for determining the focus area, an electronic device, and a storage medium, which can achieve accurate focusing on the subject being photographed.

[0005] According to a first aspect of this disclosure, a method for determining a focus area is provided, comprising:

[0006] Identify the initial focus area containing the subject in the image to be focused;

[0007] At least one target pixel value is determined based on the pixel values ​​of the pixels contained in the initial focus area, so as to cluster the pixels contained in the image to be focused according to the determined target pixel values.

[0008] The area covered by the cluster of pixels obtained by clustering is determined as the target focus area when focusing the image to be focused.

[0009] Optionally, determining the initial focus area containing the subject in the image to be focused includes:

[0010] In response to a user's trigger operation on the image to be focused, the area where the trigger operation is located is determined as the initial focus image; or,

[0011] The image to be focused is processed based on a preset algorithm to identify the region of the preset object corresponding to the preset algorithm in the image to be focused, and the identified region is determined as the initial focus region.

[0012] Optionally, determining at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area includes:

[0013] At least one target pixel is selected from the initial focus area, and the pixel value of the selected target pixel is used as the determined target pixel value.

[0014] Optionally, selecting at least one target pixel from the initial focus area includes:

[0015] Obtain the pixel values ​​of all pixels contained in the initial focus area, and select the pixels whose pixel values ​​differ from those of their neighboring pixels within a preset range as target pixels; or...

[0016] Identify a pixel located at a specific position within the initial focus area, and designate the identified pixel as the target pixel; or,

[0017] At least one target pixel is randomly selected from the initial focus area.

[0018] Optionally, selecting at least one target pixel from the initial focus area to cluster the pixels in the image to be focused based on the pixel values ​​of the selected target pixel includes: selecting at least two reference pixels from the image to be focused to cluster the pixels in the image to be focused based on the pixel values ​​of the selected reference pixels; wherein a portion of the selected reference pixels are target pixels selected from the initial focus area.

[0019] The step of determining the area covered by the clustered pixel clusters as the target focus area when focusing the image to be focused includes: determining the area covered by the pixel clusters that have a preset association with the initial focus area among the multiple clustered pixel clusters as the target focus area when focusing the image to be focused.

[0020] The preset association includes any of the following: the cluster center is located within the initial focus area; the difference between the pixel value of the cluster center and the average pixel value of the initial focus area is within a preset range; and the area of ​​the pixel cluster located within the initial focus area is larger than the area located outside the initial focus area.

[0021] Optionally, determining at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area includes:

[0022] Obtain the pixel values ​​of all pixels in the initial focus area, and calculate the average pixel value of all obtained pixels;

[0023] The calculated average pixel value is determined as the target pixel value.

[0024] Optionally, the calculated pixel mean of all pixels includes:

[0025] The position of each pixel in the initial focus area is determined, and the pixel value of all the obtained pixels is calculated by weighting the pixel values ​​based on the weight values ​​set in advance for each position to obtain the pixel mean.

[0026] Optionally, the method further includes:

[0027] The target focus area is used as the initial focus area in the next frame of the image to be focused, so as to determine the target pixel value used in the clustering of the next frame of the image to be focused.

[0028] Optionally, the method further includes: substituting the pixel values ​​of at least one pixel cluster obtained by clustering into the Gaussian mixture model to obtain several Gaussian mixture components corresponding to each pixel cluster; and adjusting the parameter values ​​of the several Gaussian mixture components to adjust the area covered by each pixel cluster until all the several Gaussian mixture components converge.

[0029] The step of determining the region covered by the clustered pixel points as the target focus region when focusing the image to be focused includes: when all the Gaussian mixture components converge, determining the region covered by the adjusted pixel point cluster as the target focus region when focusing the image to be focused.

[0030] Optionally, determining the area covered by the clustered pixel clusters as the target focus area for focusing the image to be focused includes:

[0031] The region covered by the cluster of pixels obtained by clustering is determined as the focus region to be determined, and the focus image containing the focus region to be determined is input into the pre-trained focus region optimization model so as to optimize the focus region to be determined through the focus region optimization model.

[0032] The optimized focus region contained in the image output by the focus region optimization model is determined as the target focus region to be used when focusing the image to be focused.

[0033] According to a second aspect of this disclosure, a focus area determining device is provided, comprising:

[0034] The first determining unit is used to determine the initial focus area containing the subject in the image to be focused;

[0035] A clustering unit is used to determine at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area, so as to cluster the pixels contained in the image to be focused according to the determined target pixel value.

[0036] The second determining unit is used to determine the area covered by the cluster of pixels obtained by clustering as the target focusing area when focusing the image to be focused.

[0037] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0038] processor;

[0039] Memory used to store processor-executable instructions;

[0040] The processor implements the method as described in the first aspect by running the executable instructions.

[0041] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0042] In the technical solution disclosed herein, when image capture is required, an initial focus area of ​​the subject contained in the image to be focused is first determined, and at least one target pixel value is determined based on the pixel values ​​of the pixels contained in the initial focus area. Then, the pixels contained in the image to be focused are clustered according to the at least one target pixel value. Based on this, the area covered by the clustered pixel clusters can be determined as the target focus area used when focusing the image to be focused.

[0043] It should be understood that since the initial focus area includes the subject, at least one target pixel value determined based on the pixel values ​​of the pixels in the initial focus area can largely characterize the approximate pixel values ​​of the pixels contained in the subject. Based on this, the pixel clusters obtained by clustering based on the target pixel value include pixels with pixel values ​​close to the target pixel value; correspondingly, the area covered by the pixel clusters should be the area where the subject is located. Clearly, when focusing on the image to be focused, using this area as the target focus area is focusing on the subject, which can maximize the focusing accuracy for the subject. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0045] Figure 1This is a flowchart illustrating a method for determining a focus area according to an exemplary embodiment of this disclosure;

[0046] Figure 2 This is a flowchart illustrating another method for determining the focus area, as shown in an exemplary embodiment of this disclosure;

[0047] Figure 3 This is a block diagram illustrating a focus area determination device according to an exemplary embodiment of the present disclosure;

[0048] Figure 4 This is a block diagram illustrating another focus area determining device according to an exemplary embodiment of the present disclosure;

[0049] Figure 5 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0050] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0051] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0052] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0053] Smartphones, tablets, and other electronic devices are typically equipped with cameras. Users can then use camera software to take pictures of their subjects through a combination of software and hardware. For example, after launching the camera software, the camera can capture images of the surrounding environment. Once the image is captured, the camera can focus on the objects within it to ensure they are in sharp focus.

[0054] In related technologies, when electronic devices focus on objects in a scene, they do so based on a pre-set fixed area. For example, the area in the center of the frame can be pre-set as the focus area for focusing on the subject. However, in actual image shooting, the subject is not necessarily located in the center of the frame. In this case, because focusing is done on the central area of ​​the frame, the resulting image may still be blurry, leading to lower sharpness in the final image.

[0055] To address this issue, this disclosure proposes a method for determining the focus area, thereby avoiding the problem in related technologies where focusing on the subject based on a fixed focus area leads to poor focusing performance.

[0056] Figure 1 This embodiment of the present disclosure illustrates a method for determining a focus area. For example... Figure 1 As shown, the method may include the following steps:

[0057] Step 102: Determine the initial focus area containing the subject in the image to be focused.

[0058] As can be seen from the above introduction, the reason why the image clarity of the final image obtained in the related technology is poor is because the related technology focuses on the subject in the picture based on a fixed focus area.

[0059] In view of this, this disclosure no longer focuses on the subject in the image based on a fixed focus area. Instead, it determines a specific focus area for different images to be focused, and focuses on the subject in the image to be focused based on the determined focus area. Before focusing on the subject, this disclosure can first determine an initial focus area containing the subject in the image to be focused, and determine at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area. The pixels in the image to be focused are then clustered based on the at least one target pixel value. On this basis, the area covered by the clustered pixel clusters can be determined as the target focus area used when focusing the image to be focused.

[0060] It should be understood that this disclosure determines the target pixel value (i.e., cluster center) for clustering based on the pixel values ​​of the pixels in the initial focus area where the subject is located. This cluster center can characterize the approximate range of pixel values ​​contained in the subject. Since the pixel values ​​of the pixels in the cluster obtained based on this cluster center are mostly similar to the target pixel value, the area covered by the clustered pixel cluster can accurately characterize the area of ​​the subject in the image to be focused. Clearly, using the area covered by the clustered pixel cluster as the target focus area can improve the accuracy of focusing on the subject and avoid the problem of inaccurate focusing results caused by focusing on a fixed focus area in related technologies.

[0061] In this disclosure, the initial focus area can be determined in a variety of ways.

[0062] In some embodiments, the initial focus area can be manually determined by the user. In practical applications, this disclosure can respond to a user's trigger operation on the image to be focused and determine the area where the trigger operation is located as the initial focus area. For example, after a user opens the camera software, the camera can be invoked to capture an image of the surrounding environment and display a preview image in the camera software. Then, the user can trigger any area in the preview image to determine that area as the initial focus area.

[0063] In other embodiments, the initial focus image can be automatically determined by the electronic device. In practical applications, this disclosure can, upon acquiring the image to be focused, process the image based on a preset algorithm to identify the region of a preset object corresponding to the preset algorithm within the image to be focused, and then determine the identified region as the initial focus region. For example, after acquiring a preview image, this disclosure can process the image to be focused based on a face recognition algorithm to identify the region of a face within the image to be focused, and then determine the region where the face is located as the initial focus region. Of course, this example is merely illustrative. The specific algorithm used and the region containing the object determined as the initial focus region can be determined by those skilled in the art according to actual needs. For example, the preset object can also be an animal's face, an object with a specific structure, etc., and this disclosure does not limit this.

[0064] Step 104: Determine at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area, so as to cluster the pixels contained in the image to be focused according to the determined target pixel values.

[0065] After determining the initial focus area containing the subject, at least one target pixel value can be determined based on the pixel values ​​of the pixels contained in the initial focus area, which can then be used as a cluster center to cluster the pixels contained in the image to be focused.

[0066] In this disclosure, the target pixel value can be determined in various ways based on the pixel values ​​of the pixels contained in the initial focus area.

[0067] In some embodiments, at least one target pixel can be selected from the initial focus area, and the pixel value of the selected target pixel can be used as the determined target pixel value. Based on this, the pixels in the image to be focused can be clustered. In this embodiment, different methods can be used to select target pixels from the initial focus area in different situations.

[0068] In one embodiment, the pixel values ​​of all pixels contained in the initial focus area can be acquired first, and pixels whose pixel values ​​differ from those of their neighboring pixels within a preset range can be selected as target pixels. It should be understood that the area covered by the subject is typically composed of continuous pixels with similar pixel values, rather than discrete pixels with significantly different pixel values ​​from their surroundings. Therefore, when the pixel value difference between a pixel and its neighboring pixels is small, this pixel is highly likely to be a pixel contained within the area covered by the subject. In this case, this pixel can represent the approximate pixel values ​​of the pixels contained within the subject. Accordingly, clustering based on the pixel value of this pixel can more accurately determine the area where the subject is located.

[0069] In another embodiment, a pixel located at a specific position within the initial focus area can be designated as a target pixel. For example, a pixel located at the center of the initial focus area can be used as the target pixel. It should be understood that, regardless of whether the initial focus area is determined based on the aforementioned preset algorithm or by the user's own selection, in most cases, the pixel located at the center of the initial focus area is a pixel within the subject. Designating it as a target pixel for clustering can more accurately determine the area where the subject is located.

[0070] In another embodiment, at least one target pixel can be randomly selected from the initial focus area. For example, a preset number of target pixels can be selected from the initial focus area, and the pixels in the image to be focused can be clustered based on the pixel values ​​of these target pixels. It should be understood that the determined initial focus area contains the subject, and whether it is determined by the electronic device based on a preset algorithm or actively selected by the user, the subject usually occupies most of the initial focus area, so that most of the randomly selected target pixels are located within the subject. Therefore, after clustering based on the selected target pixels, the area where the subject is located can be determined more accurately.

[0071] Of course, the above three methods for determining target pixels are all illustrative. How to select target pixels from the initial focus area can be determined by those skilled in the art according to actual needs, and this embodiment does not limit this.

[0072] In this embodiment, besides selecting target pixels from the initial focus area for clustering, clustering can also be performed based on pixels outside the initial focus area to more accurately determine the area where the subject is located. For example, this disclosure can select at least two reference pixels from the image to be focused, and cluster the pixels in the image to be processed based on the pixel values ​​of the selected reference pixels, wherein a portion of the selected reference pixels are target pixels selected from the initial focus area. Based on this, the area covered by the pixel clusters that have a preset association with the initial focus area among the multiple pixel clusters obtained based on each reference pixel can be determined as the target focus area used when focusing the image to be focused.

[0073] The preset association can be determined by those skilled in the art based on actual needs, and this embodiment does not impose any restrictions on it. For example, the preset association can be "the cluster center is located within the initial focus area." In other words, after clustering to obtain each pixel cluster, the area covered by the pixel cluster whose cluster center is located within the initial focus area can be determined as the target focus area. Another example is that the preset association can be "the difference between the pixel value of the cluster center and the pixel mean of the initial focus area is within a preset range." In other words, after clustering to obtain each pixel cluster, the pixel mean of all pixels within the initial focus area can be calculated, and the difference between the cluster center of each pixel cluster and the pixel mean can be calculated. If the difference between the cluster center and some cluster centers is within a preset range... If the pixel clusters are within a preset range, the area covered by the pixel clusters corresponding to these cluster centers is determined as the target focus area. For example, the preset association can also be "the area of ​​the pixel cluster within the initial focus area is greater than the area outside the initial focus area". In other words, after clustering each pixel cluster, it can be determined whether each pixel cluster is within or outside the initial focus area. If the area of ​​any pixel cluster within the initial focus area is greater than the area outside the initial focus area, then that pixel cluster is determined as at least a part of the area where the subject is located.

[0074] Of course, the above-mentioned preset associations are all illustrative. How to set the preset associations can be determined by those skilled in the art based on the actual situation. This embodiment does not limit this.

[0075] In other embodiments, the target pixel value for clustering can be calculated based on the pixel values ​​of all pixels in the initial focus area. For example, the average pixel value of all pixels can be calculated based on the pixel values ​​of all pixels in the initial focus area, and the calculated average pixel value can be used as the target pixel value.

[0076] In one scenario, a weighted average algorithm can be used to calculate the pixel mean. In this case, after obtaining the pixel values ​​of all pixels, the position of each pixel within the initial focus area can be further determined. Based on pre-set weight values ​​for each position, a weighted average of all obtained pixel values ​​is calculated to obtain the pixel mean. In practice, the weight value of each position within the initial focus area can be related to the distance between that position and the center of the initial focus area. For example, the weight value of each position can be negatively correlated with this distance; that is, the farther the distance from the center of the initial focus area, the smaller the weight value assigned to that position. Of course, this example is merely illustrative, and the specific determination of the weight value for each position can be determined by those skilled in the art based on the actual situation. This embodiment does not impose any limitations on this.

[0077] Step 106: Determine the area covered by the pixel point clusters obtained through clustering as the target focus area for focusing on the to-be-focused image.

[0078] In the present disclosure, after obtaining the target focus area of the to-be-focused image, on the one hand, the subject in the to-be-focused image can be focused based on this target focus area; on the other hand, when obtaining the next frame of the to-be-focused image, this target focus area can also be used as the initial focus area of the next frame of the to-be-focused image to determine the target pixel value used for clustering the next frame of the to-be-focused image.

[0079] For example, in the preview interface of the camera software, due to the inevitable slight jitter of the user, the area of the subject in the preview screen will inevitably change. In this case, the target focus area determined in the previous frame of the to-be-focused image can be determined as the initial focus area of the next frame of the to-be-focused image to determine the clustering center. It should be understood that during the preview of the screen, since the camera captures the screen at a relatively fast rate, the time difference between two frames of images is small. Correspondingly, the area of the subject in adjacent frames of the to-be-focused images is approximately the same. Therefore, the target focus area of the previous frame of the to-be-focused image can be determined as the initial focus area of the next frame of the to-be-focused image, avoiding the need to separately determine the initial focus area for different to-be-focused images.

[0080] In the present disclosure, after clustering to obtain at least one pixel point cluster, the area covered by the pixel point cluster can also be adjusted to more precisely determine the area where the subject is located. For example, the pixel values of the at least one pixel point cluster obtained through clustering can be respectively substituted into the Gaussian mixture model to obtain several Gaussian mixture components corresponding to each pixel point cluster. On this basis, the parameter values of the several Gaussian mixture components can be adjusted to adjust the area covered by each pixel point cluster until the several Gaussian mixture components all converge. Among them, when the several Gaussian mixture components all converge, the area covered by the adjusted pixel point cluster can be determined as the target focus area for focusing on the to-be-focused image.

[0081] It should be understood that some of the pixel points in the pixel point cluster obtained through clustering may be discrete, resulting in the edge area of the area covered by the pixel point cluster being not smooth. However, substituting the pixel values of the pixel points included in the pixel point cluster into the Gaussian mixture model and making the obtained Gaussian mixture components converge can make the edge area of the corresponding pixel point cluster smooth. It can be seen that the present disclosure can improve the accuracy of the target focus area through the Gaussian mixture model.

[0082] In this disclosure, in addition to optimizing the determined focus area using a Gaussian mixture model, the region covered by the clustered pixel clusters can also be optimized using a pre-trained focus area optimization model. In practice, the region covered by the clustered pixel clusters can be used as the focus area to be determined, and the image containing this focus area can be input into the pre-trained focus area optimization model for optimization. After optimization by the focus area optimization model, the optimized focus area contained in the image output by the model can be determined as the target focus area for focusing the image.

[0083] In practical applications, technicians can pre-collect several sample images and label each image to obtain target sample images with a defined focus area. Based on this, each sample image can be used as input to the model to be trained, and the corresponding target sample image can be used as output to the model to be trained. The model can then be iteratively trained until a focus area optimization model is obtained. Of course, this example is merely illustrative; how to specifically train the focus area optimization model can be determined by those skilled in the art based on the actual situation, and this disclosure does not impose any restrictions. Furthermore, during the actual training process, the specific type of model used as the focus area optimization model in this disclosure can also be determined by those skilled in the art based on the actual situation, and this disclosure does not impose any restrictions. For example, the focus area optimization model can be an image segmentation model.

[0084] It should be understood that the above two focus area optimization methods can be applied individually or simultaneously to image focusing. For example, the pixel values ​​of the pixels contained in the clustered pixel clusters can be preferentially substituted into the Gaussian mixture model to perform initial optimization of the clustered focus area. Then, the image containing the initially optimized focus area can be input into the focus area optimization model for secondary optimization, and the focus area obtained from the secondary optimization can be used as the target focus area for final focusing. Alternatively, the image containing the clustered focus area can be preferentially input into the focus area optimization model to obtain the initial focus area after initial optimization. Based on this, the pixel values ​​of the pixels contained in the initial focus area can be input into the Gaussian mixture model to perform secondary optimization of the initially optimized initial focus area. Of course, the above examples are only illustrative. When using both optimization methods simultaneously, the choice of which optimization method to prioritize can be determined by those skilled in the art based on the actual situation, and this disclosure does not impose any restrictions on this.

[0085] It should be stated that the focus area determination method of this disclosure can be used both during the display of preview images and during video recording. Of course, both application scenarios are illustrative. The specific scenario in which the technical solution of this disclosure is applied can be determined by those skilled in the art based on actual needs, and this disclosure does not impose any restrictions on this. Furthermore, the pixel values ​​in this disclosure can be determined according to actual conditions. For example, the pixel value can be a grayscale value; or, for example, the pixel value can be a color value, such as an RGB value. The specific parameters of the pixel values ​​in this disclosure can be determined by those skilled in the art based on actual conditions, and this disclosure does not impose any restrictions on this.

[0086] It should also be stated that the technical solution disclosed herein can be applied to any type of electronic device. For example, the electronic device can be a mobile terminal such as a smartphone or tablet computer, or a fixed terminal such as a smart TV or PC (Personal Computer). It should be understood that any electronic device with image capture capabilities can serve as the implementing entity of the technical solution disclosed herein. The specific type of electronic device used as the implementing entity can be determined by those skilled in the art based on actual needs, and this disclosure does not impose any restrictions in this regard.

[0087] As described above, when image capture is required, this disclosure first determines the initial focus area of ​​the subject contained in the image to be focused, and determines at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area. Then, it clusters the pixels contained in the image to be focused based on the at least one target pixel value. Based on this, the area covered by the clustered pixel clusters can be determined as the target focus area used when focusing the image.

[0088] It should be understood that since the initial focus area includes the subject, at least one target pixel value determined based on the pixel values ​​of the pixels in the initial focus area can largely characterize the approximate pixel values ​​of the pixels contained in the subject. Based on this, the pixel clusters obtained by clustering based on the target pixel value include pixels with pixel values ​​close to the target pixel value; correspondingly, the area covered by the pixel clusters should be the area where the subject is located. Clearly, when focusing on the image to be focused, using this area as the target focus area is focusing on the subject, which can maximize the focusing accuracy for the subject.

[0089] Furthermore, after clustering to obtain pixel clusters, the pixel values ​​of the pixels within each cluster can be substituted into a Gaussian mixture model to obtain the Gaussian mixture component corresponding to the pixel cluster. Based on this, the parameters of the Gaussian mixture component can be adjusted to change the area covered by the corresponding pixel cluster until the Gaussian mixture component converges. At this point, the area covered by the adjusted pixel cluster can be used as the target focus area for focusing. It should be understood that the pixels within the clustered pixel points may be discrete, resulting in edges that are not sharp and smooth. Therefore, the Gaussian mixture model can be used to adjust the corresponding area to make its edges sharp and smooth, improving the accuracy of the determined target focus area.

[0090] The following section will use focusing on the subject in the preview interface of a smartphone as an example to introduce the technical solution disclosed herein.

[0091] Figure 2 This is a flowchart illustrating another method for determining the focus area, as shown in an exemplary embodiment of this disclosure.

[0092] like Figure 2 As shown, the method may include the following steps:

[0093] Step 201: The user's triggered action in the preview screen is detected.

[0094] In this embodiment, the smartphone may have a camera application pre-installed. When the user needs to take a picture, they can launch the camera application and enter the image preview interface. Once in the preview interface, the smartphone can focus on the subject in the image. In this embodiment, the user can manually select the subject to be photographed from the preview screen.

[0095] For example, suppose a user needs to take a full-body photo of another user using their smartphone. The user can tap on the other user's face in the preview screen. Upon detecting this tap, the smartphone can determine the area where the tap is located as the initial focus area. Of course, to more accurately determine the initial focus area, simple facial recognition can be performed in the vicinity of the tap, roughly determining the coverage area of ​​the face.

[0096] Step 202: Determine the area where the trigger operation is located as the initial focus area.

[0097] Step 203: Select a preset number of target pixels both inside and outside the initial focus area.

[0098] In this embodiment, after determining the initial focus area, a preset number of target pixels can be selected from both inside and outside the initial focus area to serve as the initial cluster centers used in the clustering operation.

[0099] It should be understood that the initial focus area may not be accurately selected, and part of the subject may be located outside the initial focus area. In this case, if target pixels are selected only from the initial focus area, the portion of the subject outside the initial focus area may not be identified as the target focus area. Therefore, this embodiment uses the method of taking the same number of target pixels both inside and outside the initial focus area to cluster the pixels in the preview image. This method can, on the one hand, identify the portion of the subject outside the initial focus area as the target focus area, and on the other hand, it can form cross-validation between the pixel clusters obtained by clustering target pixels within the initial focus area and the pixel clusters obtained by clustering target pixels outside the initial focus area, thereby improving the accuracy of recognition. For example, when there is an overlap between the pixel clusters obtained by clustering target pixels within the initial focus area and the pixel clusters obtained by clustering target pixels outside the initial focus area, the overlapping area can be determined as the edge of the subject.

[0100] Following the above example, this embodiment can obtain the RGB values ​​of all pixels within the initial focus area and analyze these values ​​to select several pixels as target pixels from those with relatively stable RGB values. For example, in the above example, the hair and face are areas with relatively stable colors, i.e., areas with relatively stable RGB values. In this case, if this embodiment selects 5 pixels from both inside and outside the initial focus area as target pixels, then, based on the above analysis method, there may be 2 target pixels located within the hair area and 3 located within the face area. Outside the initial focus area, 5 target pixels can be selected randomly. Of course, the above example is only illustrative; target pixels can also be obtained by random selection within the initial focus area. How to determine the specific method can be determined by those skilled in the art according to actual needs, and this embodiment does not limit this.

[0101] Step 204: Perform K-mean clustering on the pixels in the preview image based on the target pixel.

[0102] In this embodiment, the K-means clustering algorithm can be used to cluster the pixels in the preview image. Of course, any other clustering algorithm can be used to cluster the preview image, and this embodiment does not limit this.

[0103] Step 205: Substitute the pixel values ​​of the pixels contained in each pixel cluster obtained by clustering into the Gaussian mixture model.

[0104] Continuing with the example above, after obtaining 10 pixel clusters through K-means clustering, the RGB values ​​of the pixels within each of these 10 clusters can be substituted into a Gaussian mixture model to obtain the Gaussian mixture components corresponding to the 10 pixel clusters. Specifically, for any pixel cluster, the mean and covariance of the Gaussian mixture model parameters can be calculated based on the RGB values ​​of the pixels within that cluster. These parameters and covariance are then substituted into the Gaussian mixture model to obtain the Gaussian mixture components for that pixel cluster. It should be understood that each Gaussian mixture component characterizes the distribution of RGB values ​​of the pixels within the corresponding pixel cluster.

[0105] Step 206: Adjust the model parameters to make the Gaussian mixture components corresponding to each cluster converge.

[0106] Continuing with the example above, after obtaining the Gaussian mixture components, the parameter values ​​of each Gaussian mixture component can be adjusted until convergence. It should be understood that each Gaussian mixture component corresponds to a cluster of Gaussian mixture components. When the parameter value of any Gaussian mixture component changes, the RGB distribution of the pixels in the corresponding pixel cluster also changes, leading to a change in the area covered by the pixel cluster. When all Gaussian mixture components converge, the area covered by each pixel cluster will no longer change.

[0107] Step 207: Obtain the area covered by the corresponding pixel cluster when each component converges.

[0108] Step 208: Identify the cluster of target pixels whose cluster centers are located within the initial focus area.

[0109] In this embodiment, the area covered by the cluster of target pixels whose cluster center is located within the initial focus area can be determined as the final target focus area.

[0110] It should be noted that although in the example above, 5 target pixels are selected from both inside and outside the initial focus area, the cluster centers may change during the clustering process. Therefore, after the optimization of K-means clustering and Gaussian mixture model, the number of cluster centers located within the initial focus area may not be 5. It may be more than 5 or less than 5.

[0111] Of course, the method of determining the target pixel cluster in this embodiment is only illustrative. The specific method of determining the target pixel cluster can be determined by those skilled in the art according to actual needs, and this disclosure does not limit it.

[0112] Step 209: Focus on the area covered by the target pixel cluster as the target focus area.

[0113] Following the example above, after obtaining the target pixel cluster, the area covered by the target pixel cluster can be determined as the target focus area for focusing the preview image.

[0114] As can be seen from the above technical solution, in this embodiment, during the image preview process, the user can determine the initial focus area where the subject is located by clicking, and select a preset number of target pixels both inside and outside this initial focus area to cluster the pixels in the preview image. After clustering is completed, the cluster of pixels whose cluster center is located within the initial focus area can be determined as the target pixel cluster, and the area covered by the target pixel cluster can be determined as the target focus area for focusing on the subject.

[0115] After clustering the pixels into clusters, the RGB values ​​of the pixels within each cluster can be substituted into the Gaussian mixture model. Based on this, the area covered by the pixel clusters can be optimized by converging the resulting Gaussian mixture components, making the edges of each region smoother and clearer, thus improving the accuracy of the determined target focus area.

[0116] Figure 3 This is a block diagram illustrating a focus area determination device according to an exemplary embodiment of this disclosure. (Refer to...) Figure 3 The device includes a first determining unit 301, a clustering unit 302, and a second determining unit 303.

[0117] The first determining unit 301 is used to determine an initial focus area containing the subject in the image to be focused;

[0118] Clustering unit 302 is used to determine at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area, so as to cluster the pixels contained in the image to be focused according to the determined target pixel value.

[0119] The second determining unit 303 is used to determine the area covered by the cluster of pixels obtained by clustering as the target focusing area when focusing the image to be focused.

[0120] Optionally, the first determining unit 301 is further used for:

[0121] In response to a user's trigger operation on the image to be focused, the area where the trigger operation is located is determined as the initial focus image; or,

[0122] The image to be focused is processed based on a preset algorithm to identify the region of the preset object corresponding to the preset algorithm in the image to be focused, and the identified region is determined as the initial focus region.

[0123] Optionally, clustering unit 302 is further used for:

[0124] At least one target pixel is selected from the initial focus area, and the pixel value of the selected target pixel is used as the determined target pixel value.

[0125] Optionally, clustering unit 302 is further used for:

[0126] Obtain the pixel values ​​of all pixels contained in the initial focus area, and select the pixels whose pixel values ​​differ from those of their neighboring pixels within a preset range as target pixels; or...

[0127] Identify a pixel located at a specific position within the initial focus area, and designate the identified pixel as the target pixel; or,

[0128] At least one target pixel is randomly selected from the initial focus area.

[0129] Optionally, the clustering unit 302 is further configured to: select at least two reference pixels from the image to be focused, and cluster the pixels contained in the image to be focused based on the pixel values ​​of the selected reference pixels; wherein a portion of the selected reference pixels are target pixels selected from the initial focus region;

[0130] The second determining unit 303 is further used to: determine the area covered by the pixel cluster that has a preset association with the initial focusing area in the multiple pixel clusters obtained by clustering as the target focusing area when focusing the image to be focused.

[0131] The preset association includes any of the following: the cluster center is located within the initial focus area; the difference between the pixel value of the cluster center and the average pixel value of the initial focus area is within a preset range; and the area of ​​the pixel cluster located within the initial focus area is larger than the area located outside the initial focus area.

[0132] Optionally, clustering unit 302 is further used for:

[0133] Obtain the pixel values ​​of all pixels in the initial focus area, and calculate the average pixel value of all obtained pixels;

[0134] The calculated average pixel value is determined as the target pixel value.

[0135] Optionally, clustering unit 302 is further used for:

[0136] The position of each pixel in the initial focus area is determined, and the pixel value of all the obtained pixels is calculated by weighting the pixel values ​​based on the weight values ​​set in advance for each position to obtain the pixel mean.

[0137] Optionally, the clustering unit 302 is also used to: use the target focus area as the initial focus area contained in the next frame of the image to be focused, so as to determine the target pixel value used in the clustering of the next frame of the image to be focused.

[0138] The second determining unit 303 is further used for:

[0139] The region covered by the cluster of pixels obtained by clustering is determined as the focus region to be determined, and the focus image containing the focus region to be determined is input into the pre-trained focus region optimization model so as to optimize the focus region to be determined through the focus region optimization model.

[0140] The optimized focus region contained in the image output by the focus region optimization model is determined as the target focus region to be used when focusing the image to be focused.

[0141] like Figure 4 As shown, Figure 4 This is a block diagram illustrating another focus area determining device according to an exemplary embodiment of this disclosure, which is described above. Figure 3 Based on the embodiment shown, the focus area determination device further includes a substitution unit 304.

[0142] Optionally, the substitution unit 304 is used to substitute the pixel values ​​of at least one pixel cluster obtained by clustering into the Gaussian mixture model to obtain several Gaussian mixture components corresponding to each pixel cluster; and to adjust the parameter values ​​of the several Gaussian mixture components to adjust the area covered by each pixel cluster until the several Gaussian mixture components converge.

[0143] The second determining unit 303 is further configured to: determine the area covered by the adjusted pixel cluster as the target focusing area when focusing the image to be focused, provided that all of the several Gaussian mixture components have converged.

[0144] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0145] Accordingly, this disclosure also provides a focus area determination device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the focus area determination method as described in any of the above embodiments, for example, the method may include: determining an initial focus area containing a subject in an image to be focused; determining at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area, so as to cluster the pixels contained in the image to be focused according to the determined target pixel value; and determining the area covered by the clustered pixel clusters as the target focus area used when focusing the image to be focused.

[0146] Accordingly, this disclosure also provides an electronic device, which includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for implementing the focus area determination method as described in any of the above embodiments. For example, the method may include: determining an initial focus area containing a subject in an image to be focused; determining at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area, so as to cluster the pixels contained in the image to be focused according to the determined target pixel value; and determining the area covered by the clustered pixel clusters as the target focus area used when focusing the image to be focused.

[0147] Figure 5 This is a block diagram illustrating an apparatus 500 for a focus area determination method according to an exemplary embodiment. For example, apparatus 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0148] Reference Figure 5The device 500 may include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, a multimedia component 508, an audio component 510, an input / output (I / O) interface 512, a sensor component 514, and a communication component 516.

[0149] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0150] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of such data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0151] Power supply component 506 provides power to various components of device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 500.

[0152] Multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0153] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0154] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0155] Sensor assembly 514 includes one or more sensors for providing status assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0156] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0157] In an exemplary embodiment, the apparatus 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0158] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of the device 500 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0159] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0160] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0161] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining a focus area, characterized in that, include: Identify the initial focus area containing the subject in the image to be focused; The area covered by the subject in the image to be focused consists of continuous pixels with similar pixel values; Determining at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus area includes: selecting at least one target pixel from the initial focus area, and using the pixel value of the selected target pixel as the determined target pixel value; or, obtaining the pixel values ​​of all pixels in the initial focus area, and calculating the average pixel value of all obtained pixels, and using the calculated average pixel value as the target pixel value. Cluster the pixels contained in the image to be focused based on the determined target pixel values; The area covered by the cluster of pixels obtained by clustering is determined as the target focus area when focusing the image to be focused.

2. The method according to claim 1, characterized in that, The step of determining the initial focus area containing the subject in the image to be focused includes: In response to a user's trigger operation on the image to be focused, the area where the trigger operation is located is determined as the initial focus image; or, The image to be focused is processed based on a preset algorithm to identify the region of the preset object corresponding to the preset algorithm in the image to be focused, and the identified region is determined as the initial focus region.

3. The method according to claim 1, characterized in that, Selecting at least one target pixel from the initial focus area includes: Obtain the pixel values ​​of all pixels contained in the initial focus area, and select the pixels whose pixel values ​​differ from those of their neighboring pixels within a preset range as target pixels; or... Identify a pixel located at a specific position within the initial focus area, and designate the identified pixel as the target pixel; or, At least one target pixel is randomly selected from the initial focus area.

4. The method according to claim 1, characterized in that, The step of selecting at least one target pixel from the initial focus area and clustering the pixels contained in the image to be focused based on the pixel value of the selected target pixel includes: selecting at least two reference pixels from the image to be focused and clustering the pixels contained in the image to be focused based on the pixel value of the selected reference pixels; wherein a portion of the selected reference pixels are target pixels selected from the initial focus area. The step of determining the area covered by the clustered pixel clusters as the target focus area when focusing the image to be focused includes: determining the area covered by the pixel clusters that have a preset association with the initial focus area among the multiple clustered pixel clusters as the target focus area when focusing the image to be focused. The preset association includes any of the following: the cluster center is located within the initial focus area; the difference between the pixel value of the cluster center and the average pixel value of the initial focus area is within a preset range; and the area of ​​the pixel cluster located within the initial focus area is larger than the area located outside the initial focus area.

5. The method according to claim 1, characterized in that, The calculated average pixel value of all pixels includes: The position of each pixel in the initial focus area is determined, and the pixel value of all the obtained pixels is calculated by weighting the pixel values ​​based on the weight values ​​set in advance for each position to obtain the pixel mean.

6. The method according to claim 1, characterized in that, Also includes: The target focus area is used as the initial focus area in the next frame of the image to be focused, so as to determine the target pixel value used in the clustering of the next frame of the image to be focused.

7. The method according to claim 1, characterized in that, It also includes: substituting the pixel values ​​of at least one cluster of pixels obtained by clustering into the Gaussian mixture model to obtain several Gaussian mixture components corresponding to each cluster of pixels; and adjusting the parameter values ​​of the several Gaussian mixture components to adjust the area covered by each cluster of pixels until all the several Gaussian mixture components converge. The step of determining the region covered by the clustered pixel points as the target focus region when focusing the image to be focused includes: when all the Gaussian mixture components converge, determining the region covered by the adjusted pixel point cluster as the target focus region when focusing the image to be focused.

8. The method according to claim 1, characterized in that, The step of determining the region covered by the clustered pixel clusters as the target focus region for focusing the image to be focused includes: The region covered by the cluster of pixels obtained by clustering is determined as the focus region to be determined, and the focus image containing the focus region to be determined is input into the pre-trained focus region optimization model so as to optimize the focus region to be determined through the focus region optimization model. The optimized focus region contained in the image output by the focus region optimization model is determined as the target focus region to be used when focusing the image to be focused.

9. A focusing area determining device, characterized in that, include: The first determining unit is used to determine the initial focus area containing the subject in the image to be focused; The area covered by the subject in the image to be focused consists of continuous pixels with similar pixel values; A clustering unit is configured to determine at least one target pixel value based on the pixel values ​​of the pixels contained in the initial focus region, so as to cluster the pixels contained in the image to be focused according to the determined target pixel value; wherein, the clustering unit is further configured to: select at least one target pixel from the initial focus region, so as to use the pixel value of the selected target pixel as the determined target pixel value; or, obtain the pixel values ​​of all pixels in the initial focus region, and calculate the pixel mean of all obtained pixels, so as to determine the calculated pixel mean as the target pixel value; The second determining unit is used to determine the area covered by the cluster of pixels obtained by clustering as the target focusing area when focusing the image to be focused.

10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-8 by executing the executable instructions.

11. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-8.

Images