Image processing method and related device

Abstract

The application discloses an image processing method and related equipment, the method comprises the following steps: acquiring a shooting image of a target object, and acquiring an initial blur image corresponding to the shooting image according to a blur algorithm; determining a blur parameter of a sampling feature point of the target object according to the initial blur image; determining an expected blur parameter of a blur feature point on the initial blur image according to the blur parameter of the sampling feature point; obtaining a fusion pixel weight value according to the expected blur parameter, a blur kernel of the sampling feature point indicated by the blur algorithm, and a blur kernel of the blur feature point indicated by the blur algorithm; and performing fusion processing on the shooting image and the initial blur image based on the fusion pixel weight value to obtain a target image. In this way, the target image with a better hierarchical blur effect can be obtained, and the accuracy and effectiveness of image blur processing are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to the field of computer technology, and in particular to an image processing method and related equipment. Background Technology

[0002] Image blurring is an image processing method used in the field of image processing to blur images. A common application scenario is for blurry shooting scenarios using mobile phones, tablets, cameras, and other terminal devices. In this scenario, the terminal device uses blur algorithms to blur the captured image, resulting in a layered blurring effect where the near end is sharp and the far end is blurred. However, this type of blurring often results in over-blurring of certain areas of the image, thus affecting the layered blurring effect. Summary of the Invention

[0003] This application provides an image processing method and related equipment that can optimize the hierarchical blurring effect of images and improve the accuracy and effectiveness of image blurring processing.

[0004] In a first aspect, embodiments of this application provide an image processing method, including:

[0005] Acquire a captured image of the target object, and obtain an initial blurred image corresponding to the captured image based on a blurring algorithm;

[0006] The blurring parameters of the sampled feature points of the target object are determined based on the initial blurred image;

[0007] Based on the blurring parameters of the sampled feature points, determine the expected blurring parameters of the blurring feature points on the initial blurring map;

[0008] Based on the desired blurring parameter, the blur kernel of the sampled feature points indicated by the blurring algorithm, and the blur kernel of the blurred feature points indicated by the blurring algorithm, the fused pixel weight value is obtained;

[0009] The captured image and the initial blurred image are fused based on the fused pixel weight values ​​to obtain the target image.

[0010] Secondly, embodiments of this application provide an image processing apparatus, including:

[0011] The acquisition unit is used to acquire a captured image of the target object and to acquire an initial blurred image corresponding to the captured image according to a blurring algorithm;

[0012] The first determining unit is configured to determine the blurring parameters of the sampled feature points of the target object based on the initial blurred image;

[0013] The second determining unit is used to determine the expected blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points.

[0014] The processing unit is configured to obtain a fused pixel weight value based on the desired blurring parameter, the blur kernel of the sampled feature point indicated by the blurring algorithm, and the blur kernel of the blurred feature point indicated by the blurring algorithm.

[0015] The fusion unit is used to perform fusion processing on the captured image and the initial blurred image based on the fusion pixel weight value to obtain the target image.

[0016] Thirdly, embodiments of this application provide a computing device including a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, and the processor is configured to invoke the computer program to execute the method described in the first aspect above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing program instructions that, when executed, implement the method described in the first aspect above.

[0018] Fifthly, embodiments of this application provide a computer program product, the computer program product including program instructions, which, when executed by a processor, implement the method described in the first aspect above.

[0019] This application embodiment obtains an initial blurred image by blurring the captured image of the target object according to a blur algorithm, and determines the blur parameters of the sampled feature points of the target object based on the initial blurred image. Based on the blur parameters of the sampled feature points, the expected blur parameters of the blurred feature points on the initial blurred image are more accurately determined. In order to obtain more accurate fusion pixel weight values ​​based on the expected blur parameters, the blur kernel of the sampled feature points indicated by the blur algorithm, and the blur kernel of the blurred feature points indicated by the blur algorithm, the captured image and the initial blurred image are fused based on the fusion pixel weight values, thereby optimizing and obtaining a target image with better hierarchical blurring effect, and improving the accuracy and effectiveness of image blurring processing. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1This is a schematic diagram of an interface for obtaining a portrait in a blurred shooting scene;

[0022] Figure 2 This is a schematic diagram of the structure of an image processing method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of an image processing system provided in an embodiment of this application;

[0024] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0025] Figure 5 This is a flowchart illustrating another image processing method provided in an embodiment of this application;

[0026] Figure 6 This is a planar schematic diagram of an image provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of an interface for a target image provided in an embodiment of this application;

[0028] Figure 8 This is a flowchart illustrating an interactive image processing method provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0032] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0033] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0034] It should be noted that the concepts of "first," "second," "third," and "fourth" mentioned in this application specification are only used to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0035] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0036] In specific implementations, the computing devices described in the embodiments of this application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with touch surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch surface (e.g., touchscreen displays and / or touchpads).

[0037] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0038] In a bokeh shooting scenario, a computing device can acquire a captured image of the target object. By blurring this image, an initial bokeh image with a layered blurring effect, where the near end is sharp and the far end is blurred, is obtained. The depth of field increases from shallow to deep (i.e., depth values ​​increase from small to large), corresponding to the distance from the near end to the far end. However, within the image region where the target object is located, there may be image regions with depth values ​​close to the background depth value. Therefore, when blurring the captured image to obtain the initial bokeh image, this image region may be treated as the background and blurred excessively, resulting in poor layered blurring of the initial bokeh image. In some embodiments, the depth value of an image region refers to the depth value corresponding to each pixel within that image region. In some embodiments, the bokeh involved in this application is equivalent to the blurring process described in this application, and the depth values ​​involved in this application are equivalent to the depth information described in this application.

[0039] by Figure 1 The target object shown is a person, and the image obtained by the photographer is a portrait, which will be used as an example for illustration. Figure 1 This is an interface diagram illustrating a portrait captured in a blurred shooting scene, such as... Figure 1 As shown, the target object is a person, and the entire portrait needs to exhibit a layered blurring effect, with the near end sharp and the far end blurred. Among these, Figure 1 The image includes a face region 11 and a hair region at the edge of the image 12. The face region 11 is the image region with the smallest depth value (i.e., the foremost depth of field). Therefore, the face region 11 can be regarded as a clear image region without blurring. Since the hair region at the edge of the image 12 is thin and strip-shaped, its depth value is generally calculated as the depth value of the background. This causes the hair region at the edge of the image 12 to be treated as a background region and blurred. As a result, the hair region at the edge of the image 12 is blurred excessively or mistakenly blurred, making it unclear. This results in a poor layered blurring effect for the entire image.

[0040] This application proposes an image processing scheme to address the aforementioned problems. This scheme acquires a captured image of the target object, obtains an initial blurred image corresponding to the captured image using a blurring algorithm, determines fusion pixel weight values ​​based on the captured image and the initial blurred image, and uses the fusion pixel weight values ​​to perform fusion processing on the captured image and the initial blurred image to adjust the pixel values ​​of the initial blurred image. This can optimize and obtain a target image with better hierarchical blurring effect, thereby improving the accuracy and effectiveness of image hierarchical blurring processing.

[0041] In practical applications, this image processing scheme can be applied to any scenario related to blurring, such as image bokeh shooting or image blurring processing. In the image bokeh shooting scenario, the image processing scheme can be as follows: obtain an initial bokeh image corresponding to the image captured by shooting the target object according to the blurring algorithm; determine the blurring parameters of the sampled feature points of the target object based on the initial bokeh image; determine the expected blurring parameters of the bokeh feature points on the initial bokeh image based on the sampled feature points' blurring parameters; obtain the fusion pixel weight value based on the expected blurring parameters, the blur kernel of the sampled feature points indicated by the blurring algorithm, and the blur kernel of the bokeh feature points indicated by the blurring algorithm; and perform fusion processing on the captured image and the initial bokeh image based on the fusion pixel weight value to obtain the target image.

[0042] In a practical implementation, the image processing scheme mentioned above can be executed by a computing device, which can be a terminal device or a server. Alternatively, the image processing scheme mentioned above can also be executed jointly by a terminal device and a server. For example, taking the example of this application being executed jointly by a terminal device and a server, please refer to [link to relevant documentation]. Figure 2 As shown, Figure 2 This is a schematic diagram of an image processing system provided in an embodiment of this application. The system allows a terminal device to acquire a captured image of a target object and send it to a server. The server acquires the captured image of the target object sent by the terminal device and obtains an initial blurred image corresponding to the captured image based on a fuzzing algorithm. Based on the initial blurred image, the server determines the blurring parameters of the sampled feature points of the target object. Based on the blurring parameters of the sampled feature points, the server determines the desired blurring parameters of the blurred feature points on the initial blurred image. Based on the desired blurring parameters, the blur kernel of the sampled feature points indicated by the fuzzing algorithm, and the blur kernel of the blurred feature points indicated by the fuzzing algorithm, the server obtains a fusion pixel weight value. Based on the fusion pixel weight value, the captured image and the initial blurred image are fused to obtain the target image. It is understood that this is merely an exemplary representation of two implementation logics of the above-described image processing scheme jointly executed by the terminal device and the server, and is not an exhaustive list.

[0043] The aforementioned terminal devices can be smartphones, computers (such as tablets, laptops, desktop computers, etc.), smart wearable devices (such as smartwatches, smart glasses), smart voice interaction devices, smart home appliances (such as smart TVs), vehicle terminals, or aircraft, etc.; servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc. Furthermore, terminal devices and servers can be located within or outside the blockchain network, without limitation; even further, terminal devices and servers can upload any data stored internally to the blockchain network for storage to prevent internal data from being tampered with and improve data security.

[0044] Based on the above description, the image processing method proposed in this application can be executed by a computing device (i.e., the terminal device or server mentioned above), or it can be executed jointly by the terminal device and the server. For ease of explanation, this application embodiment uses the execution of the image processing method by a computing device as an example. Please refer to... Figure 3 As shown, Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application. The image processing method may include the following steps S301-S305:

[0045] S301: Acquire the captured image of the target object, and obtain the initial blurred image corresponding to the captured image according to the blur algorithm.

[0046] The target object can be any one or more of the following, including but not limited to people, objects, plants, and animals. The fuzzy algorithm can be, but is not limited to, Gaussian fuzzy algorithm, mean fuzzy algorithm, etc.

[0047] When acquiring a captured image of a target object, the computing device may acquire a captured image of the target object obtained by the imaging device on the computing device, or it may acquire a captured image of the target object sent by other terminal devices or servers. This application embodiment does not specifically limit the method of acquiring the captured image of the target object. In some embodiments, the target object may be any one or more of a person, animal, plant, or object.

[0048] In the process of obtaining the initial blurred image corresponding to the captured image using a fuzzing algorithm, the captured image can be converted into a corresponding depth image using a preset depth calculation algorithm, and then the depth image can be blurred using the fuzzing algorithm to convert the depth image into the initial blurred image. In a specific embodiment, the computing device can first determine the depth information of each pixel in the captured image using a preset depth calculation algorithm, and then convert the captured image into a corresponding depth image based on the determined depth information of each pixel. Finally, the depth image can be blurred using a fuzzing algorithm to convert the depth image into the corresponding initial blurred image. In some embodiments, the preset depth calculation algorithm can include, but is not limited to, any one of the following: a depth calculation network, a stereo vision calculation method, a structured light method, a time-of-flight method, or a monocular depth estimation method. The depth calculation network can include, but is not limited to, a deep learning neural network.

[0049] Specifically, it can be... Figure 4 Let's take an example to illustrate. Figure 4 This is a schematic diagram of the structure of an image processing method provided in an embodiment of this application, as shown below. Figure 4 As shown, firstly, a captured image 41 of the target object can be obtained. According to a preset depth calculation algorithm, the captured image 41 is converted into a corresponding depth image 411. Then, the depth image 411 is blurred according to a blur algorithm, so that the depth image 411 is converted into an initial blurred image 42. Based on the captured image 41 and the initial blurred image 42, a fusion pixel weight value 43 is determined. The fusion pixel weight value 43 is used to fuse the captured image 41 and the initial blurred image 42 to adjust the pixel values ​​of the initial blurred image 42, thereby obtaining a target image 44 with better hierarchical blurring effect.

[0050] In one embodiment, when the computing device converts the captured image into a corresponding depth image based on the determined depth information of each pixel in the captured image, it can convert the captured image into an initial depth image based on the determined depth information of each pixel in the captured image, and detect whether the initial depth image meets the correction conditions. If the detection result is that the initial depth image meets the correction conditions, the initial depth image can be adjusted according to the depth information of the focus of the captured image to obtain the corresponding depth image.

[0051] In this process, when the computing device detects whether the initial depth image meets the correction conditions, it can obtain the pixel value of the focal point in the initial depth image and check whether the pixel value of the focal point of the initial depth image meets a pixel threshold. If the detection result is negative, it can be determined that the initial depth image meets the correction conditions, and the initial depth image is adjusted according to the depth information of the focal point of the captured image to obtain the corresponding depth image. If the pixel value of the focal point of the initial depth image does not meet the pixel threshold, it can be determined that the focal point of the initial depth image is blurred. When adjusting the initial depth image according to the depth information of the focal point of the captured image to obtain the corresponding depth image, the depth information of the focal point can be obtained, and the difference between the depth information of the initial depth image (i.e., the depth information of each pixel in the depth image) and the depth information of the focal point can be determined as the depth information of the depth image. Based on the determined depth information of the depth image, the initial depth image is adjusted to obtain the corresponding depth image.

[0052] By adjusting the acquired initial depth image to obtain the corresponding depth image, image distortion caused by shooting equipment or environmental factors can be eliminated, provided that the acquired initial depth image meets the correction conditions (i.e., correction is required).

[0053] In one embodiment, when a computing device converts a depth image into a corresponding initial blurred image according to a blurring algorithm, it can determine the blur degree value for blurring the depth image based on the blurring algorithm, and perform blurring processing on the depth image based on the blur degree value to obtain the corresponding initial blurred image. The blur degree value is data determined by the blurring algorithm to indicate the blurring effect of the initial blurred image. This data includes, but is not limited to, numerical values ​​(such as the aforementioned sigma value), linear matrices (such as the aforementioned convolution kernel), etc. For example, assuming the blurring algorithm is a Gaussian blurring algorithm, the blur degree value can be the standard deviation sigma value in the Gaussian blurring algorithm. The larger the sigma value, the more obvious the blurring effect; conversely, the smaller the sigma value, the less obvious the blurring effect. This application embodiment does not specifically limit the blur degree value.

[0054] In one embodiment, when the computing device determines the blur degree value for blurring the depth image according to the blur algorithm, it may determine the blur degree value indicated in the preset blur algorithm as the blur degree value for blurring the depth image, or it may determine the blur degree value obtained by the user for the blur algorithm as the blur degree value for blurring the depth image. This application does not specifically limit the method of obtaining the blur degree value when obtaining the initial blurred image.

[0055] By converting the depth image into a corresponding initial blurred image using a blur algorithm, preliminary blurring of the captured image can be achieved, thus realizing a gradual blurring effect in image capture scenarios with blurred backgrounds.

[0056] S302: Determine the blurring parameters of the sampled feature points of the target object based on the initial blurred image.

[0057] When determining the blurring parameters of the sampled feature points of the target object based on the initial blurred image, the computing device can sample the pixels of the target object in the initial blurred image to obtain multiple sampled feature points. The position information, depth information, and blur degree value of each sampled feature point in the initial blurred image are determined as the blurring parameters of each sampled feature point. In some embodiments, the position information and depth information of each sampled point can be coordinates in a three-dimensional coordinate system established with the focal point of the initial blurred image as the origin, the position information as two-dimensional coordinates (x-axis and y-axis), and the depth information as the z-axis.

[0058] In one implementation, when a computing device samples the pixels of a target object in an initial blurred image to obtain multiple sampled feature points, it can sample the pixels of the target object in the initial blurred image according to a preset sampling method to obtain multiple sampled feature points. The preset sampling method can include, but is not limited to, random sampling, equally spaced sampling, or any other sampling method. In one example, assuming the preset sampling method is equally spaced sampling with an interval of 1 pixel between every 3 pixels, the computing device can sample the pixels of the target object in the initial blurred image with an interval of 1 pixel between every 3 pixels to obtain multiple sampled feature points. Assuming the target object is a human image, ... Figure 1 Taking this as an example, the computing device can sample each pixel within the face region 11 in the initial blurred image by a interval of 1 pixel every 3 pixels, thus obtaining multiple sampled feature points.

[0059] This application embodiment obtains multiple sampling feature points by sampling the initial blurred image, which helps to reduce the amount of computation in the image processing process.

[0060] S303: Determine the desired blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points.

[0061] When determining the desired blurring parameters of the blurred feature points on the initial blurred image based on the blurring parameters of the sampled feature points, the computing device can use extrapolation to determine the desired blurring parameters of the blurred feature points on the initial blurred image based on the blurring parameters of each sampled feature point. Here, the blurred feature points are the pixels in the portion of the image region to be optimized outside the image region where the target object is located. Figure 1 For example, the blurred feature points are the pixels where the hair strands are located in the hair strand region 12 of the human portrait edge.

[0062] When determining the desired blurring parameters of the blurred feature points on the initial blurred image using extrapolation based on the blurring parameters of each sampled feature point, the computing device can fit each sampled feature point to obtain a fitting function. Based on this fitting function and the position information of each blurred feature point in the initial blurred image, the desired blurring parameters of each blurred feature point on the initial blurred image are determined. In some embodiments, the computing device can use any method such as a linear function, a quadratic function, or an exponential function to fit each sampled point.

[0063] For example, assuming that the position information of the sampling feature point is represented by x, the depth information by m, and the blurring degree value by n, the blurring parameters of each sampling feature point can be denoted as (x0, m0, no), (x1, m1, n1), (x2, m2, n2), ..., (xk, mk, nk), etc. The computing device can use any method such as linear function, quadratic function, exponential function, etc., to fit each sampling point based on the blurring parameters (x0, m0, no), (x1, m1, n1), (x2, m2, n2), ..., (xk, mk, nk), etc., to obtain the fitting function f(x). Substituting the position information of each blurring feature point into the fitting function f(x), the depth information and blurring degree value of each blurring feature point are obtained. The obtained depth information, blurring degree value, and position information of each blurring degree are determined as the expected blurring parameters of each blurring feature point.

[0064] Since the sampled feature points are obtained from the image region where the target object is located, the image region where the target object is located is considered a clear image region (i.e., an area that has not been blurred or a blurred region with a small degree of blur). Therefore, the blurring parameters such as the depth value of the image region where the target object is located are considered accurate. In this embodiment, the desired blurring parameters of the blurred feature points to be optimized are determined by referring to the blurring parameters of the sampled feature points of the target object. This can more accurately determine the desired degree of blurring of the blurred feature points, which helps to optimize the degree of blurring of the blurred feature points more accurately and effectively, resulting in a better hierarchical blurring effect of the optimized blurred feature points.

[0065] S304: Based on the desired blurring parameters, the blur kernel of the sampled feature point indicated by the blurring algorithm, and the blur kernel of the blurred feature point indicated by the blurring algorithm, the fused pixel weight value is obtained.

[0066] In this context, the fuzzy kernel indicated by the fuzzy algorithm for the sampled feature point is the convolution kernel for that feature point determined by the fuzzy algorithm, and the fuzzy kernel indicated by the fuzzy algorithm for the blurred feature point is the convolution kernel for that blurred feature point determined by the fuzzy algorithm. In some embodiments, each convolution kernel is a linear matrix.

[0067] When the computing device obtains the fused pixel weight value based on the desired blurring parameters, the blur kernel of the sampled feature point indicated by the blurring algorithm, and the blur kernel of the blurred feature point indicated by the blurring algorithm, it can obtain the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image; determine each sample point on the intersection line from the initial blurred image; and obtain the fused pixel weight value based on the desired blurring parameters of each blurred feature point, the blur kernel of each sample point indicated by the blurring algorithm, and the blur kernel of the blurred feature point indicated by the blurring algorithm.

[0068] In one implementation, when the target object is a person, the computing device can first detect whether the face deflection angle is less than 90 degrees. If the detection result indicates that the face deflection angle is less than 90 degrees, then the step of obtaining the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image is performed. This method can reduce the ineffectiveness caused by processing portraits with face deflections greater than or equal to 90 degrees, and improve the effectiveness of portrait processing.

[0069] This application embodiment determines the fused pixel weight value by combining the expected blurring parameters of the blurred feature points, the blur kernel of the sampled feature points of the clear image region where the target object is located as indicated by the blur algorithm, and the blur kernel of the blurred feature points as indicated by the blur algorithm, thereby improving the accuracy of the fused pixel weight value.

[0070] S305: Based on the fusion pixel weight value, the captured image and the initial blurred image are fused to obtain the target image.

[0071] When the computing device performs fusion processing on the captured image and the initial blurred image based on the fusion pixel weight value to obtain the target image, it can perform fusion processing on the captured image and the initial blurred image through the fusion pixel weight value to determine the target pixel value. By adjusting the pixel value of each target blurred feature point in the initial blurred image using the target pixel value, the blurring effect of the target blurred feature point can be optimized, resulting in a better hierarchical blurring effect in the optimized target image.

[0072] This application embodiment obtains an initial blurred image of the target object using a fuzzy algorithm, determines the blurring parameters of the sampled feature points of the target object based on the initial blurred image, and more accurately determines the expected blurring parameters of the blurred feature points on the initial blurred image based on the blurring parameters of the sampled feature points. Based on the expected blurring parameters, the blur kernels of the sampled feature points indicated by the fuzzy algorithm, and the blur kernels of the blurred feature points indicated by the fuzzy algorithm, a fusion pixel weight value is obtained. The captured image and the initial blurred image are fused based on the fusion pixel weight value to optimize and obtain a target image with better hierarchical blurring effect, thereby improving the accuracy and effectiveness of image blurring processing.

[0073] This application also proposes another image processing method. In this application embodiment, the image processing method is still described using a computing device as an example. This application embodiment and... Figure 3 The difference in the illustrated embodiments is that this application's embodiments explain how to obtain the fused pixel weight value based on the desired blurring parameters, the blur kernel of the sampled feature point indicated by the blur algorithm, and the blur kernel of the blurred feature point indicated by the blur algorithm. Please refer to... Figure 5 As shown, Figure 5 This is a schematic flowchart of another image processing method provided in this application embodiment. This application embodiment describes an image processing method that may include the following steps S501-S507:

[0074] S501: Acquire the captured image of the target object, and obtain the initial blurred image corresponding to the captured image according to the blur algorithm.

[0075] S502: Determine the blurring parameters of the sampled feature points of the target object based on the initial blurred image.

[0076] S503: Determine the desired blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points.

[0077] S504: Obtain the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image.

[0078] In a specific implementation, when the computing device obtains the intersection line between the foreground plane of the target object in the initial blurred image and the focal plane of the initial blurred image, it can acquire feature information of key feature points of the target object in the captured image through a key point detection algorithm. This feature information includes the position and depth information of the key feature points. Based on the feature information of the key feature points, the foreground plane of the target object is fitted. The position and depth information of the focal point of the initial blurred image are obtained, and the focal plane of the focal point is determined based on this information. Finally, the intersection line between the foreground plane and the focal plane is determined. In some embodiments, the key point detection algorithm may include, but is not limited to, face key point detection algorithms and object key point detection algorithms.

[0079] In one implementation, when the computing device fits the front plane of the target object based on the feature information of key feature points, it can obtain the position and depth information of each key feature point. Assuming that the position information of each key feature point is represented by x and y, and the depth information is represented by z, the position and depth information of each key feature point can be recorded as (x0, y0, z0), (x1, y1, z1), (x2, y2, z2), ..., (xk, yk, zk), etc. The computing device can fit each key feature point using any method such as linear function, quadratic function, exponential function, etc., based on the position and depth information of each key feature point (x0, y0, z0), (x1, y1, z1), (x2, y2, z2), ..., (xk, yk, zk), to obtain the fitting function f1(x), and determine the front plane of the target object based on the fitting function f1(x).

[0080] In one example, taking a human as the target object, the computing device can obtain the position and depth information of key facial feature points such as the corners of the eyebrows, eyes, nose, and mouth in the captured image through a facial landmark detection algorithm, and fit the front plane of the human face based on the position and depth information of the key facial feature points.

[0081] When determining the focal plane of the focal point based on the position and depth information, the computing device can obtain the three-dimensional coordinates corresponding to the position and depth information of the focal point using a three-dimensional coordinate system established with the focal point of the initial blurred image as the origin, the position information as two-dimensional coordinates (x-axis and y-axis), and the depth information as the z-axis. Based on the three-dimensional coordinates corresponding to the position and depth information of the focal point, the computing device can fit the focal plane of the focal point.

[0082] In one implementation, when the computing device fits the focal plane where the focal point is located based on the three-dimensional coordinates corresponding to the position and depth information of the focal point, it can obtain the position and depth information of the focal point. Assuming that the position information of the focal point is represented by x and y, and the depth information is represented by z, the position and depth information of the focal point can be recorded as (x0, y0, z0), (x1, y1, z1), (x2, y2, z2), ..., (xg, yg, zg), etc. The computing device can fit each key feature point using any method such as linear function, quadratic function, exponential function, etc., based on the position and depth information of the focal point (x0, y0, z0), (x1, y1, z1), (x2, y2, z2), ..., (xg, yg, zg), to obtain the fitting function f2(x), and determine the focal plane where the focal point is located based on the fitting function f2(x).

[0083] In one example, specifically... Figure 6 For example, the front plane where the target object is located, the focal plane where the focus is located, and the intersection of the front plane and the focal plane are illustrated. Figure 6 This is a planar schematic diagram of an image provided in an embodiment of this application. For example... Figure 6 The image shown is a front view and a top view of the initial blurred image obtained by photographing the target object (i.e., a person). The front view includes the focal point 611, the intersection line 62 of the focal plane and the front plane, the target object frame 63, and the blurred feature point 64. The top view includes the focal point 612, the front plane 65, and the focal plane 66. The sampled feature point can be obtained from within the target object frame 63. The blurred feature point can be a pixel in the image area outside the target object frame 63, such as the blurred feature point 64.

[0084] This application embodiment determines the intersection line of the front plane and the focal plane by identifying the front plane where the target object is located and the focal plane where the focus is located. Since the intersection line is on the focal plane, the pixels on the intersection line are the clearest and can be regarded as unblurred pixels, which helps to subsequently determine the sampling points on the intersection line. The sampling points on the intersection line are then used to calculate a more accurate fused pixel weight value.

[0085] S505: Determine the sampling points on the intersection line from the initial blurred image.

[0086] In one implementation, the computing device can select sampling points on the intersection line from the sampling feature points determined in the initial blurred image. Specifically, the computing device can obtain the position information of each sampling feature point in the initial blurred image and the position information of each pixel on the intersection line. The position information of each sampling feature point is compared with the position information of each pixel on the intersection line. The pixel points on the intersection line with the same comparison result are obtained, and the pixel points on the intersection line with the same comparison result are determined as the sampling points on the intersection line among the sampling feature points.

[0087] The embodiments of this application select each sampling point on the intersection line from each sampling feature point, which helps to obtain a more accurate fused pixel weight value by subsequently introducing the fuzzy kernel calculation of the sampling points on the intersection line.

[0088] S506: Based on the expected blurring parameters of each blurring feature point, the blur kernel of each sampling point indicated by the blurring algorithm, and the blur kernel of each blurring feature point indicated by the blurring algorithm, the fused pixel weight value is obtained.

[0089] In the specific implementation, when the computing device obtains the fused pixel weight value based on the expected blurring parameters of each blurred feature point, the blur kernel of each sampling point indicated by the fuzzing algorithm, and the blur kernel of each blurred feature point indicated by the fuzzing algorithm, it can determine the target sampling point corresponding to the target blurred feature point on the intersection line from each sampling point based on the position information of the target blurred feature point in each blurred feature point; and obtain the fused pixel weight value of the target blurred feature point based on the expected blurring parameters of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the fuzzing algorithm, and the blur kernel of the target sampling point indicated by the fuzzing algorithm.

[0090] In one implementation, when the computing device determines the target sampling point corresponding to the target blurred feature point on the intersection line from each sampling point based on the position information of the target blurred feature point among each blurred feature point, it can obtain the coordinate information of the target blurred feature point in the three-dimensional coordinate system among each blurred feature point, and obtain the coordinate information of each sampling point on the intersection line in the three-dimensional coordinate system. If any coordinate of any sampling point in the coordinate information of each sampling point is the same as the corresponding coordinate of the target blurred feature point, then the sampling point is determined to be the target sampling point corresponding to the target blurred feature point on the intersection line. Here, the three-dimensional coordinate system is established with the focal point of the initial blurred image as the origin, the position information as two-dimensional coordinates (i.e., the x-axis and y-axis), and the depth information as the z-axis.

[0091] In practical implementation, since the intersection line lies on the focal plane, the target sampling points on the intersection line are the clearest (i.e., there is no blurring operation). In one example, assuming the blurring algorithm is Gaussian blurring, and taking a 3x3 convolution kernel (i.e., blur kernel) as an example, the blur kernel for the target sampling points is w.src In other words, w src The kernel values ​​are shown below:

[0092]

[0093] In one example, suppose the blur kernel of the target blurred feature point is w. cur Taking a 3x3 convolution kernel (i.e., a blur kernel) as an example, the blur kernel w for the target blurred feature points cur The kernel values ​​are shown below:

[0094]

[0095] Among them, w cur The calculation is w 22 The corresponding convolution kernel for the target blurred feature points, w 11 w 12 w 13 w 21 w 23 w 31 w 32 w 33 It is w 22 Data for a 3x3 window around the perimeter.

[0096] When the computing device obtains the fused pixel weight value of the target blurred feature point based on the expected blurring parameters of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the fuzzy algorithm, and the blur kernel of the target sampling point indicated by the fuzzy algorithm, it can establish an overdetermined equation between the expected blurring degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the fuzzy algorithm, the blur kernel of the target sampling point indicated by the fuzzy algorithm, and the fused pixel weight value of the target blurred feature point; and solve the overdetermined equation by the least squares method to obtain the fused pixel weight value of the target blurred feature point.

[0097] In one embodiment, when establishing an overdetermined equation between the desired blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the fuzzing algorithm, the blur kernel of the target sample point indicated by the fuzzing algorithm, and the fused pixel weight value of the target blurred feature point, the computing device can calculate a first product of the blur kernel of the target sample point indicated by the fuzzing algorithm and the fused pixel weight value; obtain the deviation value between the fused pixel weight value and the reference pixel value, and calculate a second product of the blur kernel of the target blurred feature point indicated by the fuzzing algorithm and the deviation value; and establish an overdetermined equation by making the sum of the first product and the second product equal to the desired blur degree value. Wherein, the reference pixel value is 1, and the deviation between the fused pixel weight value and the reference pixel value is the difference between 1 and the fused pixel weight value.

[0098] Assuming the pixel weight value to be alpha is fused, and the blur kernel of the target sampling point is w.src The fuzzy kernel of the target blurred feature points is w cur The expected blurring value is w eq The overdetermined equation between the expected blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the fuzzy algorithm, the blur kernel of the target sampling point indicated by the fuzzy algorithm, and the fused pixel weight value of the target blurred feature point is as shown in the following equation (1):

[0099] alpha*w src +(1-alpha)*w cur =w eq (1)

[0100] This application embodiment uses the method of establishing overdetermined equations and solving them by least squares to remove the fusion pixel weight value, i.e., the alpha value, corresponding to each target bokeh feature point. This allows for the subsequent fusion of the pixels of the target bokeh feature points in the captured image and the initial bokeh image using the fusion pixel weight value at each target bokeh feature point.

[0101] S507: Based on the fusion pixel weight value, the captured image and the initial blurred image are fused to obtain the target image.

[0102] When the computing device performs fusion processing on the captured image and the initial blurred image based on the fusion pixel weight value to obtain the target image, it can obtain the third product of the fusion pixel weight value and the pixel value corresponding to the target blurred feature point in the captured image; calculate the fourth product of the deviation value and the pixel value corresponding to the target blurred feature point in the initial blurred image; determine the sum of the third product and the fourth product as the target pixel value; and adjust the pixel value of the target blurred feature point in the initial blurred image based on the target pixel value to obtain the target image.

[0103] Assuming the pixel value corresponding to the blurred feature point of the target in the captured image is represented by srcImg, and the pixel value corresponding to the blurred feature point of the target in the initial blurred image is represented by blurImg, then the formula for calculating the target pixel value is as follows: Equation (2):

[0104] alpha*srcImg+(1-alpha)*blurImg(2)

[0105] Combination Figure 1 and Figure 7 For example, Figure 7This is a schematic diagram of an interface for a target image provided in an embodiment of this application. Assuming that the hair region 12 at the edge of the portrait is the image region where the target blurred point is located, and the face region 11 is the image region where the sampled feature point is located, the computing device can obtain the third product of the fused pixel weight value and the pixel value corresponding to the target blurred feature point in the captured image; calculate the fourth product of the deviation value and the pixel value corresponding to the target blurred feature point in the initial blurred image; determine the sum of the third and fourth products as the target pixel value; and adjust the pixel value of the target blurred feature point in the hair region 12 at the edge of the portrait in the initial blurred image based on the target pixel value, thereby optimizing the image. Figure 7 The target image in the image has a face region 71 that remains unchanged, while the hair strands in the hair strands region 72 at the edge of the face are clearer, resulting in a better layered blurring effect in the target image.

[0106] This application embodiment obtains an initial blurred image of the target object using a fuzzing algorithm, determines the blurring parameters of the sampled feature points of the target object based on the initial blurred image, and more accurately determines the expected blurring parameters of the blurred feature points on the initial blurred image based on the blurring parameters of the sampled feature points. By obtaining the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image, each sample point located on the intersection line is determined from the initial blurred image. Based on the expected blurring parameters of each blurred feature point, the blur kernel of each sample point indicated by the fuzzing algorithm, and the blur kernel of each blurred feature point indicated by the fuzzing algorithm, a more accurate fusion pixel weight value is obtained. Based on the fusion pixel weight value, the captured image and the initial blurred image are fused to obtain a target image with better hierarchical blurring effect, thereby improving the accuracy and effectiveness of image blurring processing.

[0107] Please see Figure 8 , Figure 8 This is a flowchart illustrating an interactive image processing method provided in an embodiment of this application. The interactive image processing method can be executed jointly by a terminal device and a server, and may include the following steps:

[0108] S801: The terminal device acquires a captured image of the target object.

[0109] This can be achieved by capturing an image of the target object using the camera device of the terminal device; the user can input the captured image of the target object through the terminal device; the captured image of the target object can also be sent to this terminal device from another terminal device; or an image processing request can be sent to this terminal device from another terminal device, the image processing request carrying the captured image of the target object. It is understood that this application embodiment does not specifically limit the method by which the terminal device acquires the captured image of the target object.

[0110] S802: The terminal device sends the captured image to the server.

[0111] S803: The server obtains the initial blurred image corresponding to the captured image based on the blur algorithm.

[0112] S804: The server determines the blurring parameters of the sampled feature points of the target object based on the initial blurred image.

[0113] S805: The server determines the expected blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points.

[0114] S806: The server obtains the fused pixel weight value based on the desired blurring parameters, the blur kernel of the sampled feature point indicated by the blurring algorithm, and the blur kernel of the blurred feature point indicated by the blurring algorithm.

[0115] S807: The server performs fusion processing on the captured image and the initial blurred image based on the fusion pixel weight value to obtain the target image.

[0116] In this embodiment, the terminal device sends the captured image to the server, enabling the server to acquire the captured image and obtain an initial blurred image corresponding to the captured image based on a fuzzing algorithm. The server then determines the blurring parameters of the sampled feature points of the target object based on the initial blurred image. Based on the blurring parameters of the sampled feature points, the expected blurring parameters of the blurred feature points on the initial blurred image can be determined more accurately. Based on the expected blurring parameters, the blur kernel for the sampled feature points indicated by the fuzzing algorithm, and the blur kernel for the blurred feature points indicated by the fuzzing algorithm, a fusion pixel weight value is obtained. The captured image and the initial blurred image are then fused based on the fusion pixel weight value to optimize and obtain a target image with better blurring effect, thereby improving the accuracy and effectiveness of image hierarchical blurring processing.

[0117] Based on the above description of the image processing method embodiments, this application also discloses an image processing apparatus; the image processing apparatus may be a computer program (including one or more instructions) running on a computing device, and the image processing apparatus may execute each step in the above-described related method flow. Please refer to... Figure 9 , Figure 9 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. The image processing device can operate the following units:

[0118] The acquisition unit 901 is used to acquire a captured image of the target object and acquire an initial blurred image corresponding to the captured image according to a blurring algorithm;

[0119] The first determining unit 902 is used to determine the blurring parameters of the sampled feature points of the target object based on the initial blurred image;

[0120] The second determining unit 903 is used to determine the expected blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points.

[0121] Processing unit 904 is used to obtain fused pixel weight values ​​based on the desired blurring parameters, the blur kernel of the sampled feature points indicated by the blurring algorithm, and the blur kernel of the blurred feature points indicated by the blurring algorithm.

[0122] The fusion unit 905 is used to perform fusion processing on the captured image and the initial blurred image based on the fusion pixel weight value to obtain the target image.

[0123] Furthermore, when determining the blurring parameters of the sampled feature points of the target object based on the initial blurred image, the first determining unit 902 is specifically used for:

[0124] The pixels of the target object in the initial blurred image are sampled to obtain multiple sampled feature points;

[0125] The position information, depth information, and blur degree value of each sampled feature point on the initial blurred image are determined as the blur parameters of each sampled feature point.

[0126] Furthermore, when the second determining unit 903 determines the desired blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points, it can specifically be used for:

[0127] The sampling feature points are fitted according to the fuzzing parameters of each sampling feature point to obtain a fitting function;

[0128] Based on the fitting function and the position information of each blurred feature point in the initial blurred image, the expected blurring parameters of each blurred feature point on the initial blurred image are determined.

[0129] Furthermore, when processing unit 904 obtains the fused pixel weight value based on the desired blurring parameter, the blur kernel for the sampled feature points indicated by the blurring algorithm, and the blur kernel for the blurred feature points indicated by the blurring algorithm, it specifically performs the following:

[0130] Obtain the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image;

[0131] Determine each sampling point located on the intersection line from the initial blurred image;

[0132] The fused pixel weight value is obtained based on the expected blurring parameters of each blurring feature point, the blur kernel of each sampling point indicated by the blurring algorithm, and the blur kernel of each blurring feature point indicated by the blurring algorithm.

[0133] Furthermore, when processing unit 904 obtains the intersection line between the foreground plane of the target object in the initial blurred image and the focal plane of the initial blurred image, it specifically performs the following:

[0134] The key point detection algorithm is used to obtain the feature information of the key feature points of the target object in the captured image. The feature information includes the position information and depth information of the key feature points.

[0135] The front plane where the target object is located is fitted based on the feature information of the key feature points;

[0136] Obtain the position and depth information of the focus of the initial blurred image, and determine the focus plane where the focus is located based on the position and depth information of the focus;

[0137] The intersection line between the front plane and the focal plane is determined based on the front plane and the focal plane.

[0138] Furthermore, when processing unit 904 obtains the fused pixel weight value based on the expected blurring parameters of each blurred feature point, the blur kernel of each sampling point indicated by the blurring algorithm, and the blur kernel of each blurred feature point indicated by the blurring algorithm, it is specifically used for:

[0139] Based on the position information of the target blurred feature point among the various blurred feature points, determine the target sampling point corresponding to the target blurred feature point on the intersection line from the various sampling points;

[0140] The fused pixel weight value of the target blurred feature point is obtained based on the expected blurring parameter of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, and the blur kernel of the target sampling point indicated by the blur algorithm.

[0141] Furthermore, when processing unit 904 obtains the fused pixel weight value of the target blurred feature point based on the expected blurring parameters of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, and the blur kernel of the target sampling point indicated by the blur algorithm, it is specifically used for:

[0142] An overdetermined equation is established between the expected blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, the blur kernel of the target sampling point indicated by the blur algorithm, and the fused pixel weight value of the target blurred feature point;

[0143] The fused pixel weight values ​​of the target blurred feature points are obtained by solving the overdetermined equation using the least squares method.

[0144] Furthermore, when establishing the overdetermined equation between the expected blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, the blur kernel of the target sampling point indicated by the blur algorithm, and the fused pixel weight value of the target blurred feature point, the processing unit 904 is specifically used for:

[0145] Calculate the first product of the blur kernel of the target sampling point indicated by the blur algorithm and the weight value of the fused pixel;

[0146] Obtain the deviation value between the fused pixel weight value and the reference pixel value, and calculate the second product of the blur kernel of the target blurred feature point indicated by the blur algorithm and the deviation value;

[0147] The overdetermined equation is established by making the sum of the first product and the second product equal to the desired degree of blurring.

[0148] Furthermore, when the fusion unit 905 performs fusion processing on the captured image and the initial blurred image based on the fusion pixel weight values ​​to obtain the target image, it is specifically used for:

[0149] Obtain the third product of the fused pixel weight value and the pixel value corresponding to the target blurred feature point in the captured image;

[0150] Calculate the fourth product of the deviation value and the pixel value corresponding to the target blurred feature point in the initial blurred image;

[0151] The sum of the third product and the fourth product is determined as the target pixel value, and the pixel values ​​of the target blurred feature points in the initial blurred image are adjusted based on the target pixel value to obtain the target image.

[0152] This application embodiment obtains an initial blurred image by blurring the captured image of the target object according to a blur algorithm, and determines the blur parameters of the sampled feature points of the target object based on the initial blurred image. Based on the blur parameters of the sampled feature points, the expected blur parameters of the blurred feature points on the initial blurred image are determined more accurately. Based on the expected blur parameters, the blur kernel of the sampled feature points indicated by the blur algorithm, and the blur kernel of the blurred feature points indicated by the blur algorithm, the fusion pixel weight value is obtained. Based on the fusion pixel weight value, the captured image and the initial blurred image are fused to obtain a target image with better blur effect, thereby improving the accuracy and effectiveness of image hierarchical blur processing.

[0153] Figure 9 The various units in the image processing apparatus shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the image processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0154] This can be achieved by running a computer program (including one or more instructions) capable of executing the steps involved in any of the above method embodiments on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), to construct a system as described above. Figure 7 The image processing apparatus shown herein, and the image processing method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computing device via the computer-readable storage medium, and run therein.

[0155] It is worth noting that, in the embodiments of this application, the term "unit" refers to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can contain a portion of the overall module or unit's functionality.

[0156] Based on the description of the above method and device embodiments, this application also provides a computing device. Please refer to... Figure 10 , Figure 10 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application, such as... Figure 10 As shown, the computing device includes at least a processor 1001, an input interface 1002, an output interface 1003, and a computer-readable storage medium 1004. The processor 1001, input interface 1002, output interface 1003, and computer-readable storage medium 1004 within the computing device can be connected via a bus or other means. The computer-readable storage medium 1004 can be stored in the memory of the computing device. The computer-readable storage medium 1004 is used to store a computer program, which includes one or more instructions. The processor 1001 is used to execute one or more instructions from the computer program stored in the computer-readable storage medium 1004. The processor 1001 (or CPU (Central Processing Unit)) is the computing and control core of the computing device, adapted to implement one or more instructions, specifically adapted to load and execute one or more instructions to achieve a corresponding method flow or function.

[0157] The processor 1001 described in this application embodiment can be used to perform the following steps:

[0158] Acquire a captured image of the target object, and obtain an initial blurred image corresponding to the captured image based on a blurring algorithm;

[0159] The blurring parameters of the sampled feature points of the target object are determined based on the initial blurred image;

[0160] Based on the blurring parameters of the sampled feature points, determine the expected blurring parameters of the blurring feature points on the initial blurring map;

[0161] Based on the desired blurring parameter, the blur kernel of the sampled feature points indicated by the blurring algorithm, and the blur kernel of the blurred feature points indicated by the blurring algorithm, the fused pixel weight value is obtained;

[0162] The captured image and the initial blurred image are fused based on the fused pixel weight values ​​to obtain the target image.

[0163] Furthermore, when determining the blurring parameters of the sampled feature points of the target object based on the initial blurred image, the processor 1001 specifically performs the following:

[0164] The pixels of the target object in the initial blurred image are sampled to obtain multiple sampled feature points;

[0165] The position information, depth information, and blur degree value of each sampled feature point on the initial blurred image are determined as the blur parameters of each sampled feature point.

[0166] Furthermore, when determining the desired blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points, the processor 1001 can specifically be used to:

[0167] The sampling feature points are fitted according to the fuzzing parameters of each sampling feature point to obtain a fitting function;

[0168] Based on the fitting function and the position information of each blurred feature point in the initial blurred image, the expected blurring parameters of each blurred feature point on the initial blurred image are determined.

[0169] Furthermore, when the processor 1001 obtains the fused pixel weight value based on the desired blurring parameters, the blur kernel for the sampled feature points indicated by the blurring algorithm, and the blur kernel for the blurred feature points indicated by the blurring algorithm, it specifically performs the following:

[0170] Obtain the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image;

[0171] Determine each sampling point located on the intersection line from the initial blurred image;

[0172] The fused pixel weight value is obtained based on the expected blurring parameters of each blurring feature point, the blur kernel of each sampling point indicated by the blurring algorithm, and the blur kernel of each blurring feature point indicated by the blurring algorithm.

[0173] Furthermore, when the processor 1001 obtains the intersection line between the front plane where the target object is located in the initial blurred image and the focal plane of the initial blurred image, it specifically performs the following:

[0174] The key point detection algorithm is used to obtain the feature information of the key feature points of the target object in the captured image. The feature information includes the position information and depth information of the key feature points.

[0175] The front plane where the target object is located is fitted based on the feature information of the key feature points;

[0176] Obtain the position and depth information of the focus of the initial blurred image, and determine the focus plane where the focus is located based on the position and depth information of the focus;

[0177] The intersection line between the front plane and the focal plane is determined based on the front plane and the focal plane.

[0178] Furthermore, when the processor 1001 obtains the fused pixel weight value based on the expected blurring parameters of each blurred feature point, the blur kernel for each sampling point indicated by the blurring algorithm, and the blur kernel for each blurred feature point indicated by the blurring algorithm, it is specifically used for:

[0179] Based on the position information of the target blurred feature point among the various blurred feature points, determine the target sampling point corresponding to the target blurred feature point on the intersection line from the various sampling points;

[0180] The fused pixel weight value of the target blurred feature point is obtained based on the expected blurring parameter of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, and the blur kernel of the target sampling point indicated by the blur algorithm.

[0181] Furthermore, when the processor 1001 obtains the fused pixel weight value of the target blurred feature point based on the expected blurring parameters of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, and the blur kernel of the target sampling point indicated by the blur algorithm, it is specifically used for:

[0182] An overdetermined equation is established between the expected blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, the blur kernel of the target sampling point indicated by the blur algorithm, and the fused pixel weight value of the target blurred feature point;

[0183] The fused pixel weight values ​​of the target blurred feature points are obtained by solving the overdetermined equation using the least squares method.

[0184] Furthermore, when establishing the overdetermined equation between the expected blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, the blur kernel of the target sampling point indicated by the blur algorithm, and the fused pixel weight value of the target blurred feature point, the processor 1001 is specifically used for:

[0185] Calculate the first product of the blur kernel of the target sampling point indicated by the blur algorithm and the weight value of the fused pixel;

[0186] Obtain the deviation value between the fused pixel weight value and the reference pixel value, and calculate the second product of the blur kernel of the target blurred feature point indicated by the blur algorithm and the deviation value;

[0187] The overdetermined equation is established by making the sum of the first product and the second product equal to the desired degree of blurring.

[0188] Furthermore, when the processor 1001 performs fusion processing on the captured image and the initial blurred image based on the fusion pixel weight values ​​to obtain the target image, it specifically performs the following:

[0189] Obtain the third product of the fused pixel weight value and the pixel value corresponding to the target blurred feature point in the captured image;

[0190] Calculate the fourth product of the deviation value and the pixel value corresponding to the target blurred feature point in the initial blurred image;

[0191] The sum of the third product and the fourth product is determined as the target pixel value, and the pixel values ​​of the target blurred feature points in the initial blurred image are adjusted based on the target pixel value to obtain the target image.

[0192] This application embodiment obtains an initial blurred image by blurring the captured image of the target object according to a blur algorithm, and determines the blur parameters of the sampled feature points of the target object based on the initial blurred image. Based on the blur parameters of the sampled feature points, the expected blur parameters of the blurred feature points on the initial blurred image are determined more accurately. Based on the expected blur parameters, the blur kernel of the sampled feature points indicated by the blur algorithm, and the blur kernel of the blurred feature points indicated by the blur algorithm, the fusion pixel weight value is obtained. Based on the fusion pixel weight value, the captured image and the initial blurred image are fused to obtain a target image with better blur effect, thereby improving the accuracy and effectiveness of image hierarchical blur processing.

[0193] It should be understood that, in the embodiments of this application, the processor 1001 may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0194] In specific implementations, the processor 1001 described in this application embodiment can execute the implementation method described in the image processing method provided in this application embodiment, or it can execute the implementation method of the image processing device described in this application embodiment, which will not be repeated here.

[0195] This application also provides a computer-readable storage medium storing program instructions that, when executed, implement any of the data processing methods described above.

[0196] The computer-readable storage medium can be an internal storage unit of the device described in any of the foregoing embodiments, such as the device's hard drive or memory. The computer-readable storage medium can also be an external storage device of the device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device. Further, the computer-readable storage medium may include both internal and external storage units of the device. The computer-readable storage medium is used to store the computer program and other programs and data required by the mobile device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0197] Embodiments of this application also provide a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device to perform the methods provided in the various embodiments described above.

[0198] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0199] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the terminals and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

Claims

1. An image processing method, characterized in that, include: Acquire a captured image of the target object, and obtain an initial blurred image corresponding to the captured image based on a blurring algorithm; The blurring parameters of the sampled feature points of the target object are determined based on the initial blurred image; Based on the blurring parameters of the sampled feature points, determine the expected blurring parameters of the blurring feature points on the initial blurring map; Based on the desired blurring parameter, the blur kernel of the sampled feature points indicated by the blurring algorithm, and the blur kernel of the blurred feature points indicated by the blurring algorithm, the fused pixel weight value is obtained; The captured image and the initial blurred image are fused based on the fused pixel weight values ​​to obtain the target image.

2. The method according to claim 1, characterized in that, The step of determining the blurring parameters of the sampled feature points of the target object based on the initial blurred image includes: The pixels of the target object in the initial blurred image are sampled to obtain multiple sampled feature points; The position information, depth information, and blur degree value of each sampled feature point on the initial blurred image are determined as the blur parameters of each sampled feature point.

3. The method according to claim 2, characterized in that, The step of determining the expected blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points includes: The sampling feature points are fitted using the fuzzing parameters of each sampling feature point to obtain a fitting function; Based on the fitting function and the position information of each blurred feature point in the initial blurred image, the expected blurring parameters of each blurred feature point on the initial blurred image are determined.

4. The method according to claim 3, characterized in that, The step of obtaining the fused pixel weight value based on the desired blurring parameter, the blur kernel of the sampled feature points indicated by the blurring algorithm, and the blur kernel of the blurred feature points indicated by the blurring algorithm includes: Obtain the intersection line between the front plane of the target object in the initial blurred image and the focal plane of the initial blurred image; Determine each sampling point located on the intersection line from the initial blurred image; The fused pixel weight value is obtained based on the expected blurring parameters of each blurring feature point, the blur kernel of each sampling point indicated by the blurring algorithm, and the blur kernel of each blurring feature point indicated by the blurring algorithm.

5. The method according to claim 4, characterized in that, The step of obtaining the intersection line between the foreground plane of the target object in the initial blurred image and the focal plane of the initial blurred image includes: The key point detection algorithm is used to obtain the feature information of the key feature points of the target object in the captured image. The feature information includes the position information and depth information of the key feature points. The front plane where the target object is located is fitted based on the feature information of the key feature points; Obtain the position and depth information of the focus of the initial blurred image, and determine the focus plane where the focus is located based on the position and depth information of the focus; The intersection line of the front plane and the focal plane is determined based on the front plane and the focal plane.

6. The method according to claim 4, characterized in that, The step of obtaining the fused pixel weight value based on the expected blurring parameters of each blurred feature point, the blur kernel of each sampling point indicated by the blurring algorithm, and the blur kernel of each blurred feature point indicated by the blurring algorithm includes: Based on the position information of the target blurred feature point among the various blurred feature points, determine the target sampling point corresponding to the target blurred feature point on the intersection line from the various sampling points; The fused pixel weight value of the target blurred feature point is obtained based on the expected blurring parameter of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, and the blur kernel of the target sampling point indicated by the blur algorithm.

7. The method according to claim 6, characterized in that, The step of obtaining the fused pixel weight value of the target blurred feature point based on the expected blurring parameters of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, and the blur kernel of the target sampling point indicated by the blur algorithm includes: An overdetermined equation is established between the expected blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, the blur kernel of the target sampling point indicated by the blur algorithm, and the fused pixel weight value of the target blurred feature point; The fused pixel weight values ​​of the target blurred feature points are obtained by solving the overdetermined equation using the least squares method.

8. The method according to claim 7, characterized in that, The overdetermined equations relating the desired blur degree value of the target blurred feature point, the blur kernel of the target blurred feature point indicated by the blur algorithm, the blur kernel of the target sampling point indicated by the blur algorithm, and the fused pixel weight value of the target blurred feature point include: Calculate the first product of the blur kernel of the target sampling point indicated by the blur algorithm and the weight value of the fused pixel; Obtain the deviation value between the fused pixel weight value and the reference pixel value, and calculate the second product of the blur kernel of the target blurred feature point indicated by the blur algorithm and the deviation value; The overdetermined equation is established by making the sum of the first product and the second product equal to the desired virtualization value.

9. The method according to claim 8, characterized in that, The step of fusing the captured image and the initial blurred image based on the fused pixel weight values ​​to obtain the target image includes: Obtain the third product of the fused pixel weight value and the pixel value corresponding to the target blurred feature point in the captured image; Calculate the fourth product of the deviation value and the pixel value corresponding to the target blurred feature point in the initial blurred image; The sum of the third product and the fourth product is determined as the target pixel value, and the pixel values ​​of the target blurred feature points in the initial blurred image are adjusted based on the target pixel value to obtain the target image.

10. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire a captured image of the target object and to acquire an initial blurred image corresponding to the captured image according to a blurring algorithm; The first determining unit is configured to determine the blurring parameters of the sampled feature points of the target object based on the initial blurred image; The second determining unit is used to determine the expected blurring parameters of the blurred feature points on the initial blurred map based on the blurring parameters of the sampled feature points. The processing unit is configured to obtain a fused pixel weight value based on the desired blurring parameter, the blur kernel of the sampled feature point indicated by the blurring algorithm, and the blur kernel of the blurred feature point indicated by the blurring algorithm. The fusion unit is used to perform fusion processing on the captured image and the initial blurred image based on the fusion pixel weight value to obtain the target image.

11. A computing device, characterized in that, The device includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, and the processor is configured to invoke the computer program to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed, implement the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes program instructions that, when executed by a processor, implement the method described in any one of claims 1 to 9.

Images