Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic equipment and storage medium. The image processing method comprises: performing image analysis on an initial image to obtain a foreground image and a background image; determining a target blurring intensity corresponding to each pixel point in the background image according to a depth value of each pixel point in the background image; adjusting a pixel value of each pixel point in the background image according to each target blurring intensity to obtain a blurred background image; and performing synthesis processing on the foreground image and the blurred background image to obtain a target image. The present disclosure can determine the blurring intensity according to the depth value of the pixel point in the background image, so that the blurring effect of the blurred image is more layered compared with one fixed blurring intensity corresponding to one image. Meanwhile, each pixel point in the background image corresponds to one target blurring intensity, so that the gradualness of the blurring effect of the blurred image is stronger compared with one blurring intensity corresponding to one region of the image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision, and particularly relates to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Computer vision is a crucial technology in the field of artificial intelligence, which can replace human eyes to identify, segment, track and measure objects in images, and can also change image effects by processing images, for example, blurring the background of an image to highlight the foreground.

[0003] However, the image blurred by the traditional image blurring method has weak layering and continuity, which can result in poor effect of the blurred image. SUMMARY

[0004] To overcome the problems in the related art, the present disclosure provides an image processing method and device, electronic equipment and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, and the method comprises:

[0006] performing image analysis on an initial image to obtain a foreground image and a background image;

[0007] determining target blurring intensities corresponding to each pixel point in the background image according to depth values of the pixel points in the background image;

[0008] adjusting pixel values of each pixel point in the background image according to the target blurring intensities to obtain a blurred background image;

[0009] performing synthesis processing on the foreground image and the blurred background image to obtain a target image.

[0010] In some embodiments, the method further comprises:

[0011] inputting the initial image into a depth estimation model to obtain a depth matrix corresponding to the initial image; wherein the depth matrix comprises depth values of each pixel point in the initial image, and the positions of each depth value in the depth matrix correspond to the positions of each pixel point in the initial image;

[0012] obtaining the depth values of each pixel point in the background image from the corresponding positions of the depth matrix based on the positions of each pixel point in the initial image.

[0013] In some embodiments, the determining, according to the depth value of each pixel in the background image, a target blurring intensity corresponding to each pixel in the background image comprises:

[0014] The target blurring intensity corresponding to each pixel in the background image is determined according to the depth value of each pixel in the background image and the reference blurring intensity.

[0015] In some embodiments, the adjusting, according to each target blurring intensity, the pixel value of each pixel in the background image to obtain a blurred background image comprises:

[0016] According to each target blurring intensity, a weight matrix corresponding to each pixel in the background image is determined.

[0017] A pixel matrix corresponding to each pixel in the background image is determined with each pixel in the background image as the center, wherein the position of each pixel in the pixel matrix corresponds to the position of each weight value in the weight matrix.

[0018] Each pixel matrix is convoluted by each weight matrix to obtain the blurred background image.

[0019] In some embodiments, the convoluting each pixel matrix by each weight matrix to obtain the blurred background image comprises:

[0020] A non-zero weight value is determined from the weight matrix corresponding to the current pixel to be processed.

[0021] Each non-zero weight value is multiplied by the pixel value at the corresponding position in the pixel matrix corresponding to the current pixel to be processed to obtain a weighted pixel value.

[0022] The sum of each weighted pixel value is determined as the target pixel value of the current pixel to be processed.

[0023] The blurred background image is obtained according to the target pixel value of each pixel in the background image.

[0024] In some embodiments, the image analysis of the initial image to obtain a foreground image and a background image comprises:

[0025] According to the first input operation of the user, a foreground image contour of the foreground image is determined.

[0026] The region in the initial image within the foreground image contour is determined as the foreground image.

[0027] Determine the region in the initial image outside the foreground image contour as the background image.

[0028] In some embodiments, the method further comprises:

[0029] Determine an intermediate image based on the contour edge of the foreground image and the contour edge of the background image;

[0030] Adjust the pixel value of each pixel point in the intermediate image based on the pixel value of each pixel point in the foreground image and the pixel value of each pixel point in the blurred background image, to obtain a blurred intermediate image;

[0031] The synthesizing processing of the foreground image and the blurred background image to obtain a target image comprises:

[0032] The synthesizing processing of the foreground image, the blurred background image and the blurred intermediate image to obtain the target image.

[0033] In some embodiments, the adjusting the pixel value of each pixel point in the intermediate image based on the pixel value of each pixel point in the foreground image and the pixel value of each pixel point in the blurred background image to obtain a blurred intermediate image comprises:

[0034] Weight the pixel value of each pixel point in the foreground image by a first weight value to obtain a first weighted pixel value; wherein each pixel point in the foreground image corresponds to a first weight value;

[0035] Weight the pixel value of each pixel point in the blurred background image by a second weight value to obtain a second weighted pixel value; wherein each pixel point in the blurred background image corresponds to a second weight value;

[0036] Adjust the pixel value of each pixel point in the intermediate image based on the sum of the first weighted pixel value corresponding to each pixel point in the foreground image and the second weighted pixel value corresponding to each pixel point in the blurred background image, to obtain a blurred intermediate image

[0037] According to a second aspect of the embodiments of the present disclosure, an image processing device is provided, and the device comprises:

[0038] An image analysis module configured to perform image analysis on an initial image to obtain a foreground image and a background image;

[0039] A first determination module configured to determine a target blurring intensity corresponding to each pixel point in the background image according to the depth value of each pixel point in the background image;

[0040] The first adjusting module is configured to adjust pixel values of each pixel point in the background image according to each target blurring intensity, to obtain a blurred background image.

[0041] The synthesizing module is configured to synthesize the foreground image and the blurred background image, to obtain a target image.

[0042] In some embodiments, the device further comprises:

[0043] The depth estimation module is configured to input the initial image into a depth estimation model, to obtain a depth matrix corresponding to the initial image; wherein the depth matrix comprises depth values of each pixel point in the initial image, and positions of each depth value in the depth matrix correspond to positions of each pixel point in the initial image.

[0044] The obtaining module is configured to obtain, based on positions of each pixel point in the background image in the initial image, depth values of each pixel point in the background image from corresponding positions of the depth matrix.

[0045] In some embodiments, the first determining module is configured to:

[0046] determine target blurring intensities corresponding to each pixel point in the background image according to the depth values of each pixel point in the background image and a reference blurring intensity.

[0047] In some embodiments, the first adjusting module comprises:

[0048] The first determining unit is configured to determine a weight matrix corresponding to each pixel point in the background image according to each target blurring intensity.

[0049] The second determining unit is configured to determine, respectively, a pixel point matrix corresponding to each pixel point in the background image with each pixel point in the background image as a center; wherein positions of each pixel point in the pixel point matrix correspond to positions of each weight value in the weight matrix.

[0050] The convolution unit is configured to perform convolution processing on each pixel point matrix by using each weight matrix, to obtain the blurred background image.

[0051] In some embodiments, the convolution unit is configured to:

[0052] determine a non-zero weight value from a weight matrix corresponding to a current pixel point to be processed;

[0053] multiply each of the non-zero weight values with a pixel value at a corresponding position in a pixel matrix corresponding to the current pixel point to be processed, to obtain a weighted pixel value;

[0054] determine a sum of the weighted pixel values as a target pixel value of the current pixel point to be processed;

[0055] obtain the blurred background image according to the target pixel values of the pixel points in the background image.

[0056] In some embodiments, the image analysis module is configured to:

[0057] determine a foreground image contour of the foreground image according to a first input operation of a user;

[0058] determine the foreground image as a region in the initial image within the foreground image contour;

[0059] determine the background image as a region in the initial image outside the foreground image contour.

[0060] In some embodiments, the device further comprises:

[0061] a second determination module configured to determine an intermediate image based on a contour edge of the foreground image and a contour edge of the background image;

[0062] a second adjustment module configured to adjust a pixel value of each pixel point in the intermediate image based on a pixel value of each pixel point in the foreground image and a pixel value of each pixel point in the blurred background image, to obtain a blurred intermediate image;

[0063] the synthesis module is configured to synthesize the foreground image, the blurred background image and the blurred intermediate image to obtain the target image.

[0064] In some embodiments, the second adjustment module comprises:

[0065] a first weighting unit configured to perform weighting processing on the pixel value of each pixel point in the foreground image by using a first weight value to obtain a first weighted pixel value; each pixel point in the foreground image corresponds to a first weight value;

[0066] a second weighting unit configured to perform weighting processing on the pixel value of each pixel point in the blurred background image by using a second weight value to obtain a second weighted pixel value; each pixel point in the blurred background image corresponds to a second weight value;

[0067] The adjusting unit is configured to adjust the pixel value of each pixel point in the intermediate image based on a sum value of a first weighted pixel value corresponding to each pixel point in the foreground image and a second weighted pixel value corresponding to each pixel point in the blurred background image, to obtain a blurred intermediate image.

[0068] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, comprising:

[0069] a processor;

[0070] a memory for storing processor-executable instructions;

[0071] The processor is configured to implement the steps in any of the image processing methods of the first aspect when executed.

[0072] According to a fourth aspect of embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to implement the steps in any of the image processing methods of the first aspect.

[0073] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0074] In the embodiments of the present disclosure, after image analysis is performed on the initial image to obtain a foreground image and a background image, the target blurring intensity corresponding to each pixel point in the background image can be determined according to the depth value of each pixel point in the background image, and then the pixel value of each pixel point in the background image is adjusted according to the target blurring intensity, to obtain a blurred background image, and finally the foreground image and the blurred background image are synthesized to obtain a target image.

[0075] In the embodiments of the present disclosure, in the first aspect, the blurring intensity can be determined according to the depth value of the pixel point in the background image, compared to one fixed blurring intensity corresponding to one image, by refining the image blurring to each pixel point, the blurring effect of the blurred image can be more layered. In the second aspect, the target blurring intensity corresponding to each pixel point in the background image can be determined according to the depth value of each pixel point in the background image, and then the pixel value of the pixel point is adjusted according to the target blurring intensity corresponding to each pixel point in the background image, that is, each pixel point in the background image corresponds to a target blurring intensity, compared to one blurring intensity corresponding to a region of the image, the gradualness of the blurring effect of the blurred image can be stronger.

[0076] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0077] The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0078] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment of the present disclosure Figure One ;

[0079] Figure 2 is a flowchart of a method for blurring a background image according to an exemplary embodiment of the present disclosure

[0080] Figure 3 is a flowchart of a method for blurring an intermediate image according to an exemplary embodiment of the present disclosure

[0081] Figure 4 is a flowchart of an image processing method according to an exemplary embodiment of the present disclosure Figure Two ;

[0082] Figure 5 is a block diagram of an image processing device according to an exemplary embodiment of the present disclosure

[0083] Figure 6 is a hardware block diagram of an electronic device according to an exemplary embodiment of the present disclosure Figure One ;

[0084] Figure 7 is a hardware block diagram of an electronic device according to an exemplary embodiment of the present disclosure Figure Two . DETAILED DESCRIPTION

[0085] The exemplary embodiments will be described in detail herein below with reference to the accompanying drawings. In the following description, the same drawings refer to the same elements or similar elements. The following exemplary embodiments described in the exemplary embodiments do not represent all the embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0086] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment of the present disclosure Figure One As shown in Figure 1 , the image processing method mainly includes the following steps:

[0087] Step 101, image analysis is performed on the initial image to obtain a foreground image and a background image;

[0088] In step 102, a target blurring intensity corresponding to each pixel point in the background image is determined according to a depth value of each pixel point in the background image.

[0089] In step 103, a pixel value of each pixel point in the background image is adjusted according to the target blurring intensity, and a blurred background image is obtained.

[0090] In step 104, the foreground image and the blurred background image are synthesized to obtain a target image.

[0091] It should be noted that the image processing method provided by the present disclosure can be applied to an electronic device, and can also be applied to a server. Here, the electronic device can include a terminal device, for example, a mobile terminal or a fixed terminal. The mobile terminal can include a mobile phone, a tablet computer, a notebook computer, and the like. The fixed terminal can include a desktop computer or a smart television, and the like.

[0092] In some embodiments, the initial image can be an image with any image content. The initial image can be a photo or an image frame in a video. It should be noted that the initial image includes at least two objects. The foreground of the initial image includes at least one object, and the background of the initial image includes at least one object. Here, the foreground of the initial image can be an object that is closer to the camera when the initial image is captured, for example, a portrait, a building, and the like. The background of the initial image can be an object that is farther away from the camera when the initial image is captured, for example, a landscape, a plant, and the like.

[0093] In the case where the electronic device that executes the image processing method includes an image acquisition module, the electronic device can acquire the initial image through the image acquisition module, for example, capture an image through a camera, and take the captured image as the initial image. In the case where the electronic device includes a communication module, the electronic device can acquire the initial image from other electronic devices or a cloud server through the communication module. In the case where the electronic device includes a storage module, the initial image can be pre-stored in the storage module. When image processing is needed, the initial image can be acquired from the storage module of the electronic device.

[0094] After the initial image is acquired, image analysis can be performed on the initial image to obtain a foreground image and a background image. In some embodiments, the initial image can be input into an image segmentation network, the image analysis can be performed through the image segmentation network, the position of the foreground in the initial image can be predicted, and then the initial image can be segmented according to the position of the foreground in the initial image to output the foreground image and the background image.

[0095] In some embodiments, the image segmentation network can include a CNN (Convolutional Neural Networks), a FCN (Fully Convolutional Networks), or a U-Net (a kind of image segmentation network with a U-shaped network structure).

[0096] In some embodiments, the foreground image and the background image output by the image segmentation network can be in the form of the same representation matrix, i.e., the image segmentation network can output a representation matrix M to represent the foreground image and the background image.

[0097] In some embodiments, after obtaining the initial image, the initial image can be preprocessed, and the size of the initial image can be adjusted to a preset size, for example, the size of the initial image can be adjusted to 512*512 pixels. Then, the initial image of the preset size (which can be in the form of a matrix, such as the pixel matrix I of the initial image) is input into the image segmentation network, and the image segmentation network outputs a representation matrix of the same size as the initial image, for example, a 512*512 representation matrix M. Here, the value range of each value in the representation matrix M output by the image segmentation network can be [0, 1], wherein the value corresponding to the foreground in the representation matrix M is 1, the value corresponding to the background is 0, and the value between 0 and 1 can be used to represent the intermediate image between the foreground and the background in the initial image.

[0098] In some embodiments, the pixel matrix of the foreground image can be obtained according to formula (1):

[0099] A = I * M (1);

[0100] In formula (1), A represents the pixel matrix of the foreground image, I represents the pixel matrix of the initial image, and M represents the representation matrix output by the image segmentation network. Here, formula (1) represents multiplying the pixel value in the pixel matrix I of the initial image with the value at the corresponding position in the representation matrix M.

[0101] It can be understood that in the pixel matrix of the foreground image, the pixel value of the pixel point corresponding to the foreground is the same as the pixel value of the foreground in the initial image, and the pixel value corresponding to the background is 0.

[0102] In some embodiments, the pixel matrix of the background image can be obtained according to formula (2):

[0103] B = I * (1-M) (2);

[0104] In formula (2), B represents the pixel matrix of the background image, I represents the pixel matrix of the initial image, and M represents the representation matrix output by the image segmentation network.

[0105] It can be understood that in the pixel matrix of the foreground image, the pixel value of the pixel corresponding to the background is the same as the pixel value of the background in the initial image, and the pixel value corresponding to the foreground is 0.

[0106] Here, the size of the representation matrix M output by the image segmentation network, the pixel matrix A of the foreground image, and the pixel matrix B of the background image are all the same as the pixel matrix of the initial image, and the positions of the values in the matrix M, the pixel matrix A, and the pixel matrix B correspond to the positions of the pixel values in the pixel matrix I.

[0107] In some embodiments, the image analysis on the initial image to obtain the foreground image and the background image comprises:

[0108] According to the first input operation of the user, the foreground image contour of the foreground image is determined;

[0109] The area in the initial image within the foreground image contour is determined as the foreground image;

[0110] The area in the initial image outside the foreground image contour is determined as the background image.

[0111] In this embodiment, the first input operation can be a gesture operation, and the electronic device executing the image processing method can include an interactive module, such as a touch display. The user can outline the contour of the foreground image on the interactive module through a gesture operation, and the electronic device can obtain the foreground image and the background image based on the contour of the foreground image. Then, according to the position of the foreground in the initial image and the position of the background in the initial image, a representation matrix N corresponding to the initial image is generated.

[0112] Here, the structure and size of the representation matrix N can be the same as the representation matrix M described above, and the value range of each value in the representation matrix N can be [0, 1], wherein the value corresponding to the foreground in the representation matrix N is 1, and the value corresponding to the background is 0. The values between 0 and 1 can be used to represent the intermediate image between the foreground and the background in the initial image.

[0113] After obtaining the representation matrix N, the representation matrix N can be used to replace the representation matrix M in formula (1) and formula (2), and then the pixel matrix A of the foreground image and the pixel matrix B of the background image are obtained.

[0114] In this way, the foreground image and the background image can be determined according to the user's operation, and then the background image can be blurred, so that the foreground image is more prominent compared to the background image, and the target image obtained according to the foreground image and the blurred background image is more in line with the user's needs.

[0115] In some embodiments, the image processing method can further comprise:

[0116] inputting the initial image into a depth estimation model to obtain a depth matrix corresponding to the initial image; wherein the depth matrix comprises depth values of each pixel point in the initial image, and the position of each depth value in the depth matrix corresponds to the position of each pixel point in the initial image;

[0117] based on the position of each pixel point in the background image in the initial image, obtaining the depth value of each pixel point in the background image from the corresponding position of the depth matrix.

[0118] Here, the depth estimation model can include Monodepth (monocular depth estimation model based on self-supervised learning), SFM-Net (perception geometry neural network model, Structure From Motion-Network), BA-Net (dense BA network model, Dense Bundle Adjustment Network), etc.

[0119] After the initial image is input into the depth estimation model, the depth value of each pixel point in the initial image can be obtained, and the depth estimation model can output a depth matrix D corresponding to the initial image, wherein the depth matrix D can be a matrix composed of depth values of each pixel point in the initial image. Each depth value in the depth matrix D can have a value range of [0, 1], 0 indicates that the pixel point with the depth value of 0 is closest to the camera when shooting, 1 indicates that the pixel point with the depth value of 1 is farthest from the camera when shooting, and the depth value between 0 and 1 is a transition zone, and the closer the depth value is to 1, the farther the pixel point is from the camera when shooting.

[0120] It can be understood that the size of the depth matrix D corresponding to the initial image can be the same as the pixel matrix of the initial image, and the position of each pixel point in the depth matrix D can correspond to the position of each pixel point in the image, or the position of each pixel value of each pixel point in the pixel matrix I of the initial image.

[0121] Therefore, the depth value of each pixel point in the background image can be obtained from the corresponding position of the depth matrix D based on the position of each pixel point in the background image in the initial image. Here, the position of each pixel point in the background image in the initial image can be the position of each pixel point in the background image in the pixel matrix I of the initial image. Since the positions of each value in the matrix M, the pixel matrix A, and the pixel matrix B correspond to the positions of each pixel value in the pixel matrix I, the position of each pixel point in the background image in the pixel matrix I corresponds to the position of each pixel point in the background image in the pixel matrix B.

[0122] In this way, the depth matrix corresponding to the initial image can be output by the depth estimation model, and since the position of a depth value in the depth matrix corresponds to the position of a pixel in the initial image, the depth value of each pixel in the background image can be accurately obtained from the depth matrix based on the position of each pixel in the background image.

[0123] After obtaining the depth value of each pixel in the background image, the target blurring intensity corresponding to each pixel in the background image can be determined according to the depth value of each pixel in the background image. It can be understood that determining the blurring intensity according to the depth value of the pixel in the image can make the blurring effect of the image more layered compared to a fixed blurring intensity corresponding to an image. And each pixel in the background image corresponds to a target blurring intensity, which can make the gradualness of the blurring effect of the image stronger compared to a block of the image corresponding to a blurring intensity.

[0124] In some embodiments, a mapping relationship library representing the mapping relationship between each depth value and each blurring intensity can be pre-set, and each depth value, each blurring intensity and the mapping relationship between each depth value and each blurring intensity can be pre-stored in the mapping relationship library. Then, in the process of determining the target blurring intensity corresponding to each pixel in the background image, the blurring intensity in the mapping relationship library that has a mapping relationship with each pixel can be determined as the target blurring intensity corresponding to each pixel.

[0125] In other embodiments, the determining of the target blurring intensity corresponding to each pixel in the background image according to the depth value of each pixel in the background image can include:

[0126] determining the target blurring intensity corresponding to each pixel in the background image according to the depth value of each pixel in the background image and a reference blurring intensity.

[0127] Here, the reference blurring intensity can be a reference value, which can be obtained by experiment, according to personal experience, or according to the blurring intensity of other image blurring methods. That is, the target blurring intensity of each pixel in the background image can be obtained according to the depth value of each pixel in the background image and the same blurring intensity reference value.

[0128] In some embodiments, the depth matrix of the initial image can be represented as D, the reference blurring intensity can be represented as BH, and the target blurring intensity of each pixel in the background image can be represented as K. The target blurring intensity corresponding to each pixel in the background image can be obtained by formula (3):

[0129] K(i,j)=[D(i,j)*BH+0.5] (3);

[0130] In formula (3), K(i, j) represents the target blurring intensity of the pixel point at position (i, j) in the initial image; D(i, j) represents the depth value at position (i, j) in the depth matrix, that is, the depth value of the pixel point at position (i, j) in the initial image. Here, [] represents the rounding operation, and 0.5 is a constant parameter.

[0131] Here, in the case where (i, j) represents the position of the pixel point in the background image in the initial image, K(i, j) can represent the target blurring intensity value corresponding to the pixel point in the background image.

[0132] In this way, by taking the reference blurring intensity as the reference value when determining the target blurring intensity, the scientificity of the determined target blurring intensity can be stronger. Moreover, by determining the target blurring intensity according to the depth values of the respective pixel points in the background image, compared with one image corresponding to one fixed blurring intensity, the blurring effect of the image blurred according to the target blurring intensity can be more layered; compared with one region of an image corresponding to one blurring intensity, the progression of the image blurred according to the target blurring intensity can be stronger, so that the blurring effect of the blurred image is better.

[0133] After obtaining the target blurring intensity corresponding to each pixel point in the background image, the pixel value of each pixel point in the background image can be adjusted according to the respective target blurring intensity, and then the blurred background image can be obtained according to the adjusted pixel value of each pixel point in the background image.

[0134] After obtaining the blurred background image, the foreground image and the blurred background image can be synthesized to obtain the target image corresponding to the initial image.

[0135] Here, the synthesis of the foreground image and the blurred background image can include splicing the foreground image and the blurred background image according to the position of the foreground image in the initial image and according to the position of the background image in the initial image; can also include arranging the respective pixel points in the foreground image and the respective pixel points in the blurred background image according to the arrangement order of the respective pixel points in the initial image; and can also include superimposing the pixel matrix of the foreground image and the pixel matrix of the blurred background image. Here, the pixel value in the pixel matrix of the blurred background image is changed, and the position of the pixel value is unchanged, and the superimposition of the pixel matrix of the foreground image and the pixel matrix of the blurred background image can be realized by superimposing the pixel values at the corresponding positions in the pixel matrix of the foreground image and the pixel matrix of the blurred background image.

[0136] In the embodiments of the present disclosure, after image analysis is performed on the initial image to obtain the foreground image and the background image, the target blurring intensity corresponding to each pixel point in the background image can be determined according to the depth value of each pixel point in the background image, then the pixel value of each pixel point in the background image is adjusted according to the target blurring intensity, to obtain the blurred background image, and finally the foreground image and the blurred background image are synthesized to obtain the target image.

[0137] In the embodiments of the present disclosure, in the first aspect, the blurring intensity can be determined according to the depth value of the pixel point in the background image, compared with one fixed blurring intensity corresponding to one image, the blurring effect of the blurred image can be more layered by refining the image to each pixel point. In the second aspect, the target blurring intensity corresponding to each pixel point in the background image can be determined according to the depth value of each pixel point in the background image, then the pixel value of each pixel point in the background image is adjusted according to the target blurring intensity corresponding to each pixel point in the background image, that is, each pixel point in the background image corresponds to a target blurring intensity, compared with one blurring intensity corresponding to one region of the image, the gradualness of the blurring effect of the blurred image can be stronger.

[0138] Figure 2 is a flowchart of a blurring method of a background image according to an example embodiment of the present disclosure,

[0139] As shown in Figure 2 Step 103, adjusting the pixel value of each pixel point in the background image according to the target blurring intensity to obtain the blurred background image, comprising the following steps:

[0140] Step 1031, determining the weight matrix corresponding to each pixel point in the background image according to the target blurring intensity;

[0141] Step 1032, determining the pixel point matrix corresponding to each pixel point in the background image respectively with each pixel point in the background image as the center; wherein the position of each pixel point in the pixel point matrix corresponds to the position of each weight value in the weight matrix;

[0142] Step 1033, performing convolution processing on each pixel point matrix by using each weight matrix to obtain the blurred background image.

[0143] It can be understood that the pixel value of each pixel point in the background image can be adjusted according to the weight matrix, so as to realize the blurring of the background image. Generally, the larger the size of the weight matrix is, the stronger the blurring intensity of the background image is. In some embodiments, a mapping relationship between each blurring intensity and the size of the weight matrix can be preset, so that after the target blurring intensity corresponding to each pixel point in the background image is determined, the size of the weight matrix corresponding to each pixel point in the background image can be determined based on the preset mapping relationship between each blurring intensity and the size of the weight matrix. Here, each pixel point in the background image corresponds to a weight matrix.

[0144] In some embodiments, the type of each weight matrix can be preset, for example, the weight matrix corresponding to each pixel point in the background image is set to be a cross-shaped mean matrix (i.e., a matrix in which the middle row and the middle column have the same non-zero value, and the values of other positions are all 0), or the weight matrix corresponding to each pixel point in the background image is set to be a general mean matrix (i.e., a matrix in which the values of each position are all the same non-zero value). In this way, the size of each preset weight matrix can be determined according to the target blurring intensity corresponding to each pixel point in the background image, and then the weight matrix corresponding to each pixel point in the background image can be obtained.

[0145] It can be understood that when the pixel value of each pixel point in the background image is adjusted, the pixel value of the pixel point around each pixel point in the background image can be considered, so that the correlation between each adjusted pixel point in the background image is stronger, and thus the image effect of the blurred background image composed of each adjusted pixel point is better.

[0146] Therefore, the pixel point matrix corresponding to each pixel point in the background image can be determined respectively with each pixel point in the background image as the center; then the pixel point matrix corresponding to each pixel point is convolved by the weight matrix corresponding to each pixel point to obtain the adjusted pixel value of each pixel point in the background image, and thus the blurred background image is obtained. Here, each pixel point in the background image corresponds to a pixel point matrix.

[0147] It should be noted that the size of the pixel point matrix corresponding to each pixel point in the background image can be the same as the size of the weight matrix corresponding to each pixel point in the background image, and the position of each pixel point in the pixel point matrix corresponds to the position of each weight value in the weight matrix. The pixel value of the pixel point at the corresponding position in the pixel point matrix corresponds to the weight value at the corresponding position in the weight matrix.

[0148] In this way, the pixel values of each pixel in the pixel matrix can be corresponded to each weight value in the weight matrix. When adjusting the pixel value of each pixel in the background image, the other pixel values in the pixel matrix centered on the pixel, i.e., the pixel values around the pixel, are considered, and the pixel matrix is convoluted according to the weight values corresponding to the pixel values of each pixel in the pixel matrix, so that the correlation of each pixel in the blurred background image is stronger, thereby making the image effect of the blurred background image better.

[0149] In some embodiments, in step 203, the convolution processing of each pixel matrix by each weight matrix to obtain the blurred background image comprises:

[0150] Determine the non-zero weight values from the weight matrix corresponding to the current pixel to be processed;

[0151] Multiply each non-zero weight value with the pixel value at the corresponding position in the pixel matrix corresponding to the current pixel to be processed to obtain a weighted pixel value;

[0152] Determine the sum of each weighted pixel value as the target pixel value of the current pixel to be processed;

[0153] Obtain the blurred background image according to the target pixel values of each pixel in the background image.

[0154] Here, the non-zero weight values can include weight values that are not zero. It can be understood that when the convolution processing between two matrices is performed, the sum of the products of the values at each corresponding position of the two matrices can be used as the convolution result. However, when a matrix contains a zero value, the multiplication operation with the zero value is invalid. For example, in a cross-shaped weight matrix, the middle row and the middle column are non-zero values, and the values at other positions are 0.

[0155] Therefore, the non-zero weight values can be determined from the weight matrix corresponding to the current pixel to be processed, and then each non-zero weight value is multiplied with the pixel value of the pixel at the corresponding position in the pixel matrix corresponding to the current pixel to be processed to obtain a weighted pixel value. In this way, the efficiency of the convolution processing can be improved by reducing invalid operations.

[0156] The sum of the product of each non-zero weight value in the weight matrix corresponding to the current pixel to be processed and the pixel value at the corresponding position in the pixel matrix corresponding to the current pixel to be processed can be used as the convolution result of the convolution processing of the weight matrix and the pixel matrix corresponding to the current pixel to be processed, that is, the adjusted pixel value (target pixel value) of the current pixel to be processed. After obtaining the target pixel value of each pixel in the background image, the blurred background image can be obtained based on the target pixel value of each pixel in the background image.

[0157] In this way, the product processing of each non-zero weight value and the pixel value at the corresponding position in the pixel matrix corresponding to the current pixel to be processed can reduce invalid operations and improve the efficiency of convolution processing.

[0158] In some embodiments, the image processing method can further include:

[0159] determining an intermediate image based on the contour edge of the foreground image and the contour edge of the background image;

[0160] adjusting the pixel value of each pixel in the intermediate image based on the pixel value of each pixel in the foreground image and the pixel value of each pixel in the blurred background image to obtain a blurred intermediate image;

[0161] The synthesizing processing of the foreground image and the blurred background image to obtain a target image includes:

[0162] synthesizing processing of the foreground image, the blurred background image and the blurred intermediate image to obtain the target image.

[0163] It can be understood that in some embodiments, the contour of the foreground image and the contour of the background image in the initial image can be determined by manual or preset algorithm, but since the association between the foreground and the background in some initial images is relatively close, for example, the foreground and the background are included in a certain region in the initial image at the same time, or the actual contours of the foreground and the background are not clear enough; therefore, the contour of the foreground image and the contour of the background image determined by manual or preset algorithm can not be accurate enough. For example, there can be an overlapping part between the contour of the foreground image and the contour of the background image, or a part of the initial image is independent of the contour of the foreground image and the contour of the background image.

[0164] In the embodiments of the present disclosure, the overlapping area of the contour of the foreground image and the contour of the background image can be determined based on the contour edge of the foreground image and the contour edge of the background image, and then the overlapping area is determined as an intermediate image.

[0165] In some other embodiments, an image region in the initial image that is independent of the contour of the foreground image and the contour of the background image can be determined as the intermediate image based on the contour edge of the foreground image and the contour edge of the background image.

[0166] In some other embodiments, an image region formed by pixel points with a distance from the contour edge of the foreground image within a preset range can also be determined as the intermediate image.

[0167] For example, if the foreground image is a circle in the initial image with (cx, cy) as the center and R as the radius, an image region formed by pixel points with a distance greater than zero and less than 0.1R from the circle can be determined as the intermediate image.

[0168] As described above, in the representation matrix M of the initial image, values from 0 to 1 can be used to represent the intermediate image between the foreground and the background in the initial image. Therefore, in the above example, the representation matrix M of the initial image can be represented as formula (4):

[0169]

[0170] In formula (4), M(i, j) represents the pixel representation value of the pixel point at position (i, j) in the representation matrix M of the initial image, R represents the radius of the circle in the above example, and d represents the distance between the pixel point and the center of the circle in the above example.

[0171] Here, since the intermediate image is between the foreground image and the background image, different intensities of blurring can be performed on the intermediate image on the basis of blurring the background image, which can make the progression of the blurred image stronger.

[0172] Meanwhile, adjusting the pixel value of each pixel point in the intermediate image based on the pixel value of each pixel point in the foreground image and the pixel value of each pixel point in the blurred background image to obtain a blurred intermediate image can make the blurred intermediate image more strongly associated with the foreground image and the background image, thereby making the blurring effect of the blurred image better.

[0173] Figure 3 is a flowchart of a method for blurring an intermediate image according to an example embodiment of the present disclosure.

[0174] As Figure 3 shown, the step of adjusting the pixel value of each pixel point in the intermediate image based on the pixel value of each pixel point in the foreground image and the pixel value of each pixel point in the blurred background image to obtain a blurred intermediate image can include the following steps:

[0175] In step 301, a first weight value is used to weight the pixel value of each pixel point in the foreground image to obtain a first weighted pixel value; wherein each pixel point in the foreground image corresponds to a first weight value;

[0176] In step 302, a second weight value is used to weight the pixel value of each pixel point in the blurred background image to obtain a second weighted pixel value; wherein each pixel point in the blurred background image corresponds to a second weight value;

[0177] In step 303, based on the sum of the first weighted pixel value corresponding to each pixel point in the foreground image and the second weighted pixel value corresponding to each pixel point in the blurred background image, the pixel value of each pixel point in the intermediate image is adjusted to obtain a blurred intermediate image.

[0178] It can be understood that the foreground image, the blurred background image, the intermediate image and the initial image can all be represented by pixel matrices of the same size, and the positions of the pixel points of the images are the same in the pixel matrices. The difference between the pixel matrices is that the pixel values of the same pixel point in the pixel matrices can be different. For example, in the pixel matrix of the foreground image, the pixel value of the pixel point corresponding to the foreground is the same as the pixel value of the foreground in the initial image, and the pixel value of the pixel point corresponding to the background is 0. In the pixel matrix of the background image, the pixel value of the pixel point corresponding to the blurred background is the target pixel value, and the pixel value of the pixel point corresponding to the foreground is 0.

[0179] Therefore, weighting the pixel value of each pixel point in the foreground image by using the first weight value and weighting the pixel value of each pixel point in the blurred background image by using the second weight value can also be understood as weighting the first pixel value (i.e. the pixel value of the pixel point in the pixel matrix of the foreground image) of the same pixel point by using the first weight value and weighting the second pixel value (i.e. the pixel value of the pixel point in the pixel matrix of the background image) of the same pixel point by using the second weight value.

[0180] In some embodiments, the sum of the first weight value and the second weight value of each pixel point can be 1.

[0181] In some embodiments, the synthesis processing of the foreground image, the blurred background image and the blurred intermediate image to obtain the target image can be represented by formula (5), and the pixel matrix Z of the target image can be represented as:

[0182] Z=A+C*(1-M) (5);

[0183] In formula (5), Z represents a pixel matrix of the target image, A represents a pixel matrix of the foreground image, C represents a pixel matrix of the background image after the virtualization, and M represents a representation matrix of the target image output by the image segmentation network.

[0184] In this embodiment, the first weight value of each pixel point can be 1, and the second weight value can be 1-M.

[0185] Figure 4 is a flowchart of an image processing method according to an example embodiment of the present disclosure Figure Two . As shown in Figure 4 , the image processing method mainly includes the following steps:

[0186] Step 401, image analysis is performed on the initial image to obtain a foreground image and a background image.

[0187] In some embodiments, the initial image can be segmented by an image segmentation network to obtain the foreground image and the background image. The image segmentation network can be a network obtained by training an initial network (such as a U-Net network) through an iterative training method. When training the image segmentation network, the loss value of the initial network can be determined through a cross-entropy loss function, and then the parameters of the initial network are adjusted based on the loss value until the number of parameter adjustments reaches a threshold or the performance of the initial network meets a preset requirement, and the initial network after the last parameter adjustment is determined as the image segmentation network.

[0188] In some embodiments, after obtaining the initial image, the initial image can be preprocessed, and the size of the initial image is adjusted to a preset size, for example, the size of the initial image is adjusted to 512*512 pixels. Then, the initial image of the preset size (which can be in the form of a matrix, such as the pixel matrix I of the initial image) is input into the image segmentation network, and the image segmentation network can output a representation matrix M to represent the foreground image and the background image. Here, the value range of each value in the representation matrix M output by the image segmentation network can be [0, 1], wherein the value corresponding to the foreground in the representation matrix M is 1, the value corresponding to the background is 0, and the value between 0 and 1 can be used to represent the intermediate image between the foreground and the background in the initial image.

[0189] In some embodiments, the foreground image contour of the foreground image can also be determined according to the first input operation of the user; the area in the initial image within the foreground image contour is determined as the foreground image, and the area in the initial image outside the foreground image contour is determined as the background image.

[0190] In this embodiment, the first input operation can be a gesture operation, and the electronic device performing the image processing method can include an interactive module, such as a touch display, on which a user can outline the contour of the foreground image through a gesture operation. The electronic device can obtain the foreground image and the background image based on the contour of the foreground image.

[0191] In step 402, the initial image is input into a depth estimation model to obtain a depth matrix corresponding to the initial image, and the depth value of each pixel point in the background image is obtained from the depth matrix.

[0192] Here, the initial image can be adjusted to a preset size, for example, the size of the initial image is adjusted to 512*512 pixels. Then, the initial image of the preset size (which can be in the form of a matrix, such as the pixel matrix I of the initial image) is input into the depth estimation model, and the depth estimation model can output a 512*512 depth matrix D including the depth values of each pixel point in the initial image. Each depth value in the depth matrix D can have a value range of [0, 1], where 0 indicates that the pixel point with a depth value of 0 is closest to the camera when shooting, 1 indicates that the pixel point with a depth value of 1 is farthest from the camera when shooting, and the depth value between 0 and 1 is a transition zone, and the closer the depth value is to 1, the farther the pixel point is from the camera when shooting

[0193] In step 403, the target blur intensity corresponding to each pixel point in the background image is determined according to the depth value of each pixel point in the background image and the reference blur intensity.

[0194] In some embodiments, the depth matrix of the initial image can be represented as D, the reference blur intensity can be represented as BH, and the target blur intensity of each pixel point in the background image can be represented as K. The target blur intensity corresponding to each pixel point in the background image can be obtained by formula (3).

[0195] In step 404, the weight matrix corresponding to each pixel point in the background image is determined according to each target blur intensity.

[0196] It can be understood that the pixel value of the pixel point in the background image can be adjusted according to the weight matrix, thereby realizing the blurring of the background image. Generally, the larger the size of the weight matrix, the greater the blur intensity of the background image. In some embodiments, a mapping relationship between each blur intensity and the size of the weight matrix can be pre-set, and therefore, after determining the target blur intensity corresponding to each pixel point in the background image, the size of the weight matrix corresponding to each pixel point in the background image can be determined based on the pre-set mapping relationship between each blur intensity and the size of the weight matrix.

[0197] In some embodiments, the type of each weight matrix can be preset, for example, the weight matrix corresponding to each pixel point in the background image is set as a cross-shaped mean matrix (i.e., a matrix in which the middle row and the middle column have the same non-zero value, and the values of other positions are all 0), or the weight matrix corresponding to each pixel point in the background image is set as a general mean matrix (i.e., a matrix in which the values of each position are the same non-zero value). In this way, the size of each preset weight matrix can be determined according to the target blurring intensity corresponding to each pixel point in the background image, and then the weight matrix corresponding to each pixel point in the background image is obtained.

[0198] In step 405, a non-zero weight value is determined from the weight matrix corresponding to the current pixel point to be processed; each non-zero weight value is multiplied by the pixel value at the corresponding position in the pixel point matrix corresponding to the current pixel point to be processed to obtain the sum of the weighted pixel values, which is determined as the target pixel value of the current pixel point to be processed.

[0199] Here, a non-zero position matrix can be determined according to the position of each non-zero weight value in the weight matrix, where the non-zero position matrix includes the coordinate positions of each non-zero weight value in the weight matrix. For example, corresponding to the following weight matrix:

[0200]

[0201] The corresponding non-zero position matrix can be:

[0202] (-2, 0)

[0203] (-1, 0)

[0204] (0, -2) (0, -1) (0, 0) (0, 1) (0, 2)

[0205] (1, 0)

[0206] (2, 0)

[0207] In some embodiments, the target pixel value can be obtained by formula (6):

[0208]

[0209] In formula (6), C represents the pixel matrix of the blurred background image, K represents the weight matrix corresponding to the pixel point to be processed, C(i,j) represents the target pixel value of the pixel point at position (i,j) in the pixel matrix of the blurred background image, num represents the number of non-zero weight values in the weight matrix, K(x,y) represents the non-zero weight value at position (x,y) in the weight matrix, and B(i+x,j+y) represents the pixel value at the position corresponding to each non-zero weight value in the pixel matrix centered on the pixel point to be processed in the background image.

[0210] For the non-zero position matrix in the above example, (x,y) can be (-2,0), (-1,0), (0,0), (1,0), (2,0), (0,-2), (0,-1), (0,1), or (0,2).

[0211] Since K can be a mean matrix, each non-zero weight value of the mean matrix is equal, formula (6) can be transformed to obtain formula (7), and the target pixel value of each pixel point of the blurred background image is obtained through formula (7):

[0212]

[0213] In formula (7), C represents the pixel matrix of the blurred background image, C(i,j) represents the target pixel value of the pixel point at position (i,j) in the pixel matrix of the blurred background image, num represents the number of non-zero weight values in the weight matrix, and B(i+x,j+y) represents the pixel value at the position corresponding to each non-zero weight value in the pixel matrix centered on the pixel point to be processed in the background image.

[0214] In this way, the multiplication operation in the operation process can be reduced, and the operation speed can be improved.

[0215] Step 406: obtaining the blurred background image according to the target pixel value of each pixel point in the background image.

[0216] Step 407: synthesizing the foreground image, the blurred background image, and the blurred intermediate image to obtain the target image.

[0217] Here, the pixel matrix Z of the target image can be obtained through formula (5).

[0218] Thus, when M(i, j) = 1, Z(i, j) = A(i, j), i.e. the pixel value of the pixel point at the (i, j) position in the pixel matrix of the target image is equal to the pixel value of the pixel point at the (i, j) position in the pixel matrix of the foreground image, at this time Z(i, j) can be a pixel point belonging to the foreground image in the target image; when M(i, j) = 0, A(i, j) = I(i, j) * M(i, j) = 0, thus, Z(i, j) = C(i, j), i.e. the pixel value of the pixel point at the (i, j) position in the pixel matrix of the target image is equal to the pixel value of the pixel point at the (i, j) position in the pixel matrix of the blurred background image, at this time Z(i, j) can be a pixel point belonging to the background image in the target image; when 0 < M(i, j) < 1, i.e. the pixel value of the pixel point at the (i, j) position in the pixel matrix of the target image is obtained based on the pixel value of the pixel point in the foreground image and the pixel value of the pixel point in the blurred background image, at this time Z(i, j) can be a pixel point belonging to the intermediate image in the target image.

[0219] In the embodiments of the present disclosure, after the initial image is subjected to image analysis to obtain the foreground image and the background image, the target blurring intensity corresponding to each pixel point in the background image can be determined according to the depth value of each pixel point in the background image, then the pixel value of each pixel point in the background image is adjusted according to the target blurring intensity of each pixel point, to obtain the blurred background image, and finally the foreground image and the blurred background image are subjected to synthesis processing to obtain the target image.

[0220] In the embodiments of the present disclosure, in the first aspect, the blurring intensity can be determined according to the depth value of the pixel point in the background image, compared to one fixed blurring intensity corresponding to one image, by refining the image blurring to each pixel point, the blurring effect of the blurred image can be more layered. In the second aspect, the target blurring intensity corresponding to each pixel point in the background image can be determined according to the depth value of each pixel point in the background image, then the pixel value of each pixel point in the background image is adjusted according to the target blurring intensity of each pixel point in the background image, i.e. each pixel point in the background image corresponds to a target blurring intensity, compared to one blurring intensity corresponding to one region of the image, the gradualness of the blurring effect of the blurred image can be stronger. In the third aspect, by reducing the invalid operation and multiplication operation in the calculation process, the efficiency of data processing can be improved, and thus the efficiency of image blurring can be improved.

[0221] Figure 5 is a structural block diagram of an image processing device according to an exemplary embodiment of the present disclosure. As shown in Figure 5 the image processing device 500 mainly includes:

[0222] The image analysis module 501 is configured to perform image analysis on the initial image to obtain a foreground image and a background image.

[0223] The first determination module 502 is configured to determine target blur intensities corresponding to respective pixel points in the background image according to depth values of the respective pixel points in the background image.

[0224] The first adjustment module 503 is configured to adjust pixel values of the respective pixel points in the background image according to the target blur intensities to obtain a blurred background image.

[0225] The synthesis module 504 is configured to perform synthesis processing on the foreground image and the blurred background image to obtain a target image.

[0226] In some embodiments, the device further includes:

[0227] The depth estimation module is configured to input the initial image into a depth estimation model to obtain a depth matrix corresponding to the initial image; wherein the depth matrix includes depth values of respective pixel points in the initial image, and positions of the respective depth values in the depth matrix correspond to positions of the respective pixel points in the initial image.

[0228] The acquisition module is configured to acquire, based on positions of respective pixel points in the background image in the initial image, depth values of the respective pixel points in the background image from corresponding positions of the depth matrix.

[0229] In some embodiments, the first determination module 502 is configured to:

[0230] determine target blur intensities corresponding to respective pixel points in the background image according to the depth values of the respective pixel points in the background image and a reference blur intensity.

[0231] In some embodiments, the first adjustment module 503 includes:

[0232] The first determination unit is configured to determine a weight matrix corresponding to respective pixel points in the background image according to the target blur intensities.

[0233] The second determination unit is configured to determine, respectively, a pixel point matrix corresponding to each of the pixel points in the background image with each of the pixel points in the background image as a center; wherein positions of respective pixel points in the pixel point matrix correspond to positions of respective weight values in the weight matrix.

[0234] The convolution unit is configured to perform convolution processing on the respective pixel point matrices by using the respective weight matrices to obtain the blurred background image.

[0235] In some embodiments, the convolution unit is configured to:

[0236] determine non-zero weight values from the weight matrix corresponding to the current pixel point to be processed;

[0237] multiply each of the non-zero weight values with a pixel value at a corresponding position in a pixel matrix corresponding to the current pixel point to be processed to obtain a weighted pixel value;

[0238] determine a sum value of the weighted pixel values as a target pixel value of the current pixel point to be processed;

[0239] obtain the blurred background image according to the target pixel values of the pixel points in the background image.

[0240] In some embodiments, the image analysis module 501 is configured to:

[0241] determine a foreground image contour of the foreground image according to a first input operation of a user;

[0242] determine the foreground image as a region in the initial image within the foreground image contour;

[0243] determine the background image as a region in the initial image outside the foreground image contour.

[0244] In some embodiments, the device further comprises:

[0245] a second determination module configured to determine an intermediate image based on the contour edge of the foreground image and the contour edge of the background image;

[0246] a second adjustment module configured to adjust a pixel value of each pixel point in the intermediate image based on a pixel value of each pixel point in the foreground image and a pixel value of each pixel point in the blurred background image to obtain a blurred intermediate image;

[0247] the synthesis module is configured to synthesize the foreground image, the blurred background image and the blurred intermediate image to obtain the target image.

[0248] In some embodiments, the second adjustment module comprises:

[0249] a first weighting unit configured to perform weighting processing on the pixel value of each pixel point in the foreground image by using a first weight value to obtain a first weighted pixel value, wherein each pixel point in the foreground image corresponds to a first weight value;

[0250] a second weighting unit configured to perform weighting processing on pixel values of each pixel point in the blurred background image by using a second weight value to obtain a second weighted pixel value, wherein each pixel point in the blurred background image corresponds to a second weight value;

[0251] an adjusting unit configured to adjust pixel values of each pixel point in the intermediate image based on a sum value of the first weighted pixel value corresponding to each pixel point in the foreground image and the second weighted pixel value corresponding to each pixel point in the blurred background image to obtain a blurred intermediate image.

[0252] Figure 6 is a hardware structure block of an electronic device according to an exemplary embodiment of the present disclosure Figure One For example, the electronic device 600 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0253] Referring to Figure 6 The electronic device 600 can include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0254] The processing component 602 generally controls the overall operation of the electronic device 600 such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 can include one or more processors 620 to execute instructions to complete all or part of steps of the above methods. In addition, the processing component 602 can include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 can include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0255] The memory 604 is configured to store various types of data to support operations of the electronic device 600. Examples of these data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, videos, and the like. The memory 604 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0256] The power component 606 provides power to the various components of the electronic device 600. The power component 606 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 600.

[0257] The multimedia component 608 includes a screen providing an output interface between the electronic device 600 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0258] The audio component 610 is configured to output and / or input an audio signal. For example, the audio component 610 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting an audio signal.

[0259] The I / O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0260] The sensor component 614 includes one or more sensors for providing status assessments for various aspects of the electronic device 600. For example, the sensor component 614 can detect an open / closed position of the electronic device 600, relative positioning of components, such as a display and a keypad of the electronic device 600, a change in position of the electronic device 600 or a component of the electronic device 600, presence or absence of user contact with the electronic device 600, orientation or acceleration / deceleration / g-force and temperature of the electronic device 600. The sensor component 614 can include an optical sensor for detecting ambient light, a proximity sensor configured to detect proximity of an object, a motion sensor configured to detect motion of the electronic device 600, a position sensor configured to detect position of the electronic device 600, a temperature sensor configured to detect temperature of the electronic device 600, an acceleration sensor configured to detect acceleration of the electronic device 600, a gyroscope sensor configured to detect orientation of the electronic device 600, a magnetic sensor configured to detect magnetic field, a pressure sensor configured to detect pressure, or a chemical sensor configured to detect a chemical.

[0261] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and another device. The electronic device 600 can access a wireless network based on a communication standard, such as WI-FI, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 616 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0262] In an example embodiment, the electronic device 600 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-described methods.

[0263] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 604 including instructions, is also provided. The instructions can be executable by the processor 620 of the electronic device 600 to implement the above-described image processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0264] A kind of non-transitory computer readable storage medium, when the instruction in the storage medium is executed by the processor of electronic equipment, enable electronic equipment to execute a kind of image processing method, comprising:

[0265] Image analysis is carried out to initial image, and foreground image and background image are obtained;

[0266] According to the depth value of each pixel point in the background image, the target blurring intensity corresponding to each pixel point in the background image is determined;

[0267] According to each target blurring intensity, the pixel value of each pixel point in the background image is adjusted, and the background image after blurring is obtained;

[0268] The foreground image and the background image after blurring are synthesized and handled, and target image is obtained.

[0269] Figure 7 It is according to a kind of hardware structure frame of electronic equipment shown in an exemplary embodiment Figure Two For example, electronic equipment 700 can be provided as a server. Referring to Figure 7 Electronic equipment 700 includes processing component 722, which further includes one or more processors, and memory resource represented by memory 732, for storing the instructions, such as application program, executable by processing component 722.For example, application program stored in memory 732 can include one or more than one module each corresponding to a set of instructions.In addition, processing component 722 is configured to execute instructions to execute a kind of image processing method, comprising:

[0270] Image analysis is carried out to initial image, and foreground image and background image are obtained;

[0271] According to the depth value of each pixel point in the background image, the target blurring intensity corresponding to each pixel point in the background image is determined;

[0272] According to each target blurring intensity, the pixel value of each pixel point in the background image is adjusted, and the background image after blurring is obtained;

[0273] The foreground image and the background image after blurring are synthesized and handled, and target image is obtained.

[0274] The electronic device 700 can also include a power supply component 726 configured to perform power management for the electronic device 700, a wired or wireless network interface 750 configured to connect the electronic device 700 to a network, and an input / output (I / O) interface 757. The electronic device 700 can operate based on an operating system stored in the memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0275] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the concepts disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

[0276] It is to be understood that the disclosure is not limited to the precise construction described above and shown in the attached drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the disclosure is limited only by the claims that follow.

Claims

1. An image processing method, characterized by, The method comprises: performing image analysis on the initial image to obtain a foreground image and a background image; determining target blurring intensities corresponding to each pixel point in the background image according to depth values of each pixel point in the background image; adjusting pixel values of each pixel point in the background image according to each target blurring intensity to obtain a blurred background image; adjusting pixel values of each pixel point in an intermediate image based on pixel values of each pixel point in the foreground image and pixel values of each pixel point in the blurred background image to obtain a blurred intermediate image, wherein the intermediate image is between the foreground image and the background image; performing synthesis processing on the foreground image and the blurred background image to obtain a target image; wherein the synthesis processing on the foreground image and the blurred background image to obtain a target image comprises: performing synthesis processing on the foreground image, the blurred background image and the blurred intermediate image to obtain the target image.

2. The method of claim 1, wherein, The method further comprises: inputting the initial image into a depth estimation model to obtain a depth matrix corresponding to the initial image; wherein the depth matrix comprises depth values of each pixel point in the initial image, and positions of each depth value in the depth matrix correspond to positions of each pixel point in the initial image; obtaining depth values of each pixel point in the background image from corresponding positions of the depth matrix based on positions of each pixel point in the background image in the initial image.

3. The method of claim 1, wherein, The determination of target blurring intensities corresponding to each pixel point in the background image according to depth values of each pixel point in the background image comprises: determining target blurring intensities corresponding to each pixel point in the background image according to depth values of each pixel point in the background image and a reference blurring intensity.

4. The method of claim 1, wherein, The adjustment of pixel values of each pixel point in the background image according to each target blurring intensity to obtain a blurred background image comprises: determining a weight matrix corresponding to each pixel point in the background image according to each target blurring intensity; determining a pixel point matrix corresponding to each pixel point in the background image with each pixel point in the background image as the center; wherein positions of each pixel point in the pixel point matrix correspond to positions of each weight value in the weight matrix; performing convolution processing on each pixel point matrix by using each weight matrix to obtain the blurred background image.

5. The method of claim 4, wherein, The convolution processing on each pixel point matrix by using each weight matrix to obtain the blurred background image comprises: determining non-zero weight values from a weight matrix corresponding to a current pixel point to be processed; performing product processing on each non-zero weight value and a pixel value at a corresponding position in a pixel point matrix corresponding to the current pixel point to be processed to obtain a weighted pixel value; determining a sum value of each weighted pixel value as a target pixel value of the current pixel point to be processed; According to the target pixel value of each pixel point in the background image, the blurred background image is obtained.

6. The method of claim 1, wherein, The image analysis on the initial image to obtain the foreground image and the background image comprises: According to the first input operation of the user, a foreground image contour of the foreground image is determined; The region in the initial image within the foreground image contour is determined as the foreground image; The region in the initial image outside the foreground image contour is determined as the background image.

7. The method of claim 1, wherein, The method further comprises: Based on the contour edge of the foreground image and the contour edge of the background image, an intermediate image is determined.

8. The method of claim 7, wherein, The adjustment on the pixel value of each pixel point in the intermediate image based on the pixel value of each pixel point in the foreground image and the pixel value of each pixel point in the blurred background image to obtain a blurred intermediate image comprises: The pixel value of each pixel point in the foreground image is weighted by using a first weight value to obtain a first weighted pixel value; each pixel point in the foreground image corresponds to a first weight value; The pixel value of each pixel point in the blurred background image is weighted by using a second weight value to obtain a second weighted pixel value; each pixel point in the blurred background image corresponds to a second weight value; Based on the sum of the first weighted pixel value corresponding to each pixel point in the foreground image and the second weighted pixel value corresponding to each pixel point in the blurred background image, the pixel value of each pixel point in the intermediate image is adjusted to obtain a blurred intermediate image.

9. An image processing apparatus characterized by comprising: The device comprises: An image analysis module configured to perform image analysis on an initial image to obtain a foreground image and a background image; A first determination module configured to determine a target blurring intensity corresponding to each pixel point in the background image according to the depth value of each pixel point in the background image; A first adjustment module configured to adjust the pixel value of each pixel point in the background image according to each target blurring intensity to obtain a blurred background image; A second adjustment module configured to adjust the pixel value of each pixel point in an intermediate image based on the pixel value of each pixel point in the foreground image and the pixel value of each pixel point in the blurred background image to obtain a blurred intermediate image; the intermediate image is between the foreground image and the background image; A synthesis module configured to perform synthesis processing on the foreground image and the blurred background image to obtain a target image; The synthesis module is specifically configured to perform synthesis processing on the foreground image, the blurred background image and the blurred intermediate image to obtain the target image.

10. An electronic device, comprising: Comprise: A processor; A memory for storing processor-executable instructions; The processor is configured to implement the steps in the image processing method of any one of claims 1 to 8 when executed.

11. A non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enable the electronic device to perform the steps of any of the image processing methods of claims 1-8.

Images