Bulk scene fuzzy processing method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a bokeh fuzzy processing method and device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the invention, the background mask graph is determined according to the transparency graph of the to-be-processed image, the index calculation is performed on the background mask graph and the color value of each pixel point in the to-be-processed image to obtain the light source spread function, and the convolution mask is determined according to the light source spread function and the background mask graph. And performing mask convolution processing on the to-be-processed image according to the convolution mask and a preset convolution kernel to obtain a background blurred image, and performing fusion processing on the to-be-processed image and the background blurred image according to the transparency image to obtain a target blurred image. The unnatural fusion at the foreground and background junction is effectively reduced, the color leakage of pixel blurring at the foreground and background junction is reduced, the natural degree of the edge during the foreground and background fusion is improved, and the blurring effect of the scattered scene is effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of image processing, and in particular, to a method, device, equipment, storage medium and product for bokeh blur processing. Background Art

[0002] With the development of Internet technology and computer technology, the applications of social platforms based on pictures and videos are becoming more and more extensive, and people's requirements for the processing effects of pictures and videos are getting higher and higher. For example, it is hoped to achieve a bokeh blur simulation effect on the background while achieving clear imaging.

[0003] In photography, bokeh is called out-of-focus imaging. Since different object distances (the distance from the object to the lens) project different focal points onto the lens, for the objects on the focus plane, their light rays will accurately converge on the camera's sensor (or film) after passing through the lens to form a clear image. However, for the objects not on the focus plane, their light rays cannot accurately converge on the sensor after passing through the lens, but form a series of circles of confusion. When these circles of confusion are large enough and overlap with each other, the blurred effect we see will be produced, that is, bokeh blur. In the fields of computer vision and image processing, the simulation of bokeh blur is widely used in scenarios such as portrait mode, background blurring, and post-processing of photography. Through image processing algorithms, a bokeh effect similar to that produced by a lens is generated, and the light spots formed after the simulation of the light source points are blurred are used to enhance the expressiveness of the picture or highlight the main body of the picture. Bokeh blur needs to be carried out from three parts: image segmentation, simulation of blurring effect, and foreground-background fusion. However, the conventional bokeh blur algorithm often has the situation of color leakage when processing the pixel blurring at the foreground-background junction, resulting in unnatural edges during foreground-background fusion and poor bokeh blur effect. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, equipment, storage medium and product for bokeh blur processing, so as to solve the technical problem that the bokeh blur algorithm in the related technology often has the situation of color leakage when processing the pixel blurring at the foreground-background junction, resulting in unnatural edges during foreground-background fusion and poor bokeh blur effect, and can effectively reduce the color leakage of pixel blurring at the foreground-background junction, improve the natural degree of the edges during foreground-background fusion, and improve the bokeh blur effect.

[0005] In a first aspect, the embodiments of the present application provide a method for bokeh blur processing, including:

[0006] Determine the transparency map of the image to be processed, where the transparency map is used to reflect the degree to which each pixel point in the image to be processed belongs to the foreground;

[0007] Determine a background mask image according to the transparency map, perform an exponential calculation on the color values of each pixel point in the background mask image and the image to be processed to obtain a light source diffusion function, and multiply the light source diffusion function by the background mask image to obtain a convolution mask;

[0008] Perform mask convolution processing on the image to be processed according to the convolution mask and a preset convolution kernel to obtain a background blurred image;

[0009] Perform fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image.

[0010] In a second aspect, an embodiment of the present application provides a bokeh blur processing device, including a transparency determination module, a mask determination module, a blur processing module, and an image fusion module, where:

[0011] The transparency determination module is configured to determine a transparency map of the image to be processed, and the transparency map is used to reflect the degree to which each pixel point in the image to be processed belongs to the foreground;

[0012] The mask determination module is configured to determine a background mask image according to the transparency map, perform an exponential calculation on the color values of each pixel point in the background mask image and the image to be processed to obtain a light source diffusion function, and multiply the light source diffusion function by the background mask image to obtain a convolution mask;

[0013] The blur processing module is configured to perform mask convolution processing on the image to be processed according to the convolution mask and a preset convolution kernel to obtain a background blurred image;

[0014] The image fusion module is configured to perform fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image.

[0015] In a third aspect, an embodiment of the present application provides a bokeh blur processing device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the bokeh blur processing method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the bokeh blur processing method as described in the first aspect when executed by a computer processor.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program, so that the device executes the bokeh blur processing method as described in the first aspect.

[0020] In the embodiment of the present application, a background mask map is determined according to the transparency map of the image to be processed, an exponential calculation is performed on the color values of each pixel point in the background mask map and the image to be processed to obtain a light source diffusion function, and a convolution mask is determined according to the light source diffusion function and the background mask map. The image to be processed is subjected to masked convolution processing according to the convolution mask and a preset convolution kernel to obtain a background blurred image, and the image to be processed and the background blurred image are fused according to the transparency map to obtain a target blurred image. Among them, the convolution mask is determined according to the light source diffusion function and the background mask map. The masked convolution processing based on this convolution mask can effectively reduce the unnatural fusion at the foreground-background junction while achieving the uniform fusion of bokeh blur and defocus blur, reduce the color leakage of pixel blur at the foreground-background junction, improve the naturalness of the edge during foreground-background fusion, and effectively improve the bokeh blur effect. Description of the Drawings

[0021] Figure 1 is a flowchart of a bokeh blur processing method provided by an embodiment of the present application;

[0022] Figure 2 is a flowchart of another bokeh blur processing method provided by an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a bokeh blur effect provided by an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a material picture provided by an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of the structure of a bokeh blur processing device provided by an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of the structure of a bokeh blur processing device provided by an embodiment of the present application. Detailed Embodiments

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following provides a more detailed description of specific embodiments of this application with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application. Additionally, it should be noted that, for ease of description, only parts related to this application rather than all content are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the above-mentioned process can be terminated, but there can also be additional steps not included in the drawings. The above-mentioned process can correspond to a method, function, procedure, subroutine, subprogram, and so on.

[0028] The bokeh blur processing method provided by this application can be applied to scenarios such as video live streaming, video editing, and image editing. Its purpose is to determine a convolution mask for performing masked convolution processing on the image to be processed based on the light source diffusion function and the background mask map, so as to reduce the color leakage of pixel blurring at the foreground-background junction, improve the naturalness of the edge during foreground-background fusion, and enhance the bokeh blur effect.

[0029] In photography, bokeh is known as out-of-focus imaging. Due to different object distances (the distance from the object to the lens) projecting different focal points onto the lens, for objects on the focus plane, the light passing through the lens will accurately converge on the camera's sensor to form a clear image. However, for objects not on the focus plane, the light passing through the lens cannot accurately converge on the sensor but forms individual circles of confusion. When these circles of confusion are large enough and overlap with each other, the blurred effect we see, that is, bokeh blur, will be produced. In photography, the bokeh blur effect stems from characteristics such as the aperture shape of the camera lens, lens focal length, and optical distortion. The number and shape of the aperture blades determine the shape of the light spots in the bokeh, such as circular, hexagonal, or polygonal. The larger the aperture, the shallower the depth of field and the more obvious the background blur; the longer the focal length, the more significant the separation effect between the subject and the background. Bokeh brings a unique aesthetic feeling to the photo. For example, the blurred part presents round and soft light spots, and the edges of the light spots have varying degrees of gradual change, which may be a transition from bright to dim or may have a certain color halo, capable of creating a dreamy, soft, and layered visual atmosphere for the photo, highlighting the subject and guiding the viewer's line of sight to focus on the clearly focused part. In computer vision and image processing, the simulation of bokeh blur is widely used in scenarios such as portrait mode, background blur, and post-processing of photography. Through image processing algorithms, the computer can generate a bokeh effect similar to that produced by a lens, simulating the beautiful light spots formed after the light source points are blurred, for enhancing the expressiveness of the picture or highlighting the main subject of the picture.

[0030] In the existing bokeh blur processing solutions, it is mainly carried out based on three parts: image segmentation, virtualization effect simulation, and foreground and background fusion. Among them, image segmentation is required to separate the main part (foreground) that is desired to be highlighted from the non-main part (background) that is desired to be processed with bokeh blur in the image. Subsequently, the clarity of the foreground is protected to ensure the integrity of the foreground, and the background is processed to simulate the blurred virtualization effect. The virtualization effect simulation includes two parts: blur and fusion. The blur algorithm adopts a convolution-based solution, and a specific convolution kernel is designed to simulate the blur effect of the circle of confusion. The foreground and background fusion combines the clear foreground and the blurred and virtualized background to obtain a fused image with a clear foreground and a blurred background, completing the simulation of the camera bokeh blur. However, the blur effect of the conventional bokeh blur algorithm in the related technology is not flexible enough, and it is impossible to provide a simulation algorithm for light spots of different colors, different shapes, and different sizes through a general solution. The light spots generated by the blur are not vivid and natural enough, the virtualization effect is not uniform and beautiful enough, and color leakage often occurs when processing the pixel blur at the foreground and background junction, resulting in unnatural edges during foreground and background fusion and poor bokeh blur effect. Based on this, a bokeh blur processing method according to an embodiment of the present application is provided to solve the technical problem that color leakage often occurs when the existing bokeh blur algorithm processes the pixel blur at the foreground and background junction, resulting in unnatural edges during foreground and background fusion and poor bokeh blur effect, and to provide a bokeh blur processing solution with vivid, natural, flexible, and variable simulation effects.

[0031] Figure 1 The flowchart of a bokeh blur processing method provided by an embodiment of the present application is given. The bokeh blur processing method provided by an embodiment of the present application can be executed by a bokeh blur processing device, and the bokeh blur processing device can be implemented in a hardware and / or software manner and integrated in a bokeh blur processing device.

[0032] The following describes by taking the bokeh blur processing device executing the bokeh blur processing method as an example. Refer to Figure 1 , the bokeh blur processing method includes:

[0033] S110: Determine the transparency map of the image to be processed, and the transparency map is used to reflect the degree to which each pixel point in the image to be processed belongs to the foreground.

[0034] The image to be processed provided by the present application can be a single image waiting to be processed with bokeh blur, or an image frame in video information (the video information can be a complete video, or a video live broadcast, a video call video stream). After obtaining the image to be processed, the transparency map (alpha map) of the image to be processed can be obtained. The size of the transparency map is the same as the size of the image to be processed, and the transparency map is used to reflect the degree to which each pixel point in the image to be processed belongs to the foreground.

[0035] Among them, each pixel value (alpha value) in the transparency map represents the mixing ratio of the foreground and background at this pixel position, reflecting the degree to which each pixel belongs to the foreground. For example, 0 means the corresponding pixel is completely a background pixel, 1 means the corresponding pixel is completely a foreground pixel, and values between the two are used to represent semi-transparent or mixed areas.

[0036] Optionally, the transparency map of the image to be processed can be obtained by using the trained transparency map acquisition model, or the transparency map of the image to be processed can be determined based on the foreground and background segmentation results of the image. Among them, the foreground and background segmentation of the image can be performed based on image processing technologies such as image matting, the Segment Anything image segmentation model, the BiSeNet semantic segmentation network, the STDC real-time semantic segmentation network, and the SINet image segmentation network. For example, based on the foreground and background segmentation of the image, the foreground area can be extracted from the image to be processed based on the foreground and background segmentation of the image matting technology, and at the same time, the transparency (alpha) of each pixel is calculated, and a transparency map is generated based on the transparency of each pixel. That is, the video image segmentation based on image matting will provide a transparency map of the same size as the original image for each frame of the image (for example, training a deep neural network model based on the RobustMatting method to output the transparency map of each frame of the image to be processed in real time). This image matting technology is especially suitable for processing objects with complex boundaries (such as hair, blurred edges), making the segmentation result more accurate and natural.

[0037] S120: Determine the background mask map according to the transparency map, perform an exponential calculation on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and multiply the light source diffusion function by the background mask map to obtain the convolution mask.

[0038] Exemplarily, determine the background mask map of the image to be processed according to the transparency map determined above. The size of the background mask map is the same as the size of the image to be processed, and the background mask map can be used to reflect the degree to which each pixel point in the image to be processed belongs to the background. For example, subtracting the transparency map from 1 can obtain the background mask map. Optionally, the background mask map can be determined based on the following formula:

[0039] bg mask = 1 - aloha

[0040] where, bg mask is the background mask map, and aloha is the transparency map.

[0041] In one embodiment, an exponential calculation may be performed on the determined background mask image and the color values of each pixel point in the image to be processed to obtain a light source diffusion function. For example, after multiplying each pixel value in the background mask image by the color value of the corresponding pixel point in the image to be processed, the result is substituted into a preset exponential function to obtain the light source diffusion function.

[0042] After determining the light source diffusion function, the light source diffusion function and the background mask image may be multiplied, and the multiplication result is used as a convolution mask. The size of the convolution mask is the same as the size of the image to be processed. The value range of each pixel value in the convolution mask is 0 to 1, which can represent the diffusion fullness of different pixel points during sampling of the image to be processed. The convolution mask will participate in the bokeh blur processing of the image to be processed, and configure adaptive bokeh blur weights for pixels at different positions in the image to be processed, so as to reduce the color leakage of pixel blur at the foreground-background junction in the bokeh blur processing.

[0043] S130: Perform mask convolution processing on the image to be processed according to the convolution mask and a preset convolution kernel to obtain a background blurred image.

[0044] Exemplarily, after determining the convolution mask and the preset convolution kernel, mask convolution processing may be performed on the image to be processed according to the convolution mask and the preset convolution kernel to obtain a background blurred image. Among them, the mask convolution processing provided in this application can be understood as a variant of the convolution operation. The mask convolution processing introduces a convolution mask on the basis of the standard convolution to limit the calculation of certain parts of the preset convolution kernel on the image to be processed, that is, to control the weight application range of the preset convolution kernel through the mask mechanism, so as to achieve specific goals or constraints.

[0045] In mask convolution, which preset convolution kernel weights are applied can be controlled through the convolution mask, that is, which pixel points are sampled in the convolution. For example, a position where the convolution mask is 0 indicates that the convolution kernel weight w at this point is not applied, that is, this point is not sampled, and a convolution mask of 1 indicates that this point fully participates in the sampling of the convolution. Optionally, the following formula is used to perform mask convolution processing on the image to be processed:

[0046]

[0047] Among them, B blur (i,j) is the color value of the background blurred image B blur at the pixel point (i,j), M i+m,j+n is the mask value of the convolution mask M at the pixel point (i+m,j+n), W m,n is the element value of the preset convolution kernel W with a size of K*K at the pixel point (m,n), and x i+m,j+n is the color value of the image to be processed x at the pixel point (i+m,j+n). Assume the background blurred image Bblur If the resolutions of the convolutional mask M and the image x to be processed are both w×h, then the value range of i is [0, w), the value range of j is [0, h), the size of the convolutional kernel is K×K, and the value ranges of m and n are [-(k - 1) / 2, (k - 1) / 2]. When the pixel point (i, j) is at the boundary, the coordinate pixel point (i + m, j + n) will be outside the range of the image. Boundary value processing can be performed on the pixel point (i + m, j + n). For example, zero-padding processing can be performed on the boundary of the pixel point (i + m, j + n). M×W can be regarded as the actual weight of each pixel during convolution. Optionally, the weight M×W can be normalized. B blur (i, j) is x i+m,j+n The result of masked convolution of (i, j) with the surrounding pixel points according to the preset convolutional kernel W and the convolutional mask M.

[0048] Among them, after obtaining the transparency map from the foreground-background segmentation module, the background obtained by segmenting the transparency map can be blurred. Most of the existing implementations of bokeh blur use convolution-based methods, and a specific convolutional kernel is set to simulate the blurred effect of the circle of confusion. For example, a simple circular convolutional kernel can simulate the circular light spot of bokeh blur to a certain extent. However, an ordinary uniform filtering kernel cannot form a distinct and prominent light spot. In this application, non-uniform weighted convolution is performed on the pixels during convolution, and finally, the blurred background part after bokeh blur processing is fused with the clear foreground part to obtain an image with a clear foreground and a blurred background. By introducing a convolutional mask, masked convolution can only calculate the area allowed by the convolutional mask, select specific pixel points for sampling, and ignore the pixel points that are not desired to be sampled, thereby realizing selective processing of specific positions. By introducing masked convolution, this application can effectively handle the color leakage problem that occurs when sampling pixels at the foreground-background junction. When performing convolution on the points at the junction, some pixel values in the preset convolutional kernel are in the foreground and some are in the background. If a conventional convolutional kernel is directly used to calculate all the pixel points in the area, it will cause some foreground pixels to participate in the blurring of the background. In this application, by setting the mask value of the foreground area to 0 in the convolutional mask, only the pixel points in the background area can be calculated when simulating blurring through convolution, ignoring the foreground pixels, reducing the color leakage of pixel blurring at the foreground-background junction, and obtaining a clean and clear foreground-background edge.

[0049] S140: Perform fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain the target blurred image.

[0050] Exemplarily, after obtaining the background blurred image, the image to be processed and the background blurred image may be fused according to the determined transparency map above to obtain the target blurred image. Optionally, the fusion of the image to be processed and the background blurred image may be based on alpha blending, Poisson fusion, Laplacian pyramid fusion, etc.

[0051] For example, determine the fusion ratio of each corresponding pixel point of the image to be processed and the background blurred image according to the transparency map, and perform fusion processing on each corresponding pixel point of the image to be processed and the background blurred image according to the corresponding fusion ratio. The fused pixel points are the pixel points at the corresponding positions of the target blurred image. Since the transparency map participates in the fusion processing of the image to be processed and the background blurred image, the target blurred image can achieve the bokeh blur of the background while ensuring the clear display of the foreground.

[0052] In a possible embodiment, before determining the transparency map of the image to be processed, the bokeh blur processing method provided in this application may also convert the color space of the image to be processed from the original color space to a preset color space. Correspondingly, the bokeh blur processing method provided in this application determines the transparency map of the image to be processed, which may be: determining the transparency map of the image to be processed after being converted to the preset color space. After fusing the image to be processed and the background blurred image according to the transparency map to obtain the target blurred image, the color space of the target blurred image may be converted from the preset color space to the original color space.

[0053] Exemplarily, after obtaining the image to be processed, convert the color space of the image to be processed from the original color space to a preset color space. The preset color space provided in this application may be the RGB color space. For example, if the original color space of the image to be processed is the YUV color space (for example, in other image processing processes, the color space of the image will be converted to the YUV color space), then convert the image to be processed from the YUV color space to the RGB color space.

[0054] After completing the conversion of the color space of the image to be processed, the subsequent bokeh blur processing process may be performed based on the image to be processed after the color space conversion. After obtaining the target blurred image, the color space of the target blurred image may be converted from the preset color space to the original color space. For example, convert the target blurred image from the RGB color space to the YUV color space. Optionally, the pixel values of the image to be processed may also be normalized to the range of 0 to 1, and the subsequent bokeh blur processing process may be performed based on the image to be processed after the pixel value normalization. After obtaining the target blurred image, the pixel values of the target blurred image may be mapped back to the original pixel value range. By performing bokeh blur processing on the image in the preset color space (such as the RGB color space), this application forms more colorful light spots on the image, improving the bokeh blur effect.

[0055] As described above, by determining the background mask map according to the transparency map of the image to be processed, performing exponential calculations on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and determining the convolution mask according to the light source diffusion function and the background mask map, performing mask convolution processing on the image to be processed according to the convolution mask and a preset convolution kernel to obtain the background blurred image, and performing fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain the target blurred image. Among them, the convolution mask is determined according to the light source diffusion function and the background mask map. Based on the mask convolution processing of this convolution mask, while achieving uniform fusion of bokeh blur and defocus blur, it can effectively reduce the unnatural fusion at the foreground-background junction, reduce the color leakage of pixel blur at the foreground-background junction, improve the naturalness of the edge during foreground-background fusion, and effectively improve the bokeh blur effect.

[0056] Based on the above embodiments, Figure 2 a flowchart of another bokeh blur processing method provided by an embodiment of the present application is given. This bokeh blur processing method is a specific implementation of the above bokeh blur processing method. Refer to Figure 2 and this bokeh blur processing method includes:

[0057] S210: Determine the transparency map of the image to be processed. The transparency map is used to reflect the degree to which each pixel point in the image to be processed belongs to the foreground.

[0058] S220: Determine the background mask map according to the transparency map, perform exponential calculations on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and multiply the light source diffusion function by the background mask map to obtain the convolution mask.

[0059] In a possible embodiment, the background mask map and the pixel values of the image to be processed can be used together for exponential calculation to obtain the light source diffusion function. Based on this, for the bokeh blur processing method provided by the present application to perform exponential calculations on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, it can be: calculate the first product of the color values of the corresponding pixel points in the background mask map and the image to be processed, calculate the first parameter power of the first product, and determine the calculation result as the light source diffusion function. For example, the light source diffusion function can be determined by the following formula:

[0060] Bokeh1 blur =(x * bg mask ) θ

[0061] bg mask =1 - alpha

[0062] where Bokeh1 bluris the light source diffusion function, x is the image to be processed, and bg maak is the background mask image, θ is the first parameter, and alpha is the transparency map. Among them, the first parameter θ is used as the exponential parameter of the light source diffusion function Bokeh1 blur The larger the first parameter θ, the higher the weight obtained by the bright pixels during convolution, and the more obvious the light spot. Optionally, the point light source diffusion function can be multiplied by the background mask as the convolution mask for masked convolution:

[0063] mask = bg mask * Bokeh1 blur

[0064] Using this convolution mask to perform masked convolution processing on the image to be processed can achieve the effect of bokeh blur. Among them, the background mask image bg mask represents the proportion of pixels belonging to the background. Points that completely belong to the background are 1, points that completely belong to the foreground are 0, and values between the two are used to represent semi-transparent or foreground-background mixed areas. Exponentiating the pixel values of the background mask image and the image to be processed together can effectively prevent color leakage at the foreground-background junction. Points at the junction will have a lower weight value after exponentiating with the background mask image, thus effectively preventing foreground color leakage.

[0065] In a possible embodiment, the bokeh blur processing method provided by this application calculates the light source diffusion function by exponentiating the color values of each pixel point in the background mask image and the image to be processed. It can also be: calculating the second product of the color values of the corresponding pixel points in the background mask image and the image to be processed, adding the second parameter power of the second product to a preset fusion ratio coefficient, and determining the added result as the light source diffusion function. For example, the light source diffusion function can be determined by the following formula:

[0066] fusion blur = Bokeh2 blur + defocus blur =(x * bg mask ) β + δ

[0067] Bokeh2 blur =(x * bg mask ) β

[0068] defocus blur = δ

[0069] bg mask = 1 - alpha

[0070] Among them, fusion blur is the light source diffusion function, Bokeh2blur is the bokeh blur function, defocus blur is the defocus blur function, x is the image to be processed, bg mask is the background mask image, alpha is the transparency map, β is the second parameter, δ is the preset fusion ratio coefficient, and the value range of the preset fusion ratio coefficient is 0 to 1. For example, δ = 0.1, so that the bokeh blur can produce natural and vivid light spots without obvious color erosion problems. Among them, the second parameter β is used as the exponential parameter of the bokeh blur function Bokeh2 blur The larger the second parameter β, the higher the weight obtained by the bright pixels during convolution, and the more obvious the light spots. Optionally, the point light source diffusion function can be multiplied by the background mask as the convolution mask for masked convolution:

[0071] mask = bg mask *fusion blur

[0072] Using this convolution mask to perform masked convolution on the image to be processed can achieve a bokeh blur effect with a uniform and beautiful defocus effect.

[0073] Among them, performing exponential calculation on the pixel values of the background mask image and the image to be processed can effectively prevent color leakage at the junction of the foreground and background. However, there may be a situation where the defocus effect is not uniform and beautiful enough. Especially when there are objects with large color contrasts in the background, the bokeh blur will cause obvious color erosion of the bright area to the dark area, forming an unpleasant subjective feeling. This application can solve the above problems by performing a hybrid superposition of bokeh blur and defocus blur on the image to be processed. Among them, defocus blur is a mean blur with a circular convolution kernel. In the case where the point light source diffusion function is a constant, using a circular convolution kernel kernel and a background mask mask for masked convolution, the result of defocus blur is obtained. If the image to be processed is subjected to masked convolution twice (once for bokeh blur and once for defocus blur), and then the results of the two blurs are superimposed, the phenomenon of color erosion in the dark area can be alleviated, but there are two problems: one is that the two convolutions are time-consuming and laborious, especially when the convolution kernel is large, and doing one more convolution wastes a lot of computing power; the other is that the superposition ratio of the two blur results is difficult to control. If the proportion of bokeh blur is too high, it may cause color erosion, and if the proportion of bokeh blur is too low, obvious defocus light spots cannot be formed. This application provides a light source diffusion function to achieve the superposition of bokeh blur and defocus blur in one masked convolution, and the superposition ratio of bokeh blur and defocus blur at each point is adaptive, which can adaptively process the degree of light source diffusion of different brightness pixels without manual adjustment.

[0074] Among them, under the action of the light source diffusion function fusion blur the final convolution can be expressed as:

[0075]

[0076] For the convenience of subsequent derivations, the convolution forms of bokeh and defocus as diffusion functions are introduced here separately:

[0077]

[0078] Continue to calculate the convolution result when bokeh blur is superimposed on defocus blur:

[0079]

[0080]

[0081] Record the third parameter:

[0082]

[0083] Record the fourth parameter: μ = ∑∑bg mask *W

[0084]

[0085] From the above derivations, it can be seen that the final result y of the superimposed blur fusion is the result of the weighted fusion of bokeh blur y bokeh and defocus blur y defoucs in proportion. The fusion ratio is related to the pixel value x of each point and can adaptively process the diffusion degree of light sources with different brightness pixels.

[0086] The fusion ratio of bokeh blur y bokeh is:

[0087]

[0088] For the brighter pixel points in the image to be processed, the larger the pixel value x, the larger the third parameter is, and the larger the ratio of y fusion in y bokeh . At this time, the effect of bokeh blur dominates, ensuring that obvious light spots are formed in the bright area; for the darker pixel points in the image to be processed, the smaller the pixel value x, the smaller the third parameter is, and the larger the ratio of y fusion in y defocus . At this time, the effect of defocus blur dominates, effectively avoiding color leakage. The preset fusion ratio coefficient δ determines the maximum ratio that bokeh blur y bokeh can reach during fusion. The smaller the preset fusion ratio coefficient δ, the ratio bokehThe larger the value range is, the higher the maximum weight that the bokeh blur can reach, and the more obvious the light spots in the bright areas are; conversely, the lower the maximum weight that the bokeh blur can reach, the weaker the light spot effect, and at the same time, the defocus blur protects the dark areas from color erosion. As Figure 3 As shown in the schematic diagram of the bokeh blur effect provided, where the left side is the image to be processed, and the middle is based on the light source diffusion function Bokeh1 blur The effect of performing bokeh blur on the image to be processed, and the right side is the effect of performing superimposed blur on the image to be processed based on the light source diffusion function fusion blur The effect of performing superimposed blur on the image to be processed. It can be seen that, compared with the simple bokeh blur, the superimposed blur processing of the image to be processed based on the light source diffusion function fusion blur can effectively solve the problems of unnatural diffusion effect and color erosion in the dark areas when simulating the defocus effect.

[0089] S230: Determine the blur radius and the light spot shape, and generate a preset convolution kernel according to the blur radius and the light spot shape. The preset convolution kernel is used to perform masked convolution processing on the image to be processed.

[0090] In a possible embodiment, before performing masked convolution processing on the image to be processed according to the convolution mask and the preset convolution kernel to obtain the background blur image, the blur radius and the light spot shape for performing bokeh blur on the image to be processed can be determined. Among them, the blur radius and / or the light spot shape can be the default set blur radius and / or light spot shape, or the blur radius and / or light spot shape selected or configured by the user. By setting the size of the blur radius, the radius of the simulated circle of confusion in the bokeh blur can be configured, and the size of the blur radius determines the size of the defocused blur light spot.

[0091] Optionally, the blur radius can be set to a fixed value. For example, the blur radius radius = 10. The blur radius can also be set to a value related to the image resolution size of the image to be processed. For example, the blur radius radius = 0.02 * min(width, height), where (width, height) is the width and height of the image to be processed. The larger the image resolution, the larger the blur radius, and the smaller the image resolution, the smaller the blur radius, which can make the light spots of relatively consistent size appear at different resolutions. The blur radius can also be set to a value related to the depth information of the image. The size of the light spot is related to the distance of the pixel point from the focal plane. If you want to simulate the bokeh defocus effect in the real world as much as possible, the blur radius can be set in combination with the depth information of the picture. Among them, the greater the depth, the farther the pixel point is from the focal plane, the larger the blur radius, and the larger the light spot radius.

[0092] After determining the blur radius and the spot shape, a preset convolution kernel can be generated according to the blur radius and the spot shape, so as to perform masked convolution processing on the image to be processed using the preset convolution kernel. Among them, the size of the preset convolution kernel can be obtained by processing the blur radius based on a preset function, and the values of the elements in the preset convolution kernel can be determined according to the position relationship between the element and the spot shape. For example, the value of the element within the position of the spot shape is set to 1, and the values of other elements are set to 0.

[0093] This application can simulate the blur effect of a diffusion circle of any shape by designing a specific preset convolution kernel. The preset convolution kernel is a matrix of size K*K, for example, configured as K = 2*radius + 1. Optionally, the preset convolution kernel can be obtained by being constrained by a mathematical function or by means of a texture map.

[0094] In one embodiment, for obtaining the preset convolution kernel by being constrained by a mathematical function, the bokeh blur processing method provided by this application for generating the preset convolution kernel according to the blur radius and the spot shape can be: creating an initial matrix with the sum of the preset multiple of the blur radius and the preset coefficient as the side length of the convolution kernel; determining the position relationship between each pixel point on the initial matrix and the preset shape function corresponding to the spot shape, and determining the pixel value of each pixel point on the initial matrix according to the position relationship; performing normalization processing on the initial matrix to obtain the preset convolution kernel.

[0095] Exemplarily, create an initial matrix with the sum of the preset multiple of the blur radius and the preset coefficient as the side length of the convolution kernel. After creating the initial matrix, determine the position relationship between each pixel point on the initial matrix and the preset shape function corresponding to the spot shape, and this position relationship can reflect whether the pixel point on the initial matrix is located within the shape enclosed by the preset shape function, where the center point of the shape enclosed by the preset shape function can coincide with the center point of the initial matrix. Determine the pixel value of each pixel point on the initial matrix according to the above determined position relationship. For example, when the pixel point on the initial matrix is located within the shape enclosed by the preset shape function, the pixel value corresponding to this pixel point can be set to 1, and when the pixel point on the initial matrix is located outside the shape enclosed by the preset shape function, the pixel value corresponding to this pixel point can be set to 0. After completing the setting of the pixel values of each pixel point in the initial matrix, the initial matrix can be normalized to obtain the preset convolution kernel.

[0096] For example, the side length of the convolution kernel can be determined by the following formula: K = a * radius + b, where a is a preset multiple, for example, a = 2, and b is a preset coefficient, for example, b = 1. Taking the mathematical formula corresponding to the spot shape as the preset shape function, each pixel at each position in the initial matrix is constrained by the preset shape function to obtain an initial convolution kernel corresponding to the spot shape. For example, setting the blur radius radius = 5 and wanting to implement a circular preset convolution kernel, a K * K all-zero matrix can be created as the initial matrix, and the distance dis from each pixel point to the center of the circle (the center point of the matrix) is calculated:

[0097] dis = x 2 + y 2

[0098]

[0099] Finally, the obtained convolution kernel is normalized:

[0100]

[0101] where x is the horizontal distance from the pixel point to the center of the circle, y is the vertical distance from the pixel point to the center of the circle, kernel i,j is the pixel value of the pixel point (i, j) in the initial matrix, and r is the radius of the circle. In this application, a preset convolution kernel is obtained by constraining the shape of the convolution kernel through a preset shape function, which can achieve more flexible generation of the convolution kernel and provide a bokeh blur effect with a richer spot shape.

[0102] In one embodiment, for obtaining a preset convolution kernel by means of a material map, the bokeh blur processing method provided by this application generates a preset convolution kernel according to the blur radius and the spot shape, including:

[0103] Taking the sum of the preset multiple of the blur radius and the preset coefficient as the side length of the convolution kernel;

[0104] Scaling the material image corresponding to the spot shape according to the side length of the convolution kernel;

[0105] Rotating the scaled material image by 180°, and setting the pixel values of each pixel point in the material image according to the positional relationship between each pixel point in the rotated material image and the spot shape;

[0106] Normalizing the material image with set pixel values to obtain a preset convolution kernel.

[0107] Exemplarily, the sum of the blur radius of the preset multiple and the preset coefficient is calculated, and the calculation result is used as the side length of the preset convolution kernel. The material picture corresponding to the spot shape is obtained, and the material picture is scaled to the side length of the preset convolution kernel, so that the size of the scaled material picture is consistent with the size of the preset convolution kernel (i.e., K*K). In order to ensure that the light spot after convolution presents a positive shape (when the shape of the convolution kernel is constrained by the preset shape function, the rotation of the shape can be considered in advance in the determination of the preset shape function), the scaled material picture can be rotated 180°. After completing the rotation of the material picture, the pixel value of each pixel in the material picture can be set according to the positional relationship between each pixel in the rotated material picture and the spot shape. For example, when the positional relationship is that the pixel in the material picture is within the spot shape, the pixel value of the corresponding pixel is set to 1, and when the positional relationship is that the pixel in the material picture is outside the spot shape, the pixel value of the corresponding pixel is set to 0, and the material picture after setting the pixel value is normalized to obtain the preset convolution kernel.

[0108] For example, Figure 4 As shown in the schematic diagram of a material picture provided, if you need to implement a heart-shaped preset convolution kernel, you can provide the following Figure 4 A heart-shaped material picture is taken, the material picture is converted into a grayscale image and scaled to a convolution kernel size of K*K, the material picture is rotated 180 degrees to ensure that a positive heart shape is presented after convolution, and the pixel values ​​of the rotated material picture are processed, wherein the heart-shaped area is set to 1 and the blank part is set to 0, and then the material picture is normalized to obtain a preset convolution kernel. When bokeh blur processing is performed based on the preset convolution kernel, a heart-shaped light spot is generated. The present application generates a preset convolution kernel through a material picture, which can achieve more flexible convolution kernel generation and provide a richer bokeh blur effect with a richer light spot shape.

[0109] S240: performing mask convolution processing on the image to be processed according to the convolution mask and the preset convolution kernel to obtain a background blurred image.

[0110] S250: performing fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image.

[0111] The present application obtains the foreground transparency map alpha by segmenting the foreground and background of the processed image, and simulates the blur effect of the processed image to obtain the background blur effect map B blur Finally, the image to be processed I originalThe clear foreground and the blurred background are fused to obtain the final bokeh blur effect. Optionally, a transparency blending (alpha blending) scheme can be used to fuse the image to be processed and the background blurred image. Since the transparency map alpha provides the proportion of pixels belonging to the foreground, pixels with alpha = 1 belong entirely to the foreground, pixels with alpha = 0 belong entirely to the background, and points between 0 and 1 are at the foreground-background boundary. The foreground and background can be accurately fused based on the transparency map. Based on this, the bokeh blur processing method provided in this application fuses the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image. It can be: determining a first fusion ratio and a second fusion ratio according to the transparency map, and adding the product of the first fusion ratio and the image to be processed to the product of the second fusion ratio and the background blurred image to obtain the target blurred image.

[0112] Optionally, the first fusion ratio corresponding to each pixel point can be the pixel value at the corresponding pixel position of the transparency map, and the first fusion ratio can be the difference between 1 and the pixel value at the corresponding pixel position of the transparency map. Based on this, the target blurred image can be determined by the following formula:

[0113] I final =I original ×alpha+B blur ×(1-alpha)

[0114] Where, I final is the target blurred image, I original is the image to be processed, B blur is the background blurred image, and alpha is the transparency map. Through the above formula, a target blurred image with a clear foreground, vivid and natural blurred background highlights, and a uniform and beautiful blur effect can be obtained.

[0115] As described above, by determining the background mask map according to the transparency map of the image to be processed, performing exponential calculations on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and determining the convolution mask according to the light source diffusion function and the background mask map, performing mask convolution processing on the image to be processed according to the convolution mask and the preset convolution kernel to obtain the background blurred image, and performing fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain the target blurred image. Among them, the convolution mask is determined according to the light source diffusion function and the background mask map. The mask convolution processing based on this convolution mask can effectively reduce the unnatural fusion at the foreground-background junction while achieving uniform fusion of bokeh blur and defocus blur, reduce the color leakage of pixel blur at the foreground-background junction, improve the naturalness of the edge during foreground-background fusion, and effectively improve the bokeh blur effect. At the same time, for the anchor, bokeh blur can weaken the picture information in the background while protecting the foreground, and can protect the personal privacy of the anchor in some scenarios (such as room interior decoration, sensitive documents, etc.), guide the viewer's visual focus to the anchor, avoid the background content from distracting the viewer's attention, and make the live broadcast screen look more professional, creating a professional and high-end live broadcast atmosphere and increasing the interest of the live broadcast; for the viewer, the visual focus is more concentrated on the anchor and will not be affected by the cluttered background, enabling the live broadcast viewers to enjoy high-quality live broadcast content. In addition, applying bokeh blur in the live broadcast software reduces the rendering burden of the device on the complex background of the entire picture, making the live broadcast process more stable and providing a relatively good viewing experience for some devices with lower performance. For the service provider, the reduction of background details in video live broadcast means a reduction in the amount of data to be transmitted, reducing the bandwidth cost. Especially in the case of limited network bandwidth, this optimization of the data volume can reduce problems such as video stuttering and latency, and improve the smoothness of the live broadcast.

[0116] Figure 5 is a schematic structural diagram of a bokeh blur processing device provided by an embodiment of the present application. Refer to Figure 5 As shown in the figure, the bokeh blur processing device includes a transparency determination module 51, a mask determination module 52, a blur processing module 53, and an image fusion module 54.

[0117] Among them, the transparency determination module 51 is configured to determine the transparency map of the image to be processed, and the transparency map is used to reflect the degree to which each pixel point in the image to be processed belongs to the foreground;

[0118] The mask determination module 52 is configured to determine the background mask map according to the transparency map, perform exponential calculations on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and multiply the light source diffusion function by the background mask map to obtain the convolution mask;

[0119] The blurring module 53 is configured to perform masked convolution processing on the image to be processed according to the convolution mask and a preset convolution kernel to obtain a background blurred image;

[0120] The image fusion module 54 is configured to perform fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image.

[0121] As described above, by determining the background mask map according to the transparency map of the image to be processed, performing exponential calculation on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, determining the convolution mask according to the light source diffusion function and the background mask map, performing masked convolution processing on the image to be processed according to the convolution mask and the preset convolution kernel to obtain the background blurred image, and performing fusion processing on the image to be processed and the background blurred image according to the transparency map to obtain the target blurred image, wherein the convolution mask is determined according to the light source diffusion function and the background mask map, and the masked convolution processing based on the convolution mask can, while achieving uniform fusion of bokeh blur and defocus blur, effectively reduce the unnatural fusion at the foreground-background junction, reduce the color leakage of pixel blur at the foreground-background junction, improve the naturalness of the edge during foreground-background fusion, and effectively improve the bokeh blur effect.

[0122] In a possible embodiment, the mask determination module 52 performs exponential calculation on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and is configured to:

[0123] Calculate the first product of the color values of the corresponding pixel points in the background mask map and the image to be processed, calculate the first parameter power of the first product, and determine the calculation result as the light source diffusion function.

[0124] In a possible embodiment, the mask determination module 52 performs exponential calculation on the color values of each pixel point in the background mask map and the image to be processed to obtain the light source diffusion function, and is configured to:

[0125] Calculate the second product of the color values of the corresponding pixel points in the background mask map and the image to be processed, add the second parameter power of the second product to the preset fusion ratio coefficient, and determine the addition result as the light source diffusion function.

[0126] In a possible embodiment, the blurring module 53 performs masked convolution processing on the image to be processed according to the following formula:

[0127]

[0128] where B blur (i,j) is the color value at (i,j) in the background blurred image B blur in, M i+m,j+nis the mask value of the convolution mask M at (i + m, j + n), W m,n is the element value of the preset convolution kernel at (m, n), x i+m,j+n is the color value of the image to be processed at (i + m, j + n).

[0129] In a possible embodiment, the image fusion module 54 fuses the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image, and is configured as follows:

[0130] Determine a first fusion ratio and a second fusion ratio according to the transparency map, and add the product of the first fusion ratio and the image to be processed to the product of the second fusion ratio and the background blurred image to obtain the target blurred image.

[0131] In a possible embodiment, the bokeh blur processing device further includes a convolution kernel generation module, and the convolution kernel generation module is configured as follows: determine the blur radius and the spot shape, and generate a preset convolution kernel according to the blur radius and the spot shape, and the preset convolution kernel is used to perform mask convolution processing on the image to be processed.

[0132] In a possible embodiment, the convolution kernel generation module generates a preset convolution kernel according to the blur radius and the spot shape, and is configured as follows:

[0133] Create an initial matrix with the sum of the preset multiple of the blur radius and the preset coefficient as the side length of the convolution kernel;

[0134] Determine the positional relationship between each pixel point on the initial matrix and the preset shape function corresponding to the spot shape, and determine the pixel value of each pixel point on the initial matrix according to the positional relationship;

[0135] Perform normalization processing on the initial matrix to obtain a preset convolution kernel.

[0136] In a possible embodiment, the convolution kernel generation module generates a preset convolution kernel according to the blur radius and the spot shape, and is configured as follows:

[0137] Take the sum of the preset multiple of the blur radius and the preset coefficient as the side length of the convolution kernel;

[0138] Scale the material picture corresponding to the spot shape according to the side length of the convolution kernel;

[0139] Rotate the scaled material picture by 180°, and set the pixel value of each pixel point in the material picture according to the positional relationship between each pixel point in the rotated material picture and the spot shape;

[0140] Perform normalization processing on the material picture with the pixel values set to obtain a preset convolution kernel.

[0141] In a possible embodiment, the bokeh blur processing device further includes a color space conversion module configured to convert the color space of the image to be processed from the original color space to a preset color space;

[0142] Correspondingly, the transparency determination module 51 determines the transparency map of the image to be processed, configured to determine the transparency map of the image to be processed after conversion to the preset color space;

[0143] The color space conversion module is further configured to convert the color space of the target blurred image from the preset color space to the original color space.

[0144] It should be noted that in the above embodiments of the bokeh blur processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present application.

[0145] The embodiments of the present application also provide a bokeh blur processing device, which can integrate the bokeh blur processing device provided by the embodiments of the present application. Figure 6 is a schematic structural diagram of a bokeh blur processing device provided by the embodiments of the present application. Refer to Figure 6 , the bokeh blur processing device includes: an input device 63, an output device 64, a memory 62, and one or more processors 61; the memory 62 is used to store one or more programs; when the one or more programs are executed by the one or more processors 61, the one or more processors 61 implement the bokeh blur processing method provided by the above embodiments. The bokeh blur processing device, device, and computer provided above can be used to execute the bokeh blur processing method provided by any of the above embodiments, and have the corresponding functions and beneficial effects.

[0146] The embodiments of the present application also provide a non-volatile storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the bokeh blur processing method provided by the above embodiments when executed by a computer processor. Of course, for the non-volatile storage medium storing computer-executable instructions provided by the embodiments of the present application, the computer-executable instructions are not limited to the bokeh blur processing method provided above, and can also execute related operations in the bokeh blur processing methods provided by any embodiments of the present application. The bokeh blur processing device, device, and storage medium provided in the above embodiments can execute the bokeh blur processing method provided by any embodiment of the present application. For technical details not described in detail in the above embodiments, reference can be made to the bokeh blur processing method provided by any embodiment of the present application.

[0147] Based on the above embodiments, an embodiment of the present application further provides a computer program product. Essentially, or the part that contributes to the prior art, or all or part of the technical solution of the present application can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, a mobile terminal, or a processor therein to execute all or part of the steps of the bokeh blur processing method provided in each embodiment of the present application.

Claims

1. A bokeh blur processing method, characterized in that: include: Determining a transparency map of the image to be processed, wherein the transparency map is used to reflect the degree to which each pixel in the image to be processed belongs to the foreground; Determine a background mask image according to the transparency image, perform exponential calculation on the background mask image and the color value of each pixel in the image to be processed to obtain a light source diffusion function, and multiply the light source diffusion function and the background mask image to obtain a convolution mask; Performing mask convolution processing on the image to be processed according to the convolution mask and the preset convolution kernel to obtain a background blurred image; The image to be processed and the background blurred image are fused according to the transparency map to obtain a target blurred image.

2. The bokeh blur processing method according to claim 1, characterized in that: The step of performing exponential calculation on the color values ​​of each pixel in the background mask image and the image to be processed to obtain a light source diffusion function includes: A first product of the background mask image and the color values ​​of corresponding pixels in the image to be processed is calculated, a first parameter power of the first product is calculated, and the calculation result is determined as a light source diffusion function.

3. The bokeh blur processing method according to claim 1, characterized in that: The step of performing exponential calculation on the color values ​​of each pixel in the background mask image and the image to be processed to obtain a light source diffusion function includes: A second product of the background mask image and the color values ​​of corresponding pixels in the image to be processed is calculated, a second parameter power of the second product is added to a preset fusion ratio coefficient, and the addition result is determined as a light source diffusion function.

4. The bokeh blur processing method according to claim 1, characterized in that: The mask convolution process is performed on the image to be processed according to the following formula: Among them, B blur (i,j) is the background blurred image B blur The color value at (i,j) in M i+m,j+n is the mask value of the convolution mask M at (i+m,j+n), W m,n is the element value of the preset convolution kernel at (m,n), x i+m,j+n is the color value of the image to be processed at (i+m,j+n).

5. The bokeh blur processing method according to claim 1, characterized in that: The step of fusing the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image includes: A first fusion ratio and a second fusion ratio are determined according to the transparency map, and a product of the first fusion ratio and the image to be processed is added to a product of the second fusion ratio and the background blurred image to obtain a target blurred image.

6. The bokeh blur processing method according to claim 1, characterized in that: Before performing mask convolution processing on the image to be processed according to the convolution mask and the preset convolution kernel to obtain a background blurred image, the method further includes: A blur radius and a spot shape are determined, and a preset convolution kernel is generated according to the blur radius and the spot shape, wherein the preset convolution kernel is used to perform mask convolution processing on the image to be processed.

7. The bokeh blur processing method according to claim 6, characterized in that: The generating a preset convolution kernel according to the blur radius and the spot shape comprises: An initial matrix is ​​created by taking the sum of the blur radius of a preset multiple and a preset coefficient as the side length of the convolution kernel; Determine the positional relationship between each pixel point on the initial matrix and a preset shape function corresponding to the light spot shape, and determine the pixel value of each pixel point on the initial matrix according to the positional relationship; The initial matrix is ​​normalized to obtain a preset convolution kernel.

8. The bokeh blur processing method according to claim 6, characterized in that: The generating a preset convolution kernel according to the blur radius and the spot shape comprises: The sum of the blur radius of a preset multiple and a preset coefficient is used as the side length of the convolution kernel; Scaling the material image corresponding to the light spot shape according to the side length of the convolution kernel; Rotate the material image after the scaling process by 180°, and set the pixel value of each pixel in the material image according to the positional relationship between each pixel in the rotated material image and the light spot shape; The material image after setting the pixel value is normalized to obtain a preset convolution kernel.

9. The bokeh blur processing method according to claim 1, characterized in that: Before determining the transparency map of the image to be processed, the method further includes: Converting the color space of the image to be processed from the original color space to a preset color space; Accordingly, determining the transparency map of the image to be processed includes: Determine a transparency map of the image to be processed after conversion to a preset color space; After the image to be processed and the background blurred image are fused according to the transparency map to obtain a target blurred image, the method further includes: The color space of the target blurred image is converted from a preset color space to an original color space.

10. A bokeh blur processing device, characterized in that: It includes a transparency determination module, a mask determination module, a blur processing module and an image fusion module, wherein: The transparency determination module is configured to determine a transparency map of the image to be processed, wherein the transparency map is used to reflect the degree to which each pixel in the image to be processed belongs to the foreground; The mask determination module is configured to determine a background mask map according to the transparency map, perform exponential calculation on the background mask map and the color value of each pixel in the image to be processed to obtain a light source diffusion function, and multiply the light source diffusion function and the background mask map to obtain a convolution mask; The blur processing module is configured to perform mask convolution processing on the image to be processed according to the convolution mask and a preset convolution kernel to obtain a background blurred image; The image fusion module is configured to fuse the image to be processed and the background blurred image according to the transparency map to obtain a target blurred image.

11. A bokeh blur processing device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the bokeh blur processing method as described in any one of claims 1 to 9.

12. A non-volatile storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the bokeh blur processing method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the bokeh blur processing method according to any one of claims 1 to 9 is implemented.

Images