A method, apparatus and device for generating a blurred photo

Abstract

The present invention provides a method, apparatus and device for generating a blurred photo. By obtaining a first captured image, a binocular disparity map and the captured focus position information, the binocular disparity map is determined according to the first captured image and a second captured image captured in advance. The radius map of the circle of confusion corresponding to each pixel point in the binocular disparity map is determined according to the focus position information and the binocular disparity map; the first captured image is blurred using the radius map of the circle of confusion to obtain a blurred photo. The method provided in this embodiment uses the binocular disparity map for depth-of-field blurring. Since the dynamic range of the disparity map is small, the change in the disparity value for the same target is smaller, so that the problem of missing blurring or false blurring caused by poor depth consistency can be well alleviated, which provides convenience for capturing a blurred photo with smoother edge pixel values.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device and equipment for generating a blurred photo. Background Art

[0002] In the camera function of a rear dual-camera mobile phone, the background blur function (an image processing function when the shooting mode is the portrait mode) has gradually become a standard feature. The background blur function in the portrait mode can produce pictures with strong aesthetic feeling. The specific operation is as follows: according to the focus selected by the user, non-interested objects such as the background are blurred, so as to achieve the purpose of highlighting the main body (the interested object).

[0003] Currently, there are mainly the following two implementation methods for the rear dual-camera shooting blur in the portrait mode:

[0004] 1. Using a binocular depth map to synthesize a depth-of-field blur effect. Since the dynamic range of the binocular depth map is relatively large, there will be a relatively large difference in the depth consistency of the same object. It is easy to have missed blur and false blur when performing depth-of-field blur according to the depth at the focus.

[0005] 2. Adopting a combination of portrait segmentation and binocular depth map to synthesize a depth-of-field blur effect. Since portrait segmentation is time-consuming and requires high hardware configuration of the mobile phone, and because there are many scenes in the shooting scene, if there are errors in portrait segmentation or the segmented edge is not aligned with the edge of the person in the scene, at this time, when the binocular depth map combines with the wrong segmentation mask to perform depth-of-field blur, there will be missed blur or false blur at the edge.

[0006] Therefore, the prior art needs to be further improved. Summary of the Invention

[0007] In view of the above deficiencies in the prior art, the purpose of the present invention is to provide a method, device and equipment for generating a blurred photo, which overcomes the defects of easy occurrence of missed blur and false blur when performing depth-of-field blur processing on the method for generating a blurred photo in the prior art.

[0008] The solution disclosed in the embodiment of the present invention is as follows:

[0009] In the first aspect, the present embodiment provides a method for generating a blurred photo, which includes:

[0010] Obtaining a first captured image, a binocular disparity map, and the captured focus position information, where the binocular disparity map is determined according to the first captured image and a pre-captured second captured image;

[0011] Determining a circle of confusion radius map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map;

[0012] The first captured image is blurred using the circle of confusion radius map to obtain a blurred photo.

[0013] In a second aspect, this embodiment provides a blurred photo generation device, which includes:

[0014] An information acquisition module for acquiring a first captured image, a binocular disparity map, and the captured focus position information, where the binocular disparity map is determined based on the first captured image and a pre-captured second captured image;

[0015] A circle of confusion information calculation module for determining the circle of confusion radius map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map;

[0016] A blurring processing module for blurring the first captured image using the circle of confusion radius map to obtain a blurred photo.

[0017] In a third aspect, this embodiment further provides a terminal device, which includes a processor and a storage medium communicatively connected to the processor. The storage medium is adapted to store a plurality of instructions; the processor is adapted to call the instructions in the storage medium to execute the steps of implementing the blurred photo generation method.

[0018] In a fourth aspect, this embodiment provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to execute the steps of implementing the blurred photo generation method.

[0019] Advantageous effects: The present invention provides a blurred photo generation method, device, and equipment. By acquiring the first captured image captured by the camera, the binocular disparity map calculated based on the first captured image and the auxiliary captured image by the camera, and the focus position information of the camera during shooting. The circle of confusion radius map corresponding to each pixel point in the binocular disparity map is calculated according to the focus position information, and finally the first captured image is blurred using the circle of confusion radius map to obtain a blurred photo. Since there is a reciprocal relationship between the binocular disparity value and the depth value, the dynamic range of the disparity map is smaller, and the consistency of the disparity values for the same object is better. Therefore, the problems of missing blurring or incorrect blurring caused by poor depth consistency can be well alleviated. Moreover, there is a proportional relationship between the radius of the circle of confusion radius map and the disparity value, which is beneficial to obtaining a gradually changing blurring radius, thereby achieving the effect of gradual blurring. It provides convenience for the user to capture a blurred photo with smoother edge transition pixel values when using the portrait mode to shoot the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flowchart of the steps of the blurred photo generation method provided in this embodiment;

[0021] Figure 2It is a schematic structural diagram of the circular diffusion circle blur kernel provided by this embodiment;

[0022] Figure 3 It is a schematic principle diagram of bilinear upsampling in this embodiment;

[0023] Figure 4 It is a schematic diagram of the disparity map in this embodiment;

[0024] Figure 5 It is a schematic diagram of the defocusing effect in this embodiment;

[0025] Figure 6 It is a flowchart of the specific application embodiment steps of the defocused photo generation method provided by the present invention;

[0026] Figure 7 It is a schematic block diagram of the structural principle of the device in the embodiment of the present invention;

[0027] Figure 8 It is a schematic diagram of the principle structure of the terminal device of the present invention. Detailed implementation manners

[0028] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present invention means the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0030] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless specifically defined as here.

[0031] The inventors found that there are two methods for blurring photos in the prior art. One is implemented using a binocular depth map, and the other is to combine portrait segmentation and a binocular depth map to achieve the depth-of-field blurring effect. Since the dynamic range of the binocular depth map is relatively large, it is easy to have missed blurring and false blurring when performing depth-of-field blurring based on the depth at the focus. And there are errors in portrait segmentation or the segmented edges do not align with the edges of the people in the scene. Therefore, when using the binocular depth map in combination with an incorrect segmentation mask to perform depth-of-field blurring, missed blurring or false blurring will occur at the edges.

[0032] Since the methods for blurring photos in the prior art all have the defect of being prone to missed blurring and false blurring, this embodiment provides a method for generating a blurred photo, which uses a binocular disparity map to perform blurring processing on the photo. Specifically, since there is a reciprocal relationship between the binocular disparity value and the depth value, the dynamic range of the disparity map is smaller. For the same object (such as a person), the consistency of its disparity values is better, which can better overcome the problem of missed blurring or false blurring caused by poor depth consistency. Moreover, the radius of the defocus blur circle is in a direct proportional relationship with the disparity value, which is beneficial to obtaining a gradually changing defocus radius, thereby achieving the effect of gradual blurring; in the present invention, for a small-scale disparity map, bilinear interpolation is used for upsampling to the original size, and this processing method can well accelerate the blurring of the photo; at the same time, the present invention also adopts an operation of weight stretching, which can make the blurred background have light spots under the condition of basically not increasing the additional processing time, so as to make the obtained blurred image more realistic.

[0033] The following will explain the method disclosed in the present invention in more detail with reference to the accompanying drawings.

[0034] Exemplary method

[0035] This embodiment provides a method for generating a blurred photo, as Figure 1 shown, including:

[0036] Step S1, an image blurring device obtains a first captured image, a binocular disparity map, and the captured focus position information obtained by shooting. The binocular disparity map is determined according to the first captured image and a pre-captured second captured image.

[0037] The image blurring device may be equipped with multiple cameras with different pixels.

[0038] In one embodiment, the camera with the highest pixel among the multiple cameras can be used to capture the target subject to obtain the first captured image; the cameras other than the camera with the highest pixel among the multiple cameras are used to capture the second captured image, and then the binocular disparity map is calculated according to the first captured image and the second captured image. The focus position information is the position information of the shooting focus when shooting the first captured image and the second captured image.

[0039] In one embodiment, the image blurring device first obtains a first captured image using the camera with the highest pixel count among the multiple cameras and obtains a second captured image using the cameras other than the camera with the highest pixel count among the multiple cameras. Then, a binocular disparity map is calculated based on the first captured image and the second captured image, and the shooting focus position information when the first captured image and the second captured image are taken is obtained synchronously.

[0040] The first captured image is obtained by capturing the target subject using the camera with the highest pixel count among the multiple cameras equipped with the image blurring device. The camera with the highest pixel count among the multiple cameras is equipped with a color sensor to collect image colors. The cameras other than the camera with the highest pixel count among the multiple cameras assist in the depth of field to blur the background. Therefore, the cameras other than the camera with the highest pixel count among the multiple cameras are configured with black and white sensors to supplement the image brightness and details.

[0041] In another embodiment, in this step, the image blurring device can directly obtain from other image acquisition devices the first captured image and the second captured image respectively captured by the camera with the highest pixel count among the multiple cameras and the cameras other than the camera with the highest pixel count among the multiple cameras, as well as the binocular disparity map corresponding to the first captured image and the second captured image, and the shooting focus position information when the first captured image and the second captured image are taken.

[0042] Furthermore, the binocular disparity map is calculated by a stereo matching algorithm from the first captured image and the second captured image. Specifically, taking any one of the images as a reference, the pixel differences between the two images are compared to obtain a comparison image, which contains the distance information of the scene.

[0043] The above embodiments are only for illustrative purposes and should not be construed as limitations on the first captured image and the second captured image obtained by the image blurring device.

[0044] Step S2: The image blurring device determines the radius-of-blur circle map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map.

[0045] The image blurring device determines the radius-of-blur circle map corresponding to each pixel point in the binocular disparity map based on the focus position contained in the focus position information obtained in the above steps and the calculated binocular disparity map.

[0046] Specifically, the image blurring device determines the radius-of-blur circle map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map, including:

[0047] Step S21: The image blurring device takes a target area centered on the position where the focus is located and calculates the median of all the disparity values within the target area. The median is used as the focus disparity value.

[0048] Combined Figure 4 As shown, first, taking the position of the focus as the center point, a region is selected. The median value of all the disparity values within the target region is taken as the focus disparity value d focus . The shape of the target region can be selected and set as needed. Specifically, it can be set as a square, a circle, or a shape with a certain rule, or a shape without a rule. In one implementation, in order to accurately calculate the median of all the disparity values within the target region, the shape of the selected region is a square.

[0049] Step S22: The image blurring device determines the radius of the circle of confusion according to the focus disparity value and the binocular disparity map, and obtains the circle of confusion radius map according to the radius of the circle of confusion.

[0050] Since the blurring intensity corresponding to each pixel point is related to the disparity value, in this step, first, according to the disparity value, the focus disparity value, and the preset maximum blurring intensity corresponding to each pixel point, the blurring intensity corresponding to each pixel point is obtained, and then according to the blurring intensity corresponding to each pixel point and the position information of the non-blurred region, the radius of the circle of confusion is calculated; wherein, the position information of the non-blurred region is the position information corresponding to the non-blurred region, and the non-blurred region is a preset range region centered on the focus, and the preset range region is the region position information where the photographed target subject is located.

[0051] Centered on the position of the focus, according to the preset circle of confusion radius value and the preset blurring intensity coefficient of each pixel point within the non-blurred region, the position information of the non-blurred region is obtained; so as to achieve the clarity of the main body at the focus without being falsely blurred or missed. Since it is not necessary to blur each pixel point in the non-blurred region, the preset circle of confusion radius value is 0, and the blurring intensity coefficient is the ratio of the preset constant to the blurring intensity corresponding to each pixel point. Finally, according to the blurring intensity corresponding to each pixel point and the position information of the non-blurred region, the radius of the circle of confusion is calculated.

[0052] Specifically, for the disparity map D and the focus disparity value, the radius of the circle of confusion is calculated using formula 1;

[0053] r ij = k(d, d focus ) * max(0, |d - d focus | - d o ) (Formula 1)

[0054] Where: k(d, d focus ) represents a function of the blurring intensity related to the disparity value d, d ij is the disparity value at (i, j) in the disparity map; d o represents the effective change range without blurring.

[0055] Specifically, calculate k(d, d focus ) and d o respectively through the following formulas:

[0056] k(d, d focus ) = L * k1(d) * k2(d focus );

[0057] Where: The calculation methods of k1(d) and k2(d focus ) are as shown in Formula 3 and Formula 4;

[0058] k1(d) = (0.3 / d + 0.2)| [0.5,2.5] (Formula 3)

[0059]

[0060] Where: (·)| in Formula 3 and Formula 4 [a,b] means truncated to between [a, b].

[0061] To ensure the clarity of the main body at the focus without being falsely blurred or missed blurred, set:

[0062] [d focus - d o , d focus + d o The area within the parallax range is not blurred, that is, set the radius of the circle of confusion corresponding to each pixel point within the non-blurred area to 0.

[0063] The calculation method of d o is as shown in Formula 5:

[0064] d o = 0.5 / k(d, d focus ) (Formula 5)

[0065] Step S3. The image blurring device uses the circle of confusion radius map to perform blurring processing on the first captured image to obtain a blurred photo.

[0066] When the circle of confusion radius map is calculated in the above step S2, in this step, the calculated circle of confusion radius map can be used to perform blurring processing on the first captured image.

[0067] Specifically, the image blurring device uses the circle of confusion radius map to perform blurring processing on the first captured image to obtain a blurred photo, including:

[0068] Step S31. The image blurring device downsamples the first captured image according to a preset size to obtain a downsampled captured image, and performs interpolation processing on the circle of confusion radius map using the interpolation method to obtain an interpolated circle of confusion radius map with the same size as the downsampled captured image.

[0069] To improve the data processing rate, the binocular disparity map obtained in the embodiment is one-fourth or one-half of the size of the first captured image. To keep the obtained interpolated diffusion radius map and the blurred first captured image the same size, in this step, downsampling is performed on the first captured image to change its size to half or one-fourth of the original, and the diffusion radius map is upsampled to make the sizes of the two images consistent.

[0070] In one implementation, the selected size of the binocular disparity map is a multiple of 1 / 4 of the size of the first captured image; each pixel value in the first captured image is reduced according to a preset size to obtain a downsampled captured image; the preset size is 1 / 2 of the size of the first captured image; the diffusion circle radius map is interpolated using the nearest neighbor interpolation method to obtain an interpolated diffusion radius map, and the size of the interpolated diffusion radius map is 1 / 2 of the size of the first captured image.

[0071] Step S32: The image blurring device calculates the blur kernel corresponding to each pixel point in the diffusion circle radius map according to the diffusion circle radius value corresponding to each pixel point in the diffusion circle radius map; the image blurring device magnifies the blur kernel corresponding to each pixel point in the diffusion circle radius map according to the pixel value corresponding to each pixel point in the downsampled captured image and the magnification weight value of the blur kernel, and performs a blurring operation on the downsampled captured image using the magnified blur kernel to obtain an initial blurred photo.

[0072] To improve the blurring processing speed and the blurring presentation (light spot) effect; in this step, the diffusion circle radius map is interpolated to 1 / 2 scale for blurring by the nearest neighbor interpolation method, and then the blurred result is upsampled to the original size, and an operation of stretching based on the blur kernel weight is performed during the 1 / 2 scale blurring process to generate a light spot effect with little additional time consumption.

[0073] Furthermore, in this step, the image blurring device calculates the magnification weight value of the blur kernel according to the variance of the preset control magnification ratio, the blur kernel corresponding to each pixel point in the diffusion circle radius map, and the pixel value corresponding to each pixel in the downsampled captured image, then magnifies the blur kernel corresponding to each pixel point in the diffusion circle radius map according to the calculated magnification weight value of the blur kernel to obtain a magnified blur kernel, and finally performs a blurring operation on the downsampled captured image according to the radius of each pixel point corresponding in the diffusion circle radius map and the magnified blur kernel to obtain an initial blurred photo.

[0074] Step S33: The image blurring device upsamples the initial blurred photo and the interpolated diffusion radius map to the same size as the first captured image to obtain an upsampled initial blurred photo and an upsampled interpolated diffusion radius map after upsampling processing; the background edge of the upsampled initial blurred photo is blurred using the radius of the upsampled interpolated diffusion radius map as the blurring radius to generate a blurred photo.

[0075] After obtaining the initial blurred photo by performing a blurring operation on the downsampled captured image using the enlarged blurred kernel, the obtained initial blurred photo and the interpolated diffusion radius map are upsampled again to obtain an image of the same size as the first captured image, and the background edge of the initial blurred photo is blurred to generate a blurred photo.

[0076] Specifically, in this step, the image blurring device uses the bilinear interpolation method to upsample the initial blurred photo and the interpolated diffusion radius map to the same size as the first captured image, obtaining the upsampled initial blurred photo after upsampling the initial blurred photo and the upsampled interpolated diffusion radius map after upsampling the interpolated diffusion radius map. Then, using the method of scatter blurring, with the radius of the upsampled interpolated diffusion radius map as the blurring radius, the foreground and background edges of the upsampled initial blurred photo are blurred to generate a blurred photo, and its blurring effect is as Figure 5 shown.

[0077] Next, in combination with the specific application example of this embodiment, the method provided in this embodiment will be further described.

[0078] Combined Figure 6 As shown, in the overall process of the specific application of this embodiment, the image blurring device performs operations in the following steps:

[0079] Step H1: The image blurring device obtains the first captured image I, the binocular disparity map D, the focal position (X focus , Y focus ), and the maximum blurring intensity coefficient L; where: the binocular disparity map is calculated by using the first captured image I and the second captured image through a stereo matching algorithm;

[0080] Step S2: Calculate the median value d focus of the disparity values within a small area centered on the focal position (X focus ), Y median , and calculate the diffusion circle radius map R according to the binocular disparity map D; the specific implementation steps are as follows:

[0081] Step S21: Taking the focal point (X focus , Y focus ) as the center, taking a square area with a side length of min(W / c, H / c), and taking the median value of all the disparity values in the square area as the focal point disparity value d focus ;

[0082] Step S22: Calculate the diffusion circle radius for the disparity map D and the focal point disparity value using formula 1;

[0083] r ij = k(d, d focus)*max(0,|d - d focus |- d o ) (Formula 1)

[0084] Where: k(d, d focus ) represents a function related to the defocusing intensity corresponding to the disparity value d, and d ij is the disparity value at (i, j) in the binocular disparity map; d o represents the effective variation range without defocusing; the specific calculation method of k(d, d focus ) and d o will be described below.

[0085] Step S23: k(d) represents the defocusing intensity at the disparity value d, which is related to the scene disparity value d, the disparity value at the focus selected by the user, and the maximum defocusing intensity L; its calculation method is as shown in Formula 2;

[0086] k(d, d focus ) = L * k1(d) * k2(d focus ) (Formula 2)

[0087] Where: the calculation methods of k1(d) and k2(d focus ) are as shown in Formula 3 and Formula 4;

[0088] k1(d) = (0.3 / d + 0.2)| [0.5,2.5] (Formula 3)

[0089]

[0090] Where: L represents the maximum defocusing intensity, and (·)| [a,b] in Formula 3 and Formula 4 represents truncation to the range [a, b];

[0091] Step S24: To ensure the clarity of the main body at the focus and prevent incorrect or missed defocusing, set the area within the disparity range [d focus - d o , d focus + d o without defocusing, that is, set the defocusing blur circle radius to 0; the calculation method of d o is as shown in Formula 5:

[0092] d o = 0.5 / k(d, d focus ) (Formula 5)

[0093] Step S3: Downsample the captured image I to 1 / 2 scale to obtain I_, and use nearest-neighbor interpolation for the defocusing blur circle radius map R to obtain R_ with the same size as I_; the specific operations are as follows:

[0094] Step S31: Downsample the captured image I to 1 / 2 scale. Let (i, j) be the pixel value at any point in I_, and its corresponding position in I is (2*i, 2*j). So the downsampling operation is: I_(i, j) = I(2*i, 2*j).

[0095] Step S32: Nearest neighbor interpolation operation for the defocus circle radius map R. The size ratio between R and R_ is ratio. Then the position in R corresponding to the pixel value at (x, y) in R_ is x’ = ((x + 1) / ratio + 0.5) - 1, y’ = ((y + 1) / ratio + 0.5) - 1. So the nearest neighbor interpolation operation is: R_(x, y) = R(x’, y’).

[0096] Step S4: Based on the pixel values corresponding to each pixel point in I_, perform an amplification operation on the weights of the blur kernels corresponding to each pixel point in the defocus circle radius map, and use the amplified blur kernels to perform a blurring operation on I_ to obtain I_bokeh. The specific steps are as follows:

[0097] Step S41: Let the pixel value at a certain point in I_ be P, and its corresponding blur kernel weight be w. Then the operation of amplifying the blur kernel weight is calculated according to formula 6:

[0098]

[0099] where: w’ represents the amplified weight value, exp(*) represents the exponential function, P max = max(P r , P g , P b ), and σ represents the variance controlling the amplification factor. In this method, σ = 3;

[0100] Step S42: The radius corresponding to the pixel point (i, j) in I_ in the defocus circle radius map R_ is r. Then the size of the circular blur kernel K corresponding to (i, j) is (2*r + 1, 2*r + 1). As Figure 2 , perform a blurring operation on I_ using the following formula 7;

[0101]

[0102] where: The variation ranges of Δi and Δj are between [-r, r].

[0103] Step S5: Upsample I_bokeh and R_ to the original size (the same size as I), and then perform a scatter-based blurring process on the foreground and background edges in I_bokeh. The specific implementation steps are as follows:

[0104] Step S51: Use bilinear interpolation to upsample I_bokeh and R_ to the original size. The sampling principle diagram is as Figure 3; (i’, j’) is a point in the original I_bokeh, and (i, j) is the midpoint (i, j) in I_bokeh after its interpolation; the calculation of the interpolation method is calculated according to Formula 8;

[0105]

[0106] Where: represents the floor operation, and ratio represents the interpolation ratio coefficient.

[0107] Step H5: Use the scatter method (blurring method based on scattering) to blur the foreground and background edges of I_bokeh to obtain the final blurred effect image I_bokeh’, where the scatter method is to spread a part of a pixel value to other pixels within the same diffusion circle.

[0108] The blurring operation in this step is as Formula 9:

[0109]

[0110] Where, K i+Δi,j+Δj is the blur kernel, the variation ranges of Δi and Δj are between [-r, r], the number of blur kernels in this step is multiple, the values of i + Δi and j + Δj are different, corresponding to different blur kernels respectively, the number of blur kernels is determined by the blur radius, and the calculation formula for its number is (2r + 1)*(2r + 1). If the blur radius is r = 1 as an example, then there are nine groups of values for the subscripts i + Δi and j + Δj of the blur kernel, which are (i - 1, j - 1), (i - 1, j), (i - 1, j + 1), (i, j - 1), (i, j), (i, j + 1), (i + 1, j + 1), (i + 1, j), and (i + 1, j - 1). Therefore, there are 9 blur kernels in total. Use these 9 blur kernels to perform the blurring operation on I_bokeh in the scattering blurring method to obtain the blurred image.

[0111] Compared with the prior art, the method in this embodiment improves the effect and speed of taking pictures with blurring through the following three aspects:

[0112] (1), This method uses the binocular disparity map for depth-of-field blurring. The dynamic range of the disparity map is smaller, and the change in the disparity value of the same target is smaller; moreover, this method is based on the disparity value d focus of the focus position, and dynamically calculates d o , and sets the blur radius of the pixels within [d focus - d o , d focus + d o to be 0; overcoming the problems of false blurring and missed blurring caused by blurring;

[0113] (2) To improve the virtualization processing speed and the virtualization rendering (light spot) effect; this method performs virtualization by nearest neighbor interpolation of the circle of confusion radius map to a 1 / 2 scale, and then upsamples the virtualization result to the original size; and during the virtualization process at the 1 / 2 scale, an operation based on stretching of the blur kernel weight is performed to generate a light spot effect without adding extra time consumption, making the photo after virtualization processing more realistic;

[0114] (3) To ensure the sharpness of the virtualization edge and suppress the halo problem; this method uses the scatter method for virtualization at the edge part (scatter mode) to enhance the expressiveness of virtualization visually.

[0115] Exemplary device

[0116] Based on the disclosure of the above method, this embodiment also discloses a virtualized photo generation device, as Figure 7 shown, including:

[0117] An information acquisition module 710, configured to acquire a first captured image, a binocular disparity map, and the captured focus position information, where the binocular disparity map is determined according to the first captured image and a pre-captured second captured image; its function is as in step S1.

[0118] A dispersion information calculation module 720, configured to determine the circle of confusion radius map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map; its function is as in step S2.

[0119] A virtualization processing module 730, configured to perform virtualization processing on the first captured image by using the circle of confusion radius map to obtain a virtualized photo, and its function is as in step S3.

[0120] Further, the virtualization processing module 630 includes:

[0121] A downsampling processing unit, configured to downsample the first captured image according to a preset size to obtain a downsampled captured image, and perform interpolation processing on the circle of confusion radius map by using an interpolation method to obtain an interpolated dispersion radius map with the same size as the downsampled captured image;

[0122] A first virtualization unit, configured to calculate the blur kernel corresponding to each pixel point in the circle of confusion radius map according to the circle of confusion radius value corresponding to each pixel point in the circle of confusion radius map; perform magnification processing on the blur kernel corresponding to each pixel point in the circle of confusion radius map according to the pixel value corresponding to each pixel point in the downsampled captured image and the magnification weight value of the blur kernel, and perform a virtualization operation on the downsampled captured image by using the magnified blur kernel to obtain an initial virtualized photo;

[0123] A second blurring unit, configured to upsample the initial blurred photo and the interpolated diffusion radius map to the same size as the first captured image, so as to obtain an upsampled initial blurred photo and an upsampled interpolated diffusion radius map after the upsampling process; using the radius of the upsampled interpolated diffusion radius map as the blurring radius to blur the background edge of the upsampled initial blurred photo, thereby generating a blurred photo.

[0124] Based on the above method, this embodiment also discloses a terminal device, including a processor and a storage medium communicatively connected to the processor. The storage medium is adapted to store a plurality of instructions; the processor is adapted to call the instructions in the storage medium to execute the method for generating a blurred photo. In one implementation, the terminal device may be a mobile phone, a tablet computer, or a smart TV.

[0125] Specifically, as Figure 8 shown, the terminal device includes at least one processor 20 and a memory 22, and may further include a display screen 21, a communication interface 23, and a bus 24. Among them, the processor 20, the display screen 21, the memory 22, and the communication interface 23 can complete communication with each other through the bus 24. The display screen 21 is configured to display a user guidance interface preset in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logical instructions in the memory 22 to execute the steps of the method for generating a blurred photo in the above embodiment.

[0126] In addition, when the logical instructions in the above-mentioned memory 22 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0127] The memory 22, as a computer-readable storage medium, can be configured to store software programs and computer-executable programs, such as program instructions or modules corresponding to the method in the embodiments of the present disclosure. The processor 30 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, that is, implements the method in the above embodiment.

[0128] The memory 22 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, various media that can store program codes, such as USB flash drives, external hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, may also be transient storage media.

[0129] On the other hand, a computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the virtualized photo generation method.

[0130] The present invention provides a method, device, and equipment for generating virtualized photos. Since the binocular disparity map is used to achieve the depth-of-field virtualization of the photo, and there is an inverse relationship between the binocular disparity values and the depth values in the binocular disparity map, the dynamic range of the disparity map is smaller, and the consistency of the disparity values for the same object is better. Therefore, the problem of missing or false virtualization caused by poor depth consistency can be well alleviated. Moreover, there is a proportional relationship between the radius of the circle of confusion radius map and the disparity value. Therefore, taking the radius of the circle of confusion radius map as the circular radius can present a gradually changing virtualization effect, which provides convenience for the user to take a virtualized photo with a smoother edge transition pixel value when using the portrait mode to photograph the target object.

[0131] It can be understood that for those of ordinary skill in the art, equivalent substitutions or changes can be made according to the technical solutions and inventive concepts of the present invention, and all such changes or substitutions should fall within the protection scope of the claims appended to the present invention.

Claims

1. A method for generating a blurred photo, characterized in that, Including: Obtaining a first captured image, a binocular disparity map, and captured focus position information, where the binocular disparity map is determined based on the first captured image and a pre-captured second captured image; Determining a circle of confusion radius map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map; Performing a blurring process on the first captured image using the circle of confusion radius map to obtain a blurred photo; The step of determining a circle of confusion radius map corresponding to each pixel point in the binocular disparity map according to the focus position information and the binocular disparity map includes: Determining a circle of confusion radius according to a focus disparity value and the binocular disparity map, and obtaining a circle of confusion radius map based on the circle of confusion radius.

2. The method according to claim 1, wherein Including: Taking a target area in a preset shape with the position where the focus is located as the center, and calculating the median of all disparity values within the target area, and using the median as the focus disparity value.

3. The method according to claim 2, wherein The determining a circle of confusion radius according to the focus disparity value and the binocular disparity map includes: Obtaining a blurring intensity corresponding to each pixel point according to the disparity value corresponding to each pixel point, the focus disparity value, and a preset maximum blurring intensity; Calculating a circle of confusion radius according to the blurring intensity corresponding to each pixel point and non-blurring area position information; where the non-blurring area position information is the position information corresponding to a non-blurring area, and the non-blurring area is a preset range area centered on the focus, and the preset range area is the area position information where the captured target object is located.

4. The method according to claim 3, characterized in that, Before calculating a circle of confusion radius according to the blurring intensity corresponding to each pixel point and non-blurring area position information, the method further includes: Taking the position where the focus is located as the center, and obtaining the non-blurring area position information according to a preset circle of confusion radius value and a preset blurring intensity coefficient of each pixel point within the non-blurring area; where the blurring intensity coefficient is the ratio of a preset constant to the blurring intensity corresponding to each pixel point.

5. The method according to claim 4, wherein The performing a blurring process on the first captured image using the circle of confusion radius map to obtain a blurred photo includes: Downsampling the first captured image according to a preset size to obtain a downsampled captured image, and performing an interpolation process on the circle of confusion radius map using an interpolation method to obtain an interpolated circle of confusion radius map with the same size as the downsampled captured image; Calculating a blur kernel corresponding to each pixel point in the circle of confusion radius map according to the circle of confusion radius value corresponding to each pixel point in the circle of confusion radius map; Magnifying the blur kernel corresponding to each pixel point in the circle of confusion radius map according to the pixel value corresponding to each pixel point in the downsampled captured image and a magnification weight value of the blur kernel, and performing a blurring operation on the downsampled captured image using the magnified blur kernel to obtain an initial blurred photo; Performing an upsampling process on the initial blurred photo and the interpolated circle of confusion radius map to the same size as the first captured image to obtain an upsampled initial blurred photo and an upsampled interpolated circle of confusion radius map after the upsampling process; Using the radius of the upsampled interpolation diffusion radius map as the blurring radius, blur the background edges of the upsampled initial blurred photo to generate a blurred photo.

6. The method according to claim 5, characterized in that, The selected size of the binocular disparity map is a multiple of 1 / 4 of the size of the first captured image. Downsample the first captured image according to a preset size to obtain a downsampled captured image, and perform interpolation processing on the diffusion circle radius map using the interpolation method to obtain an interpolation diffusion radius map with the same size as the downsampled captured image, including: Reduce each pixel value in the first captured image according to a preset size to obtain the downsampled captured image; the preset size is 1 / 2 times the size of the first captured image; Use the nearest neighbor interpolation method to perform interpolation processing on the diffusion circle radius map to obtain the interpolation diffusion radius map, and the size of the interpolation diffusion radius map is 1 / 2 times the size of the first captured image.

7. The method according to claim 5 or 6, characterized in that, According to the pixel values corresponding to each pixel point in the downsampled captured image and the amplification weight value of the blur kernel, amplify the blur kernel corresponding to each pixel point in the diffusion circle radius map, and perform a blurring operation on the downsampled captured image using the amplified blur kernel to obtain an initial blurred photo, including: Calculate the amplification weight value of the blur kernel according to the preset variance of the control magnification, the blur kernel corresponding to each pixel point in the diffusion circle radius map, and the pixel values corresponding to each pixel in the downsampled captured image; Amplify the blur kernel corresponding to each pixel point in the diffusion circle radius map according to the calculated amplification weight value of the blur kernel to obtain an amplified blur kernel; Perform a blurring operation on the downsampled captured image according to the radius of each pixel point in the downsampled captured image in the diffusion circle radius map and the amplified blur kernel to obtain the initial blurred photo.

8. An apparatus for generating a blurred photo, characterized in that, Including: An information acquisition module for acquiring a first captured image, a binocular disparity map, and the focal position information of the capture. The binocular disparity map is determined according to the first captured image and a pre-captured second captured image; A diffusion information calculation module for determining the diffusion circle radius map corresponding to each pixel point in the binocular disparity map according to the focal position information and the binocular disparity map; A blurring processing module for using the diffusion circle radius map to perform blurring processing on the first captured image to obtain a blurred photo; The step of determining the diffusion circle radius map corresponding to each pixel point in the binocular disparity map according to the focal position information and the binocular disparity map includes: Determine the diffusion circle radius according to the focal disparity value and the binocular disparity map, and obtain the diffusion circle radius map according to the diffusion circle radius.

9. A terminal device, characterized in that, Including a processor and a storage medium communicatively connected to the processor. The storage medium is suitable for storing multiple instructions; the processor is suitable for calling the instructions in the storage medium to execute the steps of implementing the blurred photo generation method according to any one of claims 1-7 above.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the blurred photo generation method according to any one of claims 1-7.

Images