A depth map optimization processing method, intelligent terminal and storage medium

Abstract

The present invention discloses a depth map optimization processing method, intelligent terminal, and storage medium. The method comprises: obtaining an original depth map, performing hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map; performing edge filtering on the preliminary depth map to obtain an intermediate depth map; and performing further smoothing and median filtering on the intermediate depth map to obtain an optimized depth map. By performing two filtering and two smoothing operations on the depth map, the present invention ensures the integrity of depth edges and avoids leaking and false blurring in the image blurring effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a depth map optimization processing method, an intelligent terminal, and a storage medium. Background Art

[0002] Blurring in photos often relies on the accuracy of depth and foreground information and the integrity of edges. For example, achieving the wide-aperture effect (i.e., blurring) of a DSLR on a mobile phone requires accurate depth information. However, existing technologies often use depth information with inaccurate depth information or incomplete edges, which can lead to missing or inaccurate blurring.

[0003] Therefore, the existing technology needs to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a depth map optimization processing method, an intelligent terminal and a storage medium in response to the above-mentioned defects of the prior art, aiming to solve the problem of missing or erroneous blurring due to incorrect depth values ​​or incomplete edges in the prior art.

[0005] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0006] In a first aspect, this embodiment provides a depth map optimization processing method, which includes:

[0007] Obtain an original depth map, perform hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map;

[0008] Perform edge filtering on the preliminary depth map to obtain an intermediate depth map;

[0009] The intermediate depth map is smoothed again and median filtered to obtain an optimized depth map.

[0010] In one embodiment, obtaining an original depth map, performing hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map includes:

[0011] Obtain an original image, perform grayscale processing on the original image, and obtain a grayscale image;

[0012] The grayscale image is downsampled to obtain the original depth map;

[0013] Obtain pixel depth values ​​on the original depth map, and determine the hole area based on the pixel depth values. The hole area is the area where the pixel depth value meets the preset threshold;

[0014] Assign the pixel depth values ​​in the hole area to obtain a filled depth map;

[0015] Perform preliminary guided filtering on the filled depth map to obtain a filtered depth map;

[0016] Perform preliminary smoothing on the filtered depth map to obtain a preliminary depth map.

[0017] In one embodiment, performing preliminary guided filtering on the filled depth map to obtain a filtered depth map includes:

[0018] Determine the grayscale image corresponding to the filled depth map;

[0019] Traverse all pixels in the grayscale image to obtain multiple selected pixels;

[0020] Calculate a depth value weight set for filling the depth map based on multiple selected pixels;

[0021] Calculate the new pixel depth value on the filled depth map according to the depth value weight set;

[0022] The pixel depth values ​​of the filled depth map are updated according to the new pixel depth values ​​to obtain a filtered depth map.

[0023] In one embodiment, performing preliminary smoothing on the filtered depth map to obtain a preliminary depth map includes:

[0024] Obtaining a first gradient value of the filtered depth map, and determining a to-be-processed area according to the first gradient value; wherein the to-be-processed area is an area in the filtered depth map where the first gradient value is less than a first preset gradient threshold;

[0025] Perform preliminary smoothing on the area to be processed to obtain a preliminary depth map.

[0026] In one embodiment, edge filtering is performed on the preliminary depth map to obtain an intermediate depth map, including:

[0027] Determine an edge region mask in the preliminary depth map;

[0028] Obtaining the pixel points to be processed in the edge area mask, where the pixel points to be processed are the pixel points in the edge area mask whose pixel values ​​are not 0;

[0029] Determine the depth value weight corresponding to the preliminary depth map according to the pixel to be processed, and determine the filtered depth value according to the depth value weight;

[0030] The preliminary depth map is edge filtered according to the filtered depth value to obtain an intermediate depth map.

[0031] In one embodiment, determining an edge region mask in a preliminary depth map includes:

[0032] Calculate the second gradient value of the pixel in the preliminary depth map;

[0033] Determine an edge area in the preliminary depth map according to the second gradient value, where the edge area is an area where the second gradient value is greater than a second preset gradient threshold;

[0034] Morphological processing and dilation processing are performed on the edge area to obtain the edge area mask in the preliminary depth map.

[0035] In one embodiment, determining a depth value weight corresponding to the preliminary depth map according to the pixel to be processed, and determining a filtered depth value according to the depth value weight includes:

[0036] Obtaining a first pixel point and a second pixel point of the pixel point to be processed, where the distance between the first pixel point and the second pixel point is less than a preset distance;

[0037] Calculating the inter-pixel color difference and the Euclidean distance between the first pixel and the second pixel respectively;

[0038] The depth value weight is calculated according to the color difference and Euclidean distance between pixels, and the filtered depth value is determined according to the depth value weight. The filtered depth value is the sum of all depth value weights.

[0039] In one embodiment, the intermediate depth map is smoothed again and median filtered to obtain an optimized depth map, including:

[0040] Obtaining a third gradient value of the intermediate depth map, and determining a to-be-processed area in the intermediate depth map according to the third gradient value; wherein the to-be-processed area is an area in the intermediate depth map where the third gradient value is less than a third preset gradient threshold;

[0041] The area to be processed in the intermediate depth map is smoothed and median filtered to obtain an optimized depth map.

[0042] In a second aspect, the present invention further provides a depth map optimization processing device, wherein the device includes:

[0043] A depth map processing unit is used to obtain an original depth map, perform hole filling, preliminary guided filtering and preliminary smoothing on the original depth map to obtain a preliminary depth map;

[0044] A depth map filtering unit is used to perform edge filtering on the preliminary depth map to obtain an intermediate depth map;

[0045] The depth map smoothing unit is used to perform further smoothing and median filtering on the intermediate depth map to obtain an optimized depth map.

[0046] In a third aspect, the present invention provides an intelligent terminal, which includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include a method for executing any one of the above-mentioned schemes.

[0047] In a fourth aspect, the present invention provides a non-temporary computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute any method in the above-mentioned schemes.

[0048] Beneficial effects of the present invention: When performing depth map optimization processing, the embodiment of the present invention first obtains the original depth map, and performs hole filling, preliminary guided filtering and preliminary smoothing processing on the original depth map to obtain a preliminary depth map. Since the edges in the obtained preliminary depth map are not complete enough, the blurring effect of the image is affected. Therefore, after obtaining the preliminary depth map, this embodiment also performs edge filtering processing on the preliminary depth map to obtain an intermediate depth map, eliminating interference factors such as noise in the preliminary depth map. The intermediate depth map is then smoothed again and median filtered to obtain an optimized depth map, and edge details are guaranteed by smoothing again and median filtering. It can be seen that in the embodiment of the present invention, the original depth map is filtered twice and smoothed twice, which not only eliminates the interference factors in the original depth map, but also ensures the integrity of the edges of the depth map, avoiding the phenomenon of missing and false blurring in the image blurring effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the process flow of a depth map optimization processing method provided by an embodiment of the present invention.

[0050] Figure 2 This is a flowchart of preliminary guided filtering and preliminary smoothing processing in the depth map optimization processing method provided by an embodiment of the present invention.

[0051] Figure 3 This is a flowchart of edge filtering processing in the depth map optimization processing method provided by an embodiment of the present invention.

[0052] Figure 4 This is a flowchart of the re-smoothing process in the depth map optimization processing method provided by an embodiment of the present invention.

[0053] Figure 5 This is a comparison diagram of the effects of the depth map optimization processing method provided by an embodiment of the present invention.

[0054] Figure 6 This is a principle block diagram of a depth map optimization processing device provided by an embodiment of the present invention.

[0055] Figure 7This is a principle block diagram of an intelligent terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0057] Research has found that blurring in photos often relies on the accuracy of depth and foreground information and the integrity of edges. For example, achieving the wide-aperture effect (i.e., blurring) of a DSLR on a mobile phone requires accurate depth information. However, existing technologies often use depth information with inaccurate depth information or incomplete edges, which can lead to missing or inaccurate blurring, impacting image quality.

[0058] In order to solve the above technical problems, an embodiment of the present invention provides a depth map optimization processing method. Through the method of the embodiment of the present invention, the phenomenon of missing and false blur in the image blur effect can be avoided, and the imaging quality can be improved. Specifically, when performing depth map optimization processing, the embodiment of the present invention first obtains the original depth map, and performs hole filling, preliminary guided filtering and preliminary smoothing on the original depth map to obtain a preliminary depth map. Since the edges of the obtained preliminary depth map are not complete enough, the blur effect of the image is affected. Therefore, after obtaining the preliminary depth map, this embodiment also performs edge filtering on the preliminary depth map to obtain an intermediate depth map, eliminating interference factors such as noise in the preliminary depth map. Then, the intermediate depth map is smoothed again and median filtered to obtain an optimized depth map. By smoothing again and median filtering, the edge details are guaranteed. It can be seen that in the embodiment of the present invention, the original depth map is filtered and smoothed twice, which not only eliminates the interference factors in the original depth map, but also ensures the integrity of the depth map edge, avoids missing and false blur in the image blur effect, and improves the imaging quality.

[0059] For example, after an image is captured by a mobile phone, the original depth map is obtained from the captured image, and a filled depth map is obtained based on the original depth map. Then, the filled depth map is subjected to preliminary guided filtering and preliminary smoothing to obtain a preliminary depth map. The preliminary depth map obtained at this time has undergone one guided filtering and smoothing process. Next, the preliminary depth map is edge filtered to obtain an intermediate depth map. The intermediate depth map obtained at this time has undergone two guided filtering and one smoothing process relative to the original image (i.e., the image captured by the mobile phone). Finally, the intermediate depth map is smoothed again and median filtered to obtain an optimized depth map. The optimized depth map obtained at this time has undergone two guided filtering and two smoothing processes relative to the original image (i.e., the image captured by the mobile phone). It can be seen that in this example, by filtering the depth map twice and smoothing it twice, the integrity of the depth map edge is guaranteed, the phenomenon of missing and false blurring in the image blurring effect is avoided, and the imaging quality is improved.

[0060] Exemplary Methods

[0061] The depth map optimization processing method in this embodiment can be applied to a terminal, specifically, Figure 1 As shown in , the depth map optimization processing method in this embodiment includes the following steps:

[0062] Step S100: Obtain an original depth map, perform hole filling, preliminary guided filtering and preliminary smoothing on the original depth map to obtain a preliminary depth map.

[0063] To process the depth map, the smart terminal in this embodiment captures an image and then obtains an original depth map from the captured image. The image captured by the smart terminal is the original image, and the original depth map is obtained from the original image. Once the original depth map is obtained, this embodiment performs hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to perform preliminary processing on the grayscale image.

[0064] In one implementation, Figure 2 As shown in , in this embodiment, performing preliminary guided filtering and preliminary smoothing on the original depth map includes the following steps:

[0065] Step S101: obtaining an original image, performing grayscale processing on the original image, and obtaining a grayscale image;

[0066] Step S102: downsampling the grayscale image to obtain an original depth map;

[0067] Step S103: Obtain pixel depth values ​​on the original depth map, and determine a hole area based on the pixel depth values, where the hole area is an area where the pixel depth value meets a preset threshold;

[0068] Step S104: assigning pixel depth values ​​in the hole area to obtain a filled depth map;

[0069] Step S105: performing preliminary guided filtering on the filled depth map to obtain a filtered depth map;

[0070] Step S106 : performing preliminary smoothing processing on the filtered depth map to obtain a preliminary depth map.

[0071] In this embodiment, the image data is obtained by shooting with the main and auxiliary cameras, and then the image data is row-aligned and corrected to obtain the original image. After the original image is obtained, the original image is grayed to obtain a grayscale image. Then, based on the grayscale image, the StereoBM algorithm (stereo Block Match) is used to obtain the original depth map, and the holes in the original depth map are filled to obtain a filled depth map. In one embodiment, when obtaining the depth map, this embodiment uses downsampling to obtain 1 / 2 and 1 / 4 size data, and calculates multi-size depth maps f, f1 / 2, and f1 / 4. Since large-size depth maps are prone to holes, and small-size depth maps do not have sudden changes in pixel depth values ​​when calculated, in this embodiment, multi-size depth maps are obtained and small-size depth maps are filled, which can more accurately fill the holes in the original depth map to better restore the depth map.

[0072] In this embodiment, the hole area in the original depth map refers to the area where the pixel depth value cannot be calculated when the texture is repeated or weak in the original depth map. The pixel depth value of this area is 0, and the depth map visually has holes or is missing. Therefore, the hole can be judged by whether the pixel depth value is 0. In one embodiment, this embodiment first determines the hole in the grayscale image. Specifically, this embodiment obtains the pixel depth value on the grayscale image, and determines the hole area in the original depth map based on the pixel depth value. The hole area is an area where the pixel depth value meets a preset threshold (that is, the pixel value is 0); then, according to the hole area, the pixel depth value in the hole area is assigned to fill the hole area to obtain a filled depth map.

[0073] During specific implementation, this embodiment first obtains the pixel depth value of each pixel in the original depth map, and generates a filling mask based on the pixel depth value of each pixel. The filling mask in this embodiment is a binary image with the same width and height as the original depth map. In fact, in order to reflect whether the pixel depth value of each pixel in the subsequent filling mask is 255, it is judged whether the pixels at the corresponding position of the original depth map need to be filled. The hole area can be determined based on the filling mask. In this embodiment, this embodiment determines whether the pixel depth value of each pixel in the original depth map is equal to 0. If it is not equal to 0, the pixel value at the corresponding position in the generated filling mask is 0, otherwise it is 255. The formula for generating the filling mask in this embodiment is:

[0074]

[0075] Among them, f(i,j) is the pixel depth value of the coordinate depth map f at (i,j), and mask(i,j) is the value of the mask coordinate at (i,j).

[0076] The original depth map is accurately filled based on the filling mask. Specifically, after obtaining the filling mask, this embodiment makes a pixel-by-pixel judgment. If the pixel depth value in the filling mask of the coordinate (i, j) is 0, the value of f1 at (i, j) is assigned to F(i, j); if the pixel depth value in the filling mask of the coordinate (i, j) is 255, then it is determined whether the pixel depth value of f1 / 2 at (i, j) is 0. If it is not 0, the value of f1 / 2 at (i, j) is assigned to F(i, j); if it is 0, the value of f1 / 4 at (i, j) is assigned to F(i, j).

[0077] The corresponding formula is:

[0078]

[0079] Among them, F(i,j) is the value of the coordinate depth map F at (i,j), f(i,j) is the value of the coordinate f at (i,j), mask(i,j) is the value of the filling mask at (i,j), f k This embodiment can complete the filling of the original depth map through the above formula to obtain a filled depth map.

[0080] In one embodiment, after obtaining the filled grayscale image, this embodiment performs preliminary guided filtering and preliminary smoothing on the filled grayscale image. Specifically, this embodiment first determines the grayscale image corresponding to the filled depth map based on the filled depth map; then traverses all pixel points in the grayscale image to obtain multiple selected pixel points, and then calculates the depth value weight set of the filled depth map based on the multiple selected pixel points, and then calculates the new pixel depth value on the filled depth map based on the depth value weight set, and finally updates the pixel depth value of the filled depth map based on the new pixel depth value to obtain a filtered depth map. In one embodiment, when traversing all pixel points in the grayscale image, this embodiment first determines the pixel point to be processed, and then takes the pixel point to be processed as the center pixel point, calculates the depth value weight set of all pixel points with a distance r from the center pixel point in the four directions of up, down, left and right, calculates the pixel depth value of the filled depth map, and then obtains the filtered depth map based on the pixel depth value.

[0081] The calculation formula for pixel depth value is:

[0082]

[0083] Among them, f p is the pixel depth value of the point to be processed, f q is the pixel depth value of the four directions, W is the weight calculation function, and λ is the penalty coefficient.

[0084] In one embodiment, after the present embodiment obtains the filtered depth map, the first gradient value of the filtered depth map is obtained, and the area to be processed in the filtered depth map is determined based on the first gradient value, wherein the area to be processed is the area in the filtered depth map where the first gradient value is less than the first preset gradient threshold; the area to be processed is preliminarily smoothed to obtain a preliminary depth map. In specific implementation, the present embodiment can calculate the X-direction and Y-direction gradient values ​​in the filtered depth map, thereby obtaining the gradient of the entire depth map. Then, a first preset gradient threshold is set, and the obtained gradient is compared with the first preset gradient threshold, and gradients less than the first preset gradient threshold are screened out, and the areas corresponding to these gradients less than the first preset gradient threshold are obtained. These areas are then preliminarily smoothed to obtain a preliminary depth map.

[0085] Step S200: performing edge filtering on the preliminary depth map to obtain an intermediate depth map.

[0086] In this embodiment, after obtaining the preliminary depth map, this embodiment also needs to perform edge filtering on the preliminary depth map. Through edge filtering, the edge details of the preliminary depth map can be optimized, thereby more comprehensively retaining the main edge of the preliminary depth map.

[0087] Specifically, if Figure 3As shown in , step S200 specifically includes:

[0088] Step S201: Determine an edge area mask in a preliminary depth map;

[0089] Step S202: obtaining pixels to be processed in the edge region mask, where the pixels to be processed are pixels in the edge region mask whose pixel values ​​are not 0;

[0090] Step S203: Determine the depth value weight corresponding to the preliminary depth map according to the pixel to be processed, and determine the filtered depth value according to the depth value weight;

[0091] Step S204: performing edge filtering on the preliminary depth map according to the filtered depth value to obtain an intermediate depth map.

[0092] In one implementation, this embodiment first calculates the second gradient value of the pixel point in the preliminary depth map based on the preliminary depth map; then, based on the second gradient value, determines the edge area in the preliminary depth map, and the edge area is the area where the second gradient value is greater than the second preset gradient threshold. Finally, the edge area is morphologically processed and expanded to obtain an edge area mask. In specific implementation, the second gradient value of the pixel point in the preliminary depth map is calculated in this embodiment, mainly by calculating the gradient in the x and y directions through the sobel operator, and adding the pixel gradient in proportion to obtain the second gradient value. Then a reasonable second preset gradient threshold is set, and the calculated second gradient value is compared with the second preset gradient threshold to determine the second gradient value greater than the second preset gradient threshold, and obtain the area composed of pixel points corresponding to these second gradient values ​​greater than the second preset gradient threshold. These areas are edge areas. The calculation formula is:

[0093]

[0094] Among them, F(i,j) is the second gradient value of the coordinate preliminary depth map F at (i,j), edge(i,j) is the value of the Edge (edge ​​area mask) coordinate at (i,j), and T is the second preset gradient threshold.

[0095] After the edge area is determined, in this embodiment, morphological processing and dilation processing are performed to obtain an edge area mask. After the edge area mask is obtained, the preliminary depth map is filtered according to the edge area mask. Specifically, in this embodiment, the pixel points to be processed in the edge area mask are first obtained. The pixel points to be processed are the pixel points in the edge area mask whose pixel values ​​are not 0; then, according to the pixel points to be processed, the depth value weight corresponding to the preliminary depth map is determined, and based on the depth value weight, the filtered depth value is determined. Specifically, in this embodiment, the first pixel point and the second pixel point to be processed are first obtained. The first pixel point is the pixel point at the same coordinate position of the color image corresponding to the preliminary depth map of the pixel to be processed, and the distance between the second pixel point and the first pixel point is less than the preset distance R; the inter-pixel color difference and Euclidean distance between the first pixel point and the second pixel point are calculated respectively; the depth value weight is calculated according to the inter-pixel color difference and Euclidean distance, and the filtered depth value is determined according to the depth value weight. The filtered depth value is the sum of all depth value weights. Finally, the intermediate depth map is obtained according to the filtered pixel depth value. In this embodiment, the pixel depth values ​​of the pixels with a pixel value of 0 in the edge area mask remain unchanged.

[0096] In one embodiment, to reduce the amount of filtering calculations, downsampling is performed when selecting the second pixel point, with calculations performed at intervals in the front-back and top-bottom directions, reducing the amount of calculations to 1 / 4 of the original amount. The color image corresponding to the original depth map is used as a guide map, and the preliminary depth map is filtered to obtain an intermediate depth map, "guided." The specific calculation formula is:

[0097]

[0098] Among them, p is the pixel to be processed, q is the pixel in the window with radius R centered on p, and f c is the color weight calculation function, f s It is the spatial weight calculation function.

[0099] Step S300 : performing smoothing and median filtering on the intermediate depth map to obtain an optimized depth map.

[0100] In this embodiment, after obtaining the intermediate depth map, it is necessary to perform smoothing and median filtering on the intermediate depth map again, so that the intermediate depth map is filtered and smoothed twice relative to the depth map of the original image, thereby better repairing the original depth map, ensuring the integrity of the edge, and avoiding the phenomenon of false blur and missing blur in the blur effect.

[0101] In one embodiment, Figure 4 As shown in , step S300 includes:

[0102] Step S301: Obtain a third gradient value of the intermediate depth map, and determine a to-be-processed area in the intermediate depth map according to the third gradient value; wherein the to-be-processed area is an area in the intermediate depth map where the third gradient value is less than a third preset gradient threshold;

[0103] Step S302: Smoothing and median filtering are performed on the area to be processed in the intermediate depth map to obtain an optimized depth map.

[0104] The method for smoothing the intermediate depth map in this embodiment can be the same as the method for performing preliminary smoothing in the above-mentioned embodiment. Specifically, this embodiment first obtains the third gradient value of the intermediate depth map, and determines the area to be processed in the intermediate depth map based on the third gradient value, wherein the area to be processed is the area in the intermediate depth map where the third gradient value is less than the third preset gradient threshold. Then the area to be processed in the intermediate depth map is smoothed again and median filtered to reduce noise to obtain an optimized depth map. The depth optimization map obtained by this embodiment can well reflect the edge details, and the integrity of the edge can be restored after two filtering and two smoothing processes. As Figure 5 As shown in Figure 5 The middle one is the effect comparison chart, Figure 5 Middle A is the original expectation, Figure 5 B in the figure is the optimized depth map obtained by the method provided in this embodiment. Figure 5 The edge details can be clearly reflected in B, thereby improving the quality of the depth map.

[0105] It can be seen that when the embodiment of the present invention performs depth map optimization processing, it first obtains the original depth map, and performs hole filling, preliminary guide filtering and preliminary smoothing processing on the original depth map to obtain a preliminary depth map. Since the edges in the obtained preliminary depth map are not complete enough, it affects the blurring effect of the image. Therefore, after obtaining the preliminary depth map, this embodiment also performs edge filtering processing on the preliminary depth map to obtain an intermediate depth map, eliminating interference factors such as noise in the preliminary depth map. Then the intermediate depth map is smoothed again and median filtered to obtain an optimized depth map. The edge details are guaranteed by smoothing again and median filtering. It can be seen that in the embodiment of the present invention, the original depth map is filtered twice and smoothed twice, which not only eliminates the interference factors in the original depth map, but also ensures the integrity of the edges of the depth map, avoiding the phenomenon of missing and false blurring in the image blurring effect.

[0106] Exemplary devices

[0107] like Figure 6As shown in , an embodiment of the present invention provides a depth map optimization processing device, which includes: a depth map processing unit 10, a depth map filtering unit 20, and a depth map smoothing unit 30. Specifically, the original depth map processing unit 10 is used to obtain an original depth map, perform hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map. The original depth map filtering unit 20 is used to perform edge filtering on the preliminary depth map to obtain an intermediate depth map. The original depth map smoothing unit 30 is used to perform further smoothing and median filtering on the intermediate depth map to obtain an optimized depth map.

[0108] Based on the above embodiment, the present invention also provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 7 As shown. The smart terminal includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected via a system bus. The processor of the smart terminal is used to provide computing and control capabilities. The memory of the smart terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the smart terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a depth map optimization processing method is implemented. The display screen of the smart terminal can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor of the smart terminal is pre-set inside the smart terminal to detect the operating temperature of the internal device.

[0109] Those skilled in the art will understand that Figure 7 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention and does not constitute a limitation on the smart terminal to which the solution of the present invention is applied. The specific smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0110] In one embodiment, a smart terminal is provided, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0111] Obtain an original depth map, perform hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map;

[0112] Perform edge filtering on the preliminary depth map to obtain an intermediate depth map;

[0113] The intermediate depth map is smoothed again and median filtered to obtain an optimized depth map.

[0114] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0115] In summary, the present invention discloses a depth map optimization processing method, computer device, and storage medium. The method includes: obtaining an original depth map, performing hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map; performing edge filtering on the preliminary depth map to obtain an intermediate depth map; and performing further smoothing and median filtering on the intermediate depth map to obtain an optimized depth map. By performing two filtering and two smoothing operations on the depth map, the present invention ensures the integrity of the depth edge and avoids the phenomenon of missing or incorrect blurring in the image blurring effect.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A depth map optimization processing method, characterized in that: include: Obtaining an original depth map, performing hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map; Performing edge filtering on the preliminary depth map to obtain an intermediate depth map; Performing further smoothing and median filtering on the intermediate depth map to obtain an optimized depth map; The performing edge filtering on the preliminary depth map to obtain an intermediate depth map includes: determining an edge region mask in the preliminary depth map; Obtaining the pixel points to be processed in the edge area mask; Determine a depth value weight corresponding to the preliminary depth map according to the pixel to be processed, and determine a filtered depth value according to the depth value weight; Performing edge filtering on the preliminary depth map according to the filtered depth value to obtain an intermediate depth map; The step of determining a depth value weight corresponding to the preliminary depth map according to the pixel to be processed, and determining a filtered depth value according to the depth value weight, includes: Obtaining a first pixel point and a second pixel point of the pixel point to be processed, where the first pixel point is a pixel point at the same coordinate position of the pixel to be processed in the color image corresponding to the preliminary depth map, and a distance between the first pixel point and the second pixel point is less than a preset distance; respectively calculating an inter-pixel color difference and an inter-pixel Euclidean distance between the first pixel and the second pixel; A depth value weight is calculated according to the color difference between pixels and the Euclidean distance, and a filtered depth value is determined according to the depth value weight. The filtered depth value is the sum of all depth value weights.

2. The method according to claim 1, characterized in that The obtaining of the original depth map, performing hole filling, preliminary guided filtering, and preliminary smoothing on the original depth map to obtain a preliminary depth map includes: Acquire an original image, and perform grayscale processing on the original image to obtain a grayscale image; Downsampling the grayscale image to obtain an original depth map. Obtaining pixel depth values ​​on the original depth map, and determining a hole area according to the pixel depth values, wherein the hole area is an area where the pixel depth value meets a preset threshold; Assigning pixel depth values ​​in the hole area to obtain a filled depth map; performing preliminary guided filtering on the filled depth map to obtain a filtered depth map; Perform preliminary smoothing on the filtered depth map to obtain a preliminary depth map.

3. The method according to claim 2, characterized in that The performing preliminary guided filtering on the filled depth map to obtain a filtered depth map includes: Determining a grayscale image corresponding to the filled depth map; Traversing all pixels in the grayscale image to obtain a plurality of selected pixels; Calculating a depth value weight set of the filled depth map according to the multiple selected pixel points; Calculating a new pixel depth value on the filled depth map according to the depth value weight set; The pixel depth values ​​of the filled depth map are updated according to the new pixel depth values ​​to obtain a filtered depth map.

4. The method according to claim 3, characterized in that The performing preliminary smoothing on the filtered depth map to obtain a preliminary depth map includes: Obtaining a first gradient value of the filtered depth map, and determining a to-be-processed area according to the first gradient value; wherein the to-be-processed area is an area in the filtered depth map where the first gradient value is less than a first preset gradient threshold; Perform preliminary smoothing on the area to be processed to obtain a preliminary depth map.

5. The method according to claim 1, wherein Determining an edge area mask in the preliminary depth map includes: Calculating a second gradient value of a pixel in the preliminary depth map; determining an edge area in the preliminary depth map according to the second gradient value, the edge area being an area where the second gradient value is greater than a second preset gradient threshold; Morphological processing and dilation processing are performed on the edge region to obtain an edge region mask in the preliminary depth map.

6. The method according to claim 1, characterized in that The further smoothing and median filtering of the intermediate depth map to obtain an optimized depth map includes: Obtaining a third gradient value of the intermediate depth map, and determining a to-be-processed area in the intermediate depth map according to the third gradient value; wherein the to-be-processed area is an area in the intermediate depth map where the third gradient value is less than a third preset gradient threshold; The area to be processed in the intermediate depth map is smoothed and median filtered to obtain an optimized depth map.

7. An intelligent terminal, characterized in that: The device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, wherein the one or more programs include the method according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method according to any one of claims 1 to 6.

Images