Depth Image Denoising Method, Electronic Device and Storage Medium

Abstract

The embodiment of the present invention relates to the field of image processing, and discloses a method for denoising a deep image, an electronic device, and a storage medium, wherein the method comprises: performing hierarchical splitting of a depth image to be processed according to a preset splitting rule to form a plurality of first sub-regions; each first sub-region is a region obtained in the hierarchical splitting process that cannot be further split; hierarchical merging of the plurality of first sub-regions according to a preset merging rule to form a plurality of second sub-regions; each second sub-region is a region obtained in the hierarchical merging process that cannot be further merged; determining a second sub-region in the plurality of second sub-regions whose area is less than an area threshold as a noise region, and denoising the noise region. This solution can achieve the effect of efficiently removing noise from a depth image on the basis of retaining the depth value of the correct region as much as possible.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a depth image denoising method, electronic equipment and storage medium. Background Art

[0002] At present, the most dynamic technical branch in the field of machine vision is depth perception technology, which is widely used in applications such as 3D scene reconstruction, target detection and recognition. However, whether it is based on active structured light technology, passive stereo vision, or time-of-flight technology, the depth map obtained has more or less noise. If these noises are processed using commonly used median filtering, Gaussian filtering, or bilateral filtering, while removing the noise, they may affect the values ​​of the correct area of ​​the depth map. Therefore, the existing technology often uses a connected domain denoising algorithm to denoise the depth map, which will not destroy the depth value of the correct area while removing the noise.

[0003] The processing process of the connected domain denoising algorithm is as follows: the connected domain threshold is calculated based on the depth information of the current pixel, and the connected domain threshold is traversed from the upper left to the lower right or from the lower right to the upper left according to the threshold to form a connected domain. However, using this traversal order will cause image breakage when the target object (such as a finger) is tilted 45 degrees to the upper right in the depth map. This situation is more obvious when the area threshold is set too small. In fact, as long as a connected domain is formed by traversing from a certain point according to a certain rule, this method will have a break phenomenon under certain circumstances. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a depth image denoising method, an electronic device and a storage medium, which can achieve the effect of efficiently removing depth image noise on the basis of retaining the correct area depth value as much as possible.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a depth image denoising method, comprising:

[0006] The depth image to be processed is hierarchically split according to a preset splitting rule to form a plurality of first sub-regions; each first sub-region is a region obtained in the hierarchical splitting process that cannot be further split;

[0007] The plurality of first sub-regions are hierarchically merged according to a preset merging rule to form a plurality of second sub-regions; each second sub-region is a region obtained in the hierarchical merging process that cannot be further merged;

[0008] A second sub-region whose area is smaller than an area threshold among the multiple second sub-regions is determined as a noise region, and denoising is performed on the noise region.

[0009] An embodiment of the present invention further provides an electronic device, including:

[0010] at least one processor; and,

[0011] a memory communicatively connected to the at least one processor; wherein,

[0012] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the depth image denoising method as described above.

[0013] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned depth image denoising method when executed by a processor.

[0014] Compared with the prior art, the embodiment of the present invention forms a plurality of first sub-regions by performing hierarchical splitting on the depth image to be processed according to a preset splitting rule; each first sub-region is a region that cannot be further split obtained in the hierarchical splitting process; the plurality of first sub-regions are hierarchically merged according to a preset merging rule to form a plurality of second sub-regions; each second sub-region is a region that cannot be further merged obtained in the hierarchical merging process; a second sub-region whose area is less than an area threshold in the plurality of second sub-regions is determined as a noise region, and the noise region is denoised. This scheme firstly performs hierarchical splitting on the depth image according to the preset splitting rule to form a plurality of first sub-regions that cannot be further split; then, all the first sub-regions are hierarchically merged according to the preset merging rule to form a plurality of second sub-regions that cannot be further merged; the depth image to be processed is traversed by the "split + merge" traversal method to form the final connected domain (second sub-region), thereby effectively avoiding the phenomenon of the connected domain being broken in certain circumstances caused by the use of a single traversal method, so as to achieve the effect of efficiently removing the noise of the depth image on the basis of retaining the depth value of the correct region as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a specific flow chart of a depth image denoising method according to an embodiment of the present invention;

[0016] Figure 2 is a specific flow chart of the splitting operation according to an embodiment of the present invention;

[0017] Figure 3 is an example diagram of a split image generated by a splitting operation according to an embodiment of the present invention;

[0018] Figure 4 is an example diagram of a quadtree structure used in a splitting operation according to an embodiment of the present invention;

[0019] Figure 5 is a specific flow chart of the merging operation according to an embodiment of the present invention;

[0020] Figure 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the following will be described in detail with reference to the accompanying drawings. However, it will be appreciated by those skilled in the art that in the various embodiments of the present invention, many technical details are provided in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented.

[0022] One embodiment of the present invention relates to a method for denoising a deep image. Figure 1 As shown, the depth image denoising method provided by this embodiment includes the following steps.

[0023] Step 101: hierarchically split the depth image to be processed according to a preset splitting rule to form a plurality of first sub-regions; each first sub-region is a region obtained in the hierarchical splitting process that cannot be further split.

[0024] Among them, the splitting rules are pre-set execution rules for determining whether to split the current area to be split and how to split it. Hierarchical splitting means that after executing the splitting rules once for the current area to be split, the above splitting rules can be executed again on the sub-areas obtained after the splitting operation as new areas to be split. The iterative splitting operation performed in this way can be called hierarchical splitting. When determining whether to split the current area to be split, a corresponding splitting threshold can be set. The splitting threshold is the critical value for judging whether to split the area to be split as agreed upon by the splitting rules; the splitting threshold can be updated as the area area of ​​the depth image to be split is different each time. For example, the splitting threshold can be proportional to the area area of ​​the area to be split.

[0025] In this embodiment, the splitting rules used for hierarchical splitting of the depth image are not limited, and those skilled in the art can design the splitting rules and splitting thresholds according to their own needs. For example, the splitting rule of evenly splitting the image to be split according to the area can be used for splitting.

[0026] Specifically, the depth image to be processed is hierarchically split according to a preset splitting rule, and finally multiple first sub-regions that cannot be further split are formed. The inability to further split each first sub-region can be reflected in: the splitting metric value calculated for these first sub-regions according to the splitting rule is not greater than the corresponding splitting threshold (in this case, the splitting rule stipulates that the splitting metric value needs to be greater than the corresponding splitting threshold to further perform the splitting operation); or, the splitting metric value corresponding to the first sub-region calculated according to the splitting rule is not less than the corresponding splitting threshold (in this case, the splitting rule stipulates that the splitting metric value needs to be less than the corresponding splitting threshold to further perform the splitting operation), etc.

[0027] In one example, as Figure 2 shown, the entire region of the depth image can be used as the first region to be split, and the following splitting operations (sub-steps 1011 - 1013) are performed for each region to be split to implement the hierarchical splitting process of this step.

[0028] Sub-step 1011: Calculate the mean square deviation of the depths of the pixel points in the current region to be split.

[0029] Specifically, for all N pixel points P (coordinates are (i, j)) in the region to be split Ω, the following formulas (1) and (2) can be used to calculate the average value C of all depth values in this region, and the mean square deviation RS of the depths is calculated based on the average depth value C.

[0030]

[0031]

[0032] Sub-step 1012: If the mean square deviation of the depths is less than the splitting threshold corresponding to the current region to be split, then mark the current region to be split as a first sub-region, and end the splitting operation for the current region to be split.

[0033] Specifically, the mean square deviation of the depths can be used to judge the overall uniformity of an image region, and the smaller the mean square deviation of the depths, the better the corresponding overall uniformity. If the mean square deviation of the depths of the pixel points in the current region to be split is less than the splitting threshold corresponding to the current region to be split, it indicates that the overall uniformity of the current region to be split is relatively good. At this time, there is no need to further split the current region to be split, but directly mark the current region to be split as a first sub-region, and end the splitting operation for the current region to be split.

[0034] Sub-step 1013: If the mean square deviation of the depths is not less than the splitting threshold corresponding to the current region to be split, then split the current region to be split into multiple regions, and use each of these multiple regions as a new region to be split to perform the splitting operation.

[0035] Specifically, if the mean square deviation of the depths of the pixel points in the currently to-be-split region is not less than the splitting threshold corresponding to the currently to-be-split region, it indicates that the overall uniformity of the currently to-be-split region is relatively not good, and there may be multiple object objects or noise. At this time, it is necessary to further split the currently to-be-split region to obtain multiple split regions, and each of the obtained split regions can be used as a new to-be-split region to perform the splitting operation, that is, to perform the splitting process as shown in Figure 2 shown again.

[0036] In this way, through the above splitting operation, the original to-be-processed depth image can be split into multiple image regions of unequal sizes, and each image region is regarded as a first sub-region. For example, when splitting the currently to-be-split region into multiple regions, the currently to-be-split region can be split into M regions of equal area, where M is an integer greater than 0. For convenient management, the M regions obtained by splitting the to-be-split region each time can be managed through an M-ary tree structure.

[0037] For example Figure 3 and Figure 4 respectively show the split images ( Figure 3 ) after the original to-be-processed depth image is split when M takes the value of 4, and the quadtree structure ( Figure 4 ) for managing the split images.

[0038] Step 102: Hierarchically merge the multiple first sub-regions according to a preset merging rule to form multiple second sub-regions; each second sub-region is a region that cannot be further merged during the hierarchical merging process.

[0039] Among them, the merging rule is an execution rule preset for determining whether to merge two currently to-be-merged regions and how to merge them. Hierarchical merging means that after executing a merging rule for two currently to-be-merged regions, the sub-region obtained after this merging operation can continue to be used as a new to-be-merged region to execute the above merging rule again with other regions. Such an iterative merging operation can be called hierarchical merging. When determining whether to merge two currently to-be-merged regions, a corresponding merging threshold can be set, and this merging threshold is a judgment critical value for whether to merge two to-be-merged regions as agreed by the merging rule; the merging threshold can be updated as the area of the regions of the depth image to be merged each time is different. For example, the merging threshold can be proportional to the area of the two regions to be merged, or calculated based on the splitting thresholds of the two regions to be merged.

[0040] In this embodiment, there is no limitation on the merging rules used for hierarchical merging of the first sub-regions. Those skilled in the art can design the merging rules and merging thresholds according to their own needs. For example, the first sub-regions can be selected for merging in descending order of their areas, and the adjacent rules can also be referred to during the merging process.

[0041] Specifically, multiple first sub-regions are hierarchically merged according to the pre-set merging rules, and finally multiple second sub-regions that cannot be further merged are formed. The inability to be further merged for each second sub-region can be reflected in: when these second sub-regions are combined with any adjacent first sub-region, the merging metric value corresponding to the combined region calculated according to the merging rules is not greater than the corresponding merging threshold (in this case, the merging rule stipulates that the merging metric value needs to be greater than the corresponding merging threshold to further perform the merging operation); or, the merging metric value corresponding to the combined region calculated according to the merging rules is not less than the corresponding merging threshold (in this case, the merging rule stipulates that the merging metric value needs to be less than the corresponding merging threshold to further perform the merging operation), etc.

[0042] In one example, as Figure 5 shown, one first sub-region with the largest area can be selected from the first sub-regions that have not been assigned to any second sub-region in sequence as the region to be merged to perform the following merging operations (sub-steps 1021 - 1023) until all first sub-regions are assigned to the second sub-regions, or any two regions in the remaining first sub-regions that have not been assigned to any second sub-region are not adjacent, so as to implement the hierarchical merging process of this step 102.

[0043] Sub-step 1021: Calculate the mean square deviation of the depths of the pixel points in the combined region obtained by sequentially combining the region to be merged and each first sub-region that is adjacent to the region to be merged and does not belong to any second sub-region.

[0044] Among them, in every two adjacent calculations, if the mean square deviation of the depths obtained in the previous calculation is less than the merging threshold corresponding to the combined region, the combined region in the previous calculation is used as the region to be merged in the next calculation; if the mean square deviation of the depths obtained in the previous calculation is not less than the merging threshold corresponding to the combined region, the region to be merged in the previous calculation is used as the region to be merged in the next calculation.

[0045] Specifically, before the merging operation is performed, all the first sub-regions included in the depth image do not belong to any second sub-region (at this time, the first second sub-region has not been formed yet). Therefore, all the first sub-regions can be regarded as the first sub-regions that have not been divided into any second sub-region. From all the first sub-regions, select the first sub-region with the largest area as the current region to be merged, and start the merging operation for this region to be merged. Determine the first regions that are adjacent to the region to be merged and do not belong to any second sub-region, and calculate the depth mean square error of the pixel points in the combined region formed by combining the region to be merged and each of these first sub-regions in turn. Among them, the process of calculating the depth mean square error of the pixel points in the image region can refer to Formula (1) and Formula (2), which will not be elaborated here. And, each time the depth mean square error of the combined region is calculated, the region to be merged included in the combined region is determined based on the region to be merged or the combined region when calculating the depth mean square error of the previous combined region. The determination rule is: in two adjacent calculations, if the depth mean square error obtained in the previous time is less than the merging threshold corresponding to its combined region, then use the previous combined region as the region to be merged in the next time; if the depth mean square error obtained in the previous time is not less than the merging threshold corresponding to its combined region, then use the region to be merged in the previous time as the region to be merged in the next time.

[0046] For example, in two adjacent calculations, when calculating the depth mean square error of the combined region in the previous time, the corresponding combined region includes the region to be merged A1 and a first sub-region R1 adjacent to A1; two execution results can be generated according to the depth mean square error obtained in this calculation:

[0047] If the depth mean square error obtained in this time is less than the merging threshold corresponding to its combined region, it indicates that the combined region still has the possibility of continuing to merge with other regions. Then, at this time, the previous combined region (A1 + R1) can be used as the region to be merged in the next time, and then the new region to be merged (A1 + R1) and a first sub-region R2 that is adjacent to the region to be merged (A1 + R1) and does not belong to any second sub-region are combined to form the combined region (A1 + R1 + R2) in the next time, and continue to calculate the depth mean square error of the combined region (A1 + R1 + R2) in the next time.

[0048] If the mean square deviation of depth obtained this time is not less than the merging threshold corresponding to its combined region, it indicates that the combined effect of this combined region itself is not good, and there is even less possibility of continuing to merge with other regions. In this case, the previously waiting-to-be-merged region A1 can be continued to be used as the waiting-to-be-merged region for the next time. Then, the new waiting-to-be-merged region (A1) and a first sub-region R3 that is adjacent to this waiting-to-be-merged region (A1) and does not belong to any second sub-region are combined to form the combined region (A1 + R3) for the next time, and the mean square deviation of depth of the combined region (A1 + R3) for the next time is calculated continuously.

[0049] Thus, it can be seen that the size relationship between the mean square deviation of depth of the combined region obtained each time and the corresponding merging threshold can affect the waiting-to-be-merged regions included in the combined region when calculating the mean square deviation of depth of the combined region next time. And as the waiting-to-be-merged regions are continuously updated, the first sub-regions that are adjacent to the waiting-to-be-merged regions and do not belong to any second sub-region are also continuously updated.

[0050] Sub-step 1022: If the mean square deviation of depth obtained in the last calculation is less than the merging threshold corresponding to the combined region of that time, then the combined region of the last time is used as a second sub-region.

[0051] Specifically, if when performing the calculation of the mean square deviation of depth at a certain time, it is determined that there is no first sub-region that is adjacent to the waiting-to-be-merged region of that time and does not belong to any second sub-region, then it is considered that the task of calculating the mean square deviation of depth at that time fails, and the current merging operation is ended. The previous calculation of that calculation is determined as the last calculation. If the mean square deviation of depth obtained in the last calculation is less than the merging threshold corresponding to the combined region of that time, it indicates that the combined region in the last calculation has good uniformity, and the combined region of the last time can be used as a second sub-region.

[0052] Sub-step 1023: If the mean square deviation of depth obtained in the last calculation is not less than the merging threshold corresponding to the combined region of that time, then the waiting-to-be-merged region in the last calculation is used as a second sub-region.

[0053] Specifically, if when performing the calculation of the mean square deviation of depth at a certain time, it is determined that there is no first sub-region that is adjacent to the waiting-to-be-merged region of that time and does not belong to any second sub-region, then it is considered that the task of calculating the mean square deviation of depth at that time fails, and the current merging operation is ended. The previous calculation of that calculation is determined as the last calculation. If the mean square deviation of depth obtained in the last calculation is not less than the merging threshold corresponding to the combined region of that time, it indicates that the combined region in the last calculation does not have good uniformity, and the waiting-to-be-merged region in the last calculation can be used as a second sub-region.

[0054] After completing the current merging operation and determining the second sub-region generated by the current merging operation, one can continue to select the largest first sub-region from the first sub-regions that have not been assigned to any second sub-region as the region to be merged and perform the above merging operation. Repeat this operation until all first sub-regions are assigned to the second sub-regions, or any two regions among the remaining first sub-regions that have not been assigned to any second sub-region are not adjacent.

[0055] Finally, after multiple above merging operations, there may still be first sub-regions in the depth image that have not been assigned to any second sub-region. At this time, each of the remaining first sub-regions that have not been assigned to any second sub-region can be directly used as an independent second sub-region, thus finally realizing the division of the depth image to be processed into multiple connected regions, that is, the second sub-regions.

[0056] In addition, during each execution of the merging operation, it also includes:

[0057] Before calculating the depth mean square error of the pixel points in the combined region formed by combining the region to be merged and a first sub-region adjacent to the region to be merged during each execution of step 1021, the following steps can also be performed.

[0058] Step 1: Calculate the difference in the average depth of the pixel points between the region to be merged and the first sub-region adjacent to the region to be merged.

[0059] Step 2: If the difference is less than the difference threshold, determine to perform the operation of calculating the depth mean square error of the pixel points in the combined region formed by combining the region to be merged and a first sub-region adjacent to the region to be merged.

[0060] The smaller the difference in the average depth of the pixel points between two regions, the greater the possibility that the two regions are regions of the same target object as a whole. In this way, when the above difference is less than the difference threshold and then it is determined to perform the operation of calculating the depth mean square error, the possibility of successful combination of the two regions is greater.

[0061] In one example, the method for obtaining the above merging threshold may include:

[0062] Calculate the merging threshold using the following formula:

[0063]

[0064] where T s is the merging threshold corresponding to the combined region S, Si is the i-th first sub-region included in the combined region S, N Si is the number of pixel points included in the i-th first sub-region Si in the combined region S, T Siis the splitting threshold corresponding to the i-th first sub-region Si in the combined region S.

[0065] Step 103: Determine a second sub-region whose area is smaller than an area threshold among the plurality of second sub-regions as a noise region, and perform denoising on the noise region.

[0066] Specifically, after all the merging operations are completed, multiple second sub-regions with different areas and cannot be merged are obtained. The areas of these second sub-regions are compared with a preset area threshold, and the second sub-regions whose areas are smaller than the area threshold are determined as noise regions, and denoising is performed on the noise regions.

[0067] In one example, the depth values ​​of the pixels in the noise area may be set to 0 to implement denoising.

[0068] Compared with the related art, this embodiment forms multiple first sub-regions by performing hierarchical splitting on the depth image to be processed according to the preset splitting rules; each first sub-region is a region that cannot be further split obtained in the hierarchical splitting process; the multiple first sub-regions are hierarchically merged according to the preset merging rules to form multiple second sub-regions; each second sub-region is a region that cannot be further merged obtained in the hierarchical merging process; the second sub-region with an area less than the area threshold in the multiple second sub-regions is determined as a noise region, and the noise region is denoised. This scheme will firstly perform hierarchical splitting on the depth image according to the preset splitting rules to form multiple first sub-regions that cannot be further split; then, all the first sub-regions are continuously hierarchically merged according to the preset merging rules to form multiple second sub-regions that cannot be further merged; the depth image to be processed is traversed by the "split + merge" traversal method to form the final connected domain (second sub-region), thereby effectively avoiding the phenomenon of connected domain rupture in certain situations caused by the use of a single traversal method, so as to achieve the effect of efficiently removing the depth image noise on the basis of retaining the correct area depth value as much as possible.

[0069] Another embodiment of the present invention relates to an electronic device, such as Figure 6 As shown, it includes at least one processor 202; and a memory 201 that is communicatively connected to the at least one processor 202; wherein the memory 201 stores instructions that can be executed by the at least one processor 202, and the instructions are executed by the at least one processor 202 so that the at least one processor 202 can execute any of the above method embodiments.

[0070] Among them, the memory 201 and the processor 202 are connected in a bus manner. The bus can include any number of interconnected buses and bridges, which connect various circuits of one or more processors 202 and the memory 201 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, etc. These are well known in the art, so they will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, and provides a unit for communicating with various other devices over a transmission medium. The data processed by the processor 202 is transmitted over a wireless medium via the antenna. Further, the antenna also receives data and transmits the data to the processor 202.

[0071] The processor 202 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 201 can be used to store the data used by the processor 202 when performing operations.

[0072] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements any one of the above method embodiments.

[0073] That is, those skilled in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by a program instructing relevant hardware. The program is stored in a storage medium, including several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0074] Those of ordinary skill in the art can understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes can be made in form and details without departing from the spirit and scope of the present invention.

Claims

1. A depth image denoising method, characterized in that, Including: Performing hierarchical splitting on the depth image to be processed according to a preset splitting rule to form a plurality of first sub-regions; each first sub-region is a region that cannot be further split obtained during the hierarchical splitting process; Performing hierarchical merging on the plurality of first sub-regions according to a preset merging rule to form a plurality of second sub-regions; each second sub-region is a region that cannot be further merged obtained during the hierarchical merging process; Determining the second sub-regions with an area smaller than the area threshold among the plurality of second sub-regions as noise regions, and performing denoising processing on the noise regions; The performing hierarchical splitting on the depth image to be processed according to a preset splitting rule to form a plurality of first sub-regions includes: Taking the entire region of the depth image as the first region to be split, and performing the following splitting operation for each region to be split: Calculating the depth mean square deviation of the pixel points in the current region to be split; If the depth mean square deviation is less than the splitting threshold corresponding to the current region to be split, then recording the current region to be split as one of the first sub-regions, and ending the splitting operation on the current region to be split; If the depth mean square deviation is not less than the splitting threshold corresponding to the current region to be split, then splitting the current region to be split into a plurality of regions, and taking each of the plurality of regions as a new region to be split to perform the splitting operation.

2. The method according to claim 1, wherein The splitting the current region to be split into a plurality of regions includes: Splitting the current region to be split into M regions with equal areas, where M is an integer greater than 0.

3. The method according to claim 2, characterized in that, The method further includes: Managing the M regions obtained by splitting the regions to be split each time through an M-ary tree structure.

4. The method according to any one of claims 1 to 3, characterized in that, The performing hierarchical merging on the plurality of first sub-regions according to a preset merging rule to form a plurality of second sub-regions includes: Sequentially selecting, from the first sub-regions that have not been divided into any of the second sub-regions, the first sub-region with the largest area as the region to be merged to perform the following merging operation until all the first sub-regions are all divided into the second sub-regions, or any two regions among the remaining first sub-regions that have not been divided into any of the second sub-regions are not adjacent: Sequentially calculating the depth mean square deviation of the pixel points in the combined region after combining the region to be merged and each first sub-region that is adjacent to the region to be merged and does not belong to any of the second sub-regions; Wherein, in each adjacent two calculations, if the depth mean square deviation obtained in the previous time is less than the merging threshold corresponding to the combined region, then taking the combined region in the previous time as the region to be merged in the next time; if the depth mean square deviation obtained in the previous time is not less than the merging threshold corresponding to the combined region, then taking the region to be merged in the previous time as the region to be merged in the next time; If the depth mean square deviation obtained in the last calculation is less than the merging threshold corresponding to the combined region in that time, then taking the combined region in the last time as one of the second sub-regions; If the depth mean square deviation obtained in the last calculation is not less than the merging threshold corresponding to the combined region in that time, then taking the region to be merged in the last time as one of the second sub-regions; Each of the first sub-regions that have not been assigned to any of the second sub-regions is used as an independent second sub-region respectively.

5. The method according to claim 4, wherein The merging operation further includes: Before calculating the mean square error of the depths of the pixel points in the combined region obtained by combining the region to be merged and a first sub-region adjacent to the region to be merged each time, it further includes: Calculating the difference in the average depth of the pixel points between the region to be merged and the first sub-region adjacent to the region to be merged. If the difference is less than the difference threshold, it is determined to perform the operation of calculating the mean square error of the depths of the pixel points in the combined region obtained by combining the region to be merged and a first sub-region adjacent to the region to be merged this time.

6. The method according to claim 4, characterized in that The method for obtaining the merging threshold includes: Calculating the merging threshold using the following formula: Among them, T s is the merging threshold corresponding to the combined region S, Si is the i-th first sub-region included in the combined region S, N Si is the number of pixel points included in the i-th first sub-region Si in the combined region S, and T Si is the splitting threshold corresponding to the i-th first sub-region Si in the combined region S.

7. The method according to any one of claims 1-3, 5, and 6, characterized in that The denoising process for the noise region includes: Setting the depth value of the pixel points in the noise region to 0.

8. An electronic device, characterized in that, It includes: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the depth image denoising method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the depth image denoising method according to any one of claims 1 to 7.

Images