A noise filtering method, device, equipment and storage medium
By traversing pixels in the depth map and obtaining and judging the similarity and number thresholds of nearest neighbor points, the image distortion problem caused by inaccurate noise point judgment in the existing technology is solved, and a highly efficient noise filtering effect is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AIRLOOK TECH (BEIJING) CO LTD
- Filing Date
- 2023-04-28
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies suffer from high image distortion due to inaccurate identification of noise points when filtering depth map noise.
By traversing the pixels in the depth map, nearest neighbor points are obtained and candidate noise points are judged according to preset rules. The judgment steps are repeated until the traversal ends. When the second preset rule is met, candidate noise points are filtered out. Accurate filtering is performed by using the similarity between the pixel and the nearest neighbor point and the threshold of the number of candidate noise points.
It achieves accurate and effective filtering of noisy data in depth maps, reducing image distortion.
Smart Images
Figure CN116612020B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and more particularly to a noise filtering method, apparatus, device, and storage medium. Background Technology
[0002] With the advent and development of depth measurement technology, acquiring depth images and using them for subsequent image processing has become increasingly common. Depth images, also known as distance images, have pixel depth values that represent the distance from the image acquisition device to various points in the scene. They directly reflect the geometry of visible surfaces and are typically used in image processing scenarios such as post-processing, real-time shadows, edge detection, and ambient occlusion.
[0003] In existing technologies, researchers use various filters to reduce noise in depth maps. This traditional denoising method works reasonably well in depth maps with reasonable noise levels. However, a significant portion of the images are too noisy, and the filtered images will be very blurry, and may also lose key details in the depth map. Summary of the Invention
[0004] To address the technical problem of high image distortion caused by inaccurate identification of noise points when filtering depth map noise in existing technologies, this invention provides a noise filtering method.
[0005] In a first aspect, the present invention provides a noise filtering method, comprising:
[0006] Traverse the pixels in the depth map of the noise to be filtered, wherein the nearest neighbor of the pixel is obtained during the traversal, and the nearest neighbor is determined according to the first preset rule to determine whether the nearest neighbor is a candidate noise point, the nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point is repeated until the traversal process ends.
[0007] Determine whether the candidate noise points satisfy the second preset rule;
[0008] If so, filter the candidate noise points.
[0009] Furthermore, the step of traversing the pixels in the depth map of the noise to be filtered includes:
[0010] The first pixel in the traversal process is stored in the detection point set;
[0011] Update the pixels in the detection point set based on the candidate noise points;
[0012] The update process includes clearing pixels from the detection point set and adding candidate noise points to the detection point set.
[0013] Furthermore, determining whether the nearest neighbor is a candidate noise point according to the first preset rule includes:
[0014] The similarity between the pixel and its nearest neighbor is used to determine whether the nearest neighbor is a candidate noise point.
[0015] Furthermore, determining whether a nearest neighbor is a candidate noise point based on the similarity between the pixel and its nearest neighbor includes:
[0016] Determine whether the pixel and its nearest neighbor satisfy depth(c) i )>0 and Where, depth(c i ) represents the depth value of the i-th nearest neighbor, depth(CurP) represents the depth value of the pixel, and σ represents the preset similarity threshold, σ∈[0.001, 0.1];
[0017] If so, the nearest neighbor point is taken as the candidate noise point.
[0018] Furthermore, after determining whether the nearest neighbor is a candidate noise point according to the first preset rule, the process includes:
[0019] If the nearest neighbor is a candidate noise point, then the candidate noise point is stored in the candidate noise point set.
[0020] Furthermore, determining whether the candidate noise point satisfies the second preset rule includes:
[0021] Determine whether the number of candidate noise points in the candidate noise point set meets a preset number threshold.
[0022] Furthermore, determining whether the number of candidate noise points in the candidate noise point set meets a preset threshold includes:
[0023] Determine whether the set of candidate noise points satisfies |C|≤CT, where |C| represents the number of candidate noise points in the set of candidate noise points, CT is the preset number threshold, and CT∈[5,500].
[0024] In a second aspect, the present invention provides a noise filtering device, comprising:
[0025] The traversal module is used to traverse the pixels in the depth map to be filtered for noise. During the traversal, the nearest neighbor of the pixel is obtained, and the nearest neighbor is determined to be a candidate noise point according to a first preset rule. The nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule is repeated until the traversal process ends.
[0026] The noise determination module is used to determine whether the candidate noise point satisfies the second preset rule;
[0027] The noise filtering module is used to filter candidate noise points that meet the second preset rule.
[0028] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the noise filtering method provided in the first aspect.
[0029] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the noise filtering method provided in the first aspect.
[0030] This invention traverses the pixels in a depth map to be filtered for noise; during the traversal, it obtains the nearest neighbors of each pixel and determines whether each nearest neighbor is a candidate noise point according to a first preset rule; it then obtains the nearest neighbors of the candidate noise points and repeats the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule until the traversal process ends; finally, it determines whether the candidate noise point satisfies a second preset rule; if so, it filters the candidate noise point. This invention determines and filters candidate noise points by judging the similarity between the detection point and its nearest neighbors, achieving an accurate and effective technical effect of filtering noise data in the depth map. This solves the technical problem in the prior art where inaccurate judgment of noise points during depth map noise filtering leads to high image distortion. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 This is a schematic flowchart of the noise filtering method provided in an embodiment of the present invention;
[0033] Figure 2 This is a schematic diagram of selecting nearest neighbor points provided in an embodiment of the present invention;
[0034] Figure 3 A schematic diagram of the traversal process provided in an embodiment of the present invention;
[0035] Figure 4 This is a structural block diagram of the noise filtering device provided in an embodiment of the present invention;
[0036] Figure 5 This is a schematic diagram of the device structure provided in an embodiment of the present invention. Detailed Implementation
[0037] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0040] With the advent and development of depth measurement technology, acquiring depth images and using them for subsequent image processing has become increasingly common. However, existing technologies suffer from high image distortion due to inaccurate identification of noise points when filtering noise in depth maps. To address this issue, this invention provides a noise filtering method. Figure 1 This is a schematic flowchart of the noise filtering method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the noise filtering method includes:
[0041] S101: Traverse the pixels in the depth map of the noise to be filtered, wherein, during the traversal, the nearest neighbor of the pixel is obtained, and the nearest neighbor is determined to be a candidate noise point according to the first preset rule, the nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point is repeated until the traversal process ends.
[0042] Specifically, the process iterates through each pixel in the noise map, starting from the first pixel and capturing its nearest neighbors. Candidate noise points are then obtained based on the relationship between the pixel and its nearest neighbors. This process continues until the traversal is complete, ensuring noise detection for all pixels in the depth map.
[0043] For example, the selection of the nearest neighbors of a pixel can adopt different modes. Figure 2 This is a schematic diagram illustrating the selection of nearest neighbor points according to an embodiment of the present invention. For example... Figure 2 As shown, it can be as follows Figure 2 The "Nearest Neighbor 1 Mode" shown in (a) selects four nearest neighbor pixels as the nearest neighbors of that pixel. Alternatively, it can be done as follows: Figure 2 (b) shows the “Nearest Neighbor 2 Mode”, which selects eight nearest neighbor pixels as the nearest neighbors of the pixel.
[0044] S102: Determine whether the candidate noise point satisfies the second preset rule;
[0045] S103: If so, filter the candidate noise points.
[0046] Specifically, if the number of candidate noise points in the candidate noise point set meets the preset threshold, it indicates that the selection of the candidate noise is valid and the pixel is indeed a noise point. Therefore, the depth value of the candidate noise point is set to 0 to complete the noise filtering of the depth map. If the number of candidate noise points in the candidate noise point set does not meet the preset threshold, it indicates that the selection of the candidate noise point does not conform to the conventional selection and belongs to an invalid noise point. Therefore, the candidate noise point is discarded.
[0047] This invention traverses the pixels in a depth map to be filtered for noise; during the traversal, it obtains the nearest neighbors of each pixel and determines whether each nearest neighbor is a candidate noise point according to a first preset rule; it then obtains the nearest neighbors of the candidate noise points and repeats the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule until the traversal process ends; it then determines whether the candidate noise point satisfies a second preset rule; if so, it filters the candidate noise point. This invention determines and filters candidate noise points by judging the similarity between the detection point and its nearest neighbors, achieving an accurate and effective technical effect of filtering noise data in the depth map. This solves the technical problem in the prior art where inaccurate judgment of noise points during depth map noise filtering leads to high image distortion.
[0048] Furthermore, the step of traversing the pixels in the depth map of the noise to be filtered includes:
[0049] The first pixel in the traversal process is stored in the detection point set;
[0050] Update the pixels in the detection point set based on the candidate noise points;
[0051] The update process includes clearing pixels from the detection point set and adding candidate noise points to the detection point set.
[0052] Specifically, the first pixel can be any pixel in the depth map, or it can be a pixel at a pre-specified location. When a candidate noise point is identified among the nearest neighbors of a pixel, this candidate noise point is stored in the detection point set. There can be one or more candidate noise points. After neighbor detection is performed on the candidate noise points in the detection point set, these candidate noise points are removed. If none of the nearest neighbors of a pixel are candidate noise points, a detection point is selected from the undetected pixels. This process continues until all pixels in the depth map have been detected, at which point the traversal process ends.
[0053] Furthermore, determining whether the nearest neighbor is a candidate noise point according to the first preset rule includes:
[0054] The similarity between the pixel and its nearest neighbor is used to determine whether the nearest neighbor is a candidate noise point.
[0055] Furthermore, determining whether a nearest neighbor is a candidate noise point based on the similarity between the pixel and its nearest neighbor includes:
[0056] Determine whether the pixel and its nearest neighbor satisfy depth(c) i )>0 and Where, depth(c i ) represents the depth value of the i-th nearest neighbor, depth(CurP) represents the depth value of the pixel, and σ represents the preset similarity threshold, σ∈[0.001, 0.1];
[0057] If so, the nearest neighbor point is taken as the candidate noise point.
[0058] Furthermore, after determining whether the nearest neighbor is a candidate noise point according to the first preset rule, the process includes:
[0059] If the nearest neighbor is a candidate noise point, then the candidate noise point is stored in the candidate noise point set.
[0060] Furthermore, determining whether the candidate noise point satisfies the second preset rule includes:
[0061] Determine whether the number of candidate noise points in the candidate noise point set meets a preset number threshold.
[0062] Furthermore, determining whether the number of candidate noise points in the candidate noise point set meets a preset threshold includes:
[0063] Determine whether the set of candidate noise points satisfies |C|≤CT, where |C| represents the number of candidate noise points in the set of candidate noise points, CT is the preset number threshold, and CT∈[5,500].
[0064] Based on the above embodiments, Figure 3 This is a schematic diagram illustrating the traversal process provided in an embodiment of the present invention. For example... Figure 3 As shown, the present invention can also provide an optional embodiment, including: first performing depth map noise removal processing, initializing a candidate noise point set C, a detection point set W, and traversing each position of the depth map, as follows: Figure 3 As shown in (a), the current traversal position is StartP, which is added to the candidate noise point set and the detection point set, i.e., C = {StartP} and W = {StartP}. The initial detection point CurP = StartP. The nearest neighbors of the detection point CurP can adopt different patterns, such as... Figure 3 As shown, to select Figure 2 (a) Taking the "Nearest Neighbor 1 Mode" as an example, the four nearest neighbor locations of the detection point CurP are as follows: If among these four points, depth(c i )>0 and If the threshold σ ranges from [0.001, 0.1], then points that meet the condition will be added to the candidate noise point set, such as... Figure 3 As shown in (b), only If the condition is met, add it to the candidate noise point set. Simultaneously update the detection point set, i.e., delete StartP and add it. Update detection points Similarly, the updated detection point CurP belongs to the nearest neighbor occupancy of the set of non-candidate noise points. Determine if the above depth condition formula is satisfied, and process using the same steps as before, until the detection point set W is empty, i.e., as shown above. Figure 3 (c) All detection points have been detected, and the above process ends. This set of non-candidate noise points contains pixels in the depth map that were not detected. After the traversal process, at this point... If |C|≤CT (CT threshold range [5, 500]), then the points in the candidate noise point set C are determined to be noise points, and all depth values of the noise points are set to 0.
[0065] Based on the above embodiments and their contents, as an optional embodiment, this invention provides a noise filtering device. Figure 4 This is a structural block diagram of the noise filtering device provided in an embodiment of the present invention, as shown below. Figure 4 The noise filtering device includes:
[0066] The traversal module 201 is used to traverse the pixels in the depth map of the noise to be filtered out. During the traversal, the nearest neighbor of the pixel is obtained, and the nearest neighbor is determined to be a candidate noise point according to a first preset rule. The nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule is repeated until the traversal process ends.
[0067] Noise determination module 202 is used to determine whether the candidate noise point meets the second preset rule;
[0068] The noise filtering module 203 is used to filter candidate noise points that meet the second preset rule.
[0069] This invention traverses the pixels in a depth map to be filtered for noise; during the traversal, it obtains the nearest neighbors of each pixel and determines whether each nearest neighbor is a candidate noise point according to a first preset rule; it then obtains the nearest neighbors of the candidate noise points and repeats the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule until the traversal process ends; finally, it determines whether the candidate noise point satisfies a second preset rule; if so, it filters the candidate noise point. This invention determines and filters candidate noise points by judging the similarity between the detection point and its nearest neighbors, achieving an accurate and effective technical effect of filtering noise data in the depth map. This solves the technical problem in the prior art where inaccurate judgment of noise points during depth map noise filtering leads to high image distortion.
[0070] Furthermore, the traversal module 201 includes a traversal unit and an update unit.
[0071] A traversal unit is used to store the first pixel in the traversal process in the detection point set;
[0072] An update unit is used to update the pixels in the detection point set based on the candidate noise points.
[0073] Furthermore, the traversal module 201 also includes a noise determination unit.
[0074] The noise determination unit is used to determine whether the nearest point is a candidate noise point based on the similarity between the pixel and the nearest point.
[0075] Furthermore, the noise determination unit includes a noise determination component.
[0076] The noise determination component is used to determine whether the pixel and its nearest neighbor satisfy depth(c i )>0 and Where, depth(c i ) represents the depth value of the i-th nearest neighbor, depth(CurP) represents the depth value of the pixel, and σ represents the preset similarity threshold, σ∈[0.001, 0.1];
[0077] If so, the nearest neighbor point is taken as the candidate noise point.
[0078] Furthermore, the noise filtering device also includes a noise storage module.
[0079] The noise storage module is used to store the candidate noise point in the candidate noise point set if the nearest neighbor point is the candidate noise point.
[0080] Furthermore, the noise judgment module 202 includes a noise judgment unit.
[0081] The noise determination unit is used to determine whether the number of candidate noise points in the candidate noise point set meets a preset number threshold.
[0082] Furthermore, the noise detection unit includes a noise detection component.
[0083] The noise judgment component is used to determine whether the candidate noise point set satisfies |C|≤CT, where |C| represents the number of candidate noise points in the candidate noise point set, CT is the preset number threshold, and CT∈[5,500].
[0084] Figure 5 A block diagram of an electronic device provided in an embodiment of the present invention, such as Figure 5 As shown, the device includes: a processor 301, a memory 302, and a bus 303;
[0085] The processor 301 and the memory 302 communicate with each other via the bus 303. The processor 301 is used to call program instructions in the memory 302 to execute a noise filtering method provided in the above embodiment, which includes, for example:
[0086] Traverse the pixels in the depth map of the noise to be filtered, wherein, during the traversal, the nearest neighbor of the pixel is obtained, and it is determined whether the nearest neighbor is a candidate noise point according to a first preset rule. The nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule is repeated until the traversal process ends.
[0087] Determine whether the candidate noise points satisfy the second preset rule;
[0088] If so, filter the candidate noise points.
[0089] This invention provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the noise filtering method provided in the above embodiments. For example, it includes:
[0090] Traverse the pixels in the depth map of the noise to be filtered, wherein, during the traversal, the nearest neighbor of the pixel is obtained, and it is determined whether the nearest neighbor is a candidate noise point according to a first preset rule. The nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule is repeated until the traversal process ends.
[0091] Determine whether the candidate noise points satisfy the second preset rule;
[0092] If so, filter the candidate noise points.
[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.
[0095] Finally, specific embodiments have been used to illustrate the principles and implementation methods of this invention. The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.
Claims
1. A noise filtering method, characterized in that, include: Traverse the pixels in the depth map of the noise to be filtered, wherein, during the traversal, the nearest neighbor of the pixel is obtained, and it is determined whether the nearest neighbor is a candidate noise point according to a first preset rule. The nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule is repeated until the traversal process ends. The step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule includes: Determine whether the nearest neighbor is a candidate noise point based on the similarity between the pixel and the nearest neighbor; The step of determining whether a nearest neighbor is a candidate noise point based on the similarity between the pixel and its nearest neighbor includes: Determine whether the pixel and its nearest neighbor satisfy the following conditions: and ,in, Let be the depth value of the i-th nearest neighbor. Let σ be the depth value of the pixel, and σ be a preset similarity threshold. ∈ ; If so, the nearest neighbor point is selected as the candidate noise point; The process of traversing the pixels in the depth map of the noise to be filtered includes: The first pixel in the traversal process is stored in the detection point set; Update the pixels in the detection point set based on the candidate noise points; The update process includes clearing pixels from the detection point set and adding the candidate noise points to the detection point set. Determining whether the candidate noise point satisfies the second preset rule includes: Determine whether the number of candidate noise points in the candidate noise point set meets a preset number threshold; Determine whether the set of candidate noise points satisfies CT, among which, This represents the number of candidate noise points in the candidate noise point set, where CT is the preset number threshold, and CT∈ ; If so, filter the candidate noise points.
2. The method according to claim 1, characterized in that, After determining whether the nearest neighbor is a candidate noise point according to the first preset rule, the process includes: If the nearest neighbor is a candidate noise point, then the candidate noise point is stored in the candidate noise point set.
3. A noise filtering device, characterized in that, include: The traversal module is used to traverse the pixels in the depth map to be filtered for noise. During the traversal, the nearest neighbor of the pixel is obtained, and the nearest neighbor is determined to be a candidate noise point according to a first preset rule. The nearest neighbor of the candidate noise point is obtained, and the step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule is repeated until the traversal process ends. The step of determining whether the nearest neighbor is a candidate noise point according to the first preset rule includes: Determine whether the nearest neighbor is a candidate noise point based on the similarity between the pixel and the nearest neighbor; The step of determining whether a nearest neighbor is a candidate noise point based on the similarity between the pixel and its nearest neighbor includes: Determine whether the pixel and its nearest neighbor satisfy the following conditions: and ,in, Let be the depth value of the i-th nearest neighbor. Let σ be the depth value of the pixel, and σ be a preset similarity threshold. ∈ ; If so, the nearest neighbor point is selected as the candidate noise point; The traversal module is also used for: The first pixel in the traversal process is stored in the detection point set; Update the pixels in the detection point set based on the candidate noise points; The update process includes clearing pixels from the detection point set and adding the candidate noise points to the detection point set. The noise determination module is used to determine whether the candidate noise point satisfies the second preset rule; The noise detection module is also used for: Determine whether the number of candidate noise points in the candidate noise point set meets a preset number threshold; Determine whether the set of candidate noise points satisfies CT, among which, This represents the number of candidate noise points in the candidate noise point set, where CT is the preset number threshold, and CT∈ ; The noise filtering module is used to filter candidate noise points that meet the second preset rule.
4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the noise filtering method as described in any one of claims 1 to 2.
5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the noise filtering method as described in any one of claims 1 to 2.