Image point cloud data filtering method and device, equipment, medium and product
By combining color and depth information from image point cloud data, a Gaussian distribution algorithm is used to segment and perform depth filtering repair on edge regions, solving the problem of poor edge preservation in existing technologies and achieving better edge filtering effects and image clarity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING CO WHEELS TECH CO LTD
- Filing Date
- 2024-10-15
- Publication Date
- 2026-04-17
AI Technical Summary
Existing image filtering methods are poor at preserving edges, resulting in loss of image details, especially in applications that require high edge preservation performance.
By utilizing the color information of image point cloud data to determine edge regions and states, and combining the Gaussian distribution of depth information, the edge regions are segmented and depth filtered for repair. The gradient division of the edge regions and the repair of depth information are performed using color change difference maps and depth Gaussian distribution.
It improves the filtering effect in edge regions, maintains the edge sharpness and detail of the image, and improves the edge processing effect of the depth map.
Smart Images
Figure CN121883264A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically to a method, apparatus, device, medium, and product for filtering image point cloud data. Background Technology
[0002] Currently, the main methods for depth map filtering include mean filtering and Gaussian filtering. These methods are effective in smoothing noise and improving depth map quality, but they are poor at preserving edges. While mean filtering can reduce image noise to some extent, it treats all areas of the image equally, which can easily lead to blurring of edge areas and loss of image details. Gaussian filtering improves this by giving higher weight to the center pixel, but it still cannot effectively distinguish between edge and non-edge areas. Therefore, it is acceptable for applications where edge preservation is not critical, but it is not effective for applications requiring high edge preservation performance. Summary of the Invention
[0003] The purpose of this application is to provide a filtering method, apparatus, device, medium, and product for image point cloud data, in order to solve the problem of poor edge preservation effect of existing filtering methods.
[0004] To achieve the above objectives, the first aspect of this application provides a method for filtering image point cloud data, the method comprising:
[0005] Determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data;
[0006] The edge region is segmented so that each edge region segment has an image region on each side, and the image region is a different region in the image point cloud data with the edge region as the boundary.
[0007] The edge region segments and the depth Gaussian distribution of each image region are determined based on the depth information of the image point cloud data.
[0008] The depth information of the corresponding edge region is determined based on the depth Gaussian distribution and the edge state of the edge region.
[0009] Based on the depth information, the image point cloud depth map is repaired to obtain the edge-filtered image point cloud data.
[0010] Based on the above technical means, the edge regions and corresponding edge states in the image point cloud data are determined by using the color information of the image point cloud data. Then, the depth information of the corresponding edge region is determined according to the edge region segment, the depth Gaussian distribution of each image region, and the corresponding edge state. This allows for depth filtering and repair of the edge region separately, resulting in better edge preservation of the image point cloud data obtained after edge filtering.
[0011] In this embodiment of the application, determining the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data includes:
[0012] Based on the color information of the image point cloud data, the image point cloud data is separated into three images with a single color channel;
[0013] Calculate the difference between adjacent pixels in each image to obtain a color change difference map;
[0014] The edge regions and corresponding edge states of the image point cloud data are determined based on the color change difference map.
[0015] Based on the above technical means, the color information of image point cloud data can be used to determine the edge region and edge state, providing accurate edge information for subsequent edge region repair filtering.
[0016] In this embodiment of the application, determining the edge regions and corresponding edge states of image point cloud data based on the color change difference map includes:
[0017] Traverse the color change difference map and identify the area in any color change difference map where the difference value is greater than zero and the difference between the difference values is less than the preset difference value.
[0018] Calculate the difference between the color values of the first and last pixels in any direction of the surface region based on the color values of the corresponding image in the difference map;
[0019] If the difference is greater than the preset color threshold, the current surface area is determined to be a first-class edge area, and the corresponding edge state is a gradient edge state.
[0020] Based on the aforementioned technical methods, the edge region of the gradient can be determined according to the color change difference diagram.
[0021] In this embodiment of the application, determining the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region includes:
[0022] When the corresponding edge state is a gradient edge state, calculate the color change amplitude of the current edge region segment in the color change difference map;
[0023] Divide the color change amplitude into gradients;
[0024] Calculate the difference between the depth Gaussian distributions of two image regions adjacent to the current edge region segment;
[0025] The difference is divided into gradients based on the gradient division results of the color change amplitude.
[0026] The depth information of the corresponding color change amplitude gradient is determined based on the corresponding difference gradient division results;
[0027] The depth information is converted into depth information of different gradients in the corresponding edge region segment by using the color change difference map;
[0028] The depth information of different edge region segments within the same edge region constitutes the depth information of the corresponding edge region.
[0029] Based on the above technical means, by performing gradient division on the difference between the depth Gaussian distributions of two adjacent regions in the current edge region segment, and then performing depth filtering repair on different gradients in the current edge region segment based on the corresponding difference gradient division results, the repaired results have a higher degree of fit with the actual depth of the corresponding edge region segment and are more accurate.
[0030] In this embodiment of the application, determining the edge regions and corresponding edge states of image point cloud data based on the color change difference map includes:
[0031] Traverse the color change difference map and determine the point or line region in any color change difference map where the color difference is greater than the preset difference threshold as the second type of edge region, and the corresponding edge state is the significant edge state.
[0032] Based on the aforementioned technical methods, the edge regions with significant changes can be identified according to the color change difference map.
[0033] In this embodiment of the application, determining the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region includes:
[0034] When the corresponding edge state is a significant edge state, compare the mean depth Gaussian distribution of the current edge region segment and the two image regions adjacent to the current edge region segment.
[0035] If the difference between the mean of the depth Gaussian distribution of the current edge region segment and the mean of the depth Gaussian distribution of the two adjacent image regions is greater than the first preset depth, then the maximum value among the three mean values of the depth Gaussian distribution is determined as the mean of the depth Gaussian distribution of the current edge region segment.
[0036] If the difference between the mean of the depth Gaussian distribution of the current edge region segment and the mean of the depth Gaussian distribution of the two adjacent image regions is less than or equal to a preset depth, then the average of the three mean values of the depth Gaussian distribution is determined as the mean of the depth Gaussian distribution of the current edge region segment.
[0037] The depth information of different edge region segments within the same edge region constitutes the depth information of the corresponding edge region.
[0038] Based on the above technical means, the mean depth Gaussian distribution of the corresponding edge region segment is determined according to the difference between the mean depth Gaussian distributions of two adjacent image regions of the current edge region segment, thereby improving the reliability and accuracy of the depth data of the filtered edge region.
[0039] In this embodiment of the application, the filtering method for the image point cloud data further includes:
[0040] If the size of the image point cloud data is larger than the preset size, the image point cloud data is divided into grids to obtain the gridded image, which triggers the determination of the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data.
[0041] Based on the above technical means, image point cloud data is processed into a size that is easier for the server to handle, thereby improving the server's processing speed.
[0042] In this embodiment of the application, the edge region is segmented so that each edge region segment has one image region on each side, including:
[0043] Identify at least two image regions adjacent to the current edge region;
[0044] The edge region is segmented according to the positional relationship between the determined image region and the edge region, so that each edge region segment has one image region on each side.
[0045] Based on the above technical means, the edge region is segmented according to the positional relationship between the edge region and each image region, so that each edge region segment can be repaired separately, thereby improving the repair accuracy.
[0046] A second aspect of this application provides a filtering device for image point cloud data, the filtering device comprising:
[0047] The edge determination unit is used to determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data.
[0048] An edge segmentation unit is used to segment the edge region so that each edge region segment has an image region on each side; the image region is a different region in the image point cloud data with the edge region as the boundary;
[0049] A depth calculation unit is used to determine the edge region segments and the depth Gaussian distribution of each image region based on the depth information of the image point cloud data.
[0050] A depth update unit is used to determine the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region.
[0051] The image restoration unit is used to restore the image point cloud depth map based on the depth information to obtain edge-filtered image point cloud data.
[0052] Based on the above technical means, the device uses the color information of the image point cloud data to determine the edge regions and corresponding edge states in the image point cloud data, and then determines the depth information of the corresponding edge regions according to the edge region segments, the depth Gaussian distribution of each image region, and the corresponding edge states. It can perform depth filtering and repair on the edge regions separately, and the edge preservation effect of the image point cloud data obtained after edge filtering is better.
[0053] A third aspect of this application provides an electronic device, comprising:
[0054] The memory is configured to store instructions; and
[0055] The processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the filtering method for the image point cloud data.
[0056] A fourth aspect of this application provides a machine-readable storage medium storing instructions for causing a machine to perform the image point cloud data filtering method.
[0057] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the image point cloud data filtering method.
[0058] Through the above technical solution, the device uses the color information of the image point cloud data to determine the edge regions and corresponding edge states in the image point cloud data, and then determines the depth information of the corresponding edge regions based on the edge region segments, the depth Gaussian distribution of each image region, and the corresponding edge states. It can perform depth filtering and repair on the edge regions separately, and the edge preservation effect of the image point cloud data obtained after edge filtering is better.
[0059] By combining color images and depth maps, the accuracy of edge detection in the depth map is improved. By employing a Gaussian distribution algorithm combined with color variation information for edge filtering of the depth map, the filtering effect is improved, resulting in a clearer and more accurate depth image.
[0060] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description
[0061] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings:
[0062] Figure 1 The illustration shows a flowchart of a method for filtering image point cloud data according to an embodiment of this application;
[0063] Figure 2 This illustration schematically shows a process diagram for determining the edge region according to Embodiment 1 of this application;
[0064] Figure 3 This illustration schematically shows a process diagram for determining the edge region according to Embodiment 2 of this application;
[0065] Figure 4 The schematic diagram illustrates the structure of a filtering device for image point cloud data according to an embodiment of this application. Detailed Implementation
[0066] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0067] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0068] Terminology Explanation:
[0069] Point cloud data: Point cloud data is an important form of 3D data representation, consisting of a series of points distributed in three-dimensional space. These points typically contain three coordinates: X, Y, and Z, used to represent their positions in space. In some cases, point cloud data also includes additional information, such as the color (RGB value), intensity, and reflectivity of each point.
[0070] The Gaussian distribution, also known as the normal distribution, is one of the most important continuous probability distributions in statistics. It was named after the German mathematician and physicist Carl Friedrich Gauss. The Gaussian distribution is very common in many phenomena in nature and the social sciences, such as human height or measurement errors.
[0071] During the process of practice, the inventors discovered that areas with drastic edge changes also exhibit drastic color changes in color images. Therefore, the color change gradient in the color image can be used to help filter and repair point cloud data. Based on the limitations of existing technologies, this application proposes a filtering method for image point cloud data, which introduces color information to filter the depth map, resulting in a more accurate filtering effect. Its aim is to significantly improve the edge processing effect and solve the problem of edge differences in the depth map.
[0072] Example 1
[0073] Figure 1 The illustration schematically shows a flowchart of a method for filtering image point cloud data according to an embodiment of this application. For example... Figure 1 As shown in the embodiment of this application, a filtering method for image point cloud data is provided, the filtering method for image point cloud data including:
[0074] S11: Determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data.
[0075] In some embodiments, color information includes values for three color channels, and edge regions refer to the boundary regions of any object in the image. For example, in an image of a red pentagram on a blue background, edge regions refer to the boundary regions of the red pentagram.
[0076] In some embodiments, such as Figure 2 As shown, the edge regions and corresponding edge states in the image point cloud data are determined based on the color information of the image point cloud data, including:
[0077] The image point cloud data is separated into three images with a single color channel based on the color information of the image point cloud data. In one embodiment, the image point cloud data can be separated into images with three color channels: red (R), green (G), and blue (B).
[0078] Calculate the difference between adjacent pixels in each image to obtain a color change difference map.
[0079] The edge regions and corresponding edge states of the image point cloud data are determined based on the color change difference map.
[0080] Based on the above technical means, the color information of image point cloud data can be used to determine the edge region and edge state, providing accurate edge information for subsequent edge region repair filtering.
[0081] In some cases, the object's sides change at an angle, so the color change in the shaded area is gradual. In the resulting color change difference map, a region will show color changes, and the differences between the color difference values are very small.
[0082] In one embodiment, determining the edge regions and corresponding edge states of image point cloud data based on a color change difference map includes:
[0083] Traverse the color change difference maps and identify the areas in any color change difference map where the difference value is greater than zero and the difference between the difference values is less than a preset difference value. In some feasible embodiments, the area refers to a region with a certain area in the color change difference map.
[0084] The difference between the color values of the first and last pixels in any direction of the surface region is calculated based on the color values of the corresponding image in the difference map. In some feasible embodiments, the difference between the color values of the topmost and bottommost pixels in the vertical direction of the surface region can be calculated, or the difference between the color values of the leftmost and rightmost pixels in the horizontal direction of the surface region can be calculated.
[0085] If the difference is greater than a preset color threshold, the current surface area is determined to be a first-type edge region, and the corresponding edge state is a gradient edge state. In some feasible embodiments, the preset difference value can be set according to the gradient of the color gradient. The first preset value can be set according to the color value of the gradient region in the image point cloud data, or it can be preset according to the calculation accuracy requirements. Thus, the gradient edge region can be determined according to the color change difference map, so as to repair the depth of the gradient region.
[0086] S12: The edge region is segmented so that each edge region segment has one image region on each side. The image regions are different regions in the image point cloud data bounded by the edge region. In this embodiment, because the positions of objects are staggered, the same edge region may correspond to more than two image regions. Therefore, in order to accurately repair the edge region, it is necessary to segment the edge region. Specifically, this includes:
[0087] Identify at least two image regions adjacent to the current edge region. Generally, an edge region in image point cloud data includes two adjacent image regions. For example, in the aforementioned image of a red pentagram on a blue background, the boundary region of the pentagram includes the red area of the pentagram itself and the blue background area, because the two areas of the pentagram's boundary region are the red area of the pentagram itself and the blue background area. Similarly, if an image contains a black cardboard box placed on a white table and extending beyond the edge of the table, and the background of the image is blue, then the bottom edge of the black cardboard box has three adjacent image regions: the white tabletop area, the black cardboard box area, and the blue background area.
[0088] The edge region is segmented based on the positional relationship between the determined image region and the edge region, so that each edge region segment has one image region on each side. Based on the aforementioned cardboard box image, the bottom edge boundary of the part of the black cardboard box that contacts the white square table can be divided into a first boundary region segment, and the bottom edge boundary of the suspended part of the black cardboard box located in the blue background area can be divided into a second boundary region segment. Thus, by segmenting the edge region based on the positional relationship between the edge region and each image region, each edge region segment can be repaired separately, improving the accuracy of the repair.
[0089] S13: Determine the depth Gaussian distribution of the edge region segments and each image region based on the depth information of the image point cloud data. Using the depth information in the depth map, the Gaussian distribution of the depth information of each edge region segment and each image region can be calculated. The depth Gaussian distribution claimed in this application is the Gaussian distribution of the depth information. In this embodiment, the mean and variance of the Gaussian distribution of the depth information are mainly calculated to obtain the mean and variance of the depth Gaussian distribution of each edge region segment and each image region.
[0090] S14: Determine the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region.
[0091] Generally, edge regions in image point cloud data consist of at least two adjacent image regions. In some special cases, the edge region coincides with the boundary of the depth image, in which case depth restoration is unnecessary. Therefore, this application does not focus on the special case where the edge region coincides with the boundary of the depth image.
[0092] In this embodiment of the application, determining the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region includes:
[0093] When the corresponding edge state is a gradient edge state, calculate the color change amplitude of the current edge region segment in the color change difference map.
[0094] Gradient division is performed on the color change amplitude. The number of gradient divisions is determined based on the size of the edge region and the color change amplitude. The more gradient divisions, the more accurate the subsequent depth restoration, but the greater the computational load. The fewer gradient divisions, the lower the accuracy of the subsequent depth restoration, but the smaller the computational load.
[0095] The difference between the depth Gaussian distributions of two image regions adjacent to the current edge region segment is calculated. In this embodiment, the difference between the mean depth Gaussian distributions of two image regions adjacent to the edge region is calculated.
[0096] The difference can be divided into gradients based on the gradient division results of the color change amplitude. The difference between two adjacent gradients can be obtained. The number of parts for the difference division is the same as the number of parts for the gradient division.
[0097] The depth information of the corresponding color change amplitude gradient is determined based on the corresponding difference gradient division results.
[0098] Finally, the depth information is converted into depth information of different gradients in the corresponding edge region segments using the color change difference map; the depth information of different edge region segments within the same edge region constitutes the depth information of the corresponding edge region. For example, if the mean Gaussian distribution of depth for two adjacent image regions is 10 and 20, and the gradient is divided into 10 parts, then the difference between two adjacent gradients is 1. Therefore, it can be determined that the depth information of each color change amplitude gradient differs by 1. In the corresponding edge region segment, the mean Gaussian distribution of the first gradient adjacent to the image region with a mean Gaussian distribution of depth of 10 is 11; in the corresponding edge region segment, the mean Gaussian distribution of the first gradient adjacent to the image region with a mean Gaussian distribution of depth of 20 is 19, and so on for the remaining 8 gradients.
[0099] Based on the above technical means, by performing gradient division on the difference between the depth Gaussian distributions of two adjacent regions in the current edge region segment, and then performing depth filtering repair on different gradients in the current edge region segment based on the corresponding difference gradient division results, the repaired results have a higher degree of fit with the actual depth of the corresponding edge region segment and are more accurate.
[0100] S15: Repair the image point cloud depth map based on the depth information to obtain edge-filtered image point cloud data. In some embodiments, the depth information of the corresponding edge region is integrated back into the depth map to complete the edge filtering process.
[0101] Based on the above technical means, the edge regions and corresponding edge states in the image point cloud data are determined by using the color information of the image point cloud data. Then, the depth information of the corresponding edge region is determined according to the edge region segment, the depth Gaussian distribution of each image region, and the corresponding edge state. This allows for depth filtering and repair of the edge region separately, resulting in better edge preservation of the image point cloud data obtained after edge filtering.
[0102] In some embodiments, the filtering method for image point cloud data further includes:
[0103] When the image point cloud data size is larger than a preset size, the image point cloud data is divided into a grid to obtain a gridded image. This triggers the determination of edge regions and corresponding edge states in the image point cloud data based on its color information. Grid division reduces the image point cloud data to a size that is easier for the server to process, improving server processing speed. The preset size and the number of grid divisions can be set according to the server's computing power.
[0104] Example 2
[0105] This application provides another method for filtering image point cloud data, the method comprising:
[0106] S21: Determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data.
[0107] In some embodiments, color information includes values for three color channels, and edge regions refer to the boundary regions of any object in the image. For example, in an image of a pentagram on a blue background, edge regions refer to the boundary regions of the pentagram.
[0108] In the embodiments of this application, such as Figure 3 As shown, the edge regions and corresponding edge states in the image point cloud data are determined based on the color information of the image point cloud data, including:
[0109] The image point cloud data is separated into three images with a single color channel based on the color information of the image point cloud data. In one embodiment, the image point cloud data can be separated into images with three color channels: red (R), green (G), and blue (B).
[0110] Calculate the difference between adjacent pixels in each image to obtain a color change difference map.
[0111] The edge regions and corresponding edge states of the image point cloud data are determined based on the color change difference map.
[0112] Based on the above technical means, the color information of image point cloud data can be used to determine the edge region and edge state, providing accurate edge information for subsequent edge region repair filtering.
[0113] In some cases, objects have distinct boundaries, so the color changes at the edges in a color image can be quite dramatic, resulting in noticeable color differences in the color variation map.
[0114] In this embodiment of the application, determining the edge regions and corresponding edge states of image point cloud data based on the color change difference map includes:
[0115] By traversing the color change difference map, point or line regions in any color change difference map where the color difference exceeds a preset difference threshold are identified as second-type edge regions, and their corresponding edge states are defined as significant edge states. Therefore, significantly changing edge regions can be determined based on the color change difference map.
[0116] In some embodiments, the preset threshold can be determined based on the color range of each single-color image, and the preset threshold is generally 1 / 4 to 1 / 2 of the color range. For example, assuming that the color range of an image with a red channel is 98 to 212, if the preset threshold is 1 / 4 of the color range, then the preset threshold is (214-98)*1 / 4=29.
[0117] In some embodiments, the preset threshold can be manually determined and entered based on the color difference of the actual edges of the image.
[0118] S22: The edge region is segmented so that each edge region segment has one image region on each side. The image regions are different regions in the image point cloud data bounded by the edge region. In this embodiment, because the positions of objects are staggered, the same edge region may correspond to more than two image regions. Therefore, in order to accurately repair the edge region, it is necessary to segment the edge region. Specifically, this includes:
[0119] Identify at least two image regions adjacent to the current edge region. Generally, an edge region in image point cloud data includes two adjacent image regions. For example, in the aforementioned image of a red pentagram on a blue background, the boundary region of the pentagram includes the red area of the pentagram itself and the blue background area, because the two areas of the pentagram's boundary region are the red area of the pentagram itself and the blue background area. Similarly, if an image contains a black cardboard box placed on a white table and extending beyond the edge of the table, and the background of the image is blue, then the bottom edge of the black cardboard box has three adjacent image regions: the white tabletop area, the black cardboard box area, and the blue background area.
[0120] The edge region is segmented based on the positional relationship between the determined image region and the edge region, so that each edge region segment has one image region on each side. Based on the aforementioned cardboard box image, the bottom edge boundary of the part of the black cardboard box that contacts the white square table can be divided into a first boundary region segment, and the bottom edge boundary of the suspended part of the black cardboard box located in the blue background area can be divided into a second boundary region segment. Thus, by segmenting the edge region based on the positional relationship between the edge region and each image region, each edge region segment can be repaired separately, improving the accuracy of the repair.
[0121] S23: Determine the depth Gaussian distribution of the edge region segments and each of the image regions based on the depth information of the image point cloud data. Using the depth information in the depth map, the Gaussian distribution of the depth information of each edge region segment and each image region can be calculated. The depth Gaussian distribution claimed in this application is the Gaussian distribution of the depth information. In this embodiment, the mean and variance of the Gaussian distribution of the depth information are mainly calculated to obtain the mean and variance of the depth Gaussian distribution of each edge region segment and each image region.
[0122] S24: Determine the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region.
[0123] Generally, edge regions in image point cloud data consist of at least two adjacent image regions. In some special cases, the edge region coincides with the boundary of the depth image, in which case depth restoration is unnecessary. Therefore, this application does not focus on the special case where the edge region coincides with the boundary of the depth image.
[0124] In this embodiment of the application, determining the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region includes:
[0125] When the corresponding edge state is a significant edge state, compare the mean depth Gaussian distribution of the current edge region segment and the two image regions adjacent to the current edge region segment.
[0126] If the difference between the mean of the depth Gaussian distribution of the current edge region segment and the mean of the depth Gaussian distribution of the two adjacent image regions is greater than the first preset depth, then the maximum value among the three mean values of the depth Gaussian distribution is determined as the mean of the depth Gaussian distribution of the current edge region segment.
[0127] If the difference between the mean depth Gaussian distribution of the current edge region segment and the mean depth Gaussian distribution of the two adjacent image regions is less than or equal to a preset depth, then the average of the three mean depth Gaussian distributions is determined as the mean depth Gaussian distribution of the current edge region segment; the depth information of different edge region segments in the same edge region constitutes the depth information of the corresponding edge region.
[0128] The mean depth Gaussian distribution of the corresponding edge region is determined by the difference between the mean depth Gaussian distributions of two adjacent image regions, thus improving the reliability and accuracy of the depth data of the filtered edge region. By analyzing the mean depth Gaussian distributions on both sides, it is possible to distinguish whether the color change region is a true edge or an error caused by noise. If it is an error caused by noise, the difference between the mean depth Gaussian distributions on both sides of such an edge region will be less than the preset depth. Using the average of the three mean depth Gaussian distributions as the mean depth Gaussian distribution of the edge region can eliminate the error, thereby reducing the negative impact of noise on depth information.
[0129] In some embodiments, the first preset depth can be determined based on the depth range contained in the processed image point cloud data.
[0130] S25: Repair the image point cloud depth map based on the depth information to obtain edge-filtered image point cloud data. In some embodiments, the depth information of the corresponding edge region is integrated back into the depth map to complete the edge filtering process.
[0131] Based on the aforementioned techniques, the color information of image point cloud data is used to determine the edge regions and corresponding edge states. Then, the depth information of the corresponding edge regions is determined based on the edge region segments, the depth Gaussian distribution of each image region, and the corresponding edge states. This allows for separate depth filtering and repair of edge regions, resulting in better edge preservation in the image point cloud data obtained after edge filtering. Detailed processing of the depth information around the edges helps maintain the continuity and structural integrity of objects during depth map reconstruction, especially for complex scenes.
[0132] In some embodiments, the filtering method for image point cloud data further includes:
[0133] When the image point cloud data size is larger than a preset size, the image point cloud data is divided into a grid to obtain a gridded image. This triggers the determination of edge regions and corresponding edge states in the image point cloud data based on its color information. Grid division reduces the image point cloud data to a size that is easier for the server to process, improving server processing speed. The preset size and the number of grid divisions can be set according to the server's computing power.
[0134] It should be noted that in the actual process of filtering and repairing image point cloud data, there may be edge regions with both gradual edge states and significant edge states in the same image point cloud data. Therefore, it is necessary to use two methods to filter and repair image point cloud data in different regions at the same time.
[0135] A second aspect of this application provides a filtering device for image point cloud data, such as... Figure 4 As shown, the filtering device for the image point cloud data includes:
[0136] The edge determination unit is used to determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data.
[0137] An edge segmentation unit is used to segment the edge region so that each edge region segment has an image region on each side; the image region is a different region in the image point cloud data with the edge region as the boundary;
[0138] A depth calculation unit is used to determine the edge region segments and the depth Gaussian distribution of each image region based on the depth information of the image point cloud data.
[0139] A depth update unit is used to determine the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region.
[0140] The image restoration unit is used to restore the depth map of the image point cloud based on the depth information, thereby obtaining edge-filtered image point cloud data. Thus, this device uses the color information of the image point cloud data to determine the edge regions and corresponding edge states in the image point cloud data, and then determines the depth information of the corresponding edge regions based on the edge region segments, the depth Gaussian distribution of each image region, and the corresponding edge states. This allows for independent depth filtering and restoration of edge regions, resulting in better edge preservation in the image point cloud data obtained after edge filtering.
[0141] A third aspect of this application provides an electronic device, comprising:
[0142] The memory is configured to store instructions; and
[0143] The processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the filtering method for the image point cloud data.
[0144] A fourth aspect of this application provides a machine-readable storage medium storing instructions for causing a machine to perform the image point cloud data filtering method.
[0145] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the image point cloud data filtering method.
[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0148] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0150] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0151] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0152] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0153] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0154] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method of filtering image point cloud data, characterized in that, The filtering method for the image point cloud data includes: Determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data; The edge region is segmented so that each edge region segment has an image region on each side, and the image region is a different region in the image point cloud data with the edge region as the boundary. The edge region segments and the depth Gaussian distribution of each image region are determined based on the depth information of the image point cloud data. The depth information of the corresponding edge region is determined based on the depth Gaussian distribution and the edge state of the edge region. Based on the depth information, the image point cloud depth map is repaired to obtain the edge-filtered image point cloud data.
2. The method of filtering image point cloud data according to claim 1, characterized in that, Determine the edge regions and corresponding edge states in the image point cloud data based on the color information, including: Based on the color information of the image point cloud data, the image point cloud data is separated into three images with a single color channel; Calculate the difference between adjacent pixels in each image to obtain a color change difference map; The edge regions and corresponding edge states of the image point cloud data are determined based on the color change difference map.
3. The method of filtering image point cloud data according to claim 2, characterized in that, The edge regions and corresponding edge states of the image point cloud data are determined based on the color change difference map, including: Traverse the color change difference map and identify the area in any color change difference map where the difference value is greater than zero and the difference between the difference values is less than the preset difference value. Calculate the difference between the color values of the first and last pixels in any direction of the surface region based on the color values of the corresponding image in the difference map; If the difference is greater than the preset color threshold, the current surface area is determined to be a first-class edge area, and the corresponding edge state is a gradient edge state.
4. The method of filtering image point cloud data according to claim 3, characterized in that, Determining the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state of the edge region includes: When the corresponding edge state is a gradient edge state, calculate the color change amplitude of the current edge region segment in the color change difference map; Divide the color change amplitude into gradients; Calculate the difference between the depth Gaussian distributions of two image regions adjacent to the current edge region segment; The difference is divided into gradients based on the gradient division results of the color change amplitude. The depth information of the corresponding color change amplitude gradient is determined based on the corresponding difference gradient division results; The depth information is converted into depth information of different gradients in the corresponding edge region segment by using the color change difference map; The depth information of different edge region segments within the same edge region constitutes the depth information of the corresponding edge region.
5. The method of filtering image point cloud data of claim 2, wherein, The edge regions and corresponding edge states of the image point cloud data are determined based on the color change difference map, including: Traverse the color change difference map and determine the point or line region in any color change difference map where the color difference is greater than the preset difference threshold as the second type of edge region, and the corresponding edge state is the significant edge state.
6. The image point cloud data filtering method according to claim 5, characterized in that, Determining the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state of the edge region includes: When the corresponding edge state is a significant edge state, compare the mean depth Gaussian distribution of the current edge region segment and the two image regions adjacent to the current edge region segment. If the difference between the mean of the depth Gaussian distribution of the current edge region segment and the mean of the depth Gaussian distribution of the two adjacent image regions is greater than the first preset depth, then the maximum value among the three mean values of the depth Gaussian distribution is determined as the mean of the depth Gaussian distribution of the current edge region segment. If the difference between the mean of the depth Gaussian distribution of the current edge region segment and the mean of the depth Gaussian distribution of the two adjacent image regions is less than or equal to a preset depth, then the average of the three mean values of the depth Gaussian distribution is determined as the mean of the depth Gaussian distribution of the current edge region segment. The depth information of different edge region segments within the same edge region constitutes the depth information of the corresponding edge region.
7. The image point cloud data filtering method according to claim 1, characterized in that, The filtering method for the image point cloud data further includes: If the size of the image point cloud data is larger than the preset size, the image point cloud data is divided into grids to obtain the gridded image, which triggers the determination of the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data.
8. The image point cloud data filtering method according to claim 1, characterized in that, The edge region is segmented so that each edge region segment has one image region on each side, including: Identify at least two image regions adjacent to the current edge region; The edge region is segmented according to the positional relationship between the determined image region and the edge region, so that each edge region segment has one image region on each side.
9. A filtering device for image point cloud data, characterized in that, The filtering device for the image point cloud data includes: The edge determination unit is used to determine the edge regions and corresponding edge states in the image point cloud data based on the color information of the image point cloud data. An edge segmentation unit is used to segment the edge region so that each edge region segment has an image region on each side; the image region is a different region in the image point cloud data with the edge region as the boundary; A depth calculation unit is used to determine the edge region segments and the depth Gaussian distribution of each image region based on the depth information of the image point cloud data. A depth update unit is used to determine the depth information of the corresponding edge region based on the depth Gaussian distribution and the edge state corresponding to the edge region. The image restoration unit is used to restore the image point cloud depth map based on the depth information to obtain edge-filtered image point cloud data.
10. An electronic device, characterized in that, include: The memory is configured to store instructions; as well as A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the filtering method for image point cloud data according to any one of claims 1 to 8.
11. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform the filtering method for image point cloud data according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image point cloud data filtering method according to any one of claims 1 to 8.