An image noise cleaning method and device and a storage medium

Abstract

The application relates to the technical field of image processing, and discloses a kind of image noise cleaning method, device and storage medium.The method comprises: for the input image to be cleaned, a multidimensional vector is constructed;Based on the multidimensional vector, the connectivity segmentation is carried out to output regional data, and the data to be removed in the regional data is determined;The data to be removed in the regional data is eliminated.Through the construction of multidimensional vector and the connectivity segmentation, the data to be removed is determined for the regional data after segmentation, and then the noise data is eliminated, which can efficiently, quickly and accurately remove the noise of depth map, point cloud and other three-dimensional images without damaging the source image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for cleaning up image noise. Background Technology

[0002] With the rapid development of 3D vision technology and 3D acquisition equipment, depth images and 3D point cloud data are being used more and more widely in industrial measurement, inspection and matching positioning.

[0003] High-precision acquisition equipment often contains outliers, noise, environmental points, and non-target area points in 3D images, which can affect target tasks such as recognition, measurement, and detection. To overcome this problem, noise removal can be used to repair and optimize depth images to ensure the accuracy of target tasks. Currently, depth image repair and optimization are mainly achieved through filtering or global denoising. However, filtering can easily damage the source image, while global denoising struggles to repair edge distortions; therefore, existing noise removal methods are not ideal. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an image noise removal method, apparatus, and storage medium, which solves the problems that existing noise removal methods for 3D images easily damage the source image and are difficult to repair edge distortion.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for cleaning up image noise includes:

[0007] For the input image to be cleaned, construct a multidimensional vector;

[0008] Connectivity segmentation is performed based on the multidimensional vector to output regional data, and data to be removed from the regional data is determined.

[0009] Remove the data to be removed from the regional data.

[0010] Optionally, the step of performing connectivity segmentation based on the multidimensional vector to output regional data, and determining the data to be removed from the regional data, includes:

[0011] A local segmentation method is used to segment regions based on the multidimensional vector, outputting the segmented first regional data, and determining the data to be removed from the first regional data; and / or

[0012] The full-image segmentation method is used to perform regional segmentation based on the multidimensional vector, outputting the segmented second regional data, and determining the data to be removed in the second regional data.

[0013] Optionally, the step of performing connectivity segmentation based on the multidimensional vector to output regional data, and determining the data to be removed from the regional data, includes:

[0014] First, a local segmentation method is used to segment the region based on the multidimensional vector, and the segmented first regional data is output. The region of interest is determined in the first regional data, and the data outside the region of interest is determined as data to be removed.

[0015] Then, a full-image segmentation method is used to perform regional segmentation based on the multidimensional vector, and the segmented second regional data is output. Noise points are identified in the second regional data and are determined as data to be removed.

[0016] Optionally, the method of using local segmentation to segment regions based on the multidimensional vector includes:

[0017] 1) Determine the seed point based on the multidimensional vector;

[0018] 2) Based on a predetermined segmentation threshold, add point data with the same attributes as the seed point to the same region until all other point data except the seed point have been traversed.

[0019] Repeat steps 1) and 2) until the region segmentation is complete.

[0020] Optionally, determining the seed point based on the multidimensional vector includes:

[0021] The region of interest is selected based on the multidimensional vector, and the region of interest is used as a seed point;

[0022] or,

[0023] Based on a predetermined segmentation threshold, point data that does not have the same attributes as the seed point are used as new seed points.

[0024] Optionally, the method of performing regional segmentation based on the multidimensional vector using the full-image segmentation method includes:

[0025] Construct a four-neighbor template, and traverse the entire graph based on the four-neighbor template;

[0026] Construct four state diagrams, determine the connectivity of point data based on a predetermined segmentation threshold, and fill each point data into the four state diagrams according to the connectivity.

[0027] Construct a set of regions, traverse the entire graph based on the four state graphs, assign connected point data to the same region, and assign disconnected point data to a new region, until the region segmentation is completed.

[0028] Optionally, constructing a multi-dimensional vector for the input image to be cleaned includes:

[0029] Fill the locations in the image to be cleaned where no data was collected with preset invalid values;

[0030] Calculate the fast point set features based on the image data of the image to be cleaned;

[0031] Based on the aforementioned fast point set features, and combined with the spatial coordinates of the points, a multidimensional vector weighted dot multiplication operator is defined to complete the construction of the multidimensional vector.

[0032] Optionally, calculating the fast point set features based on the image data of the image to be cleaned includes:

[0033] Calculate the normal vector of the current point;

[0034] Calculate the spherical integral and volume integral values ​​of the linear correlation of point curvature;

[0035] A multidimensional vector is constructed based on the spherical integral volume integral value that is linearly related to the current point's spatial location, the current point's normal vector, and the point's curvature.

[0036] The present invention also provides an image noise removal apparatus for implementing the image noise removal method as described in any of the preceding claims, comprising:

[0037] The computational unit is used to construct a multidimensional vector for the input image to be cleaned;

[0038] A data determination unit is used to perform connectivity segmentation based on the multidimensional vector to output regional data, and to determine the data to be removed from the regional data.

[0039] The clearing unit is used to remove data to be removed from the regional data.

[0040] The present invention also provides a storage medium including computer-readable instructions that, when read and executed by a computer, cause the computer to perform the image noise removal method as described in any of the preceding claims.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This invention provides a method, apparatus, and storage medium for cleaning up image noise. By constructing a multidimensional vector and performing connectivity separation, the method determines the data to be removed from the segmented regional data and then removes the noise data. This method can efficiently, quickly, and accurately remove noise from three-dimensional images such as depth maps and point clouds without damaging the source image. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating an image noise removal method provided in Embodiment 1 of the present invention.

[0045] Figure 2 This is a flowchart illustrating step S2 in an image noise removal method provided in Embodiment 1 of the present invention.

[0046] Figure 3 This is a flowchart illustrating step S21 of an image noise removal method provided in Embodiment 1 of the present invention.

[0047] Figure 4 This is a flowchart illustrating step S22 of an image noise removal method provided in Embodiment 1 of the present invention.

[0048] Figure 5 This is the depth image to be segmented in Embodiment 1 of the present invention;

[0049] Figure 6 This is the depth image image after being segmented using the full-image segmentation method in Embodiment 1 of the present invention;

[0050] Figure 7 This is a depth image after noise cleanup in Embodiment 1 of the present invention;

[0051] Figure 8 This is another depth image to be segmented in Embodiment 1 of the present invention;

[0052] Figure 9 This is a depth image of the region of interest in Embodiment 1 of the present invention;

[0053] Figure 10 This is another depth image of the region of interest drawn in Embodiment 1 of the present invention;

[0054] Figure 11 This is another depth image of the region of interest drawn in Embodiment 1 of the present invention;

[0055] Figure 12 This is a depth image image after being segmented using a local segmentation method in Embodiment 1 of the present invention;

[0056] Figure 13This is another depth image image after noise removal in the image noise removal method provided by the present invention;

[0057] Figure 14 This is a further depth image to be segmented in Embodiment 1 of the present invention;

[0058] Figure 15 This is another depth image image after being segmented by the full-image segmentation method in Embodiment 1 of the present invention;

[0059] Figure 16 This is another depth image image after being segmented by the local segmentation method in Embodiment 1 of the present invention;

[0060] Figure 17 This is a schematic diagram of an image noise removal device provided in Embodiment 2 of the present invention.

[0061] In the above diagram: 10, calculation unit; 20, data determination unit; 30, clearing unit. Detailed Implementation

[0062] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0063] It should be understood that in the description of this invention, specific embodiments are merely used to explain the invention and not to limit it. The exemplary embodiments are described as processes or methods depicted as flowcharts; although the flowcharts describe the operations or steps in a certain order, many of these operations or steps can be performed in parallel, concurrently, or simultaneously, and the order of the operations can be rearranged. When an operation or step is completed, the corresponding process can be terminated, and additional steps not included in the drawings may also be included. The processes described above can correspond to methods, functions, procedures, subroutines, subroutines, etc., and the embodiments and features in the embodiments of this invention can be combined with each other without conflict.

[0064] The term "comprising" and its variations used in this invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The technical solutions of this invention will be further described below with reference to the accompanying drawings and specific embodiments; it should be understood that, for ease of description, only the parts related to this invention are shown in the drawings, not the entire structure.

[0065] High-precision acquisition equipment often contains data such as outliers, noise, environmental points, and non-target area points in the 3D images it acquires. This data can affect target tasks such as identification, measurement, and detection.

[0066] To overcome this problem, noise cleanup can be used to repair and optimize depth images to ensure the accuracy of the target task.

[0067] Currently, depth images are mainly repaired and optimized through filtering or global denoising. However, filtering can easily damage the source image, while global denoising is difficult to repair edge distortion. Therefore, the existing noise removal methods are not ideal.

[0068] To address the aforementioned problems, this invention provides an image noise removal scheme that can clean up and remove noise in non-target areas and unimportant data, thereby improving the accuracy and precision of industrial measurement algorithms and enabling efficient, fast, and accurate cleaning of 3D images such as depth maps and point clouds.

[0069] Example 1

[0070] Please refer to Figure 1 This invention provides a method for cleaning up image noise, comprising:

[0071] S1. Construct a multidimensional vector for the input image to be cleaned.

[0072] In this step, firstly, preset invalid values ​​are filled into the positions in the image to be cleaned where no data has been collected, and then fast point set features are calculated based on the image data of the image to be cleaned.

[0073] Finally, based on the features of the fast point set, a multidimensional vector weighted dot multiplication operator is defined by combining the spatial coordinates of the points, thus completing the construction of the multidimensional vector.

[0074] Specifically, calculating the fast point set features includes: calculating the normal vector of the current point; and calculating the spherical integral and volume integral values ​​of the linear correlation of the point curvature.

[0075] A multidimensional vector is constructed based on the spherical integral volume integral value that is linearly related to the current point's spatial location, the current point's normal vector, and the point's curvature.

[0076] In this embodiment, the specific process for calculating the fast point set features is as follows:

[0077] Specifically, calculating the normal vector of the current point includes:

[0078] Define the normal vector N: ,in, This indicates the actual spatial location of the image to be cleaned.

[0079] To eliminate the influence of errors, in step S1, the average value of the neighborhood normal vectors is used to replace the normal vector of the current point: ;

[0080] in, The result of solving for the normal vector at any position in the image to be cleaned. This indicates normalization.

[0081] Different curvatures have different calculation methods, but they all share the common characteristic of being an expression of spatial geometric features. Generally, when characterizing curvature features, the volume integral of the spherical convolution symbol is constructed to estimate the curvature.

[0082] Based on this, in this embodiment, the calculation of the linearly related spherical integral volume integral value of the point curvature includes:

[0083] Constructing the volume integral of the spherical convolution notation: ;

[0084] Among them, the integral kernel Defined as a sphere with center point p and radius r, S is the surface of the object, and the volume is the volume of the overlapping part of the sphere and the object.

[0085] Describing curvature shape based on volume integral: ;

[0086] in, Represents any point in space. This represents the volume integral with the current sphere radius r. This represents the average curvature at that point;

[0087] Calculate the spherical integral and volume integral values ​​of the linear correlation of point curvature: , and It is a function that describes the correlation between signals before and after.

[0088] Specifically, constructing multidimensional vectors includes:

[0089] Define the eigenvector of a point set: , Indicates the spatial location of a point. Indicates the normal direction of a point. The integral value of the sphere volume representing the linear correlation of the curvature at a point;

[0090] In this step, the purpose of defining the feature vector of the point set is to quantify the differences between points at different locations.

[0091] To measure the significance of the difference, a multidimensional vector weighted dot product operator is defined: ;

[0092] in, All are real numbers less than 1 and greater than 0, and satisfy the following conditions: Weight Used to measure the significant contribution of a point's spatial location. and These are used to measure the significant contributions of the normal and curvature, respectively. Let represent the norm of vector 2.

[0093] Following step S1, the image noise removal method provided by the present invention further includes:

[0094] S2. Perform connectivity segmentation based on multidimensional vectors to output regional data and determine the data to be removed from the regional data.

[0095] In this step, connectivity segmentation based on multidimensional vectors is used, and the segmentation threshold is determined by the dot product of two adjacent pixels. That's for you to decide.

[0096] Please refer to Figure 2 Specifically, connectivity segmentation based on multi-dimensional vectors is used to output regional data, and the data to be removed from the regional data is determined. This mainly includes two methods: local segmentation and whole-image segmentation, as detailed below:

[0097] S21. Use a local segmentation method based on multidimensional vectors to segment regions, output the first regional data after segmentation, and determine the data to be removed in the first regional data.

[0098] S22. Use the full-image segmentation method to perform regional segmentation based on multi-dimensional vectors, output the segmented second regional data, and determine the data to be removed in the second regional data.

[0099] Understandably, steps S21 or S22 can be selected for segmentation based on actual needs, and if necessary, steps S21 and S22 can be executed sequentially for segmentation.

[0100] Taking the sequential execution of S21 and S22 for segmentation as an example, connectivity segmentation based on multidimensional vectors is used to output regional data, and the data to be removed from the regional data is determined, including the following steps:

[0101] First, a local segmentation method is used to segment regions based on multidimensional vectors, and the segmented first regional data is output. The region of interest is determined in the first regional data, and the data outside the region of interest is determined as the data to be removed.

[0102] Then, a full-image segmentation method is used to perform regional segmentation based on multi-dimensional vectors, and the segmented second regional data is output. Noise points are identified in the second regional data and are determined as data to be removed.

[0103] Local segmentation is a method that avoids full-map segmentation by providing a region of interest and segmenting out all connected regions with the same attributes as the selected region.

[0104] Please refer to Figure 3 The local segmentation method, based on multi-dimensional vectors, for region segmentation includes the following steps:

[0105] S211. Determine seed points based on multidimensional vectors;

[0106] S212. According to the predetermined segmentation threshold, add point data with the same attributes as the seed point to the same region until all the remaining point data except the seed point are traversed.

[0107] S213, repeat steps S211 and S212 until the region segmentation is complete.

[0108] Among them, determining seed points based on multidimensional vectors can be done by selecting regions of interest based on multidimensional vectors and using these regions of interest as seed points, or by using point data that does not have the same attributes as the seed points as new seed points based on a pre-determined segmentation threshold.

[0109] Specifically, the local segmentation method selects a region of interest containing seed points, traverses the first seed point region, finds points with the same attributes in the remaining neighborhoods and adds them into the first region, and repeats this process until there are no similar points in the connected components.

[0110] Then, iterate through all the remaining point data except the seed point, and use the point data that does not have the same attributes as the seed point as the new seed point. Then add the point data that has the same attributes as the seed point to the same region. Repeat the above steps until the segmentation of the entire selected region is completed. Finally, output the regional data of the local segmentation.

[0111] Taking local segmentation as an example, the regional data obtained after segmentation can be represented as:

[0112] ;

[0113] in, Represents the set of the entire depth image. This indicates that the image can be divided into A limited area This represents the image size. Clearly, the result of local connectivity segmentation must be... and The result of any combination.

[0114] Based on the selected features, the basic idea of ​​controlling feature differences through a set threshold is used to perform region growth, thereby completing the connectivity segmentation of the entire image.

[0115] In this embodiment, the result of connectivity segmentation is represented by the following set relationship:

[0116] ;

[0117] in, Represents any region after partitioning. This represents the point that has been assigned to a given set. Indicates no more than The point index position value, The 4-neighborhood points are represented. The measure value representing the feature. The threshold representing the difference in features.

[0118] Please refer to Figure 4 In this embodiment, a full-image segmentation method is used to perform regional segmentation based on multi-dimensional vectors, including:

[0119] S221. Construct a four-neighbor template and traverse the entire graph based on the four-neighbor template;

[0120] S222. Construct four state diagrams, determine the connectivity of point data according to a predetermined segmentation threshold, and fill each point data into the four state diagrams according to the connectivity.

[0121] S223. Construct a set of regions. Based on the four state diagrams, traverse the entire graph, assign connected point data to the same region, and assign disconnected point data to the newly added region, until the region segmentation is completed.

[0122] As can be understood, whole-image segmentation involves traversing the entire image and assigning corresponding connected regions to each point based on its different attributes, thereby completing the segmentation task. A detailed description follows:

[0123] First, construct a four-neighbor Mask (template) in the lower left corner, construct four state graphs, then use the Mask to traverse the entire graph, determine the connectivity of the four neighboring points through the threshold T, fill the four state graphs in turn, and finally output the four state graphs.

[0124] Then, a region set is constructed. Using four state graphs, the entire image is traversed to allocate regions. When a disconnected point is encountered, a new region is added. The above operation is repeated until all regions are segmented.

[0125] Finally, output the segmented regional data.

[0126] S3. Remove the data to be removed from the regional data.

[0127] After segmentation, for the regional data obtained from the segmentation, a threshold for the number of points to be cleaned is set according to the characteristics of noise points and outliers. Noise points or non-target points that need to be cleaned are cleaned up, thereby finally obtaining the target region within the region of interest that meets the feature conditions.

[0128] The following sections, with reference to the accompanying diagrams, will provide a detailed introduction to the full-image segmentation method and the local segmentation method, respectively.

[0129] For depth images requiring global cleanup, this invention employs a full-image segmentation method, as detailed below:

[0130] First, input a depth image with a fixed resolution and offset, such as... Figure 5 As shown, taking a noisy grain depth image as an example, the purpose of cleaning is to ensure that the identified grain size is not affected by noise;

[0131] Then, the normal map and spherical integral map of the image are calculated, a K-dimensional vector is constructed for segmentation, weight parameters are set, and the K-dimensional vector is used for segmentation to obtain the following results: Figure 6 The results show that the dark background represents regions with more than 500 connected component points, the light-colored area represents regions with fewer than 100 component points after segmentation, and the white circle contains regions with between 100 and 500 connected component points after segmentation.

[0132] Finally, based on the requirements, the area to be cleaned was determined to be less than 500, a cleaning threshold was set, and the edge noise was cleaned up. The final result after cleaning is as follows. Figure 7 As shown.

[0133] For depth images requiring local cleaning, this invention employs a local segmentation method, as detailed below:

[0134] First, input a depth image with a fixed resolution and offset, such as... Figure 8 As shown, taking the weld image during welding as an example, the purpose of local cleaning is to ensure the accuracy of the fitting plane for subsequent defect detection;

[0135] Then, determine the areas of the image that need cleaning and draw the local ROI (Region of Interest), such as... Figures 9-11 As shown;

[0136] Next, parameters were set to perform region growing segmentation and obtain all noise points. The results are as follows: Figure 12 As shown;

[0137] Finally, the growing noise areas are cleaned up, completing the cleanup of the specified location. The resulting image is as follows. Figure 13 As shown.

[0138] For some complex cleaning needs, such as Figure 14 Taking the fan blade shown as an example, the local segmentation method and the full image segmentation method are executed sequentially for segmentation, as detailed below:

[0139] First, use local segmentation cleanup to remove the background and other irrelevant areas outside the fan blades, such as... Figure 15 As shown;

[0140] Then, global segmentation is used to clean up the remaining small areas, completing the cleanup task and obtaining the image as shown. Figure 16 As shown.

[0141] The image noise removal method proposed in this invention is applicable to the rapid removal of noise from 3D images based on point set features and region connectivity, and can also be used with other 3D image processing methods. Furthermore, the image noise removal method proposed in this invention, based on a point set feature-weighted dot product algorithm, makes noise removal from 3D images simple and fast. In addition, it can also be used as a means of extracting other salient features.

[0142] Example 2

[0143] Please refer to Figure 17 Based on the foregoing embodiments, the present invention also provides an image noise removal apparatus for implementing the image noise removal method as described in any of the above embodiments, including a calculation unit 10, a data determination unit 20, and a removal unit 30.

[0144] The computing unit 10 is used to construct a multidimensional vector for the input image to be cleaned.

[0145] Specifically, the calculation unit 10 first fills the positions in the image to be cleaned where no data has been collected with preset invalid values, and then calculates the fast point set features based on the image data of the image to be cleaned; finally, based on the fast point set features, it defines a multidimensional vector weighted dot multiplication operator in combination with the spatial coordinates of the points to complete the construction of the multidimensional vector.

[0146] Specifically, the calculation unit 10 is used to: calculate the normal vector of the current point; calculate the spherical integral volume integral value of the linear correlation of the point curvature; and construct a multidimensional vector based on the spatial position of the current point, the normal vector of the current point, and the spherical integral volume integral value of the linear correlation of the point curvature.

[0147] In this embodiment, the calculation unit 10 calculates the fast point set features based on the image data of the image to be cleaned, through the following steps.

[0148] Calculate the normal vector of the current point, including:

[0149] Define the normal vector N: ;

[0150] in, This indicates the actual spatial location of the image to be cleaned.

[0151] To eliminate the influence of errors, in step S1, the average value of the neighborhood normal vectors is used to replace the normal vector of the current point: ;

[0152] in, The result of solving for the normal vector at any position in the image to be cleaned. This indicates normalization.

[0153] Different curvatures have different calculation methods, but they all share the common characteristic of being an expression of spatial geometric features. Generally, when characterizing curvature features, the volume integral of the spherical convolution symbol is constructed to estimate the curvature.

[0154] Based on this, in this embodiment, the calculation unit 10 calculates the spherical integral volume integral value of the linear correlation of point curvature, including:

[0155] Constructing the volume integral of the spherical convolution notation: ;

[0156] Among them, the integral kernel Defined as a sphere with center point p and radius r, S is the surface of the object, and the volume is the volume of the overlapping part of the sphere and the object.

[0157] Describing curvature shape based on volume integral: ;

[0158] in, Represents any point in space. This represents the volume integral with the current sphere radius r. This represents the average curvature at that point;

[0159] Calculate the spherical integral and volume integral values ​​of the linear correlation of point curvature: ;

[0160] in, and It is a function that describes the correlation between signals before and after.

[0161] Computation unit 10 constructs multidimensional vectors, including:

[0162] Define the eigenvector of a point set: , Indicates the spatial location of a point. Indicates the normal direction of a point. The integral value of the sphere volume representing the linear correlation of the curvature of a point.

[0163] In this embodiment, the purpose of defining the feature vector of the point set is to quantify the differences between points at different locations.

[0164] To measure the significance of the difference, computation unit 10 defines a multidimensional vector weighted dot product operator: ;

[0165] in, All are real numbers less than 1 and greater than 0, and satisfy the following conditions: Weight Used to measure the significant contribution of a point's spatial location. and These are used to measure the significant contributions of the normal and curvature, respectively. Let represent the norm of vector 2.

[0166] The data determination unit 20 is used to perform connectivity segmentation based on multidimensional vectors to output regional data and determine the data to be removed from the regional data.

[0167] Specifically, the data determination unit 20 performs connectivity segmentation based on multidimensional vectors, and the segmentation threshold is determined by the dot product of two adjacent pixels. That's for you to decide.

[0168] The process involves connectivity segmentation based on multi-dimensional vectors to output regional data and identify data to be removed from these regional data. This primarily includes two methods: local segmentation and full-image segmentation, as detailed below:

[0169] The data determination unit 20 uses a local segmentation method to perform region segmentation based on multidimensional vectors, outputs the first regional data after segmentation, and determines the data to be removed in the first regional data;

[0170] The data determination unit 20 uses a full-image segmentation method to perform regional segmentation based on multi-dimensional vectors, outputs the segmented second regional data, and determines the data to be removed from the second regional data.

[0171] Understandably, you can choose either a local segmentation method or a full-image segmentation method to perform segmentation based on actual needs. If necessary, you can also perform segmentation by executing the local segmentation method and the full-image segmentation method in sequence.

[0172] Taking the sequential execution of local segmentation and full-image segmentation methods as an example, connectivity segmentation based on multi-dimensional vectors is used to output regional data. The process of identifying data to be removed from these regional data includes the following steps:

[0173] First, the data determination unit 20 uses a local segmentation method to perform region segmentation based on multidimensional vectors, outputs the segmented first regional data, determines the region of interest in the first regional data, and determines the data outside the region of interest as data to be removed.

[0174] Then, the data determination unit 20 uses the full-image segmentation method to perform regional segmentation based on multi-dimensional vectors, outputs the segmented second regional data, identifies noise points in the second regional data, and determines the noise points as data to be removed.

[0175] Understandably, local segmentation methods avoid full image segmentation by providing a region of interest and segmenting all connected regions with the same attributes as the selected region. Local segmentation methods, based on multi-dimensional vectors, involve the following steps for region segmentation:

[0176] Data determination unit 20 determines seed points based on multidimensional vectors;

[0177] The data determination unit 20 adds point data with the same attributes as the seed point to the same region according to a predetermined segmentation threshold, until all the remaining point data except the seed point are traversed.

[0178] The data determination unit 20 repeats the aforementioned steps until the region segmentation is completed.

[0179] Among them, determining seed points based on multidimensional vectors can be done by selecting regions of interest based on multidimensional vectors and using these regions of interest as seed points, or by using point data that does not have the same attributes as the seed points as new seed points based on a pre-determined segmentation threshold.

[0180] Specifically, the local segmentation method selects a region of interest containing seed points, traverses the first seed point region, finds points with the same attributes in the remaining neighborhoods and adds them into the first region, and repeats this process until there are no similar points in the connected components.

[0181] Then, iterate through all the remaining point data except the seed point, and use the point data that does not have the same attributes as the seed point as the new seed point. Then add the point data that has the same attributes as the seed point to the same region. Repeat the above steps until the segmentation of the entire selected region is completed. Finally, output the regional data of the local segmentation.

[0182] Taking local segmentation as an example, the regional data obtained after segmentation can be represented as:

[0183] ;

[0184] in, Represents the set of the entire depth image. This indicates that the image can be divided into A limited area This represents the image size. Clearly, the result of local connectivity segmentation must be... and The result of any combination.

[0185] Based on the selected features, the basic idea of ​​controlling feature differences through a set threshold is used to perform region growth, thereby completing the connectivity segmentation of the entire image.

[0186] In this embodiment, the result of connectivity segmentation is represented by the following set relationship:

[0187] ;

[0188] in, Represents any region after partitioning. This represents the point that has been assigned to a given set. Indicates no more than The point index position value, The 4-neighborhood points are represented. The measure value representing the feature. The threshold representing the difference in features.

[0189] In this embodiment, the data determination unit 20 uses a full-image segmentation method to perform regional segmentation based on multi-dimensional vectors, including:

[0190] Data determination unit 20 constructs a four-neighbor template and traverses the entire graph based on the four-neighbor template;

[0191] The data determination unit 20 constructs four state diagrams, determines the connectivity of point data based on a pre-determined segmentation threshold, and fills each point data into the four state diagrams according to the connectivity.

[0192] The data determination unit 20 constructs a set of regions, traverses the entire graph based on four state graphs, assigns connected point data to the same region, and assigns disconnected point data to newly added regions, until the region segmentation is completed.

[0193] As can be understood, whole-image segmentation involves traversing the entire image and assigning corresponding connected regions to each point based on its different attributes, thereby completing the segmentation task. A detailed description follows:

[0194] First, the data determination unit 20 constructs a four-neighbor Mask (template) in the lower left corner, constructs four state graphs, then uses the Mask to traverse the entire graph, determines the connectivity of the four neighboring points through the threshold T, fills the four state graphs in turn, and finally outputs the four state graphs.

[0195] Then, the data determination unit 20 constructs a region set, and uses four state diagrams to traverse the entire image to allocate regions. When a disconnected point is encountered, a new region is added. The above operation is repeated until all regions are segmented.

[0196] Finally, the data determination unit 20 outputs the segmented regional data.

[0197] The cleaning unit 30 is used to remove data to be removed from the regional data. After segmentation, the cleaning unit 30 sets a threshold for the number of points to be cleaned based on the characteristics of noise points and outliers in the segmented regional data, and cleans up the noise points or non-target points that need to be cleaned, thereby finally obtaining the target region within the region of interest that meets the feature conditions.

[0198] Example 3

[0199] Based on the foregoing embodiments, the present invention also provides a storage medium including computer-readable instructions, which, when read and executed by a computer, cause the computer to perform an image noise removal method, as follows:

[0200] For the input image to be cleaned, a multidimensional vector is constructed; connectivity segmentation is performed based on the multidimensional vector to output regional data, and the data to be removed in the regional data is determined; the data to be removed in the regional data is then removed.

[0201] In the step of constructing a multidimensional vector for the input image to be cleaned, firstly, preset invalid values ​​are filled in the positions in the image to be cleaned where no data has been collected. Then, fast point set features are calculated based on the image data of the image to be cleaned. Finally, based on the fast point set features, a multidimensional vector weighted dot multiplication operator is defined in combination with the spatial coordinates of the points to complete the construction of the multidimensional vector.

[0202] Specifically, calculating the fast point set features includes: calculating the normal vector of the current point; calculating the spherical integral volume integral value of the linear correlation of the point curvature; and finally, constructing a multidimensional vector based on the spatial position of the current point, the normal vector of the current point, and the spherical integral volume integral value of the linear correlation of the point curvature.

[0203] Furthermore, when identifying data to be removed from regional data, connectivity segmentation based on multidimensional vectors is used, with the segmentation threshold determined by the dot product of two adjacent pixels. That's for you to decide.

[0204] Understandably, you can choose either a local segmentation method or a full-image segmentation method to perform segmentation based on actual needs. If necessary, you can also perform segmentation by executing the local segmentation method and the full-image segmentation method in sequence.

[0205] The local segmentation method based on multi-dimensional vectors for region segmentation includes the following steps:

[0206] Seed points are determined based on multidimensional vectors; according to a predetermined segmentation threshold, point data with the same attributes as the seed point are added to the same region until all other point data except the seed point are traversed; the above steps are repeated until the region segmentation is completed.

[0207] A full-image segmentation method is employed, based on multi-dimensional vectors, to perform regional segmentation, including:

[0208] Construct a four-neighborhood template and traverse the entire graph based on the four-neighborhood template; construct four state graphs, determine the connectivity of point data according to a pre-determined segmentation threshold, and fill each point data into the four state graphs according to the connectivity; construct a region set, traverse the entire graph based on the four state graphs, assign connected point data to the same region, and assign disconnected point data to the newly added region, until the region segmentation is completed.

[0209] The process involves sequentially performing local segmentation and full-image segmentation methods for segmentation, performing connectivity segmentation based on multi-dimensional vectors to output regional data, and identifying data to be removed from the regional data. This includes the following steps:

[0210] First, a local segmentation method is used to segment regions based on multidimensional vectors, and the segmented first regional data is output. The region of interest is determined in the first regional data, and the data outside the region of interest is determined as the data to be removed.

[0211] Then, a full-image segmentation method is used to perform regional segmentation based on multi-dimensional vectors, and the segmented second regional data is output. Noise points are identified in the second regional data and are determined as data to be removed.

[0212] Taking local segmentation as an example, the regional data obtained after segmentation can be represented as:

[0213] ;

[0214] in, Represents the set of the entire depth image. This indicates that the image can be divided into A limited area This represents the image size. Clearly, the result of local connectivity segmentation must be... and The result of any combination.

[0215] Based on the selected features, the basic idea of ​​controlling feature differences through a set threshold is used to perform region growth, thereby completing the connectivity segmentation of the entire image.

[0216] In this embodiment, the result of connectivity segmentation is represented by the following set relationship:

[0217] ;

[0218] in, Represents any region after partitioning. This represents the point that has been assigned to a given set. Indicates no more than The point index position value, The 4-neighborhood points are represented. The measure value representing the feature. The threshold representing the difference in features.

[0219] Finally, after segmentation, based on the characteristics of noise points and outliers in the segmented regional data, a threshold for the number of points to be cleaned is set, and noise points or non-target points that need to be cleaned are cleaned up, thereby finally obtaining the target region within the region of interest that meets the feature conditions.

[0220] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for cleaning up image noise, characterized in that, include: For the input image to be cleaned, construct a multidimensional vector; Connectivity segmentation is performed based on the multidimensional vector to output regional data, and data to be removed from the regional data is determined; wherein, the segmentation threshold for connectivity segmentation based on the multidimensional vector is the dot product of two adjacent pixels. To decide; Remove the data to be removed from the aforementioned regional data; The process of constructing a multi-dimensional vector for the input image to be cleaned includes: Fill the locations in the image to be cleaned where no data was collected with preset invalid values; Calculate the fast point set features based on the image data of the image to be cleaned; Based on the aforementioned fast point set features, and combined with the spatial coordinates of the points, a multidimensional vector weighted dot multiplication operator is defined to complete the construction of the multidimensional vector; The feature vector of the point set is defined as follows: , Indicates the spatial location of a point. Indicates the normal direction of a point. The integral value of the sphere volume representing the linear correlation of the curvature at a point; The weighted dot product operator for multidimensional vectors is defined as follows: ； All are real numbers less than 1 and greater than 0, and satisfy the following conditions: Weight Used to measure the significant contribution of a point's spatial location. and These are used to measure the significant contributions of the normal and curvature, respectively. Let represent the norm of vector 2.

2. The image cleaning method according to claim 1, characterized in that, The process of performing connectivity segmentation based on the multidimensional vector to output regional data, and determining the data to be removed from the regional data, includes: A local segmentation method is used to segment regions based on the multidimensional vector, outputting the segmented first regional data, and determining the data to be removed from the first regional data; and / or The full-image segmentation method is used to perform regional segmentation based on the multidimensional vector, outputting the segmented second regional data, and determining the data to be removed in the second regional data.

3. The image noise removal method according to claim 1, characterized in that, The process of performing connectivity segmentation based on the multidimensional vector to output regional data, and determining the data to be removed from the regional data, includes: First, a local segmentation method is used to segment the region based on the multidimensional vector, and the segmented first regional data is output. The region of interest is determined in the first regional data, and the data outside the region of interest is determined as data to be removed. Then, a full-image segmentation method is used to perform regional segmentation based on the multidimensional vector, and the segmented second regional data is output. Noise points are identified in the second regional data and are determined as data to be removed.

4. The image noise removal method according to claim 2 or 3, characterized in that, The method of using local segmentation to segment regions based on the multidimensional vector includes: 1) Determine the seed point based on the multidimensional vector; 2) Based on a predetermined segmentation threshold, add point data with the same attributes as the seed point to the same region until all other point data except the seed point have been traversed. Repeat steps 1) and 2) until the region segmentation is complete.

5. The image noise removal method according to claim 4, characterized in that, The process of determining the seed point based on the multidimensional vector includes: The region of interest is selected based on the multidimensional vector, and the region of interest is used as a seed point; or, Based on a predetermined segmentation threshold, point data that does not have the same attributes as the seed point are used as new seed points.

6. The image noise removal method according to claim 2 or 3, characterized in that, The method of using full-image segmentation to perform regional segmentation based on the multi-dimensional vector includes: Construct a four-neighbor template, and traverse the entire graph based on the four-neighbor template; Construct four state diagrams, determine the connectivity of point data based on a predetermined segmentation threshold, and fill each point data into the four state diagrams according to the connectivity. Construct a set of regions, traverse the entire graph based on the four state graphs, assign connected point data to the same region, and assign disconnected point data to a new region, until the region segmentation is completed.

7. The image noise removal method according to claim 1, characterized in that, The step of calculating fast point set features based on the image data of the image to be cleaned includes: Calculate the normal vector of the current point, including: Define the normal vector N: ,in, Indicates the actual spatial location of the image to be cleaned; Calculate the sphere volume integral value of the linear correlation of point curvature, including: Constructing the volume integral of the spherical convolution notation: ; Among them, the integral kernel Defined as a sphere with center point p and radius r, S is the surface area of ​​the object, and the volume is the volume of the overlapping part of the sphere and the object; Describing curvature shape based on volume integral: ; in, Represents any point in space. This represents the volume integral with the current sphere radius r. This represents the average curvature at that point; Calculate the sphere volume integral value of the linear correlation of point curvature: , and It is a function that describes the correlation between signals before and after; A multidimensional vector is constructed based on the sphere volume integral value that is linearly related to the current point's spatial location, the current point's normal vector, and the point's curvature.

8. An image noise removal device, characterized in that, A method for implementing image noise removal as described in any one of claims 1 to 7, comprising: The computational unit is used to construct a multidimensional vector for the input image to be cleaned; The data determination unit is used to perform connectivity segmentation based on the multidimensional vector to output regional data, and to determine the data to be removed from the regional data; wherein, in the connectivity segmentation based on the multidimensional vector, the segmentation threshold is the dot product of two adjacent pixels. To decide; The clearing unit is used to remove data to be removed from the regional data.

9. A storage medium, characterized in that, Includes computer-readable instructions that, when read and executed by a computer, cause the computer to perform the image noise removal method as described in any one of claims 1 to 7.

Images