Image denoising method and system based on point sparsity and straight-through filter fusion

Abstract

The present invention discloses an image denoising method and system based on the fusion of point sparsity and a straight-through filter, and relates to the technical field of image denoising. The technical points of the present invention include: collecting a depth image containing an object to be detected; performing image processing on the depth image to obtain the pixel coordinates of the object to be detected, and performing coordinate conversion to obtain the three-dimensional coordinates of the object to be detected in the world coordinate system; using a point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates to obtain a core point cloud set; using Euclidean distance to traverse the core point cloud set point by point to perform point cloud clustering; performing point feature screening on the clustered point cloud set to obtain target point cloud data; using a straight-through filter to remove background point clouds from the target point cloud data. The present invention can efficiently remove noise point clouds and background point clouds, has a better average denoising rate and a higher accuracy rate, and can obtain a more accurate target point cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular to an image denoising method and system based on the fusion of point sparsity and a straight-through filter. Background Art

[0002] 3D vision technology is a technology that uses computer vision and image processing technology to perceive and reconstruct the shape, structure, position and motion of targets in three-dimensional space by fusing multiple data such as camera-collected images, depth information and the camera's own parameters. It can accurately obtain the three-dimensional shape and position information of the target object without touching it.

[0003] With the rapid development of emerging technologies such as artificial intelligence, machine vision and image processing technologies are being used more and more widely in intelligent manufacturing. In intelligent sorting systems, laser triangulation can be used to collect point cloud data of stacked workpieces. However, due to the interference between stacked workpieces, the accuracy of the acquisition equipment, external environmental factors, etc., noisy point clouds will be introduced, resulting in incomplete collected point cloud data, and noisy point clouds will have a serious impact on the accuracy of three-dimensional reconstruction and the accuracy of workpiece identification. Therefore, how to effectively denoise the original point cloud data is of great significance to the accuracy and subsequent processing of point cloud data. Summary of the invention

[0004] In view of the above problems, the present invention proposes an image denoising method and system based on the fusion of point sparsity and straight-through filter.

[0005] According to one aspect of the present invention, an image denoising method based on point sparsity and through-filter fusion is proposed, the method comprising:

[0006] Acquire a depth image containing the object to be detected;

[0007] Performing image processing on the depth image to obtain pixel coordinates of the object to be detected;

[0008] Perform coordinate conversion on the pixel coordinates of the object to be detected to obtain the three-dimensional coordinates of the object to be detected in the world coordinate system;

[0009] The point sparsity denoising algorithm is used to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set;

[0010] Using Euclidean distance to traverse the core point cloud set point by point to perform point cloud clustering;

[0011] The point features of the clustered point cloud set are screened to obtain the target point cloud data.

[0012] Furthermore, the point sparsity denoising algorithm is used to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain the core point cloud set, including: selecting the distance of the nearest Nth point, sorting the distances by size, and recording the distance of the Nth point as l N ; At the same time, the proportional coefficient σ is selected according to the density of the point cloud, and the distance l of the Nth point N The product of the proportional coefficient σ determines the maximum distance L of the neighboring points. distance ; Traverse the point clouds in the core point cloud set to obtain the distances of each neighbor point in the point cloud neighborhood; set the maximum distance threshold of the neighbor points to divide the point cloud into multiple point cloud blocks; remove the discrete point clouds based on the point quantity characteristics of the nearest neighbor points of each point cloud block, and retain the core point cloud blocks with concentrated distribution and large number of points, thereby completing the preliminary screening and obtaining the core point cloud set.

[0013] Furthermore, the point feature screening of the clustered point cloud set to obtain target point cloud data includes: counting the number of point clouds of multiple point cloud blocks, presetting a point number threshold interval, and comparing the point cloud number of each point cloud block with the preset point number threshold interval: if the point cloud number is less than the minimum value of the preset point number threshold interval, deleting the corresponding point cloud block to obtain the target point cloud.

[0014] Furthermore, after acquiring the target point cloud data, the background point cloud is further removed, specifically including: according to the camera field of view, the distance between the surface of the object to be detected and the measuring plane, a pass filter is used to remove the background point cloud from the target point cloud data to acquire the point cloud data of the object to be detected in the target area.

[0015] Furthermore, the distance between the surface of the object to be detected and the measuring plane is calculated as follows: the camera and the measuring plane always maintain a vertical posture, and the distance between the surface of the object to be detected and the measuring plane is measured by normal incidence. The calculation formula is:

[0016]

[0017] Wherein, L2 represents the object distance; f represents the focal length of the lens; α represents the angle between the light beam and the imaging plane; β represents the angle between the reflected light and the normal of the object surface; A′B′ represents the projection distance from the laser emitting the laser to the surface of the object to be detected and the measuring plane, and reflected to the imaging plane of the 3D camera.

[0018] According to another aspect of the present invention, an image denoising system based on point sparsity and through-filter fusion is proposed, the system comprising:

[0019] An image acquisition module, configured to acquire a depth image containing an object to be detected;

[0020] An original point cloud computing module is configured to perform image processing on the depth image to obtain pixel coordinates of the object to be detected; perform coordinate conversion on the pixel coordinates of the object to be detected to obtain three-dimensional coordinates of the object to be detected in a world coordinate system;

[0021] A core point cloud computing module, which is configured to use a point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set;

[0022] The target point cloud computing module is configured to traverse the core point cloud set point by point using Euclidean distance to perform point cloud clustering; perform point feature screening on the clustered point cloud set to obtain target point cloud data.

[0023] Furthermore, the core point cloud computing module uses a point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system, and obtains a core point cloud set including: selecting the distance of the nearest Nth point, sorting the distances by size, and recording the distance of the Nth point as l N ; At the same time, the proportional coefficient σ is selected according to the density of the point cloud, and the distance l of the Nth point N The product of the proportional coefficient σ determines the maximum distance L of the neighboring points. distance ; Traverse the point clouds in the core point cloud set to obtain the distances of each neighbor point in the point cloud neighborhood; set the maximum distance threshold of the neighbor points to divide the point cloud into multiple point cloud blocks; remove the discrete point clouds based on the point quantity characteristics of the nearest neighbor points of each point cloud block, and retain the core point cloud blocks with concentrated distribution and large number of points, thereby completing the preliminary screening and obtaining the core point cloud set.

[0024] Furthermore, the target point cloud computing module performs point feature screening on the clustered point cloud set to obtain target point cloud data, including: counting the number of point clouds of multiple point cloud blocks, presetting a point number threshold interval, and comparing the point cloud number of each point cloud block with the preset point number threshold interval: if the point cloud number is less than the minimum value of the preset point number threshold interval, deleting the corresponding point cloud block to obtain the target point cloud.

[0025] Furthermore, it also includes a background removal module, which is configured to further remove the background point cloud after acquiring the target point cloud data, specifically including: according to the camera field of view, the distance between the surface of the object to be detected and the measuring plane, using a pass filter to remove the background point cloud in the target point cloud data, and obtaining the point cloud data of the object to be detected in the target area.

[0026] Furthermore, the distance between the surface of the object to be detected and the measuring plane is calculated as follows: the camera and the measuring plane are always kept in a vertical posture, and the distance d between the surface of the object to be detected and the measuring plane is measured by normal incidence. The calculation formula is:

[0027]

[0028] Wherein, L2 represents the object distance; f represents the focal length of the lens; α represents the angle between the light beam and the imaging plane; β represents the angle between the reflected light and the normal of the object surface; A′B′ represents the projection distance from the laser emitting the laser to the surface of the object to be detected and the measuring plane, and reflected to the imaging plane of the 3D camera.

[0029] The beneficial technical effects of the present invention are:

[0030] The present invention proposes an image denoising method and system based on the fusion of point sparsity and direct-pass filter. First, a depth image containing an object to be detected is collected; the depth image is processed to obtain the pixel coordinates of the object to be detected; the pixel coordinates of the object to be detected are converted to obtain the three-dimensional coordinates of the object to be detected in the world coordinate system; the three-dimensional coordinates of the object to be detected in the world coordinate system are preliminarily screened using a point sparsity denoising algorithm to obtain a core point cloud set; the core point cloud set is traversed point by point using Euclidean distance to perform point cloud clustering; point feature screening is performed on the clustered point cloud set to obtain target point cloud data; the background point cloud is removed from the target point cloud data using a direct-pass filter to obtain point cloud data of the object to be detected in the target area. The advantages of the present invention are as follows:

[0031] (1) The point sparsity denoising algorithm has relatively stable results in processing point cloud data, and can effectively reduce the number of points in the noisy point cloud, further improve the average denoising rate of the point cloud, and obtain a more accurate point cloud set.

[0032] (2) After the point cloud set is processed by the point sparsity algorithm, the accuracy of the point cloud model after triangulation reconstruction is lower than that of the improved point sparsity algorithm. The improved point sparsity algorithm can improve the accuracy of the point cloud model, providing a good foundation for the accuracy of subsequent workpiece recognition and pose estimation.

[0033] (3) The point sparsity denoising algorithm and the straight-through filter denoising algorithm can effectively remove noise point clouds and background point clouds, have a better average denoising rate and higher accuracy, and can obtain more accurate target point clouds to meet the actual needs of the intelligent sorting system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification, and are used to further illustrate the preferred embodiments of the present invention and explain the principles and advantages of the present invention.

[0035] Figure 1It is a flow chart of an image denoising method based on point sparsity and through-filter fusion according to an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of the design principle of 3D camera laser triangulation depth image imaging in an embodiment of the present invention;

[0037] Figure 3 1 is a comparative analysis diagram of the image denoising method based on point sparsity in an embodiment of the present invention; wherein (a) corresponds to the original point cloud; (b) corresponds to the core point cloud; and (c) corresponds to the target point cloud.

[0038] Figure 4 1 is a comparative analysis diagram of the image denoising algorithm based on the straight-through filter in the embodiment of the present invention; wherein (a) corresponds to the core point cloud after denoising based on the improved point sparsity algorithm; and (b) corresponds to the point cloud after removing the background. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, exemplary implementations or embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described implementations or embodiments are only implementations or embodiments of a part of the present invention, not all of them. Based on the implementations or embodiments of the present invention, all other implementations or embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0040] The embodiment of the present invention provides an image denoising method based on point sparsity and through-filter fusion, such as Figure 1 As shown, the method comprises the following steps:

[0041] S1, collecting a depth image containing an object to be detected;

[0042] S2. Perform image processing on the depth image to obtain pixel coordinates of the object to be detected;

[0043] S3, performing coordinate conversion on the pixel coordinates of the object to be detected to obtain the three-dimensional coordinates of the object to be detected in the world coordinate system;

[0044] S4, using a point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set;

[0045] S5. Using Euclidean distance to traverse the core point cloud set point by point to perform point cloud clustering;

[0046] S6, performing point feature screening on the clustered point cloud set to obtain target point cloud data;

[0047] S7. According to the camera field of view and the distance between the surface of the object to be detected and the measuring plane, a pass filter is used to remove the background point cloud from the target point cloud data to obtain the point cloud data of the object to be detected in the target area.

[0048] The method starts at S1. In S1, a depth image containing an object to be detected is acquired.

[0049] According to an embodiment of the present invention, the object to be detected is a stacked workpiece, and the stacked workpiece is, for example, a palletizing application in a logistics warehouse. A depth image is acquired by using a normal projection method on the stacked workpiece.

[0050] The AT-integrated 3D camera (C5-2040-CS30-330) is used to obtain the depth image by the positive injection method. The acquisition principle is: a continuous infrared laser line perpendicular to the surface of the stacked workpiece is emitted to the stacked workpiece, and the infrared laser reflected by the stacked workpiece is captured by the sensor. Due to the different heights of the stacked workpieces, the scanning laser line will deform when passing through the surfaces of workpieces at different heights, and the position of each pixel point and the degree of deformation of the vertical laser scanning line are collected. The depth information of each pixel point of the stacked workpiece can be calculated by laser triangulation. The laser triangulation method uses the reflection law of light in the process of spatial propagation and the principle of similar triangles to calculate the displacement to be measured.

[0051] As an example, the AT-integrated 3D camera built in the intelligent sorting system experimental platform is fixed on the end effector of the six-axis manipulator. Figure 2 As shown. Each time an image is collected, the 3D camera and the measuring plane are always kept in a vertical posture, and the normal incidence method is selected to measure the displacement of the workpiece. The infrared light emitted by the laser is focused by the emitting lens and vertically incident on the surface of the stacked workpiece, that is, the reference plane FG. The incident light is parallel to the normal of the stacked workpiece surface, and the angle between the reflected light and the normal of the object surface is set to β. The incident light is imaged at point A' of the 3D camera, that is, on the imaging plane DE, and the angle between the light beam and the DE imaging plane is α. The stacked workpiece is displaced relative to the camera to point B, and the imaging point of the incident light beam moves to point B' on the imaging plane DE. According to the law of optical paths and the principle of similar triangles, it is obtained that ΔOBC is similar to ΔOB'C', and formula (1) is obtained:

[0052]

[0053] From the known angle relationship parameters: l AB =m,l OC′ =L1,l OA =L2, substituting the above into formula (1) yields (2):

[0054]

[0055] Wherein, L1 represents the image distance, L2 represents the object distance, and d represents the distance between the surface of the object and the measurement plane.

[0056] According to the Gaussian formula in the law of optics, we can get formula (3):

[0057]

[0058] Where f represents the focal length of the lens. Substituting equation (3) into equation (2), we get equation (4):

[0059]

[0060] By transforming formula (4), we can obtain the distance d between the surface of the object and the measurement plane, as shown in formula (5):

[0061]

[0062] It is known that the height of the camera remains constant when collecting point clouds. When the distance between the workpiece to be detected and the 3D camera changes, the scanning laser line will be offset, and the angle at which the reflected light beam is transmitted to the sensor will also change. By measuring the distance between point A' and point B' on the imaging plane ED, the depth information of the workpiece surface is recorded, that is, the distance d between the surface of the object and the measurement plane is obtained.

[0063] Then, S2 is executed, in which the depth image is processed to obtain pixel coordinates of the object to be detected.

[0064] According to the embodiment of the present invention, the pixel coordinates are acquired by using an existing image processing method.

[0065] Then, S3 is executed, in which the pixel coordinates of the object to be detected are transformed to obtain the three-dimensional coordinates of the object to be detected in the world coordinate system.

[0066] According to an embodiment of the present invention, by calculating a camera parameter matrix and an image coordinate matrix, the data in the depth image is converted into three-dimensional point coordinates in space to obtain original point cloud data.

[0067] The point cloud data of the stacked workpiece is obtained by converting the depth image through the internal and external parameter transformation formula of the camera calibration. A pixel point K is randomly selected from the image collected by the 3D camera, and the coordinate of the pixel point k in the pixel coordinate system is set to (u k , v k ), the origin of the image physical coordinate system is set to (u0, v0) in the image coordinate system, and the physical size of any point on the depth image on the X and Y axes is defined as dx, dy. Then point k satisfies equation (6) in the pixel coordinate system:

[0068]

[0069] The origin of the world coordinate system coincides with the origin of the camera. When performing rigid body transformation, no rotation or translation transformation between coordinate systems is involved. Therefore, equation (7) can be obtained:

[0070]

[0071] Through matrix calculation, we can get the transformation from each pixel point collected by the camera to the midpoint of the world coordinate. The transformation result is shown in formula (8):

[0072]

[0073] Among them, (u k , v k ) represents the kth coordinate point position in the image coordinate system, (u0, v0) represents the image center coordinates, (X w ,Y w ,Z w ) represents the position of the three-dimensional coordinate point in the world coordinate system, Z c Indicates the distance from the target workpiece to the camera.

[0074] At the same time, the visual processing software is used to retrieve the calibration file of the 3D camera, obtain the internal and external parameters of the camera and process the relevant parameters, and use threshold processing to extract the effective points in the target area and filter the depth information of the effective area. The values ​​of the X and Y axes are defined as the row and column coordinates of the target area, and the value of the Z axis is the grayscale value of each point. The parameter matrix and image coordinate matrix are calculated, and all the data in the depth map are converted into three-dimensional point coordinates in space, and finally the original point cloud data of the stacked workpiece is obtained.

[0075] Then, S4 is executed, in which a point sparsity denoising algorithm is used to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set.

[0076] According to an embodiment of the present invention, a point sparsity denoising algorithm is used to perform preliminary screening on the original point cloud data to obtain a core point cloud set.

[0077] The original point cloud data collected by laser triangulation method contains target point cloud and noise point cloud, such as Figure 3 (a) shows. The point cloud of stacked workpieces presents a regular and uniform distribution with a large number, while the noise point cloud presents an irregular and messy distribution with a small number. The point sparsity denoising algorithm first pre-processes the point cloud data of stacked workpieces collected by laser triangulation, sets reasonable thresholds for related parameters, selects the distance to the nearest Nth point, sorts them by distance, and records the distance of the Nth point as l N At the same time, the proportional coefficient σ is reasonably selected according to the density of the point cloud, and then the distance l of the Nth point is NThe product of the proportional coefficient σ determines the maximum distance L of the neighboring points. distance According to the distribution of core point cloud and discrete noise point cloud, the core point cloud is more concentrated and has a large number of points, while the discrete point cloud is scattered and has a small number of points. The collected point cloud is traversed to obtain the distance between each neighbor point in the point cloud neighborhood, set the maximum distance between neighbor points, divide the point cloud into modules, and remove the discrete point clouds of some small modules according to the number of points of the nearest neighbor points of each module. The core point cloud module with concentrated distribution and large number of points is retained to complete the preliminary screening of the original point cloud data, and finally obtain the core point cloud set as shown in Figure 3 (b) as shown.

[0078] Then, S5 is executed, in which the core point cloud set is traversed point by point using the Euclidean distance to perform point cloud clustering.

[0079] According to an embodiment of the present invention, the maximum distance l between points is set max , and then according to the set maximum distance l max Traverse point by point to find the nearest point between each point, and cluster the point cloud data of each module, so as to filter the noise point cloud set with a smaller number of points in the original point cloud data by the order of points.

[0080] Then, S6 is executed, in which point features are screened for the clustered point cloud set to obtain target point cloud data.

[0081] According to an embodiment of the present invention, point features are screened for the clustered point cloud set to obtain relatively accurate target point cloud data.

[0082] Although the initial screening of the point cloud by the improved point sparsity algorithm can effectively remove sparse and isolated point cloud sets, it fails to effectively remove noise point clouds in the neighborhood of the target point cloud. Therefore, secondary screening is required. Point cloud point feature screening improves the denoising process based on the initial screening of the point cloud. The basic idea of ​​the algorithm is to first cluster the core point cloud data to obtain the point cloud data of each large module, and then count the number of points in the point cloud data of each large module to screen the core point cloud set based on the point quantity feature. It is known that the experimental test sample is a small cylindrical part with a radius of 18mm. The single workpiece point cloud data collected and generated by the laser triangulation method is above 2W points, and the discrete and small number of noise point cloud points is in the range of [10-1k]. According to the statistics, screening and analysis of multiple experimental data, the point number interval for the point feature screening of a single workpiece of the stacked workpiece is set to [2w, ∞]. By screening the number of points in the point feature, a more accurate point cloud set of the stacked workpiece can be obtained, that is, the target point cloud, such as Figure 3 (c).

[0083] The point feature screening of the improved point sparsity algorithm effectively removes irrelevant noise point clouds in the neighborhood of the stacked workpiece point cloud, reduces the number of irrelevant point cloud points, speeds up the reconstruction accuracy of the point cloud model, and improves the accuracy of workpiece recognition.

[0084] Then S7 is executed. In S7, after the target point cloud data is obtained, the background point cloud is further removed, specifically including: according to the camera field of view and the distance between the surface of the object to be detected and the measuring plane, a pass filter is used to remove the background point cloud from the target point cloud data to obtain the point cloud data of the object to be detected in the target area.

[0085] According to an embodiment of the present invention, a specific value range of the three dimensions X, Y, and Z in the pass-through filter algorithm is set to separate a large number of irrelevant noise point clouds and background point clouds, further screen and remove point cloud data outside the value range, and retain the target workpiece point cloud data within the value range of the X, Y, and Z dimensions. This method can efficiently remove background point clouds and noise point clouds, obtain accurate workpiece point clouds in the target area, and further speed up the speed of image preprocessing.

[0086] The straight-through filter is an efficient and practical filter for removing noise point clouds, and is suitable for removing background point clouds. The straight-through filter point cloud denoising used in this embodiment is to remove the background point cloud from the target point cloud data. Since the field of view of the AT integrated camera is relatively fixed, the collected point cloud data is Figure 4 As shown in (a), it is necessary to set reasonable value ranges in the X, Y, and Z dimensions as shown in formula (9) to remove the background point cloud in the target point cloud and obtain a more accurate workpiece point cloud in the target area, such as Figure 4 (b) as shown.

[0087]

[0088] There are a large number of irrelevant off-body point clouds and background point clouds in the original point cloud data collected by the AT-integrated 3D camera. These noise point clouds will affect the 3D reconstruction accuracy and workpiece recognition accuracy of the subsequent point cloud data. The direct filter denoising method can effectively remove a large number of irrelevant off-body point clouds and background point clouds, retain more accurate target point cloud data, speed up the subsequent point cloud processing, and improve the accuracy of workpiece recognition.

[0089] In summary, the present invention studies the influence of noise point clouds in stacked workpieces on the three-dimensional reconstruction accuracy and target recognition accuracy, and solves the problems of low target recognition accuracy and large pose estimation deviation of existing denoising methods when the stacking degree of stacked workpieces is large.

[0090] Another embodiment of the present invention provides an image denoising system based on point sparsity and straight-through filter fusion, the system comprising:

[0091] An image acquisition module, configured to acquire a depth image containing an object to be detected;

[0092] An original point cloud computing module is configured to perform image processing on the depth image to obtain pixel coordinates of the object to be detected; perform coordinate conversion on the pixel coordinates of the object to be detected to obtain three-dimensional coordinates of the object to be detected in a world coordinate system;

[0093] A core point cloud computing module, which is configured to use a point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set;

[0094] The target point cloud computing module is configured to traverse the core point cloud set point by point using Euclidean distance to perform point cloud clustering; perform point feature screening on the clustered point cloud set to obtain target point cloud data.

[0095] In this embodiment, preferably, the core point cloud computing module uses a point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system, and obtaining the core point cloud set includes: selecting the distance of the nearest Nth point, sorting the distances by size, and recording the distance of the Nth point as l N ; At the same time, the proportional coefficient σ is selected according to the density of the point cloud, and the distance l of the Nth point N The product of the proportional coefficient σ determines the maximum distance L of the neighboring points. distance ; Traverse the point clouds in the core point cloud set to obtain the distances of each neighbor point in the point cloud neighborhood; set the maximum distance threshold of the neighbor points to divide the point cloud into multiple point cloud blocks; remove the discrete point clouds based on the point quantity characteristics of the nearest neighbor points of each point cloud block, and retain the core point cloud blocks with concentrated distribution and large number of points, thereby completing the preliminary screening and obtaining the core point cloud set.

[0096] In this embodiment, preferably, the target point cloud computing module performs point feature screening on the clustered point cloud set, and obtaining the target point cloud data includes: counting the number of point clouds of multiple point cloud blocks, presetting a point number threshold interval, and comparing the number of point clouds of each point cloud block with the preset point number threshold interval: if the point cloud number is less than the minimum value of the preset point number threshold interval, deleting the corresponding point cloud block, thereby obtaining the target point cloud.

[0097] In this embodiment, preferably, it also includes a background removal module, which is configured to further remove the background point cloud after acquiring the target point cloud data, specifically including: according to the camera field of view, the distance between the surface of the object to be detected and the measuring plane, using a pass filter to remove the background point cloud in the target point cloud data, and obtaining the point cloud data of the object to be detected in the target area.

[0098] In this embodiment, preferably, the distance between the surface of the object to be detected and the measurement plane is calculated as follows: the camera and the measurement plane always maintain a vertical posture, and the distance d between the surface of the object to be detected and the measurement plane is measured by normal incidence, and the calculation formula is:

[0099]

[0100] Wherein, L2 represents the object distance; f represents the focal length of the lens; α represents the angle between the light beam and the imaging plane; β represents the angle between the reflected light and the normal of the object surface; A′B′ represents the projection distance from the laser emitting the laser to the surface of the object to be detected and the measuring plane, and reflected to the imaging plane of the 3D camera.

[0101] The function of the image denoising system based on the fusion of point sparsity and straight-through filter described in this embodiment can be explained by the aforementioned image denoising method based on the fusion of point sparsity and straight-through filter. Therefore, for the parts not described in detail in this embodiment, please refer to the above method embodiments and will not be repeated here.

[0102] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments are contemplated within the scope of the invention thus described. The disclosure of the present invention is intended to be illustrative rather than restrictive of the scope of the invention, which is defined by the appended claims.

Claims

1. An image denoising method based on point sparsity and straight-through filter fusion, characterized in that: include: Acquire a depth image containing the object to be detected; Performing image processing on the depth image to obtain pixel coordinates of the object to be detected; Perform coordinate conversion on the pixel coordinates of the object to be detected to obtain the three-dimensional coordinates of the object to be detected in the world coordinate system; The point sparsity denoising algorithm is used to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set; including: selecting the distance of the nearest Nth point, sorting the distances by size, and recording the distance of the Nth point as l N ; At the same time, the proportional coefficient σ is selected according to the density of the point cloud, and the distance l of the Nth point N The product of the proportional coefficient σ determines the maximum distance L of the neighboring points. distance ; Traverse the point clouds in the core point cloud set to obtain the distances of each neighbor point in the point cloud neighborhood; set the maximum distance threshold of the neighbor points to divide the point cloud into multiple point cloud blocks; remove the discrete point clouds based on the point quantity characteristics of the nearest neighbor points of each point cloud block, and retain the core point cloud blocks with concentrated distribution and large number of points, thereby completing the preliminary screening and obtaining the core point cloud set; Using Euclidean distance to traverse the core point cloud set point by point to perform point cloud clustering; Perform point feature screening on the clustered point cloud set to obtain target point cloud data; After acquiring the target point cloud data, the background point cloud is further removed, specifically including: according to the camera field of view, the distance between the surface of the object to be detected and the measuring plane, a pass filter is used to remove the background point cloud from the target point cloud data to acquire the point cloud data of the object to be detected in the target area.

2. The image denoising method based on point sparsity and straight-through filter fusion according to claim 1, characterized in that: The point feature screening of the clustered point cloud set to obtain target point cloud data includes: counting the number of point clouds in multiple point cloud blocks, presetting a point number threshold interval, and comparing the point cloud number of each point cloud block with the preset point number threshold interval: if the point cloud number is less than the minimum value of the preset point number threshold interval, deleting the corresponding point cloud block to obtain the target point cloud.

3. The image denoising method based on point sparsity and straight-through filter fusion according to claim 1, characterized in that: The distance between the surface of the object to be detected and the measurement plane is calculated as follows: when collecting the depth image, the camera and the measurement plane always maintain a vertical posture, and the distance between the surface of the object to be detected and the measurement plane is measured by normal incidence. The calculation formula is: Wherein, L2 represents the object distance; f represents the focal length of the lens; α represents the angle between the light beam and the imaging plane; β represents the angle between the reflected light and the normal of the object surface; A'B' represents the projection distance from the laser emitting the laser to the surface of the object to be detected and the measuring plane, and reflected to the imaging plane of the 3D camera.

4. An image denoising system based on point sparsity and straight-through filter fusion, characterized in that: include: An image acquisition module, configured to acquire a depth image containing an object to be detected; An original point cloud computing module is configured to perform image processing on the depth image to obtain pixel coordinates of the object to be detected; perform coordinate conversion on the pixel coordinates of the object to be detected to obtain three-dimensional coordinates of the object to be detected in a world coordinate system; The core point cloud computing module is configured to use the point sparsity denoising algorithm to preliminarily screen the three-dimensional coordinates of the object to be detected in the world coordinate system to obtain a core point cloud set; including: selecting the distance of the nearest Nth point, sorting the distances by size, and recording the distance of the Nth point as l N ; At the same time, the proportional coefficient σ is selected according to the density of the point cloud, and the distance l of the Nth point N The product of the proportional coefficient σ determines the maximum distance L of the neighboring points. distance ; Traverse the point clouds in the core point cloud set to obtain the distances of each neighbor point in the point cloud neighborhood; set the maximum distance threshold of the neighbor points to divide the point cloud into multiple point cloud blocks; remove the discrete point clouds based on the point quantity characteristics of the nearest neighbor points of each point cloud block, and retain the core point cloud blocks with concentrated distribution and large number of points, thereby completing the preliminary screening and obtaining the core point cloud set; A target point cloud computing module is configured to traverse the core point cloud set point by point using Euclidean distance to perform point cloud clustering; perform point feature screening on the clustered point cloud set to obtain target point cloud data; The background removal module is configured to further remove the background point cloud after acquiring the target point cloud data, specifically including: according to the camera field of view, the distance between the surface of the object to be detected and the measuring plane, using a straight-through filter to remove the background point cloud in the target point cloud data, and acquiring the point cloud data of the object to be detected in the target area.

5. The image denoising system based on point sparsity and straight-through filter fusion according to claim 4, characterized in that: The target point cloud computing module performs point feature screening on the clustered point cloud set, and obtaining the target point cloud data includes: counting the number of point clouds of multiple point cloud blocks, presetting a point number threshold interval, and comparing the number of point clouds of each point cloud block with the preset point number threshold interval: if the point cloud number is less than the minimum value of the preset point number threshold interval, deleting the corresponding point cloud block, thereby obtaining the target point cloud.

6. The image denoising system based on point sparsity and straight-through filter fusion according to claim 4, characterized in that: The distance between the surface of the object to be detected and the measurement plane in the background removal module is calculated as follows: when collecting the depth image, the camera and the measurement plane always maintain a vertical posture, and the distance d between the surface of the object to be detected and the measurement plane is measured by normal incidence. The calculation formula is: Wherein, L2 represents the object distance; f represents the focal length of the lens; α represents the angle between the light beam and the imaging plane; β represents the angle between the reflected light and the normal of the object surface; A'B' represents the projection distance from the laser emitting the laser to the surface of the object to be detected and the measuring plane, and reflected to the imaging plane of the 3D camera.

Images