Method and device for filtering point cloud flying points noise in 3D reconstruction

Abstract

The present application discloses a method and device for filtering out cloud point noise in three-dimensional reconstruction. In the present application, by obtaining first phase data and identifying discontinuity points in the first phase data, a number of phase sequences are obtained; the phase sequences are composed of at least one and continuous phase points; according to the longest phase sequence, from all the phase sequences, a first phase sequence without noise and a second phase sequence containing noise are screened and obtained; by combining all the first phase sequences, the noise points in all the current second phase sequences are filtered out. Through the present application, all potential noise points can be accurately identified, and correct phase data misidentified as noise points can be identified from all potential noise points, thereby ensuring the accuracy of noise data filtering.

Description

Technical Field

[0001] The present application relates to the field of noise processing for structured light three-dimensional reconstruction, and particularly to a method and device for filtering out flying point noise in three-dimensional reconstruction point clouds. Background Art

[0002] The phase shift method is a commonly used coding method for structured light three-dimensional reconstruction. This method not only has low hardware requirements but also fast calculation speed, so it is widely used in actual scenarios. However, in the actual calculation process, due to factors such as object reflection, illumination interference, occlusion, and multiple reflections, errors occur in phase calculation. These errors will form noise flying points with no obvious distribution characteristics in the finally reconstructed point cloud. When filtering out this noise flying point, a large amount of computing power is often required for filtering, and the filtering effect is poor.

[0003] Since in the phase diagram, the phase data has the property of monotonicity in the region or globally along the pixel direction, based on this constraint, incorrect phase values can be effectively filtered out, thereby reducing the noise in the point cloud. Existing technologies are based on the above principle to filter out the phase points that do not satisfy monotonicity in the phase data to achieve the purpose of fast denoising. However, when facing non - continuous planar objects or objects with occlusions, such as in a scene with occlusions, the angles of projection and the camera are different, resulting in partial phase loss, so that the correct phase data does not satisfy the monotonicity constraint, and finally the correct phase data is incorrectly filtered out.

[0004] Therefore, how to solve the problem of filtering out flying point noise in three - dimensional reconstruction point clouds of non - continuous planar objects or objects with occlusions has become a problem that needs to be solved currently. Summary of the Invention

[0005] The present application provides a method and device for filtering out flying point noise in three - dimensional reconstruction point clouds to solve the technical problem of filtering out flying point noise in three - dimensional reconstruction point clouds of non - continuous planar objects or objects with occlusions.

[0006] To solve the above - mentioned technical problem, in a first aspect, an embodiment of the present application provides a method for filtering out flying point noise in three - dimensional reconstruction point clouds, including:

[0007] Obtain first - phase data and identify the discontinuous points in the first - phase data to obtain a number of phase sequences; the phase sequences are composed of at least one and continuous phase points;

[0008] According to the longest phase sequence, select and obtain a first phase sequence without noise and a second phase sequence containing noise from all the phase sequences;

[0009] Combine all the first phase sequences to filter out the noise points in all the current second phase sequences.

[0010] Compared with the prior art, the embodiments of the present application have the following beneficial effects: For discontinuous planar objects or objects with occlusions, the phase pixel map of their phase data is often not a straight line, but a broken line with a large number of break points and noise points. By identifying the break points in the broken line, not only can the break points on the discontinuous plane be obtained, but also the noise points can be accurately identified because the noise points often exist separately from the broken line. Since the longest sequence that satisfies the monotonicity constraint must be the correct sequence, based on this premise and the principle that sequences must also satisfy the monotonicity constraint, the correct sequence can be used as a reference to further identify the incorrect sequences containing noise and the correct sequences without noise. At this time, when there is an occlusion, some phase sequences in the obtained phase data are misidentified as incorrect sequences. At this time, in the correct sequence, there must be a blank phase data area with the same length as the misidentified sequence, and the phase data in the misidentified sequence will not exist in the correct sequence. Based on the above characteristics, it can be identified to further accurately filter the preliminarily screened noise data, improving the accuracy of fly point noise filtering.

[0011] In some embodiments of the first aspect of the present application, the obtaining of the first phase data and the identification of the break points in the first phase data to obtain a plurality of phase sequences include:

[0012] Obtain a first phase point with the maximum phase value from the first phase data, and the pixel value of the first phase point;

[0013] Taking the ratio of the phase value and the pixel value of the first phase point as a base, respectively obtain a step determination threshold and a similarity determination threshold;

[0014] According to the step determination threshold and the similarity determination threshold, identify the break points in the first phase data;

[0015] Divide the first phase data according to all the break points to obtain a plurality of phase sequences.

[0016] Compared with the prior art, the embodiments of the present application have the following beneficial effects: Since the phase data of a continuous plane, the change trend of its phase value and pixel value is a straight line with a determined slope. For the phase data of a non - continuous plane, compared with the phase data of a continuous plane, it is equivalent to dividing a straight line into multiple line segments with different slopes or non - connected line segments, but its change trend will not change. Therefore, the slope of the line formed by the data in each phase sequence of the non - continuous plane will definitely fluctuate within the range of the slope of the straight line obtained under the assumption of a continuous plane. Based on this characteristic, according to the phase data and pixel points of the first phase point, obtain the slope under the assumption of a continuous plane, and use this slope as a base to simply determine which phase points belong to the same phase sequence.

[0017] In some embodiments of the first aspect of the present application, identifying the discontinuity points in the first phase data according to the step determination threshold and the similarity determination threshold includes:

[0018] Sort the phase points in the first phase data in ascending order according to the pixel values of the phase points in the first phase data;

[0019] Successively determine whether the phase points in the previously traversed first phase data are discontinuity points according to the difference in phase values between the currently traversed phase points in the first phase data and the previous phase point of the currently traversed phase point;

[0020] If the magnitude relationship between the phase value of the currently traversed phase point and the previous phase point is decreasing, determine that the currently traversed phase point is a discontinuity point; if the phase difference between the currently traversed phase point and the previous phase point is greater than the step determination threshold or less than the similarity determination threshold, determine that the currently traversed phase point is a discontinuity point.

[0021] Compared with the prior art, the embodiments of the present application have the following beneficial effects: Since in the phase data, the phase values of the phase points must increase monotonically with the pixel values, and because the slope of the straight line of the phase data obtained under a continuous plane is fixed. Therefore, after sorting the phase points according to the pixel values, successively judge the relationship between the currently traversed phase point and the previous traversed phase point. If it is decreasing, then the two phase points violate the aforementioned characteristics, and the currently traversed phase point must be a phase point of potential noise, and then this phase point can be determined as a discontinuity point. In addition, if the two phase points meet the increasing constraint requirements, and their difference is too far from the slope of the straight line obtained under the continuous plane assumption, it can be determined that the currently traversed phase point corresponds to a break point of a non-continuous plane, and then determine that the currently traversed phase point is a discontinuity point, thereby improving the correctness of subsequent phase sequence segmentation.

[0022] In some embodiments of the first aspect of the present application, screening and obtaining the first phase sequence without noise and the second phase sequence containing noise from all the phase sequences according to the longest phase sequence includes:

[0023] Sort the phase sequences in descending order according to the length of each phase sequence;

[0024] Determine the longest phase sequence as the first phase sequence, and starting from the longest phase sequence, traverse each phase sequence; when the currently traversed phase sequence satisfies the monotonicity constraint and the similarity determination threshold with respect to the previously determined first phase sequence, determine that the currently traversed phase sequence is the first phase sequence;

[0025] Take the phase sequences that are not the first phase sequence among all the said phase sequences as the second phase sequence.

[0026] Compared with the prior art, the embodiments of the present application have the following beneficial effects: Since the longest phase sequence that satisfies the monotonicity constraint can be first determined as the phase sequence that does not contain noise data, based on this premise, and the characteristic that the phase data corresponding to other plane regions without occluders also satisfies the monotonicity constraint with respect to the first phase sequence, the phase sequences corresponding to the plane regions without occluders can be quickly screened out, so as to obtain all the first phase sequences that definitely do not contain noise, and further determine the second phase sequences that may contain noise based on this, realizing preliminary noise filtering and improving the correctness of subsequent noise filtering.

[0027] In some embodiments of the first aspect of the present application, the step of determining the currently traversed phase sequence as the first phase sequence includes:

[0028] Obtain the second phase point with the maximum phase value in the currently traversed phase sequence and the third phase point with the minimum phase value in the previously determined first phase sequence;

[0029] When the phase value of the second phase point is less than the phase value of the third phase point, and the difference between the phase values of the second phase point and the third phase point is greater than the similarity determination threshold, determine the currently traversed phase sequence as the first phase sequence.

[0030] Compared with the prior art, the embodiments of the present application have the following beneficial effects: According to the characteristic of the change trend of the phase value and the pixel value of each phase point in the above phase data, by judging whether the change trend of the phase value and the pixel value of each phase point in the currently traversed phase sequence satisfies the above characteristic, the first phase data that definitely does not contain noise can be accurately obtained, improving the accuracy and efficiency of noise filtering.

[0031] In some embodiments of the first aspect of the present application, the step of combining all the first phase sequences to filter out the noise points in all the currently determined second phase sequences containing noise includes:

[0032] Traverse each phase point in each of the second phase sequences, and judge whether the currently traversed phase point has the same phase value as at least one phase point in at least one of the first phase sequences;

[0033] If they are the same, filter out all the phase points in the currently traversed second phase sequence; otherwise, judge whether to filter out all the phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence.

[0034] Compared with the prior art, the embodiments of the present application have the following beneficial effects: From the characteristics of the change trends of the phase values and pixel values of each phase point in the above phase data, it can be concluded that if the phase data of a current phase point coincides with the phase data of a phase point in a phase sequence that does not contain noise and is definitely correct, it indicates that this phase point does not satisfy the trend characteristic of the monotonic change of the phase value with the pixel value. Further, such phase points can be quickly and accurately identified as noise data.

[0035] In some embodiments of the first aspect of the present application, the determining whether to filter all phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence includes:

[0036] Determining whether there are a third phase sequence and a fourth phase sequence in all the first phase sequences, and there is no phase data between the phase point with the minimum phase value of the third phase sequence and the phase point with the maximum phase value of the fourth phase sequence; wherein, the minimum phase value of the third phase sequence is greater than the maximum phase value of the currently traversed second phase sequence; the maximum phase value of the fourth phase sequence is less than the minimum phase value of the currently traversed second phase sequence;

[0037] If so, retain the currently traversed second phase sequence; otherwise, filter all phase points in the currently traversed second phase sequence.

[0038] Compared with the prior art, the embodiments of the present application have the following beneficial effects: Since the phase data of the occluder will be misidentified as noise data, resulting in a situation where there is a missing phase data in the correct phase data obtained after initially filtering the noise. And the misidentified noise data can definitely be filled into the area where there is a missing phase data in the current correct phase data. According to the above characteristics, by traversing the first phase sequence, based on the starting phase point and the phase value at the end of each first phase sequence, it can be quickly determined whether there is the above-mentioned area with missing phase data, which is consistent with the second phase sequence, so as to quickly and accurately restore the phase data misidentified as noise data and improve the accuracy of noise filtering.

[0039] In a second aspect, the embodiments of the present application provide a device for filtering flying-point noise of three-dimensional reconstruction point clouds, including: a phase data partitioning module, a phase sequence screening module, and a denoising module;

[0040] Among them, the phase data partitioning module is used to obtain first-phase data and identify discontinuous points in the first-phase data to obtain a plurality of phase sequences; the phase sequence is composed of at least one and continuous phase points;

[0041] The phase sequence screening module is configured to screen and obtain a first phase sequence without noise and a second phase sequence containing noise from all the phase sequences according to the longest phase sequence;

[0042] The denoising module is configured to combine all the first phase sequences to filter out the noise points in all the current second phase sequences.

[0043] In some embodiments of the second aspect of the present application, the phase data partitioning module includes: a first phase point obtaining unit, a threshold obtaining unit, a discontinuity point obtaining unit, and a phase sequence obtaining unit;

[0044] Among them, the first phase point obtaining unit is configured to obtain a first phase point with the maximum phase value from the first phase data, and the pixel value of the first phase point;

[0045] The threshold obtaining unit is configured to respectively obtain a step determination threshold and a similarity determination threshold based on the ratio of the phase value and the pixel value of the first phase point;

[0046] The discontinuity point obtaining unit is configured to identify the discontinuity points in the first phase data according to the step determination threshold and the similarity determination threshold;

[0047] The phase sequence obtaining unit is configured to partition the first phase data according to all the discontinuity points to obtain a plurality of phase sequences.

[0048] In some embodiments of the second aspect of the present application, the discontinuity point obtaining unit is configured to identify the discontinuity points in the first phase data according to the step determination threshold and the similarity determination threshold, including:

[0049] Sort the phase points in the first phase data in ascending order according to the pixel values of the phase points in the first phase data;

[0050] Successively judge whether the phase points in the currently traversed first phase data are discontinuity points according to the difference in phase values between the currently traversed phase point in the first phase data and the previous phase point of the currently traversed phase point;

[0051] If the magnitude relationship between the currently traversed phase point and the previous phase point is decreasing, determine that the currently traversed phase point is a discontinuity point; if the phase difference between the currently traversed phase point and the previous phase point is greater than the step determination threshold or less than the similarity determination threshold, determine that the currently traversed phase point is a discontinuity point.

[0052] In some embodiments of the second aspect of the present application, the phase sequence screening module includes: a first sorting unit, a first phase sequence obtaining unit, and a second phase sequence obtaining unit;

[0053] Among them, the first sorting unit is configured to sort the phase sequences in descending order according to the lengths of the phase sequences.

[0054] The first phase sequence obtaining unit is configured to determine the longest phase sequence as the first phase sequence, and starting from the longest phase sequence, traverse each of the phase sequences; when the currently traversed phase sequence satisfies the monotonicity constraint and the similarity determination threshold with respect to the previously determined first phase sequence, determine the currently traversed phase sequence as the first phase sequence.

[0055] The second phase sequence obtaining unit is configured to use the phase sequences that are not the first phase sequence among all the phase sequences as the second phase sequences.

[0056] In some embodiments of the second aspect of the present application, the step of determining the currently traversed phase sequence as the first phase sequence includes:

[0057] Obtain a second phase point with the maximum phase value in the currently traversed phase sequence and a third phase point with the minimum phase value in the previously determined first phase sequence.

[0058] When the phase value of the second phase point is less than the phase value of the third phase point, and the difference between the phase values of the second phase point and the third phase point is greater than the similarity determination threshold, determine the currently traversed phase sequence as the first phase sequence.

[0059] In some embodiments of the second aspect of the present application, the denoising module includes: a first judgment unit and a first execution unit;

[0060] Among them, the first judgment unit is configured to traverse each phase point in each of the second phase sequences and judge whether the currently traversed phase point has the same phase value as at least one phase point in at least one of the first phase sequences.

[0061] The first execution unit is configured to, if they are the same, filter out all the phase points in the currently traversed second phase sequence; otherwise, judge whether to filter out all the phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence.

[0062] In some embodiments of the second aspect of the present application, the first judgment unit is configured to judge whether to filter out all the phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence, including:

[0063] Determine whether there is a third phase sequence and a fourth phase sequence in all the first phase sequences, and there is no phase data between the phase point of the minimum phase value of the third phase sequence and the phase point of the maximum phase value of the fourth phase sequence; wherein, the minimum phase value of the third phase sequence is greater than the maximum phase value of the currently traversed second phase sequence; the maximum phase value of the fourth phase sequence is less than the minimum phase value of the currently traversed second phase sequence;

[0064] If so, retain the currently traversed second phase sequence; otherwise, filter out all phase points in the currently traversed second phase sequence. Description of the Drawings

[0065] Figure 1 It is a schematic flowchart of a method for filtering out cloud point noise in 3D reconstruction point clouds provided in some embodiments of the present application;

[0066] Figure 2 It is a phase distribution comparison diagram of continuous plane phase data and discontinuous plane phase data;

[0067] Figure 3 It is a schematic diagram of discontinuous points identified by the method for filtering out cloud point noise in 3D reconstruction point clouds provided in some embodiments of the present application;

[0068] Figure 4 It is a schematic diagram of phase data in a scene with an occluder;

[0069] Figure 5 It is a schematic structural diagram of a device for filtering out cloud point noise in 3D reconstruction point clouds provided in some embodiments of the present application. Detailed Description of the Invention

[0070] When performing 3D reconstruction, there are a large number of noise data in its phase data, especially when facing the 3D reconstruction tasks of objects with occluders and discontinuous planes. When facing an occluder, the noise filtering step will filter out some phase data that is not noise, and it takes a huge amount of computing power to identify such phase data that has been wrongly filtered out, otherwise it will lead to the wrong recovery of real noise data. When facing the 3D reconstruction tasks of objects with occluders or discontinuous plane objects, how to quickly and accurately distinguish real non-noise data and noise data has become a technical problem to be solved currently.

[0071] To solve the above technical problems, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0072] Embodiment 1

[0073] Please refer to Figure 1 , a method for filtering out the flying point noise of three-dimensional reconstruction point clouds provided by the embodiments of the present application, including S10 to S30, specifically:

[0074] S10: Obtain first-phase data and identify the discontinuous points in the first-phase data to obtain a number of phase sequences; the phase sequences are composed of at least one and continuous phase points.

[0075] In some embodiments of the present application, the obtaining of the first-phase data and the identification of the discontinuous points in the first-phase data to obtain a number of phase sequences include:

[0076] Obtain the first-phase point with the maximum phase value from the first-phase data, and the pixel value of the first-phase point;

[0077] Take the ratio of the phase value and the pixel value of the first-phase point as the base number, and respectively obtain the step determination threshold and the similarity determination threshold;

[0078] Identify the discontinuous points in the first-phase data according to the step determination threshold and the similarity determination threshold;

[0079] Divide the first-phase data according to all the discontinuous points to obtain a number of phase sequences.

[0080] Refer to Figure 2 , since the phase data of the continuous plane (corresponding to the straight line in Figure 2 ), the change trend of its phase value and pixel value is a straight line with a determined slope. For the phase data of the non-continuous plane (corresponding to the broken line in Figure 2 ), compared with the phase data of the continuous plane, it is equivalent to dividing a straight line into multiple segments with different slopes or non-connected line segments, but its change trend will not change. Therefore, the slope of the line formed by the data in each phase sequence of the non-continuous plane will definitely fluctuate within the range of the slope of the straight line obtained under the assumption of the continuous plane. Based on this characteristic, according to the phase data and pixel points of the first-phase point, obtain the slope under the assumption of the continuous plane, and use this slope as the base number to simply determine which phase points belong to the same phase sequence.

[0081] Preferably, in some embodiments of the present application, the step determination threshold and the similarity determination threshold can be determined by the following preferred embodiments: taking the ratio of the phase maximum value in the phase data to the number of horizontal pixels as the base number, taking 2 to 4 times the base number as the step determination threshold, and taking 0.1 to 0.25 times the base number as the similarity determination threshold. When the difference between the phase values of two consecutive phase points is greater than the step threshold or less than the same threshold, the currently traversed phase point is determined as a discontinuous point.

[0082] In some embodiments of the present application, the identifying the discontinuous points in the first phase data according to the step determination threshold and the similarity determination threshold includes:

[0083] Sorting the phase points in the first phase data in ascending order according to the pixel values of the phase points in the first phase data;

[0084] Successively judging whether the phase points in the first phase data traversed previously are discontinuous points according to the difference between the phase value of the currently traversed phase point in the first phase data and the phase value of the previous phase point of the currently traversed phase point;

[0085] If the size relationship between the phase value of the currently traversed phase point and the previous phase point is decreasing, it is determined that the currently traversed phase point is a discontinuous point; if the phase difference between the currently traversed phase point and the previous phase point is greater than the step determination threshold or less than the similarity determination threshold, it is determined that the currently traversed phase point is a discontinuous point.

[0086] To better illustrate the above discontinuous point judgment process, the following is combined with Figure 3 , for detailed description: Among them, for point 1, compared with the right point of area A, the phase value difference between the two is significantly greater than the step determination threshold, and this point is marked as a discontinuous point; point 2 is marked as a discontinuous point because it is smaller than the phase value of its previous point; for area 3, since the difference between each point and the previous point is extremely small, that is, the difference between the phase values of two points is less than the similarity determination threshold, it is determined to be the same, so each point is marked as a discontinuous point; for area 4, because of continuous decrease, each point is marked as a discontinuous point.

[0087] As can be seen from the above process, since there must be a monotonic increase between the phase values of each phase point and the pixel values in the phase data, and because the slope of the straight line of the phase data obtained under a continuous plane is fixed. Therefore, after sorting the phase points according to the pixel values, the relationship between the currently traversed phase point and the previous traversed phase point is judged in turn. If it is decreasing, then the two phase points violate the aforementioned characteristics, and the currently traversed phase point must be a phase point of potential noise, and then this phase point can be determined as a discontinuity point. In addition, if the two phase points meet the increasing constraint requirements and their difference is too far from the slope of the straight line obtained under the continuous plane assumption, it can be determined that the currently traversed phase point corresponds to a break point of a non-continuous plane, so as to determine that the currently traversed phase point is a discontinuity point, thereby improving the correctness of subsequent phase sequence segmentation.

[0088] S20: According to the longest phase sequence, from all the phase sequences, screen and obtain a first phase sequence without noise and a second phase sequence containing noise.

[0089] In some embodiments of the present application, the step of screening and obtaining a first phase sequence without noise and a second phase sequence containing noise from all the phase sequences according to the longest phase sequence includes:

[0090] Sort all the phase sequences in descending order according to the length of each phase sequence;

[0091] Determine the longest phase sequence as the first phase sequence, and starting from the longest phase sequence, traverse all the phase sequences; when the currently traversed phase sequence satisfies the monotonicity constraint and the similarity determination threshold with respect to the previously determined first phase sequence, determine the currently traversed phase sequence as the first phase sequence;

[0092] Take all the phase sequences that are not the first phase sequence among all the phase sequences as the second phase sequence.

[0093] Since the longest phase sequence that satisfies the monotonicity constraint can be first determined as the phase sequence without noise data, based on this premise, and the characteristic that the phase data corresponding to other plane regions without occlusions also satisfies the monotonicity constraint with respect to the first phase sequence, the phase sequences corresponding to the plane regions without occlusions can be quickly screened out, so as to obtain all the first phase sequences that definitely do not contain noise, and further determine the second phase sequences that may contain noise based on this, realizing preliminary noise filtering and improving the correctness of subsequent noise filtering.

[0094] Preferably, refer to Figure 3 , where each phase sequence can be composed of a single or multiple continuous phase points.

[0095] In some embodiments of the present application, the step of determining that the currently traversed phase sequence is the first phase sequence includes:

[0096] Obtain a second phase point with the maximum phase value in the currently traversed phase sequence and a third phase point with the minimum phase value in the previously determined first phase sequence;

[0097] When the phase value of the second phase point is less than the phase value of the third phase point, and the difference between the phase values of the second phase point and the third phase point is greater than the similarity determination threshold, determine that the currently traversed phase sequence is the first phase sequence.

[0098] According to the characteristics of the change trend of the phase value and the pixel value of each phase point in the above phase data, by the phase value and the pixel value of each phase point in the currently traversed phase sequence, judge whether its change trend meets the above characteristics, accurately obtain the first phase data that must not contain noise, and improve the accuracy and efficiency of noise filtering.

[0099] To better illustrate the screening process of the above first phase sequence and second phase sequence, the following will be further described in conjunction with Figure 3 , as follows:

[0100] Step 1: First, traverse each phase sequence to determine the length of each phase sequence; among them, the length of each phase sequence can be determined according to the number of phase points in the phase sequence. For example, Figure 3 in A4, it represents that the length of phase sequence A is 4, which includes 4 consecutive phase points;

[0101] Step 2: For each phase sequence, combine its length and perform a descending order sorting, and set the longest phase sequence as the first phase sequence (equivalent to the longest phase sequence being the reference sequence for subsequent determination of the correct sequence, and it has correctness);

[0102] Step 3: According to the sorting, starting from the longest phase sequence, determine in turn whether the currently traversed phase sequence and the previously traversed first phase sequence (that is, the phase sequence that has been determined as the correct sequence) meet the monotonicity constraint and whether the difference in the phase values of two similar phase points in the two phase sequences is too small, that is, less than the similarity determination threshold;

[0103] Step 4: If the judgment conditions in Step 3 are all met, then determine that the currently traversed phase sequence is also the first phase sequence.

[0104] The judgment process of Steps 1-4 is actually: In Figure 3Among them, the F sequence is the longest. Therefore, the phase values in the F sequence are correct data. Based on this, for the second-longest sequence E sequence relative to the F sequence, since the E sequence is on the left side of the F sequence, the maximum phase value of the E sequence is less than the minimum phase value in the F sequence, and the difference between the two values is greater than the similarity determination threshold and meets the monotonicity constraint. So the E sequence is the first phase sequence; the second-longest sequence relative to the E sequence is the C sequence. The C sequence is on the left side of the E sequence. The maximum phase value of the C sequence is less than the minimum phase value of the E sequence, and the difference is greater than the similarity determination threshold and meets the monotonicity constraint. Therefore, C is the first phase sequence; similarly, A is also the first phase sequence; for the B sequence, on the right side of the A sequence and on the left side of the C sequence, the minimum phase value of the B sequence is greater than the maximum phase value of the A sequence, and the maximum phase value of the B sequence is less than the minimum phase value of the C sequence, and all phase differences are greater than the similarity determination threshold. Therefore, B is the first phase sequence; similarly, D is the first phase sequence. In addition, each phase point in regions 1 to 4 is also an independent phase sequence. It can be seen that these sequences do not meet the monotonicity constraint or the phase difference from the previous and subsequent sequences is less than the similarity determination threshold. Therefore, the phase sequences represented by each phase point in regions 1 to 4 are all classified as the second phase sequences.

[0105] S30: Combine all the first phase sequences and filter out the noise points in all the current second phase sequences.

[0106] In some embodiments of the present application, the combining all the first phase sequences and filtering out the noise points in all the currently determined second phase sequences containing noise includes:

[0107] Traverse each phase point in each of the second phase sequences and determine whether the currently traversed phase point has the same phase value as at least one phase point in at least one of the first phase sequences;

[0108] If they are the same, filter out all the phase points in the currently traversed second phase sequence; otherwise, according to the maximum phase value and the minimum phase value in the second phase sequence, determine whether to filter out all the phase points in the currently traversed second phase sequence.

[0109] Specifically, reference can be made to Figure 3 , where for the phase point in region 1, its phase data coincides with the phase data in sequence C. Considering that it does not meet the monotonicity constraint, this phase point must be noise data; for one of the phase points in region 2, its phase data coincides with the phase data in sequence C and neither meets the monotonicity constraint. Therefore, this phase point is noise data; similarly, the phase points corresponding to regions 3 and 4 also have the same problem as the phase point in region 1. Therefore, they are all noise data and need to be filtered out.

[0110] As can be seen from the above, by combining the characteristics of the change trends of the phase values and pixel values of each phase point in the above phase data, the conclusion can be drawn that: if the phase data of a current phase point coincides with the phase data of the phase point in a phase sequence that does not contain noise and is definitely correct, it indicates that this phase point does not satisfy the trend characteristic of the monotonic change of the phase value with the pixel value. Further, such phase points can be quickly and accurately identified as noise data.

[0111] In some embodiments of the present application, the determining whether to filter all the phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence includes:

[0112] Determining whether there is a third phase sequence and a fourth phase sequence in all the first phase sequences, and there is no phase data between the phase point with the minimum phase value of the third phase sequence and the phase point with the maximum phase value of the fourth phase sequence; wherein, the minimum phase value of the third phase sequence is greater than the maximum phase value of the currently traversed second phase sequence; the maximum phase value of the fourth phase sequence is less than the minimum phase value of the currently traversed second phase sequence;

[0113] If so, retain the currently traversed second phase sequence; otherwise, filter all the phase points in the currently traversed second phase sequence.

[0114] To better illustrate the process of identifying the second phase sequence that is wrongly determined as noise data, the following will be further described in combination with Figure 4 for further illustration:

[0115] When there is an occluder scene, due to the different angles of projection and the camera, there will be data within the red circle and blank data within the green circle. The data within the red circle corresponds to the phase data of the occluder. The reason why the data within the green circle is blank is that when the light shines at an angle different from the camera, the part where the projection should appear within the green circle is blocked by the occluder. And the camera is at another angle and can only obtain the light on the occluder, that is, the phase data corresponding to the red circle, while there is no light in the corresponding part within the green circle, so the phase data is missing.

[0116] According to the monotonicity constraint, the data within the red circle fails to meet the monotonicity constraint and is misidentified as noise data during the initial noise filtering process, that is, the second phase sequence. At this time, there is no phase data within the red circle and no phase data within the green circle in all the first phase sequences. In fact, the phase data within the red circle can be filled into the blank area of the green circle. There must be two first phase sequences before and after the blank area of the green circle: one first phase sequence is on the right side of the blank area, and the minimum phase value of this first phase sequence is greater than the maximum phase value of the phase sequence within the red circle; the other first phase sequence is on the left side of the blank area, and the maximum phase value of this first phase sequence is greater than the minimum phase value of the phase sequence within the red circle.

[0117] Therefore, based on the above characteristics, by traversing the first phase sequence and according to the starting phase point and the phase value at the end point of each first phase sequence, it is possible to quickly determine whether there is the above-mentioned phase data missing area, which is consistent with the second phase sequence, so as to quickly and accurately restore the phase data misidentified as noise data and improve the accuracy of noise filtering.

[0118] In summary, a method for filtering flying point noise in 3D reconstruction point clouds provided by some embodiments of the present application has the following beneficial effects compared with the prior art: for discontinuous planar objects or objects with occlusions, the phase pixel map of their phase data is often not a straight line, but a broken line with a large number of break points and noise points. By identifying the break points in the broken line, not only can the break points on the discontinuous plane be obtained, but also the noise points can be accurately identified because the noise points often exist separately from the broken line. Since the longest sequence that satisfies the monotonicity constraint must be the correct sequence, based on this premise and the principle that sequences must also satisfy the monotonicity constraint, the correct sequence can be used as a reference to further identify the incorrect sequences containing noise and the correct sequences without noise. At this time, some phase sequences in the phase data obtained when there is an occlusion are misidentified as incorrect sequences. At this time, in the correct sequence, there must be a blank phase data area with the same length as the misidentified sequence, and the phase data in the misidentified sequence does not exist in the correct sequence. Based on the above characteristics, it is possible to identify and further accurately filter the initially screened noise data, improving the accuracy of flying point noise filtering.

[0119] Embodiment 2

[0120] Please refer to Figure 5, a three-dimensional reconstruction point cloud flying point noise filtering device provided by an embodiment of the present application, includes: a phase data division module 11, a phase sequence screening module 12, and a denoising module 13; wherein, the phase data division module 11 includes: a first phase point acquisition unit 111, a threshold acquisition unit 112, a discontinuity point acquisition unit 113, and a phase sequence acquisition unit 114; the phase sequence screening module 12 includes: a first sorting unit 121, a first phase sequence acquisition unit 122, and a second phase sequence acquisition unit 123; the denoising module 13 includes: a first judgment unit 131 and a first execution unit 132.

[0121] In some embodiments of the present application, the phase data division module 11 is configured to obtain first phase data and identify discontinuity points in the first phase data to obtain a plurality of phase sequences; the phase sequences are composed of at least one and continuous phase points; the phase sequence screening module 12 is configured to screen and obtain a first phase sequence without noise and a second phase sequence containing noise from all the phase sequences according to the longest phase sequence; the denoising module 13 is configured to combine all the first phase sequences to filter out noise points in all the current second phase sequences.

[0122] In some embodiments of the present application, the first phase point acquisition unit 111 is configured to obtain a first phase point with the maximum phase value and the pixel value of the first phase point from the first phase data; the threshold acquisition unit 112 is configured to respectively obtain a step determination threshold and a similarity determination threshold based on the ratio of the phase value and the pixel value of the first phase point; the discontinuity point acquisition unit 113 is configured to identify discontinuity points in the first phase data according to the step determination threshold and the similarity determination threshold; the phase sequence acquisition unit 114 is configured to divide the first phase data according to all the discontinuity points to obtain a plurality of phase sequences.

[0123] In some embodiments of the present application, the discontinuity point acquisition unit 113 is configured to identify discontinuity points in the first phase data according to the step determination threshold and the similarity determination threshold, including: sorting the phase points in the first phase data in ascending order according to the pixel values of the phase points in the first phase data; sequentially judging whether the phase points in the currently traversed first phase data are discontinuity points according to the phase value difference between the currently traversed phase point in the first phase data and the previous phase point of the currently traversed phase point; if the magnitude relationship between the currently traversed phase point and the previous phase point is decreasing, determining that the currently traversed phase point is a discontinuity point; if the phase difference between the currently traversed phase point and the previous phase point is greater than the step determination threshold or less than the similarity determination threshold, determining that the currently traversed phase point is a discontinuity point.

[0124] In some embodiments of the present application, the first sorting unit 121 is configured to sort the phase sequences in descending order according to the lengths of the phase sequences; the first phase sequence obtaining unit 122 is configured to determine the longest phase sequence as the first phase sequence, and starting from the longest phase sequence, traverse each of the phase sequences; when the currently traversed phase sequence satisfies the monotonicity constraint and the similarity determination threshold with respect to the previously determined first phase sequence, determine the currently traversed phase sequence as the first phase sequence; the second phase sequence obtaining unit 123 is configured to use the phase sequences that are not the first phase sequence among all the phase sequences as the second phase sequences.

[0125] In some embodiments of the present application, the step of determining the currently traversed phase sequence as the first phase sequence includes: obtaining a second phase point with the maximum phase value in the currently traversed phase sequence and a third phase point with the minimum phase value in the previously determined first phase sequence; when the phase value of the second phase point is less than the phase value of the third phase point, and the difference between the phase values of the second phase point and the third phase point is greater than the similarity determination threshold, determine the currently traversed phase sequence as the first phase sequence.

[0126] In some embodiments of the present application, the first judgment unit 131 is configured to traverse each phase point in each of the second phase sequences and determine whether the phase value of the currently traversed phase point is the same as the phase value of at least one phase point in at least one of the first phase sequences; the first execution unit 132 is configured to, if they are the same, filter out all the phase points in the currently traversed second phase sequence; otherwise, determine whether to filter out all the phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence.

[0127] In some embodiments of the present application, the first judgment unit 131 is configured to determine whether to filter out all the phase points in the currently traversed second phase sequence according to the maximum phase value and the minimum phase value in the second phase sequence, including: determining whether there are a third phase sequence and a fourth phase sequence among all the first phase sequences, and there is no phase data between the phase point with the minimum phase value of the third phase sequence and the phase point with the maximum phase value of the fourth phase sequence; wherein, the minimum phase value of the third phase sequence is greater than the maximum phase value of the currently traversed second phase sequence; the maximum phase value of the fourth phase sequence is less than the minimum phase value of the currently traversed second phase sequence; if so, retain the currently traversed second phase sequence; otherwise, filter out all the phase points in the currently traversed second phase sequence.

[0128] In summary, a device for filtering flying point noise in 3D reconstruction point clouds provided by some embodiments of the present application has the following beneficial effects compared with the prior art: For discontinuous planar objects or objects with occlusions, the phase pixel map of their phase data is often not a straight line, but a broken line with a large number of break points and noise points. By identifying the discontinuous points in the broken line, not only can the break points on the discontinuous plane be obtained, but also the noise points can be accurately identified because the noise points often exist separately from the broken line. Since the longest sequence that satisfies the monotonicity constraint must be the correct sequence, based on this premise and the principle that sequences must also satisfy the monotonicity constraint, the correct sequence can be used as a reference to further identify the incorrect sequences containing noise and the correct sequences without noise. At this time, when there is an occlusion, some phase sequences in the obtained phase data are misidentified as incorrect sequences. At this time, in the correct sequence, there must be a blank phase data area with the same length as the misidentified sequence, and the phase data in the misidentified sequence does not exist in the correct sequence. Based on the above characteristics, it is possible to further accurately filter the preliminarily screened noise data, improving the accuracy of flying point noise filtering.

[0129] Embodiment III

[0130] Based on the above embodiment of the method for filtering flying point noise in 3D reconstruction point clouds, another embodiment of the present application provides a terminal device for filtering flying point noise in 3D reconstruction point clouds. The terminal device for filtering flying point noise in 3D reconstruction point clouds includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the method for filtering flying point noise in 3D reconstruction point clouds according to any embodiment of the present application is implemented.

[0131] Exemplarily, in this embodiment, the computer program can be divided into one or more modules. The one or more modules are stored in the memory and executed by the processor to complete the present application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the device for filtering flying point noise in 3D reconstruction point clouds.

[0132] The device for filtering flying point noise in 3D reconstruction point clouds can be a computing device such as a desktop computer, a notebook, a palm computer, or a cloud server. The terminal device for filtering flying point noise in 3D reconstruction point clouds may include, but is not limited to, a processor and a memory.

[0133] The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the three-dimensional reconstruction point cloud flying-point noise filtering device, and connects various parts of the entire three-dimensional reconstruction point cloud flying-point noise filtering device through various interfaces and circuits. The memory can be used to store the computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory, the processor realizes various functions of the three-dimensional reconstruction point cloud flying-point noise filtering device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash device, or other volatile solid-state storage devices.

[0134] Embodiment 4

[0135] Based on the embodiments of the above three-dimensional reconstruction point cloud flying-point noise filtering method, another embodiment of the present application provides a storage medium. The storage medium includes a stored computer program. When the computer program runs, it controls the device where the storage medium is located to execute the three-dimensional reconstruction point cloud flying-point noise filtering method of any embodiment of the present application.

[0136] In this embodiment, the above storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0137] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of the present application. It should be understood that the above description is only for the specific embodiments of the present application and is not used to limit the protection scope of the present application. In particular, it is pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for filtering flying point noise in three-dimensional reconstruction point cloud, characterized in that: include: Acquire first phase data, and identify discontinuity points in the first phase data to acquire a plurality of phase sequences; the phase sequence is composed of at least one and continuous phase point; According to the longest phase sequence, a first phase sequence without noise and a second phase sequence containing noise are screened out from all the phase sequences; Combine all the first phase sequences and filter out noise points in all the current second phase sequences; The combining all the first phase sequences and filtering out noise points in all the second phase sequences currently determined to contain noise comprises: Traversing each phase point in each of the second phase sequences, and determining whether a currently traversed phase point has the same phase value as at least one phase point in at least one of the first phase sequences; If they are the same, all phase points in the second phase sequence currently traversed are filtered out; otherwise, according to the maximum phase value and the minimum phase value in the second phase sequence, it is determined whether to filter out all phase points in the second phase sequence currently traversed; The determining whether to filter out all phase points in the second phase sequence currently traversed according to the maximum phase value and the minimum phase value in the second phase sequence includes: Determine whether there is a third phase sequence and a fourth phase sequence in all the first phase sequences, and there is no phase data between the phase point of the minimum phase value of the third phase sequence and the phase point of the maximum phase value of the fourth phase sequence; wherein the minimum phase value of the third phase sequence is greater than the maximum phase value of the second phase sequence currently traversed; and the maximum phase value of the fourth phase sequence is less than the minimum phase value of the second phase sequence currently traversed; If it exists, the second phase sequence currently traversed is retained; otherwise, all phase points in the second phase sequence currently traversed are filtered out.

2. A method for filtering flying point noise in a three-dimensional reconstruction point cloud according to claim 1, characterized in that: The step of acquiring first phase data and identifying discontinuity points in the first phase data to acquire a plurality of phase sequences includes: Acquire a first phase point with a maximum phase value and a pixel value of the first phase point from the first phase data; Taking the ratio of the phase value and the pixel value of the first phase point as a base, respectively obtaining a step determination threshold and a similarity determination threshold; identifying a discontinuity point in the first phase data according to the step determination threshold and the similarity determination threshold; The first phase data is divided according to all the discontinuity points to obtain a plurality of phase sequences.

3. A method for filtering flying point noise in a three-dimensional reconstruction point cloud as claimed in claim 2, characterized in that: The step of identifying the discontinuity point in the first phase data according to the step determination threshold and the similarity determination threshold comprises: sorting the phase points in the first phase data in ascending order according to the pixel value of each phase point in the first phase data; Determining whether a phase point in the first phase data previously traversed is a discontinuity point according to a phase value difference between a phase point in the currently traversed first phase data and a phase point before the currently traversed phase point; If the phase value relationship between the current traversal phase point and the previous phase point is decreasing, the current traversal phase point is determined to be a discontinuity point; if the phase difference between the current traversal phase point and the previous phase point is greater than the step judgment threshold or less than the similarity judgment threshold, the current traversal phase point is determined to be a discontinuity point.

4. A method for filtering flying point noise in a three-dimensional reconstruction point cloud according to claim 1 or 2, characterized in that: The method of selecting and acquiring a first phase sequence without noise and a second phase sequence containing noise from all the phase sequences according to the longest phase sequence includes: According to the length of each phase sequence, the phase sequences are sorted in descending order; Determine the longest phase sequence as the first phase sequence, and traverse each phase sequence starting from the longest phase sequence; when the currently traversed phase sequence satisfies the monotonicity constraint and the similarity determination threshold relative to the last determined first phase sequence, determine the currently traversed phase sequence as the first phase sequence; The phase sequence that is not the first phase sequence among all the phase sequences is used as the second phase sequence.

5. A method for filtering flying point noise in three-dimensional reconstruction point cloud according to claim 4, characterized in that: The step of determining that the phase sequence currently traversed is a first phase sequence comprises: Acquire the second phase point with the maximum phase value in the currently traversed phase sequence and the third phase point with the minimum phase value in the last determined first phase sequence; When the phase value of the second phase point is smaller than the phase value of the third phase point, and the difference between the phase values ​​of the second phase point and the third phase point is greater than the similarity determination threshold, the currently traversed phase sequence is determined to be the first phase sequence.

6. A device for filtering flying point noise of three-dimensional reconstruction point cloud, characterized in that: include: Phase data division module, phase sequence screening module and denoising module; The phase data division module is used to obtain first phase data and identify discontinuity points in the first phase data to obtain a plurality of phase sequences; the phase sequence is composed of at least one and continuous phase point; The phase sequence screening module is used to screen and obtain a first phase sequence without noise and a second phase sequence containing noise from all the phase sequences according to the longest phase sequence; The denoising module is used to combine all the first phase sequences and filter out noise points in all the current second phase sequences; The denoising module comprises: a first judging unit and a first executing unit; The first judgment unit is used to traverse each phase point in each second phase sequence, and judge whether the phase value of the currently traversed phase point is the same as the phase value of at least one phase point in at least one first phase sequence; The first execution unit is configured to filter out all phase points in the second phase sequence currently traversed if they are the same; otherwise, determine whether to filter out all phase points in the second phase sequence currently traversed according to the maximum phase value and the minimum phase value in the second phase sequence; The first judgment unit is used to judge whether to filter out all phase points in the second phase sequence currently traversed according to the maximum phase value and the minimum phase value in the second phase sequence, including: Determine whether there is a third phase sequence and a fourth phase sequence in all the first phase sequences, and there is no phase data between the phase point of the minimum phase value of the third phase sequence and the phase point of the maximum phase value of the fourth phase sequence; wherein the minimum phase value of the third phase sequence is greater than the maximum phase value of the second phase sequence currently traversed; and the maximum phase value of the fourth phase sequence is less than the minimum phase value of the second phase sequence currently traversed; If it exists, the second phase sequence currently traversed is retained; otherwise, all phase points in the second phase sequence currently traversed are filtered out.

7. The device for filtering flying point noise of three-dimensional reconstruction point cloud according to claim 6, characterized in that: The phase data division module includes: a first phase point acquisition unit, a threshold acquisition unit, a discontinuity point acquisition unit and a phase sequence acquisition unit; The first phase point acquisition unit is used to acquire a first phase point with a maximum phase value and a pixel value of the first phase point from the first phase data; The threshold acquisition unit is used to respectively acquire a step determination threshold and a similarity determination threshold based on a ratio of a phase value and a pixel value of the first phase point; The discontinuity point acquisition unit is used to identify the discontinuity points in the first phase data according to the step determination threshold and the similarity determination threshold; The phase sequence acquisition unit is used to divide the first phase data according to all the discontinuity points to acquire a plurality of phase sequences.

8. The device for filtering flying point noise of three-dimensional reconstruction point cloud according to claim 7, characterized in that: The discontinuity point acquisition unit is used to identify the discontinuity point in the first phase data according to the step determination threshold and the similarity determination threshold, including: sorting the phase points in the first phase data in ascending order according to the pixel value of each phase point in the first phase data; Determining whether a phase point in the first phase data previously traversed is a discontinuity point according to a phase value difference between a phase point in the currently traversed first phase data and a phase point before the currently traversed phase point; If the phase value relationship between the current traversal phase point and the previous phase point is decreasing, the current traversal phase point is determined to be a discontinuity point; if the phase difference between the current traversal phase point and the previous phase point is greater than the step judgment threshold or less than the similarity judgment threshold, the current traversal phase point is determined to be a discontinuity point.

9. A device for filtering flying point noise of three-dimensional reconstruction point cloud according to claim 6 or 7, characterized in that: The phase sequence screening module includes: a first sorting unit, a first phase sequence acquiring unit and a second phase sequence acquiring unit; Wherein, the first sorting unit is used to sort each of the phase sequences in descending order according to the length of each of the phase sequences; The first phase sequence acquisition unit is used to determine that the longest phase sequence is the first phase sequence, and traverse each phase sequence starting from the longest phase sequence; when the currently traversed phase sequence satisfies the monotonicity constraint and the similarity determination threshold relative to the last determined first phase sequence, the currently traversed phase sequence is determined to be the first phase sequence; The second phase sequence acquiring unit is used to use the phase sequences that are not the first phase sequences in all the phase sequences as the second phase sequences.

10. The device for filtering flying point noise of three-dimensional reconstruction point cloud according to claim 9, characterized in that: The step of determining that the phase sequence currently traversed is a first phase sequence comprises: Acquire the second phase point with the maximum phase value in the currently traversed phase sequence and the third phase point with the minimum phase value in the last determined first phase sequence; When the phase value of the second phase point is smaller than the phase value of the third phase point, and the difference between the phase values ​​of the second phase point and the third phase point is greater than the similarity determination threshold, the currently traversed phase sequence is determined to be the first phase sequence.

Images