Three-dimensional shape measurement noise reduction method and device, computer equipment and storage medium

Abstract

The invention provides a three-dimensional shape measurement noise reduction method and device, computer equipment and a storage medium. According to the scheme, firstly, each measurement pair is controlled to obtain a calibration image, and then multiple groups of first sets are obtained according to the calibration image; and respectively calculating an epipolar constraint error and a monotonicity constraint error of each first coordinate pair, synthesizing the two errors to determine a comprehensive error, and removing the first coordinate pair of which the comprehensive error is lower than a first threshold value. And determining a transformation relation between the coordinate system of the projection equipment and the unified coordinate system, updating the coordinate pairs into a second set, carrying out two-time fusion on the second set to obtain a third set, and finally generating a three-dimensional point cloud according to the coordinate pairs in the third set. According to the scheme, the coordinate pairs are screened through the combination of the epipolar constraint and the monotonicity constraint, noise data generated by factors such as optical pollution are effectively removed, compared with a traditional epipolar constraint method, errors caused in the epipolar line direction can be further considered, and the quality of measured data is remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional shape measurement, and in particular to a method, apparatus, computer equipment and storage medium for three-dimensional shape measurement noise reduction. Background Art

[0002] The demand for high-precision 3D profile measurement technology has grown significantly in industrial and research applications, including defect detection, automated manufacturing, and robotic vision. Among the many 3D profile measurement methods, Fringe Projection Profilometry (FPP) has attracted widespread attention due to its excellent accuracy, while Multi-View Figure 3 3D profile measurement further improves its stability. Figure 3 3D profilometry includes multiple measurement pairs consisting of a projection device and a camera device. The projection device projects a structured pattern (usually sinusoidal stripes) onto the object, and the camera device captures the deformed stripes to establish a correspondence between the projection device and the camera device. Ideally, the object is directly illuminated only by the projection device and accurately captured by the camera. However, indirect lighting (such as multiple reflections, subsurface scattering, and high reflectivity) can introduce optical contamination, resulting in errors in the captured stripes, which in turn leads to distortion of the reconstructed 3D shape in traditional technologies. Summary of the Invention

[0003] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the defect that optical contamination is difficult to remove during the three-dimensional topography measurement process in the prior art.

[0004] In a first aspect, the present application provides a 3D shape measurement noise reduction method, which is applied to a 3D shape measurement system. The 3D shape measurement system includes a projection device and multiple camera devices, each camera device and the projection device forming a measurement pair. The 3D shape measurement noise reduction method includes:

[0005] Controlling each measurement pair separately to obtain a calibration image;

[0006] Obtaining a first set of measurement pairs according to the calibration image; the first set includes a plurality of first coordinate pairs, each first coordinate pair including a first coordinate of a corresponding feature point in a projection device coordinate system and a second coordinate in a camera device coordinate system;

[0007] The epipolar constraint error and the monotonicity constraint error of each group of first coordinate pairs are calculated respectively, and the corresponding comprehensive error is determined based on the epipolar constraint error and the monotonicity constraint error, and the first coordinate pairs whose comprehensive error is lower than a first threshold are removed; the epipolar constraint error reflects the distance between the first coordinate and the corresponding epipolar line in the projection device coordinate system, and the monotonicity constraint error reflects the distance between the second coordinate and the first coordinate in the target direction corresponding to the monotonic fitting signal after the second coordinate is mapped to the projection device coordinate system according to the monotonic fitting signal;

[0008] For any first set, determining a transformation relationship between the corresponding projection device coordinate system and the unified coordinate system, and transforming each first coordinate into a third coordinate according to the transformation relationship to update the first coordinate pair into a second coordinate pair, thereby obtaining a corresponding second set;

[0009] For any second set, if there exists a second coordinate pair with the same third coordinate, the second coordinate pair with the smallest comprehensive error is retained;

[0010] Merge the second sets and retain the second coordinate pairs with the same third coordinates and the ones with the smallest comprehensive error to obtain a third set;

[0011] A three-dimensional point cloud is obtained according to each second coordinate pair in the third set.

[0012] In one embodiment, the process of determining the epipolar constraint error includes:

[0013] For any first coordinate pair, determining the epipolar line of the corresponding feature point in the projection device coordinate system;

[0014] The epipolar constraint error is determined based on the first coordinate of the feature point and the epipolar line.

[0015] In one embodiment, a projection device coordinate system includes a first axis and a second axis that are perpendicular to each other, a camera device coordinate system includes a third axis and a fourth axis that are perpendicular to each other, the camera device includes a first camera, the measurement pair includes a first measurement pair, the first camera is disposed on the first axis, the third axis is inclined at an angle to the first axis, and the fourth axis is parallel to the second axis, the first camera and the projection device constitute a first measurement pair, and for the first measurement pair, a process of determining a monotonicity constraint error includes:

[0016] Performing epipolar correction on the calibration image captured by the projection device in the first measurement pair in the direction of the first axis of phase unwrapping to obtain a first target image;

[0017] In the first target image, determining the mapping coordinates corresponding to each second coordinate, and obtaining a monotonic fitting signal by fitting the mapping coordinates and the second coordinates;

[0018] For the first coordinate pair, the coordinate value of the second coordinate in the direction of the third axis is substituted into the monotonic fitting signal to obtain the first ideal coordinate value, and the monotonicity constraint error is obtained according to the difference between the coordinate value of the first coordinate in the direction of the first axis and the first ideal coordinate value.

[0019] In one embodiment, the projection device coordinate system includes a first axis and a second axis that are perpendicular to each other, the camera device coordinate system includes a third axis and a fourth axis that are perpendicular to each other, the camera device includes a second camera, the measurement pair includes a second measurement pair, the second camera is arranged on the second axis, the third axis is parallel to the first axis, and the fourth axis is inclined at an angle to the second axis, the second camera and the projection device constitute a second measurement pair, and for the second measurement pair, a process of determining a monotonicity constraint error includes:

[0020] Performing epipolar correction on the calibration image captured by the projection device in the second measurement pair in the direction of the second axis of phase unwrapping to obtain a second target image;

[0021] In the second target image, a mapping coordinate corresponding to each second coordinate is determined, and a monotonic fitting signal is obtained by fitting each mapping coordinate and the second coordinate;

[0022] For the first coordinate pair, the coordinate value of the second coordinate in the direction of the fourth axis is substituted into the second monotonic fitting signal to obtain the second ideal coordinate value, and the monotonicity constraint error is obtained according to the difference between the coordinate value of the first coordinate in the direction of the second axis and the second ideal coordinate value.

[0023] In one embodiment, obtaining a monotonic fitting signal by fitting the mapping coordinates and the second coordinates includes:

[0024] Perform polynomial fitting using the coordinate value corresponding to the target direction in the second coordinate as the independent variable and the coordinate value of the target direction in the mapped coordinate as the dependent variable to obtain an initial fitting signal;

[0025] Determine the monotonic direction of the initial fitting signal;

[0026] Derivative the initial fitting signal to obtain the derivative signal;

[0027] Substitute the coordinate value corresponding to the target direction in the second coordinate into the current derivative signal to determine whether there is a new second coordinate whose derivative value does not match the monotonic direction;

[0028] If so, the corresponding second coordinate is added to the set of violation points, and the monotonic constraint is updated based on each point in the set of violation points. A polynomial fit is performed based on the monotonic constraint, each second coordinate, and the mapped coordinate to update the initial fitting signal, and the step of taking the derivative of the initial fitting signal to obtain the derivative signal is continued; the monotonic constraint is used to constrain the derivative value corresponding to the point in the set of violation points to be zero;

[0029] If not, the current initial fitting signal is used as the monotonic fitting signal.

[0030] In one embodiment, obtaining a monotonic fitting signal by fitting the mapping coordinates and the second coordinates includes:

[0031] The coordinate value corresponding to the target direction in the second coordinate is used as an independent variable, and the coordinate value of the target direction in the mapped coordinate is used as a dependent variable;

[0032] Sort the mapping coordinates according to the size of the independent variable coordinate value, and determine the monotonic direction according to the change of the dependent variable;

[0033] Determine the difference of the dependent variable between two adjacent mapping coordinates respectively to obtain a first gradient set; the first gradient set includes the gradient corresponding to each mapping coordinate;

[0034] sequentially determining the gradients in the first gradient set that do not conform to the monotonic direction as the gradients to be compensated;

[0035] The gradient to be compensated is set to zero, and the total amount to be compensated is determined based on the gradient to be compensated. The gradient to be compensated is traversed to the left and right sides respectively. If the traversed gradient does not conform to the monotonic direction, the traversed gradient is set to zero and accumulated in the total amount to be compensated based on the traversed gradient. If the traversed gradient conforms to the monotonic direction, the total amount that can be contributed is accumulated based on the traversed gradient until the total amount that can be contributed is greater than or equal to the total amount to be compensated. The allocation ratio is determined based on the proportion of the gradients that conform to the monotonic direction in the traversed gradients in the total amount that can be contributed. The gradients that conform to the monotonic direction in the traversed gradients are reduced based on the allocation ratio and the total amount to be compensated to update the first gradient set.

[0036] The mapping coordinates are updated according to the updated first gradient set, and a monotonic fitting signal is obtained by fitting according to the updated mapping coordinates.

[0037] In one embodiment, determining the corresponding comprehensive error based on the epipolar constraint error and the monotonicity constraint error includes:

[0038] The comprehensive error is obtained according to the second norm of the epipolar constraint error and the monotonicity constraint error.

[0039] In a second aspect, the present application provides a 3D shape measurement noise reduction device, which is applied to a 3D shape measurement system. The 3D shape measurement system includes a projection device and multiple camera devices, each camera device and the projection device forming a measurement pair. The 3D shape measurement noise reduction device includes:

[0040] An image acquisition module, used to control each measurement pair to obtain a calibration image;

[0041] a coordinate determination module, configured to obtain a first set of measurement pairs corresponding to the calibration image; the first set comprising a plurality of first coordinate pairs, each first coordinate pair comprising a first coordinate of a corresponding feature point in a projection device coordinate system and a second coordinate in a camera device coordinate system;

[0042] a filtering module for respectively calculating the epipolar constraint error and the monotonicity constraint error of each group of first coordinate pairs, determining a corresponding comprehensive error based on the epipolar constraint error and the monotonicity constraint error, and removing first coordinate pairs whose comprehensive error is lower than a first threshold; the epipolar constraint error reflects the distance between the first coordinate and the corresponding epipolar line in the projection device coordinate system, and the monotonicity constraint error reflects the distance between the second coordinate and the first coordinate in the target direction corresponding to the monotonic fitting signal after the second coordinate is mapped to the projection device coordinate system according to the monotonic fitting signal;

[0043] a transformation module, configured to determine, for any first set, a transformation relationship between the corresponding projection device coordinate system and the unified coordinate system, and transform each first coordinate into a third coordinate according to the transformation relationship, so as to update the first coordinate pair into a second coordinate pair to obtain a corresponding second set;

[0044] A first fusion module is configured to retain, for any second set, the second coordinate pair with the smallest comprehensive error if there exists a second coordinate pair with the same third coordinate;

[0045] A second fusion module is used to merge the second sets and retain the second coordinate pairs with the same third coordinates and the ones with the smallest comprehensive error to obtain a third set;

[0046] The point cloud generation module is used to obtain a three-dimensional point cloud according to each second coordinate pair in the third set.

[0047] In a third aspect, a computer device includes one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the three-dimensional morphology measurement noise reduction method in any of the above embodiments are executed.

[0048] In a fourth aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the three-dimensional morphology measurement noise reduction method in any of the above embodiments.

[0049] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0050] This solution addresses the noise reduction problem in 3D topography measurement by proposing a comprehensive workflow. First, each measurement pair is controlled to acquire calibration images. Multiple first sets are generated based on these calibration images. Subsequently, the epipolar constraint error and monotonicity constraint error are calculated for each coordinate pair. The combined error is then determined, and coordinate pairs with combined errors below a first threshold are removed to improve data accuracy. The transformation between the projection device coordinate system and the unified coordinate system is then determined. The coordinate pairs are updated to form a second set, and the pairs with the same third coordinate in the second set with the lowest combined error are retained. The second sets are then merged and filtered again to form a third set. Finally, a 3D point cloud is generated based on the coordinate pairs in the third set. This solution significantly improves the quality of the measurement data. By combining epipolar and monotonic constraints to filter coordinate pairs, it effectively removes noise data caused by factors such as optical contamination. Compared to traditional epipolar constraint methods, it further considers errors introduced in the epipolar direction. The final coordinate transformation and multi-round filtering mechanism fully utilize the redundant information of multiple measurement pairs, enhancing data reliability. The resulting 3D point cloud is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A schematic diagram of the layout of a three-dimensional shape measurement system provided in one embodiment of the present application;

[0053] Figure 2 A schematic flow chart of a three-dimensional shape measurement method according to an embodiment of the present application;

[0054] Figure 3 A schematic diagram of optical pollution caused by multiple reflections in one embodiment of the present application;

[0055] Figure 4 This is a schematic diagram of the principle of monotonicity constraint in one embodiment of the present application;

[0056] Figure 5 This is a schematic diagram of the principle of monotonicity constraint in an embodiment of the present application;

[0057] Figure 6 This is a flow chart of determining a monotonicity constraint error in one embodiment of the present application;

[0058] Figure 7 This is a flow chart of determining a monotonicity constraint error in another embodiment of the present application;

[0059] Figure 8 This is a schematic diagram of a process for obtaining a monotone fitting signal in one embodiment of the present application;

[0060] Figure 9 This is a flow chart of obtaining a monotonic fitting signal in another embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0062] This application provides a 3D shape measurement noise reduction method, which is applied to 3D shape measurement system. Figure 1 ,The 3D morphology measurement system includes a projection device and multiple camera devices, each camera device and the ,projection device form a measurement pair, such as Figure 1 As shown, camera devices 1-4 and a projection device form four measurement pairs, each of which can also be referred to as a binocular projector-camera pair (BPC pair). Adjusters can be provided on the camera devices to facilitate lens orientation adjustment. Multiple measurement pairs can focus on the same volumetric surface, creating a confocal volume. The 3D topography measurement noise reduction method includes steps S202 to S214.

[0063] S202, controlling each measurement pair to obtain a calibration image.

[0064] It can be understood that during the measurement pairing, the projection device projects a structured pattern onto an object. The light reflected from the projection device is captured by the camera device and formed on the camera device's image sensor, such as a CMOS (Complementary Metal Oxide Semiconductor) chip. The image obtained on the camera device is the calibration image corresponding to the camera device. The projection device is also provided with an image sensor, specifically a DMD (Digital Micromirror Device) chip. The image obtained on the projection device is the calibration image corresponding to the projection device. The calibration images corresponding to the two devices in the measurement pair can be used to determine the corresponding relationship between the camera device and the projection device.

[0065] S204: Obtain a first set of measurement pairs according to the calibration image. The first set includes multiple groups of first coordinate pairs, each group of first coordinate pairs including a first coordinate of a corresponding feature point in the projection device coordinate system and a second coordinate in the camera device coordinate system.

[0066] It can be understood that the first set is a data structure that stores the coordinate correspondence between the measurement pairs, where the first coordinate pair is a key element in establishing the coordinate mapping between the projection device and the camera device. Feature points are spatial points on the surface of the object to be measured. Points with obvious features that are easy to identify and match can be selected. Their coordinates in different coordinate systems constitute the first coordinate pair. This principle is based on triangulation. Through the geometric relationship between the projection device and the camera device, the coordinates of the feature points in the calibration image in the two coordinate systems are matched. The deformation information of the fringe pattern in the calibration image is combined with the calibration parameters of the measurement system to calculate the coordinates of the feature points in the coordinate systems of the projection device and the camera device. This coordinate correspondence is the basis for subsequent error calculation and three-dimensional reconstruction. By establishing accurate coordinate pairs, the conversion from two-dimensional image information to three-dimensional spatial information can be achieved. After acquiring the calibration image, determining the first coordinate pairs corresponding to multiple points from it is a relatively mature technology in three-dimensional topography measurement and will not be discussed in detail here.

[0067] Specifically, let u and v represent the two axes of the projector coordinate system, respectively. The first coordinate can be expressed as (u, v). Let x and y represent the two axes of the camera coordinate system, respectively. The second coordinate can be expressed as (x, y). The corresponding ((x, y), (u, v)) is a projector-camera coordinate pair (PC-CP), which is referred to as the first coordinate pair in this article. The three-dimensional coordinates (X, Y, Z) of the object to be measured in the spatial coordinate system can be expressed as follows: (X, Y, Z) = S((x, y), (u, v)). Here, the S(·) function represents the model function of the measurement system, which includes all parameters related to system calibration and geometric relationships. It is generally obtained through a calibration process to obtain the intrinsic parameters of the camera and projector device, as well as the extrinsic parameters between the camera and projector devices. Therefore, the accuracy of each first coordinate pair will determine the effectiveness of the 3D shape measurement.

[0068] S206: Calculate the epipolar constraint error and monotonicity constraint error for each set of first coordinate pairs, determine a corresponding comprehensive error based on the epipolar constraint error and the monotonicity constraint error, and remove first coordinate pairs whose comprehensive error is lower than a first threshold. The epipolar constraint error reflects the distance between the first coordinate and the corresponding epipolar line in the projection device coordinate system, and the monotonicity constraint error reflects the distance between the second coordinate and the first coordinate in the target direction corresponding to the monotonic fitting signal after the second coordinate is mapped to the projection device coordinate system according to the monotonic fitting signal.

[0069] It can be understood that for the first set corresponding to any measurement pair, ideally only the light reflected by the object to be measured is received by the camera device, but in the actual environment, other objects may participate in the propagation of light, thus generating optical pollution. Figure 3 As shown, taking a point Q on the target reflection plane as an example, the optical center O of the projection device P The emitted light v1 passes through the DMD chip q p When point Q is reached, the reflected light v is formed after being reflected by the target reflection plane. c Q through CMOS chip c After calibrating the optical centers of the two devices, the three-dimensional coordinates of point Q can be obtained based on q p dot and q c The first coordinate pair formed by the point is reconstructed. qc Represents q c The light intensity at the point, I qc Ideally, it is only affected by light v1, but other nearby reflection planes will affect q c The light intensity at the point causes pollution, such as point A on other reflecting planes in the figure, and the optical center O of the projection device. P The emitted light v2 is emitted to point A of the other reflection plane and forms the reflected light v r2 to the Q point of the target reflection plane, resulting in I qc affected and deviates from the ideal value.

[0070] According to epipolar geometry, any point on the camera ray must correspond to an illumination point on the epipolar line. Figure 3 As shown, the optical center O of the projection device P 、Camera equipment optical center O c The plane formed by the feature point Q is the epipolar plane, and the intersection line between the epipolar plane and the DMD chip of the projection device is the epipolar line l, q corresponding to the feature point Q. c The ideal q corresponding to the point p The point should be on the epipolar line l. However, optical contamination may cause q p The point is offset. This offset can be specifically divided into two cases, corresponding to Figure 3 q' in p and q'' p .q' p Corresponding to case 1, q'' p This corresponds to case 2. The difference is that the problem in case 1 is that the coordinates deviate from the epipolar line, which can be solved by using q' pThe distance from the epipolar line is used to determine the magnitude of this deviation error, which is referred to as the epipolar constraint error in this step. However, the error distributed along the epipolar line, as in Case 2, cannot be determined using the traditional epipolar constraint. Based on this, the present application further introduces a monotonicity constraint error.

[0071] The monotonicity constraint error is generated based on the monotonic phenomenon on the physical anchor surface. Figure 4 and Figure 5 As shown, for any epipolar surface, the epipolar line obtained by intersecting it with the DMD chip is called l P , the epipolar line obtained by intersecting with the CMOS chip is called l C , the cross section obtained by intersecting with the object to be measured is called l CsP These three lines are essentially related based on epipolar geometry. Any line passing through the epipolar line l P The light will only illuminate the object to be measured at l CsP The points on it will only pass through the epipolar line l C Received by the camera device. This application divides the object surface into physical anchoring surface and non-physical anchoring surface, and the method in this application is mainly applied to the physical anchoring surface. The physical anchoring surface here refers to the object surface without suspended objects, otherwise there are suspended objects. That is to say, all areas above the physical anchoring surface are invisible to any sub-area below it. Based on this, it is assumed that the epipolar line l P There are five consecutive points 0, a, b, e, l. Map these five points to the epipolar line l C Four intervals of monotonically changing coordinate values will be obtained, such as 0-a, ab, el, etc. in the figure. From this, it can be found that the coordinate values on the epipolar lines corresponding to the two coordinate systems should have a monotonically changing relationship. This monotonically changing relationship can be used to detect the error in the above situation 2.

[0072] Specifically, a monotonic fitting signal can be fitted using the variation of the second coordinate corresponding to the first coordinate on the same epipolar line. This allows the ideal coordinate value of any point on the camera device's epipolar line mapped to the projection device coordinate system to be obtained. The monotonicity constraint error is intended to account for deviations along the projection device's epipolar line. Therefore, the target direction corresponding to the monotonic fitting signal is the extension direction of the projection device's epipolar line. Therefore, after projecting the second coordinate onto the projection device coordinate system using the monotonic fitting signal to obtain the ideal coordinate, the monotonicity constraint error can be derived based on the distance between the second coordinate and the ideal coordinate in the target direction.

[0073] Epipolar constraints detect errors perpendicular to the epipolar line, while monotonicity constraints detect errors along the epipolar line. Combining these two forms a more comprehensive error detection mechanism. After obtaining the monotonicity and epipolar constraint errors, the combined error is calculated and compared with the first threshold to remove unreliable coordinate pairs and improve data quality.

[0074] S208: For any first set, determine the transformation relationship between the corresponding projection device coordinate system and the unified coordinate system, and transform each first coordinate into a third coordinate according to the transformation relationship to update the first coordinate pair into a second coordinate pair to obtain the corresponding second set.

[0075] As can be understood, since each camera device observes the object to be measured from a different oblique perspective, its coordinate data needs to undergo a perspective conversion to convert from its unique perspective to the vertical perspective used by the projection device. After the perspective conversion, the first coordinate (u, v) in each coordinate pair is transformed into the third coordinate (u', v'). Furthermore, after applying the transformation relationship, the obtained coordinate values can be rounded using the round function to simplify calculations. A first coordinate pair ((x, y), (u, v)) is updated to become a second coordinate pair ((x, y), (u', v')). Once all first coordinate pairs are updated, the first set becomes the second set. Regarding obtaining the transformation relationship, a common method is to determine the transformation relationship through calibration of the measurement system. For example, the Zhang calibration method is used to calibrate the projection device and camera device, obtaining their intrinsic and extrinsic parameter matrices, and then calculating the transformation matrix between the projection device coordinate system and the unified coordinate system.

[0076] S210: For any second set, if there is a second coordinate pair with the same third coordinate, retain the second coordinate pair with the smallest comprehensive error.

[0077] It can be understood that after the perspective is unified, each measurement pair corresponds to a second set. After obtaining the second set, due to the influence of light pollution, there may be multiple second coordinate pairs with the same third coordinate (u', v'), but the accuracy of the second coordinate pairs may be different. The comprehensive error can reflect the reliability of the coordinate pair. Selecting the coordinate pair with the smallest comprehensive error can minimize the impact of measurement error and perform the first fusion of the detection results within the same measurement pair, thereby improving the accuracy of 3D reconstruction. This step can be expressed mathematically as:

[0078]

[0079] Here D includes any second coordinate pair and its corresponding comprehensive error data, It represents the i-th second set. It means traversal The j-th D data traversed in the process, It represents The corresponding comprehensive error. It represents the second set obtained after the fusion of the i-th second set.

[0080] S212: Merge the second sets and retain the second coordinate pairs with the same third coordinates and the ones with the smallest comprehensive error to obtain a third set.

[0081] It can be understood that after achieving fusion within the same measurement pair, it is necessary to fuse different measurement pairs. The merging of the second set is to traverse all second coordinate pairs, retain the second coordinate pairs with the smallest comprehensive error from different measurement pairs but with the same third coordinate (u', v'), and discard the rest, thereby merging multiple second sets into the third set. This step can be expressed mathematically as:

[0082]

[0083] Here C includes any second set, It represents the j-th C data traversed during the traversal process. It represents The corresponding comprehensive error. It represents the third set obtained by fusion.

[0084] S214: Obtain a three-dimensional point cloud according to each second coordinate pair in the third set.

[0085] It can be understood that a 3D point cloud is the final result of 3D topography measurement, representing the 3D geometric information of an object's surface in the form of a discrete set of points. Generating a 3D point cloud based on the second coordinate pairs in the third set is a key step in converting 2D coordinate information into 3D spatial information. Each second coordinate pair in the third set is obtained through dual-constraint error screening, fusion within the same measurement pair, and fusion between different measurement pairs. This process greatly reduces the errors caused by optical contamination and achieves excellent noise reduction. Based on the denoised second coordinate pairs, a 3D point cloud can be obtained using any 3D reconstruction method.

[0086] This solution addresses the noise reduction problem in 3D topography measurement by proposing a comprehensive workflow. First, each measurement pair is controlled to acquire calibration images. Multiple first sets are generated based on these calibration images. Subsequently, the epipolar constraint error and monotonicity constraint error are calculated for each coordinate pair. The combined error is then determined, and coordinate pairs with combined errors below a first threshold are removed to improve data accuracy. The transformation between the projection device coordinate system and the unified coordinate system is then determined. The coordinate pairs are updated to form a second set, and the pairs with the same third coordinate in the second set with the lowest combined error are retained. The second sets are then merged and filtered again to form a third set. Finally, a 3D point cloud is generated based on the coordinate pairs in the third set. This solution significantly improves the quality of the measurement data. By combining epipolar and monotonic constraints to filter coordinate pairs, it effectively removes noise data caused by factors such as optical contamination. Compared to traditional epipolar constraint methods, it further considers errors introduced in the epipolar direction. The final coordinate transformation and multi-round filtering mechanism fully utilize the redundant information of multiple measurement pairs, enhancing data reliability. The resulting 3D point cloud is more accurate.

[0087] In one embodiment, the epipolar constraint error determination process includes: for any first coordinate pair, determining the epipolar line of the corresponding feature point in the projection device coordinate system, and determining the epipolar constraint error based on the first coordinate of the feature point and the epipolar line.

[0088] It is understood that in epipolar geometry, the determination of the epipolar line equation for any feature point is relatively mature, and any method can be used to determine the epipolar line expression. The expression can be in the form of a polar coordinate equation or a parametric equation. If the polar coordinate equation is used, the expression for the epipolar constraint error can be:

[0089]

[0090] in, Represents the epipolar constraint error corresponding to a first coordinate pair ((x, y), (u, v)). Substituting the first coordinate (u, v) in the first coordinate pair into , the epipolar constraint error can be obtained. 、 and They represent the three parameters of the polar coordinate equation of the epipolar line.

[0091] In one embodiment, the projection device coordinate system includes a first axis and a second axis that are perpendicular to each other, such as Figure 2 As shown, the first axis is the u axis and the second axis is the v axis. The coordinate system of the camera device includes a third axis and a fourth axis that are perpendicular to each other, such as Figure 2As shown, the third axis is the x-axis and the fourth axis is the y-axis. The camera device includes a first camera, and the measurement pair includes a first measurement pair. The first camera is set on the first axis, and the third axis is inclined at an angle to the first axis, and the fourth axis is parallel to the second axis. Figure 2 For example, the XYZ spatial coordinate system defined in [1] has the X-axis parallel to the u-axis, the first axis of the projection device, and the Y-axis parallel to the v-axis, the second axis of the projection device. The first camera is placed on the u-axis, effectively aligning it with the X-axis of the spatial coordinate system. The first camera is also referred to as the X-aligned camera. The y-axis, the fourth axis of the first camera, is parallel to the v-axis, the second axis of the projection device. However, the x-axis, the third axis, is tilted relative to the u-axis. Figure 1 The camera device 2 and the camera device 4 shown in FIG are the first cameras. The first camera and the projection device form a first measurement pair. For the first measurement pair, please refer to Figure 6 The process of determining the monotonicity constraint error includes steps S602 to S606.

[0092] S602 : Perform epipolar correction on the calibration image captured by the projection device in the first measurement pair in the direction of the first axis of phase unwrapping to obtain a first target image.

[0093] It can be understood that phase unwrapping is a technical means to unpack the phase ambiguity caused by periodic phase changes during the measurement process and obtain continuous phase values. It is particularly critical in structured light 3D measurement. Epipolar line correction is to align the epipolar lines of images from different perspectives in a stereo vision system through geometric transformation so that corresponding points are on the same horizontal scan line, thereby simplifying the feature point matching process. This can be achieved using mature technologies in this field. The first measurement is to align the image captured by the projection device. The first target image after epipolar correction , its epipolar distribution is more regular, facilitating subsequent feature analysis and coordinate calculation. To more easily determine the monotonic relationship between epipolar lines, the calibration image is epipolar corrected so that all epipolar lines of the projection device are aligned with the third axis of the camera device coordinate system, i.e., the x-axis. This way, all first coordinate pairs can be used for signal fitting and all first coordinate pairs can be mapped using the same monotonic fitting signal.

[0094] S604: In the first target image, a mapping coordinate corresponding to each second coordinate is determined, and a monotonic fitting signal is obtained by fitting the second coordinates and the mapping coordinates.

[0095] It can be understood that the mapped coordinates are the coordinate positions of the second coordinate (x, y) in the first target image after epipolar correction. That is, before epipolar correction, the second coordinate (x, y) in the calibration image corresponds to the first coordinate (u, v). However, after epipolar correction, the calibration image changes, and the original corresponding first coordinate (u, v) becomes the mapped coordinate. At this point, all mapped coordinates are fitted, meaning a monotonic function is found to approximate the relationship between these coordinates. This monotonic function is called the monotonic fitting signal, which can convert discrete coordinate data into a continuous function relationship.

[0096] S606: For the first coordinate pair, substitute the coordinate value of the second coordinate in the direction of the third axis into the monotonic fitting signal to obtain the first ideal coordinate value, and obtain the monotonicity constraint error based on the difference between the coordinate value of the first coordinate in the direction of the first axis and the first ideal coordinate value.

[0097] It can be understood that since the second axis is parallel to the fourth axis in the first measurement pair, and the epipolar line of the camera coordinate system is nearly perpendicular to the y-axis, the first coordinate only changes significantly along the first axis after monotonic fitting signal mapping. Therefore, the distance of the first coordinate in the target direction can be simplified to the difference between the first axis and the first ideal coordinate value. This can be expressed mathematically as: . That is, the monotonicity constraint error corresponding to the first coordinate pair ((x, y), (u, v)) under the first measurement pair, That is, the coordinate value of the first coordinate of the first coordinate pair in the direction of the first axis, represents the monotonic fitting signal under the first measurement pair.

[0098] In one embodiment, the projection device coordinate system includes a first axis and a second axis that are perpendicular to each other, such as Figure 2 As shown, the first axis is the u axis and the second axis is the v axis. The coordinate system of the camera device includes a third axis and a fourth axis that are perpendicular to each other, such as Figure 2 As shown, the third axis is the x-axis and the fourth axis is the y-axis. The camera device includes a second camera, the measurement pair includes a second measurement pair, the second camera is set on the second axis, the third axis is parallel to the first axis, and the fourth axis is inclined at an angle to the second axis. The second camera and the projection device form a second measurement pair. Figure 2 For example, the XYZ spatial coordinate system defined in [1] has the X-axis parallel to the projection device's first axis, the u-axis, and the Y-axis parallel to the projection device's second axis, the v-axis. The second camera is placed on the v-axis, effectively aligning it with the spatial coordinate system's Y-axis. This second camera can also be called a Y-aligned camera. The second camera's third axis, the x-axis, is parallel to the projection device's first axis, the u-axis, but the fourth axis, the y-axis, is tilted at an angle to the second axis, the v-axis. Figure 1The camera device 1 and the camera device 3 shown in FIG are the second camera. The second camera and the projection device form a second measurement pair. For the second measurement pair, please refer to Figure 7 The process of determining the monotonicity constraint error includes steps S702 to S706.

[0099] S702 , performing epipolar correction on the calibration image captured by the projection device in the second measurement pair in the direction of the second axis of phase unwrapping to obtain a second target image.

[0100] It can be understood that phase unwrapping is a technical means to unpack the phase ambiguity caused by periodic phase changes during the measurement process and obtain continuous phase values. It is particularly critical in structured light 3D measurement. Epipolar line correction is to align the epipolar lines of images from different perspectives in a stereo vision system through geometric transformation so that corresponding points are on the same horizontal scan line, thereby simplifying the feature point matching process. This can be achieved using mature technologies in this field. The second measurement is to align the image captured by the projection device. The second target image after epipolar correction , its epipolar distribution is more regular, facilitating subsequent feature analysis and coordinate calculation. To more easily determine the monotonic relationship between epipolar lines, epipolar correction is performed on the calibration image so that all epipolar lines of the projection device are aligned with the fourth axis of the camera device coordinate system, i.e., the y-axis. This way, all first coordinate pairs can be used for signal fitting and all first coordinate pairs can be mapped using the same monotonic fitting signal.

[0101] S704 : In the second target image, determine the mapping coordinates corresponding to the second coordinates, and obtain a monotonic fitting signal by fitting the second coordinates and the mapping coordinates.

[0102] It can be understood that the mapped coordinates are the coordinate positions of the second coordinate (x, y) in the second target image after epipolar correction. That is, before epipolar correction, the second coordinate (x, y) in the calibration image corresponds to the first coordinate (u, v). However, after epipolar correction, the calibration image changes, and the original corresponding first coordinate (u, v) becomes the mapped coordinate. At this point, all mapped coordinates are fitted, meaning a monotonic function is found to approximate the relationship between these coordinates. This monotonic function is called the monotonic fitting signal, which can convert discrete coordinate data into a continuous function relationship.

[0103] S706, for the first coordinate pair, substitute the coordinate value of the second coordinate in the direction of the fourth axis into the second monotonic fitting signal to obtain the second ideal coordinate value, and obtain the monotonicity constraint error based on the difference between the coordinate value of the first coordinate in the direction of the second axis and the second ideal coordinate value.

[0104] It can be understood that since the first axis and the third axis are parallel in the second measurement pair, and the epipolar line of the camera coordinate system is nearly perpendicular to the x-axis, the first coordinate only changes significantly along the second axis after monotonic fitting signal mapping. Therefore, the distance of the first coordinate in the target direction can be simplified to the difference between the second ideal coordinate value and the second axis direction. This can be expressed mathematically as: . That is, the monotonicity constraint error corresponding to a first coordinate pair ((x, y), (u, v)) under the second measurement pair, That is, the coordinate value of the first coordinate of the first coordinate pair in the direction of the second axis, represents the monotonic fitting signal under the second measurement pair.

[0105] In one embodiment, a monotonic fitting signal is obtained by fitting the second coordinates and the mapped coordinates. Figure 8 , including steps S802 to S812.

[0106] S802 , performing polynomial fitting on the coordinate value corresponding to the target direction in the second coordinate as an independent variable and the coordinate value of the target direction in the mapped coordinate as a dependent variable to obtain an initial fitting signal.

[0107] It can be understood that the two coordinate values contained in the mapping coordinates are mapped from the two coordinate values of the second coordinates respectively. The two coordinate values of the mapping coordinates are respectively in the target direction and the non-target direction perpendicular to the target direction. Since the monotonicity constraint error only needs to consider the coordinate value difference in the target direction, only the coordinate values related to the target in the mapping coordinates and the second coordinates can be associated during fitting. Specifically, the coordinate value in which direction of the second coordinate value is mapped to obtain the coordinate value of the target direction in the mapping coordinates, the coordinate value of that direction is used as the independent variable. For example, in the first measurement pair, the coordinate value x in the direction of the third axis of the second coordinate is used as the independent variable, and the coordinate value u in the direction of the first axis of the mapping coordinate is used as the dependent variable. In the second measurement pair, the coordinate value y in the direction of the fourth axis of the second coordinate is used as the independent variable, and the coordinate value v in the direction of the second axis of the mapping coordinate is used as the dependent variable. Polynomial fitting is a mathematical method that constructs a polynomial function to make the function as close as possible to a given data point. Its basic form is: , where z represents the dependent variable, t represents the independent variable, n+1 represents the number of polynomials used for fitting, and a k Represents the coefficient of the kth subterm in the polynomial. Based on a large amount of specific value data of the independent and dependent variables, common polynomial fitting methods can be used to obtain the various polynomial coefficients, such as the least squares method, to obtain the initial fitting signal.

[0108] S804: Determine the monotonic direction of the initial fitting signal.

[0109] It can be understood that the monotonic direction refers to the predominant trend of change in the initial fitting signal within its domain. For a monotonically increasing function, the dependent variable increases as the independent variable increases; for a monotonically decreasing function, the dependent variable decreases as the independent variable increases. In data processing for 3D topography measurement, determining the monotonic direction of the initial fitting signal is a key step in subsequently determining whether the fitting signal meets the monotonicity requirement and is an important basis for optimizing the fitting signal. Because optical contamination may affect the raw data used for fitting, some points may not conform to the overall trend of the function. The resulting initial fitting signal may not necessarily be a monotonic function within the entire domain. The monotonic direction here refers to the trend of change in the initial fitting signal. For example, if the trend is increasing, the monotonic direction is increasing; otherwise, it is decreasing. The monotonic direction can be determined by first fitting a straight line using all the data. If the coefficient of this line is greater than 0, the monotonic direction is determined to be increasing; otherwise, it is decreasing. Alternatively, the data can be sorted by the size of the independent variable and divided into multiple intervals. The monotonicity of each interval is determined separately, and then all the monotonicity determination results are summarized, with the one with the highest percentage being the monotonic direction.

[0110] S806: derive the initial fitting signal to obtain a derivative signal.

[0111] It can be understood that for the initial fitting signal in the form of a polynomial function, calculations can be performed using the derivative formula. The derivative signal is the function obtained by differentiating the initial fitting signal. According to the properties of derivatives, if the derivative of a function is greater than zero within a certain interval, the function is monotonically increasing within that interval; if the derivative is less than zero, the function is monotonically decreasing within that interval. By differentiating the initial fitting signal to obtain the derivative signal, we can mathematically analyze the changing trend of the initial fitting signal at different points, thereby accurately determining whether it meets the monotonicity requirement. The derivative signal provides a quantitative standard for subsequently identifying mapping coordinates that do not conform to the monotonic direction.

[0112] S808: Substitute the coordinate value corresponding to the target direction in the second coordinate into the current derivative signal to determine whether there is a new second coordinate whose derivative value does not match the monotonic direction.

[0113] It can be understood that in order to determine which mapping results do not meet the monotonicity, it can be determined based on whether the value of the derivative signal is consistent with the monotonic direction. Therefore, each variable is substituted into the current derivative signal to obtain the derivative value. If the determined monotonic direction is increasing, and the derivative value obtained by substituting the coordinate value is less than zero, then there is a problem with the mapping result of the second coordinate. Otherwise, the next coordinate value can be substituted. If the determined monotonic direction is decreasing, and the derivative value obtained by substituting the coordinate value is greater than zero, then there is a problem with the mapping result of the second coordinate. Otherwise, the next coordinate value can be substituted. If a new second coordinate that does not meet the requirements appears, it means that the current optimization is not enough to meet the monotonicity constraint, and the optimization should be continued in step S810. If no new second coordinate that does not meet the requirements appears, it means that the current optimization can meet the monotonicity constraint, and the optimization should be continued in step S812.

[0114] S810, if yes, then add the corresponding second coordinate to the violation point set, and update the monotonic constraint based on each point in the violation point set. Perform polynomial fitting based on the monotonic constraint and each second coordinate and the mapped coordinate to update the initial fitting signal, and return to the step of taking the derivative of the initial fitting signal to continue execution; the monotonic constraint is used to constrain the derivative value corresponding to the point in the violation point set to be zero.

[0115] It can be understood that the violation point set is a collection of second coordinates that do not conform to the monotonic direction. The resulting mapping of these coordinates violates the monotonicity requirement of the monotonic fitting signal and is therefore a key issue that requires attention during the signal fitting optimization process. Monotonic constraints are constraints set based on the violation point set. Their core function is to adjust the shape of the fitted signal curve to meet the monotonicity requirement by forcing the derivative values of the derivative signal corresponding to the points in the violation point set to zero. When a new second coordinate appears, a corresponding monotonic constraint is added based on the second coordinate, thereby updating the monotonic constraint. This ensures that when performing polynomial fitting, the updated polynomial coefficients ensure that the derivative values corresponding to all violation point sets are zero, thereby modifying the curve shape. The specific method for adding monotonic constraints depends on the polynomial fitting method used. For example, when using the least squares method for polynomial fitting, rows corresponding to each point in the violation point set can be added to the least squares solution matrix. Specifically, the least squares solution matrix consists of a condition matrix, a parameter matrix, and a result matrix. The parameter matrix is a column matrix that includes all polynomial coefficients. The condition matrix is a matrix with m+v rows and n+1 columns, and the result matrix is a matrix with m+v rows. The result matrix is obtained by multiplying the condition matrix with the parameter matrix. The first m rows of the condition matrix correspond to the values of the variables to be fitted. The i-th row and k+1th column of these rows can be expressed as , that is, the i-th independent variable o iThe first m rows of the result matrix are the values of the dependent variable corresponding to each variable. In this way, the parameter matrix needs to ensure that the fitted polynomial can obtain a value close to the corresponding dependent variable after substituting the fitted polynomial into each variable. The last v rows of the condition matrix correspond to each monotonic constraint condition one by one. The i-th row and k+1 column of these rows can be expressed as , that is, the i-th violation point p i The value of the k-1th power term in the derivative signal. The last v rows of the resulting matrix are all 0. Therefore, the parameter matrix must ensure that the derivative value of the fitted polynomial is 0 when the independent variable corresponding to each offending point is substituted into the derivative signal of the fitted polynomial. This means that the shape of each point in the offending point set is forcibly repaired.

[0116] S812: If not, the current initial fitting signal is used as a monotonic fitting signal.

[0117] It can be understood that when the derivative values of all mapped coordinates are consistent with the monotonic direction, the current initial fitting signal satisfies the monotonicity requirement within its domain of definition and is then determined as the final monotonic fitting signal. This monotonic fitting signal accurately describes the monotonic mapping relationship of the mapped coordinates in the target direction and can be used in subsequent 3D topography measurement data processing steps such as monotonicity constraint error calculation, providing a reliable functional relationship basis for the entire measurement system.

[0118] In one embodiment, a monotonic fitting signal is obtained by fitting each mapping coordinate, see Figure 9 , including steps S902 to S912. To distinguish, Figure 8 The independent variable in a scenario can be called the first independent variable, and the dependent variable can be called the first dependent variable. Figure 9 The independent variable in a scenario can be called the second independent variable, and the dependent variable can be called the second dependent variable.

[0119] S902: The coordinate value corresponding to the target direction in the second coordinate is used as an independent variable, and the coordinate value of the target direction in the mapped coordinate is used as a dependent variable.

[0120] It can be understood that the two coordinate values contained in the mapped coordinates are mapped from the two coordinate values of the second coordinates, respectively. The two coordinate values of the mapped coordinates are in the target direction and a non-target direction perpendicular to the target direction, respectively. Since the monotonicity constraint error only considers the difference in coordinate values in the target direction, the mapped coordinates and the target-related coordinate values in the second coordinates can be associated during fitting. Specifically, the coordinate value in the direction of the second coordinate value that maps to the coordinate value in the target direction in the mapped coordinates is used as the independent variable. For example, in the first measurement pair, the coordinate value x in the direction of the third axis of the second coordinate is used as the independent variable, and the coordinate value u in the direction of the first axis of the mapped coordinate is used as the dependent variable. In the second measurement pair, the coordinate value y in the direction of the fourth axis of the second coordinate is used as the independent variable, and the coordinate value v in the direction of the second axis of the mapped coordinate is used as the dependent variable.

[0121] S904: Sort the independent variables according to their coordinate values, and determine the monotonic direction according to the change of the dependent variable.

[0122] It can be understood that sorting here involves rearranging the independent variables (i.e., the coordinate values corresponding to the target direction in the second coordinate system) in order of magnitude. This ensures monotonic changes in the independent variables, facilitating subsequent analysis of data trends. A monotonic direction refers to whether the function exhibits a monotonically increasing or monotonically decreasing trend within its domain. In this step, this is determined by observing how the dependent variable (the coordinate value of the target direction in the mapped coordinate system) changes after the independent variables are sorted. If the dependent variable generally increases with the independent variable, the monotonic direction is monotonically increasing; if the dependent variable generally increases with the independent variable, the monotonic direction is monotonically decreasing.

[0123] Because the original data used for fitting may be affected by optical contamination, some points may not conform to the overall trend of the function. The resulting initial fitted signal may not be a monotonic function within the entire domain. The monotonic direction here should be the trend of change in the initial fitted signal. For example, if the trend is increasing, the monotonic direction is increasing, and vice versa. The monotonic direction can be determined by fitting a straight line using all the data. If the coefficient of this line is greater than 0, the monotonic direction is increasing, and vice versa, it is decreasing. Alternatively, the independent variables can be sorted by size and divided into multiple intervals. The monotonicity of each interval is determined separately, and then all the monotonicity determination results are summarized, with the one with the highest percentage being the monotonic direction.

[0124] S906 , respectively determining the difference between two adjacent dependent variables to obtain a first gradient set.

[0125] It can be understood that the difference between two adjacent dependent variables reflects the degree of signal change in the adjacent interval, namely the gradient. The first gradient set is the set consisting of all gradients. By calculating the gradient of the dependent variable difference between adjacent mapping coordinates, it is possible to quantify the change of the function in the target direction. The first gradient set contains the gradient information of all mapping coordinates, describing the overall trend of the function. This gradient information is an important basis for subsequently determining whether the data conforms to a monotonic direction. By analyzing the sign and magnitude of the gradient, it is possible to identify parts of the data that do not conform to the monotonic trend, providing data support for subsequent gradient compensation and fitting signal optimization.

[0126] S908 , sequentially determining the gradients in the first gradient set that do not conform to the monotonic direction as gradients to be compensated.

[0127] It can be understood that gradients to be compensated refer to those in the first gradient set that are inconsistent with the monotonic direction determined in step S904. In the case of monotonically increasing gradients, if the gradient is negative, then the gradient is considered to be compensated; in the case of monotonically decreasing gradients, if the gradient is positive, then the gradient is considered to be compensated. Measurement errors, data noise, or inaccuracies in the fitting process may cause some gradients to deviate from the overall monotonic direction. By identifying these gradients to be compensated, it is possible to locate portions of the data that do not conform to the monotonic trend, providing targets for subsequent gradient adjustment and fitting signal optimization.

[0128] S910: The gradient to be compensated is set to zero, and the total amount to be compensated is determined based on the gradient to be compensated. The left and right sides of the gradient to be compensated are traversed respectively. If the traversed gradient does not conform to the monotonic direction, the traversed gradient is set to zero and accumulated in the total amount to be compensated based on the traversed gradient. If the traversed gradient conforms to the monotonic direction, the total amount that can be contributed is accumulated based on the traversed gradient until the total amount that can be contributed is greater than or equal to the total amount to be compensated. The allocation ratio is determined based on the proportion of the gradients that conform to the monotonic direction in the traversed gradients in the total amount that can be contributed. The gradients that conform to the monotonic direction in the traversed gradients are reduced based on the allocation ratio and the total amount to be compensated to update the first gradient set.

[0129] It can be understood that the first gradient set is generated one by one according to the sorting order of the respective variables, so the dependent variables that produce adjacent gradients are also located adjacently in the coordinate system. The total amount to be compensated is the sum of the absolute values of all gradients to be compensated, representing the total amount of adjustment required for gradients that do not conform to the monotonic direction. The total amount that can be contributed is the cumulative sum of the gradient values that conform to the monotonic direction when traversing both sides of the gradient to be compensated. By calculating the allocation ratio and reducing the gradients that conform to the monotonic direction, the first gradient set is updated to make the updated gradient set more consistent with the monotonic direction.

[0130] Setting the gradient to be compensated to zero eliminates abnormal changes that are obviously not in line with the monotonic direction, but this may cause local discontinuity in the data. In order to minimize data mutations while ensuring monotonicity, the gradient that conforms to the monotonic direction needs to be adjusted. By traversing both sides of the gradient to be compensated, the total amount to be compensated and the total amount that can be contributed are accumulated according to whether the gradient conforms to the monotonic direction. If the traversed gradient conforms to the monotonic direction, it can be used to supplement the gradient that does not conform to the monotonic direction. If the traversed gradient does not conform to the monotonic direction, it will also become the object of supplementation. When the total amount that can be contributed is greater than or equal to the total amount to be compensated, the more a gradient can be used to compensate, the greater its output should be. Therefore, according to the ratio of these traversed gradients that conform to the monotonic direction to the total amount that can be contributed, the distribution ratio of the gradient can be obtained, and the total amount to be compensated is the total amount that needs to be distributed. By multiplying the distribution ratio by the total amount to be compensated, it can be obtained how much these gradients ultimately need to be reduced to compensate for the gradients that do not conform to the monotonic direction. It can be expressed in mathematical terms as follows: , where g represents the gradient that needs to be reduced, S1 is the total amount to be compensated, and S2 is the total amount that can be contributed. Smoothing the data near points that do not conform to the monotonic direction can make the data more consistent with the monotonic trend overall.

[0131] For example, if the dependent variable sequence is {0, 1, 2, 4, 1, 2, 6, 7}, and its calculated gradient is {1, 1, 2, −3, 1, 4, 1}, the gradient at -3 is detected as non-monotonous and set to 0. The total amount to be compensated is 3, and a search is then performed to the left and right of the gradient. Gradients 2 and 1, which are monotonic, are found. The sum of these two gradients equals the total amount to be compensated, and can be used for compensation. The compensated gradient becomes {1, 1, 0, 0, 0, 4, 1}.

[0132] S912: Update each dependent variable according to the updated first gradient set, and obtain a monotonic fitting signal by fitting the updated dependent variable.

[0133] It can be understood that the updated first gradient set contains information about the adjusted rates of change of adjacent dependent variables. The starting point for reconstructing the dependent variable sequence can be selected based on the monotonic direction. For example, for increasing, the first point in the original dependent variable sequence can be used as the starting point, while for decreasing, the last point can be used as the starting point. Once the starting point is determined, the dependent variable sequence can be updated sequentially using the gradient corresponding to each dependent variable in the first gradient set. Finally, the updated dependent variable sequence is fitted with the original independent variable sequence to obtain a monotonic fitted signal. The fitting method here can also be a polynomial fit. Continuing with the above example, if the updated first gradient set is {1, 1, 0, 0, 0, 4, 1} and the original monotonic direction is increasing, then the first point in the sequence {0, 1, 2, 4, 1, 2, 6, 7} is used as the starting point for the update, resulting in {0, 1, 2, 2, 2, 2, 6, 7}.

[0134] In one embodiment, determining the corresponding integrated error according to the epipolar constraint error and the monotonicity constraint error includes: obtaining the integrated error according to the second norm of the epipolar constraint error and the monotonicity constraint error.

[0135] The present application provides a 3D shape measurement noise reduction device, which is applied to a 3D shape measurement system. The 3D shape measurement system includes a projection device and multiple camera devices, each camera device and the projection device forming a measurement pair. The 3D shape measurement noise reduction device includes:

[0136] The image acquisition module is used to control each measurement pair to obtain a calibration image.

[0137] A coordinate determination module is configured to obtain a first set of measurement pairs corresponding to the calibration image. The first set includes multiple sets of first coordinate pairs, each set of first coordinate pairs including a first coordinate of a corresponding feature point in a projection device coordinate system and a second coordinate in a camera device coordinate system.

[0138] The filtering module is configured to calculate the epipolar constraint error and the monotonicity constraint error for each set of first coordinate pairs, determine a corresponding comprehensive error based on the epipolar constraint error and the monotonicity constraint error, and remove first coordinate pairs whose comprehensive error is lower than a first threshold. The epipolar constraint error reflects the distance between the first coordinate and the corresponding epipolar line in the projection device coordinate system, and the monotonicity constraint error reflects the distance between the second coordinate and the first coordinate in the target direction corresponding to the monotonic fitting signal after the second coordinate is mapped to the projection device coordinate system according to the monotonic fitting signal.

[0139] The transformation module is used to determine the transformation relationship between the corresponding projection device coordinate system and the unified coordinate system for any first set, and transform each second coordinate into a third coordinate according to the transformation relationship to update the first coordinate pair into the second coordinate pair to obtain the corresponding second set.

[0140] The first fusion module is used to retain the second coordinate pair with the smallest comprehensive error if there is a second coordinate pair with the same third coordinate in any second set.

[0141] The second fusion module is used to merge the second sets and retain the second coordinate pairs with the same third coordinates and the ones with the smallest comprehensive error to obtain a third set.

[0142] The point cloud generation module is used to obtain a three-dimensional point cloud according to each second coordinate pair in the third set.

[0143] For the specific limitations of the text recognition device, please refer to the limitations of the three-dimensional shape measurement noise reduction method above, which will not be repeated here. The various modules in the above-mentioned text recognition device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0144] The present application provides a computer device comprising one or more processors and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the steps of the three-dimensional morphology measurement denoising method in any of the above embodiments are executed.

[0145] The present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the three-dimensional shape measurement noise reduction method in any of the above embodiments.

[0146] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0147] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0148] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A 3D shape measurement noise reduction method, characterized in that: Applied to a three-dimensional shape measurement system, the three-dimensional shape measurement system includes a projection device and multiple camera devices, each of the camera devices and the projection device forming a measurement pair, the three-dimensional shape measurement noise reduction method includes: respectively controlling each of the measurement pairs to obtain a calibration image; Obtaining a first set corresponding to the measurement pairs according to the calibration image; the first set includes a plurality of first coordinate pairs, each first coordinate pair including a first coordinate of a corresponding feature point in a projection device coordinate system and a second coordinate in a camera device coordinate system; calculating the epipolar constraint error and the monotonicity constraint error of each group of the first coordinate pairs respectively, determining a corresponding comprehensive error based on the epipolar constraint error and the monotonicity constraint error, and removing the first coordinate pairs whose comprehensive error is lower than a first threshold; the epipolar constraint error reflects the distance between the first coordinate and the corresponding epipolar line in the projection device coordinate system, and the monotonicity constraint error reflects the distance between the second coordinate and the first coordinate in the target direction corresponding to the monotonic fitting signal after the second coordinate is mapped to the projection device coordinate system according to the monotonic fitting signal; For any of the first sets, determining a transformation relationship between the corresponding projection device coordinate system and the unified coordinate system, and transforming each of the first coordinates into a third coordinate according to the transformation relationship, so as to update the first coordinate pair into a second coordinate pair, thereby obtaining a corresponding second set; For any second set, if there exists a second coordinate pair with the same third coordinate, retaining the second coordinate pair with the smallest comprehensive error; Merging the second sets, and retaining the second coordinate pairs with the same third coordinates and having the smallest comprehensive error, to obtain a third set; A three-dimensional point cloud is obtained according to each of the second coordinate pairs in the third set.

2. The 3D shape measurement noise reduction method according to claim 1, characterized in that: The process of determining the epipolar constraint error includes: For any first coordinate pair, determining the epipolar line of the corresponding feature point in the projection device coordinate system; The epipolar constraint error is determined according to the first coordinate of the feature point and the epipolar line.

3. The 3D shape measurement noise reduction method according to claim 1, characterized in that: The projection device coordinate system includes a first axis and a second axis that are perpendicular to each other, the camera device coordinate system includes a third axis and a fourth axis that are perpendicular to each other, the camera device includes a first camera, the measurement pair includes a first measurement pair, the first camera is arranged on the first axis, the third axis is inclined at an angle to the first axis, and the fourth axis is parallel to the second axis, the first camera and the projection device constitute the first measurement pair, and for the first measurement pair, the process of determining the monotonicity constraint error includes: Performing epipolar correction on the calibration image captured by the projection device in the first measurement pair in the direction of the first axis of phase unwrapping to obtain a first target image; In the first target image, determining a mapping coordinate corresponding to each second coordinate, and obtaining the monotonic fitting signal by fitting each second coordinate and the mapping coordinate; For the first coordinate pair, the coordinate value of the second coordinate in the direction of the third axis is substituted into the monotonic fitting signal to obtain the first ideal coordinate value, and the monotonicity constraint error is obtained based on the difference between the coordinate value of the first coordinate in the direction of the first axis and the first ideal coordinate value.

4. The 3D shape measurement noise reduction method according to claim 1, characterized in that: The projection device coordinate system includes a first axis and a second axis that are perpendicular to each other, the camera device coordinate system includes a third axis and a fourth axis that are perpendicular to each other, the camera device includes a second camera, the measurement pair includes a second measurement pair, the second camera is arranged on the second axis, the third axis is parallel to the first axis, and the fourth axis is inclined at an angle to the second axis, the second camera and the projection device constitute a second measurement pair, and for the second measurement pair, the process of determining the monotonicity constraint error includes: performing epipolar correction on the calibration image captured by the projection device in the second measurement pair in the direction of the second axis of phase unwrapping to obtain a second target image; In the second target image, determining a mapping coordinate corresponding to each second coordinate, and obtaining the monotonic fitting signal by fitting each second coordinate and the mapping coordinate; For the first coordinate pair, the coordinate value of the second coordinate in the direction of the fourth axis is substituted into the second monotonic fitting signal to obtain a second ideal coordinate value, and the monotonicity constraint error is obtained based on the difference between the coordinate value of the first coordinate in the direction of the second axis and the second ideal coordinate value.

5. The 3D shape measurement noise reduction method according to claim 3 or 4, characterized in that: The step of obtaining the monotonic fitting signal by fitting the second coordinates and the mapped coordinates includes: Performing polynomial fitting on the coordinate value corresponding to the target direction in the second coordinate as an independent variable and the coordinate value of the target direction in the mapped coordinate as a dependent variable to obtain an initial fitting signal; determining a monotonic direction of the initial fitting signal; Derivative the initial fitting signal to obtain a derivative signal; Substituting the coordinate value corresponding to the target direction in the second coordinate into the current derivative signal, and determining whether there is a new second coordinate whose derivative value does not conform to the monotonic direction; If so, adding the corresponding second coordinate to the violation point set, and updating the monotonic constraint condition based on each point in the violation point set, performing polynomial fitting based on the monotonic constraint condition, each second coordinate, and the mapped coordinate to update the initial fitting signal, and returning to the step of differentiating the initial fitting signal to obtain a derivative signal; the monotonic constraint condition is used to constrain the derivative value corresponding to the point in the violation point set to be zero; If not, the current initial fitting signal is used as the monotonic fitting signal.

6. The 3D shape measurement noise reduction method according to claim 3 or 4, characterized in that: The step of obtaining the monotonic fitting signal by fitting the second coordinates and the mapped coordinates includes: The coordinate value corresponding to the target direction in the second coordinate is used as an independent variable, and the coordinate value of the target direction in the mapped coordinate is used as a dependent variable; Sort by the size of the independent variable coordinate value, and determine the monotonic direction according to the change of the dependent variable; Determine the difference between two adjacent dependent variables respectively to obtain the first gradient set; sequentially determining the gradients in the first gradient set that do not conform to the monotonic direction as gradients to be compensated; The gradient to be compensated is set to zero, and the total amount to be compensated is determined based on the gradient to be compensated. The gradient to be compensated is traversed to the left and right sides respectively. If the traversed gradient does not conform to the monotonic direction, the traversed gradient is set to zero and accumulated in the total amount to be compensated based on the traversed gradient. If the traversed gradient conforms to the monotonic direction, the total amount that can be contributed is accumulated based on the traversed gradient until the total amount that can be contributed is greater than or equal to the total amount to be compensated. An allocation ratio is determined based on the proportion of the gradients that conform to the monotonic direction in the traversed gradients in the total amount that can be contributed. The gradients that conform to the monotonic direction in the traversed gradients are reduced based on the allocation ratio and the total amount to be compensated to update the first gradient set. Each of the dependent variables is updated according to the updated first gradient set, and the monotonic fitting signal is obtained by fitting according to the updated dependent variables.

7. The 3D shape measurement noise reduction method according to claim 1, characterized in that: Determining a corresponding comprehensive error according to the epipolar constraint error and the monotonicity constraint error includes: The comprehensive error is obtained according to the second norm of the epipolar constraint error and the monotonicity constraint error.

8. A 3D shape measurement noise reduction device, characterized in that: Applied to a three-dimensional shape measurement system, the three-dimensional shape measurement system includes a projection device and multiple camera devices, each of the camera devices and the projection device forming a measurement pair, the three-dimensional shape measurement noise reduction device includes: An image acquisition module, configured to control each of the measurement pairs to acquire a calibration image; a coordinate determination module, configured to obtain a first set corresponding to the measurement pairs based on the calibration image; the first set comprising a plurality of first coordinate pairs, each first coordinate pair comprising a first coordinate of a corresponding feature point in a projection device coordinate system and a second coordinate in a camera device coordinate system; a filtering module, configured to respectively calculate an epipolar constraint error and a monotonicity constraint error for each group of the first coordinate pairs, determine a corresponding comprehensive error based on the epipolar constraint error and the monotonicity constraint error, and remove the first coordinate pairs whose comprehensive error is lower than a first threshold; the epipolar constraint error reflects the distance between the first coordinate and the corresponding epipolar line in the projection device coordinate system; and the monotonicity constraint error reflects the distance between the second coordinate and the first coordinate in a target direction corresponding to the monotonic fitting signal after the second coordinate is mapped to the projection device coordinate system according to the monotonic fitting signal; a transformation module, configured to determine, for any of the first sets, a transformation relationship between the corresponding projection device coordinate system and the unified coordinate system, and transform each of the first coordinates into a third coordinate according to the transformation relationship, so as to update the first coordinate pair into a second coordinate pair, thereby obtaining a corresponding second set; a first fusion module, configured to, for any second set, retain the second coordinate pair with the smallest comprehensive error if there exists a second coordinate pair with the same third coordinate; A second fusion module is configured to merge the second sets and retain the second coordinate pairs with the same third coordinates having the smallest comprehensive error to obtain a third set; A point cloud generation module is used to obtain a three-dimensional point cloud according to each second coordinate pair in the third set.

9. A computer device, characterized in that: The method comprises one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the three-dimensional shape measurement denoising method according to any one of claims 1 to 7 are executed.

10. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to perform the steps of the three-dimensional shape measurement denoising method according to any one of claims 1 to 7.

Images