A method and device for correcting structured light distortion based on volume error decomposition

Abstract

This application discloses a structured light distortion correction method and apparatus based on volume error decomposition. The method obtains a preliminary correction point cloud after initial distortion correction of the structured light system, fits a reference plane using the preliminary correction point cloud under multiple attitude planes, and generates a volume error training data set. Based on the imaging geometry of the camera and projector, the volume error is decomposed into depth direction error components, camera-side radial error components, and projector-side radial error components. Depth direction error models and radial direction error models are established respectively, and the model parameters are estimated using a regularized regression method. The estimated parameters are used to predict the residual volume error point by point in the preliminary correction point cloud and superimpose it onto the corresponding coordinates to obtain the final correction point cloud. This invention can effectively compensate for residual errors in 3D reconstruction and improve the measurement accuracy and geometric consistency of the point cloud in the structured light system.

Description

Technical Field

[0001] This application relates to the field of optical correction technology, specifically to a structured light distortion correction method and apparatus based on volume error decomposition. Background Technology

[0002] Structured light 3D measurement systems consist of a projector and a camera. They project coded stripes and acquire images of deformed stripes, then reconstruct the 3D point cloud of an object through phase decoding and triangulation. Ideally, the system can use a pinhole imaging model to describe the geometric projection relationship between spatial points and the image projection coordinates. However, in real-world systems, both the camera and projector lenses exhibit distortion due to manufacturing errors, assembly errors, and optical structure deviations. This causes the image coordinates to deviate from the ideal projection position, directly affecting the accuracy of phase decoding, pixel matching, and 3D reconstruction. Therefore, existing technologies typically use distortion models such as Brown-Conrady to calibrate and correct distortion for the camera and projector separately, or incorporate distortion parameters into the reconstruction equation for joint correction using nonlinear optimization methods. Alternatively, they may use high-order polynomials, neural networks, or other models to directly fit the error relationship between the distorted point cloud and the true point cloud in the 3D point cloud space.

[0003] However, existing methods still have the following problems: after traditional separate and joint correction, there are still obvious residual systematic errors in the 3D reconstructed point cloud; while the direct 3D error fitting method lacks imaging geometric constraints, the model has insufficient interpretability, and the parameter stability is poor. Summary of the Invention

[0004] This application provides a method and apparatus for correcting structured light distortion based on volume error decomposition.

[0005] This application provides a structured light distortion correction method based on volume error decomposition, including: A preliminary correction point cloud is obtained, which is obtained by the structured light system performing three-dimensional reconstruction of the object under test after completing the preliminary distortion correction of the camera lens and the projector lens; A reference plane is fitted using preliminary correction point clouds under multiple attitude planes. The corresponding point of each preliminary correction point on the reference plane is determined as the reference real point. A volume error is generated based on the deviation between the preliminary correction point and the reference real point, and a volume error training data set is generated. The volume error training data set records the preliminary correction point and the volume error corresponding to the preliminary correction point. The volume error in the volume error training dataset is decomposed according to the camera imaging geometry and projector imaging geometry in the structured light system. The volume error is decomposed into depth direction error component, camera side radial error component and projector side radial error component. The depth direction error component is along the object point direction in world coordinates, the camera side radial error component is along the camera radial direction, and the projector side radial error component is along the projector radial direction. A depth direction error model and a radial direction error model are established. The depth direction error model uses the depth direction error component as the fitting target and combines the depth information and radial distance information of the preliminary correction point to construct a feature vector. The radial direction error model uses the radial error component on the camera side and the radial error component on the projector side as the fitting target and combines their respective radial distances to construct a feature vector. The model parameters of the depth direction error model and the radial direction error model are estimated using a regularized regression method. The estimated model parameters are then used to predict the residual volume error of points in the preliminary correction point cloud. The predicted residual volume error is then superimposed onto the coordinates of the corresponding preliminary correction points to obtain the final correction point cloud.

[0006] This application also provides a structured light distortion correction system based on volume error decomposition, including: processor; Storage device, on which computer programs are stored, When the computer program is executed by the processor, the processor implements any of the structured light distortion correction methods based on volume error decomposition.

[0007] This application also provides a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of the structured light distortion correction method based on volume error decomposition.

[0008] Therefore, the embodiments of this application have the following beneficial effects: By constructing a volume error direction decomposition and parameterized compensation mechanism based on imaging geometric constraints, a systematic leap in accuracy from the initial corrected point cloud to the final corrected point cloud is achieved. This scheme does not perform empirical overall fitting of the residual error, but uses multi-pose planar data to explicitly decompose the volume error into depth direction, camera side radial, and projector side radial components, making the error model directly correspond to the physical imaging process of the structured light system. Based on this, feature vectors with clear geometric meaning are constructed for each directional component, and a parameterized error model is established. Combined with regularized regression methods, the model parameters are stably estimated, effectively avoiding parameter inflation and overfitting problems caused by high-order polynomial overall fitting. Finally, the initial corrected point cloud is compensated point-by-point by predicting the residual volume error, so that the 3D reconstructed point cloud obtains consistent correction in both the depth and radial directions, significantly improving the full field-of-view measurement accuracy and point cloud geometric consistency. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the steps of a structured light distortion correction method based on volume error decomposition provided in an embodiment of this application.

[0010] Figure 2 This is a flowchart illustrating the technical concept of a structured light distortion correction method based on volume error decomposition, as provided in an embodiment of this application.

[0011] Figure 3 This is a schematic diagram of directional decomposition provided in an embodiment of this application.

[0012] Figure 4 This is a schematic diagram (planar depth direction error diagram) provided for an embodiment of this application before and after correction.

[0013] Figure 5 This is a schematic diagram of the basic structure of a structured light distortion correction system based on volume error decomposition provided in an embodiment of this application. Detailed Implementation

[0014] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0015] In structured light 3D measurement scenarios, a projector projects an coded fringe pattern onto the surface of the object being measured, and a camera acquires the deformed fringe image modulated by the object's surface. The system obtains 3D point cloud data of the object's surface through processes such as fringe decoding, phase unwrapping, and triangulation. Due to manufacturing and assembly errors in both the camera and projector lenses, residual volumetric errors still exist in the reconstructed 3D point cloud after the system completes preliminary distortion correction based on a traditional distortion model. This application aims to further improve the reconstruction accuracy of the 3D point cloud by directionally decomposing the residual volumetric errors and establishing a parametric compensation model.

[0016] See Figure 1 As shown, this figure is a flowchart of a structured light distortion correction method based on volume error decomposition provided in an embodiment of this application. This method is executed by a structured light distortion correction system based on volume error decomposition. Figure 1 As shown, the method includes steps 110-150.

[0017] Step 110: Obtain the preliminary correction point cloud. The preliminary correction point cloud is obtained by the structured light system performing three-dimensional reconstruction of the object under test after completing the preliminary distortion correction of the camera lens and the projector lens.

[0018] The structured light system first completes system calibration. The calibration process uses a planar calibration board. The camera acquires images of the calibration board in multiple poses, and the camera's intrinsic, extrinsic, and distortion parameters are calibrated based on these images. For the projector, since it cannot directly acquire images, it is treated as a reverse camera for calibration. The projector projects an coded pattern onto the calibration board, and the camera acquires the pattern projected onto the board. Based on the correspondence between the camera image coordinates, the projector image coordinates, and the calibration board's world coordinates, the projector's intrinsic, extrinsic, and distortion parameters are calibrated.

[0019] After calibration, the system uses the Brown-Conrady lens distortion model to perform preliminary distortion correction on the camera and projector. The Brown-Conrady distortion model includes radial and tangential distortion. Radial distortion is described using a polynomial with respect to radial distance, while tangential distortion describes the distortion caused by misalignment between the lens optical center and the image sensor center. During the preliminary correction, the pixel coordinates of the image acquired by the camera are converted to normalized image plane coordinates, and then the distortion-free coordinates are derived from the distorted coordinates using an iterative solution method. Similarly, the image plane coordinates of the pattern projected by the projector are also distorted. After completing the preliminary distortion correction for the camera and projector, the system uses the corrected camera-projector geometric relationship based on triangulation principles to perform 3D reconstruction, obtaining a preliminary corrected point cloud of the object's surface. Each point in the preliminary corrected point cloud contains 3D coordinate components in the world coordinate system and the corresponding acquisition posture identifier.

[0020] Step 120: Use the preliminary correction point cloud under multiple attitude planes to perform reference plane fitting, determine the corresponding point of each preliminary correction point on the reference plane as the reference real point, generate volume error based on the deviation between the preliminary correction point and the reference real point, and generate a volume error training data set. The volume error training data set records the preliminary correction point and the volume error corresponding to the preliminary correction point.

[0021] Since preliminary distortion correction cannot completely eliminate lens distortion and system assembly errors, residual errors still exist between the preliminary correction point cloud and the real spatial points. To construct a training dataset for residual volumetric errors, this embodiment collects planar calibration board data under multiple different postures. In each posture, the calibration board occupies a different position and orientation in space. The system performs preliminary distortion correction and 3D reconstruction on the surface of the calibration board in each posture to obtain the preliminary correction point cloud for that posture.

[0022] Step 121: Based on the acquisition attitude identifier associated with each point in the preliminary correction point cloud, divide the preliminary correction point cloud into multiple point cloud subsets corresponding to different attitude planes.

[0023] During data acquisition, for each acquired calibration board data pose, the system assigns a unique pose identifier and records this identifier on each reconstructed 3D point. Based on the pose identifiers, all preliminary calibration point clouds are grouped, with all points belonging to the same pose identifier forming a subset of the point cloud.

[0024] Step 122: For each subset of point clouds, extract the three-dimensional coordinates of all preliminary correction points in that subset of point clouds to form a coordinate matrix.

[0025] For a subset of point clouds in a given pose, which contains Num points, the x-coordinate, y-coordinate, and y-coordinate values ​​of each point in the world coordinate system are extracted to form a coordinate matrix of shape Num rows and three columns. Each row of this coordinate matrix corresponds to the three spatial coordinate components of a 3D point.

[0026] Step 123: Fit the spatial plane equation using the coordinate matrix to obtain the fitting reference plane under the attitude plane, and use the fitting reference plane as the geometric reference surface corresponding to each preliminary correction point in the point cloud subset.

[0027] Since the calibration board surface is considered an ideal plane in space, the initial correction point cloud for this pose should be distributed on a spatial plane. This embodiment employs a least-squares plane fitting method, with the optimization objective of minimizing the sum of squared distances from points to the plane, to solve for the normal vector and intercept of the plane equation. Specifically, during the fitting process, the covariance matrix of the coordinate matrix is ​​constructed, and eigenvalue decomposition is performed on the covariance matrix. The eigenvector corresponding to the smallest eigenvalue is used as the plane normal vector. Combining the mean coordinates of all points in the point cloud subset as a point on the plane, the fitting reference plane corresponding to this pose is determined. The fitting reference plane is stored in the form of a normal vector and a plane intercept for subsequent projection calculations.

[0028] Step 124: For each preliminary correction point in the point cloud subset, calculate the coordinates of its projection point on the fitting reference plane along the direction of the normal vector of the fitting reference plane, and determine the coordinates of the projection point as the reference true point coordinates of the preliminary correction point.

[0029] For any initial calibration point in the point cloud subset, its position in the world coordinate system is represented by a coordinate vector. The fitting reference plane for this point is determined by the normal vector and the plane intercept. A perpendicular line is drawn from the initial calibration point to the fitting reference plane; the foot of the perpendicular is the projection point. The calculation process for the projection point coordinates is as follows: First, calculate the position vector difference between the initial calibration point and any point on the fitting reference plane. Then, project this vector difference onto the direction of the normal vector to obtain a vertical distance scalar, which reflects the directed distance from the initial calibration point to the reference plane. Finally, subtract the product of the normal vector and the vertical distance scalar from the coordinates of the initial calibration point to obtain the coordinates of the projection point. This projection point is considered the true reference point for the initial calibration point.

[0030] Step 125: Calculate the vector difference between the coordinates of the preliminary correction point and the coordinates of the reference true point to obtain the volume error vector corresponding to the preliminary correction point.

[0031] The reference true point coordinates are considered as the actual spatial position, and the preliminary calibration point coordinates are considered as the measurement point position with errors. The vector pointing from the reference true point to the preliminary calibration point is the volume error vector. This vector contains the direction and magnitude of the deviation of the preliminary calibration point from the true position in three-dimensional space. The three components of the volume error vector correspond to the deviation in the three coordinate axes of the world coordinate system.

[0032] Step 126: Pair the coordinates of the preliminary correction point with the volume error vector and store them to generate a single-point training record.

[0033] Each training record contains two parts: the first part is the three-dimensional coordinates of the initial calibration point, and the second part is the three components of the volume error vector corresponding to that point. The training records are stored in a structured data format for easy reading and processing in subsequent steps.

[0034] Step 127: Traverse all preliminary correction points in the point cloud subset to form the training record group corresponding to the point cloud subset.

[0035] For each point in the point cloud subset under the current pose, the above projection calculation and vector difference calculation are performed. All the training records obtained are collected into the training record group corresponding to the pose. Each record in the training record group is associated with the same pose identifier.

[0036] Step 128: Summarize the training record groups corresponding to all point cloud subsets to obtain the volume error training data set.

[0037] If M planar data points of pose number are collected, and N sampling points are collected for each pose, then the total volume error training dataset contains M multiplied by N training records. Each training record stores the coordinates of the initial correction point and its corresponding volume error vector. This training dataset is used for parameter estimation of the subsequent error model. During the aggregation process, all training records are merged into a unified training dataset, regardless of the source pose.

[0038] Step 130: The volume error in the volume error training dataset is decomposed according to the camera imaging geometry and projector imaging geometry in the structured light system. The volume error is decomposed into depth direction error component, camera side radial error component and projector side radial error component. The depth direction error component is along the object point direction in world coordinates, the camera side radial error component is along the camera radial direction, and the projector side radial error component is along the projector radial direction.

[0039] Volumetric errors are not randomly distributed; their generation is closely related to the imaging geometry of the camera and projector. Errors related to triangulation depth variations are categorized into depth direction error components, errors related to the radial position of the camera image plane are categorized into camera-side radial error components, and errors related to the radial position of the projector image plane are categorized into projector-side radial error components. By decomposing volumetric errors into directions with clear geometric and physical meaning, the subsequently established error model can have lower complexity and better interpretability. During the decomposition process, the magnitude of the volumetric error vector is assigned to three directions with different orthogonality, and the component scalar in each direction reflects the magnitude of the error contribution in that direction.

[0040] Step 131: Obtain the coordinates of the calibrated camera optical center in the world coordinate system, and obtain the coordinates of the calibrated projector optical center in the world coordinate system.

[0041] The optical centers of the camera and projector were determined during the system calibration phase, and their coordinates are stored as camera optical center coordinate vectors and projector optical center coordinate vectors, respectively. These two optical center coordinates serve as the fundamental reference points for all subsequent direction vector calculations.

[0042] Step 132: Read each training record one by one from the volume error training data set, and extract the coordinates of the preliminary correction point and the volume error vector from the training record.

[0043] Iterate through all records in the training dataset and perform steps 133 to 139 on each record. During processing, the processing result of each record is independent of the other records, which facilitates parallel execution.

[0044] Step 133: Based on the camera optical center coordinates and the coordinates of the preliminary correction point, calculate the unit vector pointing from the camera optical center to the preliminary correction point, which is used as the unit vector corresponding to the object point direction in world coordinates.

[0045] First, calculate the direction vector from the camera's optical center to the initial correction point. Specifically, subtract the camera's optical center coordinates from the coordinates of the initial correction point to obtain the unnormalized direction vector. Calculate the magnitude of this direction vector and divide each component of the direction vector by the magnitude to obtain a unit vector with a magnitude of 1. This unit vector represents the line-of-sight direction of the camera observing this spatial point.

[0046] Step 134: Based on the coordinates of the projector's optical center and the coordinates of the preliminary correction point, calculate the unit vector pointing from the projector's optical center to the preliminary correction point, and use it as the unit vector of the projector's observation direction.

[0047] Using the same method as in step 133, the direction vector pointing to the preliminary correction point is calculated using the coordinate position of the projector's optical center and normalized to obtain the unit vector of the projector's observation direction. This unit vector represents the direction of the projector's projected light at that spatial point.

[0048] Step 135: Project the volume error vector onto the unit vector corresponding to the object point direction in world coordinates to obtain the scalar of the depth direction error component.

[0049] Projection operations are performed by calculating the dot product of the volume error vector and the unit vector corresponding to the object point's direction in world coordinates. The dot product operation involves multiplying each of the three components of the volume error vector by the corresponding component of the unit vector corresponding to the object point's direction in world coordinates and then summing the results. This dot product value represents the magnitude of the volume error component along the camera's line of sight, with positive or negative values ​​indicating whether the error deviates outward or inward along the line of sight.

[0050] Step 136: Determine the radial reference direction of the camera based on the coordinates of the camera's optical center and the principal point of the camera's imaging plane, and calculate the unit vector of the camera's radial direction by combining the coordinates of the preliminary correction points.

[0051] The camera radial direction refers to the direction from the intersection of the camera's optical axis and the imaging plane to a point on the imaging plane. For a preliminary correction point in space, it is first projected onto the camera's normalized image plane using the camera projection matrix, obtaining normalized image coordinates. The direction from the camera's principal point to these normalized image coordinates is the camera radial direction at that point. Normalizing the vector representing this direction by dividing it by its magnitude yields the unit vector of the camera radial direction.

[0052] Step 137: Project the volume error vector onto the unit vector in the radial direction of the camera to obtain the scalar of the radial error component on the camera side.

[0053] The projection operation is achieved by calculating the dot product of the volume error vector and the unit vector in the radial direction of the camera. This dot product value represents the magnitude of the volume error component in the radial direction of the camera image plane, reflecting the error contribution caused by the lens residual radial distortion on the camera side.

[0054] Step 138: Determine the radial reference direction of the projector based on the coordinates of the projector's optical center and the principal point of the projector's imaging plane, and calculate the unit vector of the projector's radial direction in conjunction with the coordinates of the preliminary calibration point.

[0055] Using the same method as in step 136, the preliminary correction point is projected onto the normalized image plane of the projector. The direction from the principal point of the projector to the coordinates of the normalized image is the radial direction of the projector at that point. The radial direction vector of the projector is normalized to obtain the unit vector of the radial direction of the projector.

[0056] Step 139: Project the volume error vector onto the unit vector in the radial direction of the projector to obtain the scalar of the radial error component on the projector side.

[0057] Projection operations are performed by calculating the dot product of the volume error vector and the projector's radial unit vector. This dot product value represents the magnitude of the volume error component in the radial direction of the projector's image plane. For each training record, the calculated scalar values ​​of the depth direction error component, the camera-side radial error component, and the projector-side radial error component are associated with the coordinates of the initial correction point and stored to form a direction decomposition record. The direction decomposition record contains the three-dimensional coordinates of the initial correction point, the scalar values ​​of the depth direction error component, the camera-side radial error component, and the projector-side radial error component. After the above decomposition process, the original volume error vector is decomposed into three scalar components with clear geometric meanings. Each component corresponds to a different error source direction, providing a data foundation for subsequently establishing an anisotropic error model.

[0058] Step 140: Establish a depth direction error model and a radial direction error model. The depth direction error model uses the depth direction error component as the fitting target and combines the depth information and radial distance information of the preliminary correction point to construct a feature vector. The radial direction error model uses the radial error component on the camera side and the radial error component on the projector side as the fitting target and combines their respective radial distances to construct a feature vector.

[0059] Depth orientation error is directly related to the principle of triangulation. In structured light triangulation, the depth of a spatial point is determined by the baseline distance between the camera and the projector, the system focal length, and the parallax. Differentiating the depth expression reveals a positive correlation between the depth error and the square of the depth term; therefore, the depth error increases non-linearly with the increase of the spatial point depth. Simultaneously, the depth error is also affected by the radial position of the camera and projector, as residual radial distortion introduces pixel coordinate errors, which are transmitted to the depth dimension through triangulation. The radial orientation error is mainly caused by residual radial distortion from the lens, and the error is more significant when the radial distance is large. Therefore, it is necessary to establish separate models for depth orientation error and radial orientation error, using different eigenvector construction methods for each model to adapt to their respective error physical characteristics.

[0060] Step 141: Extract the depth direction error component scalar from the direction decomposition record for each preliminary correction point, and use it as the target value for depth model fitting.

[0061] The scalar component of the depth direction error has been calculated in step 139 and is used as the dependent variable of the depth direction error model. The value of this scalar may be positive or negative, and the positive or negative sign indicates the positive or negative deviation of the error along the camera's line of sight.

[0062] Step 142: For each preliminary correction point, determine its depth value based on its coordinates in the camera coordinate system, and calculate the depth squared term based on the depth value to obtain the depth squared feature quantity.

[0063] The coordinates of the initial calibration point in the world coordinate system are transformed to the camera coordinate system using the rotation matrix and translation vector in the camera extrinsic parameter matrix. Specifically, the world coordinates are multiplied by the rotation matrix, and then the translation vector is added to obtain the coordinates in the camera coordinate system. The depth component in the camera coordinate system is extracted, which is the depth value of the point. Multiplying this depth value by itself yields the squared depth term, a feature that describes the nonlinear trend of the depth error increasing rapidly with increasing depth.

[0064] Step 143: Obtain the camera-side radial distance corresponding to the preliminary correction point. The camera-side radial distance is defined as the radial distance from the projection point of the point on the camera imaging plane to the principal point.

[0065] The initial correction points are projected onto the camera's normalized image plane to obtain normalized image coordinates. The Euclidean distance from these coordinates to the principal point is then calculated. The radial distance is calculated as follows: the difference between the normalized image coordinates and the principal point coordinates along the horizontal and vertical axes is calculated. The squares of these two differences are then added together, and the square root is taken to obtain the camera's radial distance.

[0066] Step 144: Obtain the projector-side radial distance corresponding to the preliminary correction point. The projector-side radial distance is defined as the radial distance from the projection point to the principal point on the projector imaging plane.

[0067] Project the initial correction point onto the normalized image plane of the projector to obtain the normalized image coordinates. Calculate the Euclidean distance from these coordinates to the principal point in the same way as in step 143 to obtain the radial distance to the projector side.

[0068] Step 145: Calculate the cubic term of the radial distance on the camera side as the first radial feature quantity of the camera side, calculate the fifth term of the radial distance on the camera side as the second radial feature quantity of the camera side, calculate the cubic term of the radial distance on the projector side as the first radial feature quantity of the projector side, and calculate the fifth term of the radial distance on the projector side as the second radial feature quantity of the projector side.

[0069] The cubic term of the radial distance is obtained by multiplying the radial distance itself three times, and the fifth term is obtained by multiplying the radial distance itself five times. These two higher-order terms correspond to the third and fifth-order terms of radial distortion in the Brown-Conrady model, respectively, and are used to describe the nonlinear change in radial distortion that intensifies with increasing radial distance. The combination of cubic and fifth-order terms can cover the residual error of radial distortion from low to high orders.

[0070] Step 146: Construct the feature vector of the depth error feature vector, which includes the constant term, depth value, depth square feature quantity and the product of the first camera radial feature quantity, the feature quantity of the product of the depth square feature quantity and the second camera radial feature quantity, the feature quantity of the product of the depth square feature quantity and the first projector radial feature quantity, and the feature vector of the product of the depth square feature quantity and the second projector radial feature quantity.

[0071] Each component in this depth error feature vector serves as an independent variable for the subsequent depth orientation error model. The feature vector has six dimensions, with constant terms that are always fixed and used as bias terms in the model. All feature components are scalars, and their combination forms an ordered feature vector structure.

[0072] Step 147: The depth direction error component scalar is defined as being represented by a linear weighted combination of the depth error feature vector and the depth model parameter vector to be solved, thereby establishing the depth direction error model.

[0073] The depth orientation error scalar is equal to the sum of the products of each component in the depth error eigenvector and its corresponding depth model parameter. Specifically, the depth model parameter vector contains six parameters to be solved, corresponding to the constant term, the depth value, the eigenvalue of the product of the squared depth feature and the radial feature of the first camera, the eigenvalue of the product of the squared depth feature and the radial feature of the second camera, the eigenvalue of the product of the squared depth feature and the radial feature of the first projector, and the weighted coefficients of the eigenvalues ​​of the depth error eigenvector of the product of the squared depth feature and the radial feature of the second projector. The goal of the depth orientation error model is to determine the optimal values ​​of these six parameters using training data, so that the scalar values ​​of the depth orientation error components predicted by the model are as close as possible to the scalar values ​​of the depth orientation error components actually calculated.

[0074] For the radial direction error model, the scalar of the camera-side radial error component corresponding to each preliminary correction point is extracted from the direction decomposition record and used as the target value for fitting the camera radial model. The camera-side radial distance corresponding to this preliminary correction point is obtained. The cubic term of the camera-side radial distance is calculated to construct the first feature term of the camera radial model, and the fifth term of the camera-side radial distance is calculated to construct the second feature term of the camera radial model. The scalar of the camera-side radial error component is defined as the product of the first feature term of the camera radial model and the first coefficient, plus the product of the second feature term of the camera radial model and the second coefficient. The first coefficient and the second coefficient constitute the camera-side radial model parameters. This linear combination is derived from the radial distortion part of the Brown-Conrady model.

[0075] Similarly, the projector-side radial error component scalar is extracted from the direction decomposition record for each preliminary correction point, serving as the target value for fitting the projector radial model, and the projector-side radial distance corresponding to that preliminary correction point is obtained. The cubic term of the projector-side radial distance is calculated to construct the first characteristic term of the projector radial model, and the fifth term is calculated to construct the second characteristic term. The projector-side radial error component scalar is defined as the product of the first characteristic term and the third coefficient, plus the product of the second characteristic term and the fourth coefficient. The third and fourth coefficients constitute the projector-side radial model parameters, and this linear combination also does not contain a constant term.

[0076] Step 150: The regularized regression method is used to estimate the model parameters of the depth direction error model and the radial direction error model. The estimated model parameters are used to predict the residual volume error of the points in the preliminary correction point cloud. The predicted residual volume error is superimposed on the coordinates of the corresponding preliminary correction points to obtain the final correction point cloud.

[0077] Because the feature vector of the depth orientation error model contains depth values, depth squared terms, and multiple higher-order radial distance terms, there is strong multicollinearity among the features. Directly applying ordinary least squares may lead to unstable parameter estimation and large variance. Regularized regression methods, by introducing a penalty term for the parameter vector into the loss function, can effectively reduce the impact of multicollinearity on parameter estimation and improve the stability and generalization ability of model parameters. This embodiment uses ridge regression for model parameter estimation. Ridge regression adds an L2 regularization term to the least squares loss, that is, it adds the sum of squares of all model parameters multiplied by a regularization coefficient to the optimization objective.

[0078] Step 151: Use the ridge regression method to estimate the parameters of the depth direction error model, and solve the depth model parameter vector by introducing a regularization term.

[0079] The optimization objective of ridge regression is to minimize the sum of squared residuals between the actual values ​​of the depth orientation error scalar and the model predictions, plus the sum of the regularization term. For a dataset containing all training samples, a feature matrix is ​​constructed, with each row corresponding to the depth error feature vector of a single sample. A target value vector is also constructed, with each row corresponding to the depth orientation error scalar of a single sample. The solution for ridge regression is as follows: First, calculate the product of the transpose of the feature matrix and the feature matrix, add the product of the regularization coefficient and the identity matrix, then invert the matrix and multiply it by the product of the transpose of the feature matrix and the target value vector to obtain the depth model parameter vector. The regularization coefficient is determined through cross-validation, selecting the coefficient value that minimizes the validation set error from a pre-defined set of candidate coefficients.

[0080] Step 152: Use the ridge regression method to estimate the parameters of the camera-side radial error model and obtain the parameter vector of the camera-side radial model.

[0081] Each row of the feature matrix of the camera-side radial model contains two elements: a first feature term and a second feature term. The target value vector is the scalar of the camera-side radial error component. The first and second coefficients in the camera-side radial model are estimated using the same ridge regression solution method as in step 151.

[0082] Step 153: Use the ridge regression method to estimate the parameters of the projector-side radial error model and obtain the parameter vector of the projector-side radial model.

[0083] Each row of the feature matrix of the projector-side radial model contains two elements: the first radial feature term and the second radial feature term. The objective value vector is the scalar of the projector-side radial error component. The third and fourth coefficients in the projector-side radial model are solved using ridge regression.

[0084] Step 154: For each point to be corrected in the preliminary correction point cloud, determine the depth value corresponding to that point.

[0085] The world coordinates of the point to be corrected are transformed to the camera coordinate system using the camera extrinsic parameter matrix. The depth component in the camera coordinate system is then extracted. This depth value is used for the subsequent calculation of the depth direction error prediction scalar.

[0086] Step 155: Calculate the camera-side radial distance and the projector-side radial distance corresponding to the point to be corrected.

[0087] The points to be corrected are projected onto the camera's normalized image plane and the projector's normalized image plane, respectively. The Euclidean distance from each projection point to the corresponding principal point is calculated to obtain the camera's side radial distance and the projector's side radial distance.

[0088] Step 156: Calculate the depth direction error prediction scalar based on the depth model parameter vector, the depth value of the point to be corrected, the depth square term, the cubic and quintic terms of the radial distance on the camera side, and the cubic and quintic terms of the radial distance on the projector side.

[0089] The depth value of the point to be corrected, the squared depth term, the cubic and quintic terms of the radial distance from the camera side, the cubic and quintic terms of the radial distance from the projector side, and the constant term are arranged in the order described in step 146 to form a feature vector. Each feature component is multiplied by the corresponding parameter in the depth model parameter vector estimated in step 151, and then all products are added together. The resulting scalar is the depth direction error prediction scalar.

[0090] Step 157: Calculate the camera-side radial error prediction scalar based on the camera-side radial model parameter vector and the cubic and quintic terms of the camera-side radial distance to the point to be corrected.

[0091] Multiply the cubic term of the camera-side radial distance to be corrected by the first coefficient estimated in step 152, multiply the fifth term of the camera-side radial distance by the second coefficient, and add the two products to obtain the camera-side radial error prediction scalar.

[0092] Step 158: Calculate the projector-side radial error prediction scalar based on the projector-side radial model parameter vector and the cubic and quintic terms of the projector-side radial distance of the point to be corrected.

[0093] Multiply the cubic term of the projector-side radial distance of the point to be corrected by the third coefficient estimated in step 153, multiply the fifth term of the projector-side radial distance by the fourth coefficient, and add the two products to obtain the projector-side radial error prediction scalar.

[0094] Step 159: Based on the unit vector corresponding to the object point direction in the world coordinates of the point to be corrected, the unit vector of the camera radial direction, and the unit vector of the projector radial direction, the depth direction error prediction scalar, the camera side radial error prediction scalar, and the projector side radial error prediction scalar are combined into a predicted residual volume error vector.

[0095] The synthesis process follows the inverse of error decomposition. First, the depth direction error prediction scalar is scalar-multiplied by the unit vector corresponding to the object point's direction in world coordinates, i.e., the scalar is multiplied by each component of the unit vector to obtain the depth direction error component vector. Then, the camera-side radial error prediction scalar is scalar-multiplied by the camera-side radial direction unit vector to obtain the camera-side radial error component vector. Next, the projector-side radial error prediction scalar is scalar-multiplied by the projector-side radial direction unit vector to obtain the projector-side radial error component vector. Finally, the three component vectors are added together, i.e., the corresponding components of the three vectors are added separately, to obtain a three-dimensional composite vector. This three-dimensional composite vector is the predicted residual volume error vector for the point to be corrected.

[0096] Step 160: Perform vector addition on the predicted residual volume error vector and the coordinates of the point to be corrected to obtain the corrected point coordinates.

[0097] The compensated three-dimensional coordinates are obtained by adding the component corresponding to the predicted residual volume error vector to each component of the original three-dimensional coordinates of the point to be corrected.

[0098] Step 161: Traverse all points in the preliminary correction point cloud and perform the operations from Step 154 ​​to Step 160 for each point. After all points are compensated, the final correction point cloud is obtained.

[0099] The residual volume error in the final corrected point cloud is significantly suppressed. In typical industrial structured light measurement applications, when the above method is used to measure a planar calibration plate, the root mean square error of the final corrected point cloud relative to the fitted reference plane is reduced by more than an order of magnitude compared to the initial corrected point cloud, and the error distribution is more uniform. The error consistency between the edge region and the center region is significantly improved, and the accuracy of 3D reconstruction is enhanced.

[0100] In a preferred embodiment of this application, the method further includes the following steps to evaluate the reconstruction performance of the model and optimize the subsequent correction strategy: After obtaining the final corrected point cloud, points located within the calibration plate plane region are extracted. Plane fitting is performed on these points, and the root mean square error and mean absolute error of the residuals from each point to the fitted plane are calculated. These two indicators are used as parameters for evaluating reconstruction accuracy. Simultaneously, the corresponding evaluation indicators of the final corrected point cloud and the initial corrected point cloud are compared, and the improvement ratio of each indicator is calculated to generate correction effectiveness evaluation parameters. Based on the correction effectiveness evaluation parameters, it is determined whether the compensation effect of the current volume error model reaches the preset effectiveness threshold. If it does not, the regularization coefficient of the ridge regression is adjusted, and the model parameters are re-estimated. This process enables adaptive optimization of the error model.

[0101] In another optional embodiment, the method further includes verifying the geometric dimensions of the reconstructed 3D point cloud: extracting a subset of point clouds containing standard geometries of known dimensions from the final corrected point cloud, fitting key dimension parameters of the geometries, comparing the fitted dimension parameters with the nominal dimensions of the standard geometries, and calculating the dimension deviation. This dimension deviation is then compared with the corresponding dimension deviation extracted from the preliminary corrected point cloud to generate dimension accuracy improvement features. Based on the dimension accuracy improvement features, the consistency of compensation for the volume error model in the edge and center regions is determined. If the dimension deviation improvement ratio in the edge region is lower than a specified ratio in the center region, it is determined that the complexity of the radial direction error model needs to be increased. In this case, the tangential distortion portion of the Brown-Conrady model can also be included in the modeling considerations to improve accuracy.

[0102] In another optional embodiment, the method further includes statistical analysis of the predicted residuals of the volume error model: In the final corrected point cloud, the actual volume error of each corrected point is calculated, defined as the vector difference between the coordinates of the corrected point and the coordinates of the reference true point. The actual volume error of the corrected point is compared with the predicted residual volume error vector predicted by the model to calculate the predicted residual vector. The mean and standard deviation of the predicted residual vectors of all corrected points are statistically analyzed to generate residual distribution features. If the mean residual value in a certain direction of the residual distribution features exceeds a preset residual mean threshold, it is determined that there is a systematic bias in the error model in that direction, and the error model parameters corresponding to that direction are biased and corrected. This process enables fine-tuning of the error model.

[0103] In another optional embodiment, the method further includes generating a three-dimensional topographic model of the object under test using the final calibrated point cloud: the points in the final calibrated point cloud are meshed according to their spatial coordinates, and a triangulation algorithm is used to connect the point cloud into a triangular mesh surface to generate a three-dimensional digital model of the object under test. Geometric measurement parameters such as volume, surface area, flatness, and roundness of the object under test are calculated based on the three-dimensional digital model. These geometric measurement parameters are compared with the corresponding parameters generated using the preliminary calibrated point cloud to calculate the improvement margin of each measurement parameter. Based on the improvement margin of each measurement parameter, task-adaptive recommendation parameters for different measurement tasks are generated. These task-adaptive recommendation parameters include recommended error model configurations for different measurement task types. This process can provide differentiated calibration strategy configurations for different measurement tasks.

[0104] In another preferred embodiment, the method further includes multi-view fusion processing of the final corrected point cloud: acquiring preliminary corrected point clouds collected from multiple different viewpoints, performing volume error prediction and compensation on the preliminary corrected point cloud for each viewpoint to obtain the final corrected point cloud corresponding to each viewpoint. The final corrected point clouds from each viewpoint are registered and fused using an iterative nearest-point algorithm to generate a multi-view fused point cloud. Point pairs in overlapping areas of the multi-view fused point cloud are extracted, and the root mean square value of the spatial distance between point pairs is calculated as a multi-view consistency evaluation parameter. The multi-view consistency evaluation parameter is compared with the corresponding evaluation parameter obtained by direct registration of the preliminary corrected point cloud to generate consistency improvement features. The confidence weights of the error model parameters for each viewpoint are determined based on the consistency improvement features, and the training data volume is increased for views with low confidence. This process can improve the global consistency of 3D reconstruction in multi-view measurement scenarios.

[0105] The complete workflow of this application embodiment is described below through a specific application scenario. In an industrial online inspection scenario, a structured light system is installed at the end effector of a robotic arm to perform 3D topographic measurement of workpieces on a production line. The system first uses a ceramic planar calibration plate to calibrate the camera and projector, obtaining the intrinsic parameter matrix, extrinsic parameter matrix, and Brown-Conrady distortion parameters. Then, the system acquires data from the calibration plate in twelve different spatial orientations, each with a different pitch and yaw angle relative to the system. The system performs fringe projection, image acquisition, phase decoding, and 3D reconstruction on the calibration plate surface in each orientation, obtaining twelve sets of preliminary correction point clouds. Each point cloud contains approximately 100,000 3D points. The system performs least-squares plane fitting on each point cloud to determine the reference plane corresponding to each orientation, projects each 3D point onto the reference plane to obtain the reference ground truth point, and calculates the volume error vector. All points from the twelve point clouds are aggregated to construct a volume error training dataset containing 1.2 million training records.

[0106] The system reads the camera and projector optical center coordinates from the calibration data, iterates through each record in the training dataset, and calculates the unit vectors corresponding to the object point direction, camera radial direction, projector observation direction, and projector radial direction in world coordinates. It then projects the volume error vector onto these four directions to obtain the scalar components of the depth direction error, camera-side radial error, and projector-side radial error. The system uses all training records to build a depth direction error model. The model's feature vector includes a constant term, depth value, depth squared term, and cubic and quintic terms of the camera-side radial distance, as well as cubic and quintic terms of the projector-side radial distance. Simultaneously, a camera-side radial error model and a projector-side radial error model are established. The former includes cubic and quintic terms of the camera-side radial distance, and the latter includes cubic and quintic terms of the projector-side radial distance. The system uses ridge regression to solve for the parameter vectors of the three models, and the regularization coefficients are determined to preset values ​​through cross-validation.

[0107] In actual workpiece measurement, the system first completes fringe projection, image acquisition, and preliminary distortion correction to obtain a preliminary corrected point cloud of the workpiece. For each 3D point in the preliminary corrected point cloud, the system calculates the depth value, camera-side radial distance, and projector-side radial distance of that point. It then calculates the depth direction error prediction scalar using the trained depth model parameter vector, the camera-side radial error prediction scalar using the camera-side radial model parameter vector, and the projector-side radial error prediction scalar using the projector-side radial model parameter vector. Finally, the system synthesizes the three prediction scalars into a predicted residual volume error vector based on the geometric direction unit vector of that point. This vector is then superimposed onto the original coordinates to obtain the corrected 3D point. After processing all points, the final corrected point cloud is generated. Experimental results show that after the above compensation processing, the root mean square error of planar measurement is reduced to a significantly smaller value from the uncorrected state, and the measurement accuracy is significantly improved.

[0108] Based on the above content, please refer to the following: Figures 2-4 ,Depend on Figure 2 As can be seen, the lens distortion joint correction method of this application is generally divided into two major processing stages: model construction and point cloud compensation. The model construction stage takes multi-pose data acquisition as the input end, and obtains a preliminary corrected point cloud through initial lens distortion correction and point cloud reconstruction. Then, the sampling points on each pose plane are extracted by the plane fitting and sampling module, and the volume error vector is generated by the error construction module. Then, the volume error is decomposed into three directional components: depth direction, camera side radial direction, and projector side radial direction by the error decomposition module. Finally, the volume error model is obtained by the parameter estimation module. The point cloud compensation stage takes the reconstructed and corrected point cloud to be corrected as input. First, the direction calculation module obtains the unit vectors of depth direction, camera-side radial direction, and projector-side radial direction for each point. Then, the input is fed into the model application module to calculate the error in combination with the trained volume error model. The error prediction module outputs the error prediction values ​​for the three types of directions. Finally, the compensation module superimposes the predicted residual volume error onto the original point cloud coordinates and outputs the final corrected point cloud. This flowchart clearly presents the complete technical link from training data construction, error direction decomposition modeling to point-by-point compensation of the point cloud, and intuitively demonstrates the two-stage processing logic of the volume error decomposition correction method and the data flow relationship between each module.

[0109] Furthermore, by Figure 3It can be seen that the volume error direction decomposition is carried out independently based on the imaging geometry of the camera and the projector. In the left-hand camera-side diagram, the light rays emitted from the camera's optical center intersect at a spatial point via the distorted and undistorted projection points on the camera's imaging plane. The volume error vector at this spatial point is decomposed into a depth error component along the object point direction in world coordinates and into a radial error component along the radial direction of the camera's imaging plane. The depth component is parallel to the object point direction in world coordinates, reflecting the depth deviation along the line of sight of the camera's optical axis, while the radial component is perpendicular to the object point direction in world coordinates, reflecting the lateral offset caused by radial distortion within the imaging plane. The right-hand projector-side diagram uses a symmetrical decomposition logic. The light rays emitted from the projector's optical center intersect at the same spatial point via the distorted and undistorted projection points on the projector's imaging plane. The volume error vector is decomposed into a depth error component along the projector's light ray direction and into a radial error component along the radial direction of the projector's imaging plane. The schematic diagrams on both sides show that the residual volume error in three-dimensional space can be decomposed into two components, depth and radial, based on the imaging geometry of the camera and projector, providing a geometric basis for establishing depth error models and radial error models respectively.

[0110] In addition, by Figure 4 As can be seen, the planar depth direction error goes through three stages: uncorrected, traditional preliminary distortion correction, and volume error decomposition and joint correction of this application. It shows a progressively converging improvement trend, clearly highlighting the core value of the volume error decomposition and compensation step in correcting residual errors. In the uncorrected state on the left, the error heatmap color scale range is -4mm to 4mm. The error distribution shows a significant non-uniformity with low error at the center and high error at the edges. The error in the central region of the field of view is close to 0mm, while the error amplitude in the edge region can reach ±4mm. The overall error fluctuation is large, and the consistency of the measurement across the entire field of view is poor. In the intermediate preliminary correction stage, after traditional lens distortion correction, the error heatmap color scale range converges to -2.5mm to 2.5mm. The overall error amplitude is reduced, but there are still obvious systematic errors in the field of view. The problem of high error in the edge region has not been effectively solved, and the correction efficiency has a significant bottleneck. In the final correction result after volume error decomposition and compensation on the right, the error heatmap color scale range further converges to -1mm to 1mm. The error amplitude is significantly reduced across the entire field of view, the error difference between the center and the edge is greatly reduced, and the error distribution tends to be uniform and smooth. The significant reduction and uniformity of error distribution from preliminary to final correction fully demonstrate that the volume error decomposition correction step can effectively break through the accuracy limit of traditional preliminary correction, strongly compensate for residual volume errors in 3D reconstruction, and significantly improve the full field-of-view measurement accuracy and point cloud geometric consistency of the structured light system.

[0111] In summary, this application achieves a systematic leap in accuracy from the initial corrected point cloud to the final corrected point cloud by constructing a volumetric error direction decomposition and parameterized compensation mechanism based on imaging geometric constraints. This scheme does not empirically fit the residual error as a whole, but instead uses multi-pose planar data to explicitly decompose the volumetric error into depth, camera-side radial, and projector-side radial components, making the error model directly correspond to the physical imaging process of the structured light system. Based on this, eigenvectors with clear geometric meaning are constructed for each directional component, and a parameterized error model is established. Combined with regularized regression methods, the model parameters are stably estimated, effectively avoiding parameter inflation and overfitting problems caused by high-order polynomial global fitting. Finally, the initial corrected point cloud is compensated point-by-point by predicting the residual volumetric error, ensuring consistent correction of the 3D reconstructed point cloud in both the depth and radial directions, significantly improving the full-field measurement accuracy and point cloud geometric consistency.

[0112] See Figure 5 As shown in the figure, this is a schematic diagram of the basic structure of a structured light distortion correction system 20 based on volume error decomposition provided in an embodiment of this application. The structured light distortion correction system 20 based on volume error decomposition includes: Processor 201; Storage device 202, on which computer program 2020 is stored; When the computer program 2020 is executed by the processor 201, the processor 201 implements any of the structured light distortion correction methods based on volume error decomposition.

[0113] Based on the above, a readable storage medium is provided, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the above method are implemented.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes: USB flash drive, portable hard drive, read-only memory (ROM). Various media that can store program code, such as ROM (ROM only memory), RAM (random access memory), magnetic disks, or optical disks.

[0118] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A structured light distortion correction method based on volume error decomposition, characterized in that, include: A preliminary correction point cloud is obtained, which is obtained by the structured light system performing three-dimensional reconstruction of the object under test after completing the preliminary distortion correction of the camera lens and the projector lens; A reference plane is fitted using preliminary correction point clouds under multiple attitude planes. The corresponding point of each preliminary correction point on the reference plane is determined as the reference real point. A volume error is generated based on the deviation between the preliminary correction point and the reference real point, and a volume error training data set is generated. The volume error training data set records the preliminary correction point and the volume error corresponding to the preliminary correction point. The volume error in the volume error training dataset is decomposed according to the camera imaging geometry and projector imaging geometry in the structured light system. The volume error is decomposed into depth direction error component, camera side radial error component and projector side radial error component. The depth direction error component is along the object point direction in world coordinates, the camera side radial error component is along the camera radial direction, and the projector side radial error component is along the projector radial direction. A depth direction error model and a radial direction error model are established. The depth direction error model uses the depth direction error component as the fitting target and combines the depth information and radial distance information of the preliminary correction point to construct a feature vector. The radial direction error model uses the radial error component on the camera side and the radial error component on the projector side as the fitting target and combines their respective radial distances to construct a feature vector. The model parameters of the depth direction error model and the radial direction error model are estimated using a regularized regression method. The estimated model parameters are then used to predict the residual volume error of points in the preliminary correction point cloud. The predicted residual volume error is then superimposed onto the coordinates of the corresponding preliminary correction points to obtain the final correction point cloud.

2. The method according to claim 1, characterized in that, The process involves fitting a reference plane using preliminary correction point clouds across multiple attitude planes, determining the corresponding point on the reference plane for each preliminary correction point as a reference ground truth point, generating a volume error based on the deviation between the preliminary correction points and the reference ground truth points, and generating a volume error training dataset, including: Based on the acquisition attitude identifier associated with each point in the preliminary correction point cloud, the preliminary correction point cloud is divided into multiple point cloud subsets corresponding to different attitude planes; For each subset of point cloud, extract the three-dimensional coordinates of all preliminary correction points in that subset of point cloud to form a coordinate matrix; By fitting the spatial plane equation with a coordinate matrix, a fitting reference plane under the attitude plane is obtained, and the fitting reference plane is used as the geometric reference plane corresponding to each preliminary correction point in the point cloud subset. For each preliminary correction point in the cloud subset, calculate the coordinates of its projection point on the fitting reference plane along the direction of the normal vector of the fitting reference plane, and determine the coordinates of the projection point as the reference true point coordinates of the preliminary correction point. Calculate the vector difference between the coordinates of the preliminary correction point and the coordinates of the reference true point to obtain the volume error vector corresponding to the preliminary correction point; The coordinates of the initial correction point are paired and stored with the volume error vector to generate a single-point training record; Traverse all preliminary correction points in the point cloud subset to form the training record group corresponding to the point cloud subset; By summing up the training record groups corresponding to all point cloud subsets, a volume error training data set is obtained. The volume error training data set consists of the coordinates of all preliminary correction points and their corresponding volume error vectors.

3. The method according to claim 1, characterized in that, The volume errors in the volume error training dataset are decomposed directionally based on the camera imaging geometry and projector imaging geometry in the structured light system, decomposing the volume errors into depth direction error components, camera-side radial error components, and projector-side radial error components, including: Obtain the coordinates of the calibrated camera optical center in the world coordinate system; obtain the coordinates of the calibrated projector optical center in the world coordinate system. Read each training record one by one from the volume error training dataset and extract the coordinates of the preliminary correction point and the volume error vector from the training record; Based on the camera optical center coordinates and the coordinates of the initial correction point, calculate the unit vector from the camera optical center to the initial correction point, which is used as the unit vector corresponding to the object point direction in world coordinates. Based on the coordinates of the projector's optical center and the coordinates of the preliminary correction point, calculate the unit vector pointing from the projector's optical center to the preliminary correction point, and use it as the unit vector of the projector's observation direction. Projecting the volume error vector onto the unit vector corresponding to the object point direction in world coordinates yields the scalar of the depth direction error component. Based on the coordinates of the camera's optical center and the principal point of the camera's imaging plane, the radial reference direction of the camera is determined, and the unit vector of the camera's radial direction is calculated in conjunction with the coordinates of the preliminary correction points. Projecting the volume error vector onto a unit vector in the camera's radial direction yields a scalar of the camera-side radial error component. Based on the coordinates of the projector's optical center and the principal point of the projector's imaging plane, the radial reference direction of the projector is determined, and the unit vector of the projector's radial direction is calculated in conjunction with the coordinates of the preliminary calibration points. Projecting the volume error vector onto a unit vector in the radial direction of the projector yields a scalar of the radial error component on the projector side. The scalar values ​​of the depth direction error component, the scalar values ​​of the camera-side radial error component, and the scalar values ​​of the projector-side radial error component are associated with and stored with the coordinates of the initial correction point to obtain the direction decomposition record.

4. The method according to claim 1, characterized in that, The establishment of the depth direction error model and the radial direction error model includes: Extract the scalar of the depth direction error component corresponding to each preliminary correction point from the direction decomposition record, and use it as the target value for the depth model fitting. For each initial correction point, its depth value is determined based on its coordinates in the camera coordinate system, and the depth squared term is calculated based on the depth value to obtain the depth squared feature. Obtain the camera-side radial distance corresponding to the preliminary correction point, whereby the camera-side radial distance is defined as the radial distance from the projection point of the point on the camera imaging plane to the principal point. Obtain the projector-side radial distance corresponding to the preliminary correction point. The projector-side radial distance is defined as the radial distance from the projection point of this point to the principal point on the projector imaging plane. The cubic term of the radial distance on the camera side is used as the first radial feature of the camera, the fifth term of the radial distance on the camera side is used as the second radial feature of the camera, the cubic term of the radial distance on the projector side is used as the first radial feature of the projector, and the fifth term of the radial distance on the projector side is used as the second radial feature of the projector. Construct the feature vector of depth error eigenvectors, which includes constant terms, depth values, depth squared feature quantities and the product of the first camera radial feature quantities, the product of the depth squared feature quantities and the second camera radial feature quantities, the product of the depth squared feature quantities and the first projector radial feature quantities, and the product of the depth squared feature quantities and the second projector radial feature quantities. The depth direction error component scalar is defined as being represented by a linear weighted combination of the depth error feature vector and the depth model parameter vector to be solved, thereby establishing the depth direction error model.

5. The method according to claim 1, characterized in that, The process involves estimating the model parameters of the depth direction error model and the radial direction error model using a regularized regression method, predicting the residual volume error of points in the preliminary correction point cloud using the estimated model parameters, and then superimposing the predicted residual volume error onto the corresponding coordinates of the preliminary correction points to obtain the final correction point cloud. This includes: The ridge regression method is used to estimate the parameters of the depth direction error model. The depth model parameter vector is solved by introducing a regularization term. The ridge regression method is used to estimate the parameters of the camera-side radial error model to obtain the camera-side radial model parameter vector. The ridge regression method is used to estimate the parameters of the projector-side radial error model to obtain the projector-side radial model parameter vector. For each point to be corrected in the preliminary correction point cloud, determine the depth value corresponding to that point; Calculate the radial distance between the camera and the projector sides corresponding to the point to be corrected; The depth direction error prediction scalar is calculated based on the depth model parameter vector, the depth value of the point to be corrected, the depth square term, the cubic and quintic terms of the radial distance on the camera side, and the cubic and quintic terms of the radial distance on the projector side. The camera-side radial error prediction scalar is calculated based on the camera-side radial model parameter vector and the cubic and quintic terms of the camera-side radial distance to the point to be corrected. Calculate the projector-side radial error prediction scalar based on the projector-side radial model parameter vector and the cubic and quintic terms of the projector-side radial distance of the point to be corrected. Based on the unit vectors corresponding to the object point direction in the world coordinates of the point to be corrected, the unit vectors of the camera radial direction and the projector radial direction, the depth direction error prediction scalar, the camera side radial error prediction scalar and the projector side radial error prediction scalar are combined into a predicted residual volume error vector. The predicted residual volume error vector is then added to the coordinates of the point to be corrected to obtain the corrected point coordinates. By traversing all points in the preliminary corrected point cloud, the final corrected point cloud is obtained.

6. The method according to claim 5, characterized in that, The process of synthesizing the depth direction error prediction scalar, the camera-side radial error prediction scalar, and the projector-side radial error prediction scalar into a predicted residual volume error vector based on the unit vector corresponding to the object point direction in world coordinates, the camera radial direction unit vector, and the projector radial direction unit vector of the point to be corrected includes: The depth direction error prediction scalar is multiplied by the unit vector corresponding to the object point direction in world coordinates to obtain the depth direction error component vector. The camera-side radial error prediction scalar is multiplied by the camera-side radial direction unit vector to obtain the camera-side radial error component vector. The radial error prediction scalar of the projector side is multiplied by the unit vector of the projector radial direction to obtain the radial error component vector of the projector side. The depth direction error component vector, the camera-side radial error component vector, and the projector-side radial error component vector are added together to obtain a three-dimensional composite vector. The three-dimensional synthesized vector is determined as the predicted residual volume error vector for the point to be corrected.

7. The method according to claim 1, characterized in that, The establishment of depth direction error model and radial direction error model, wherein the radial direction error model uses the camera-side radial error component and the projector-side radial error component as fitting targets and combines their respective radial distances to construct feature vectors, includes: Extract the camera-side radial error component scalar corresponding to each preliminary correction point from the orientation decomposition record, use it as the target value for fitting the camera radial model, and obtain the camera-side radial distance corresponding to the preliminary correction point. Calculate the cubic term of the camera-side radial distance to construct the first radial feature term of the camera, and calculate the fifth term of the camera-side radial distance to construct the second radial feature term of the camera; The camera-side radial error model is defined as a linear combination of the first radial characteristic term and the second radial characteristic term of the camera, and the coefficients in the linear combination constitute the parameters of the camera-side radial model. Extract the projector-side radial error component scalar corresponding to each preliminary correction point from the direction decomposition record, use it as the target value for fitting the projector radial model, and obtain the projector-side radial distance corresponding to the preliminary correction point. Calculate the cubic term of the radial distance on the projector side to construct the first radial characteristic term of the projector, and calculate the fifth term of the radial distance on the projector side to construct the second radial characteristic term of the projector. The projector-side radial error model is defined as a linear combination of the first radial characteristic term and the second radial characteristic term of the projector, and the coefficients in the linear combination constitute the parameters of the projector-side radial model.

8. The method according to claim 7, characterized in that, The camera-side radial error model is defined as a linear combination of the first radial characteristic term and the second radial characteristic term of the camera, including: The scalar representation of the camera-side radial error component is the product of the first radial characteristic term and the first coefficient of the camera, plus the product of the second radial characteristic term and the second coefficient of the camera; The first coefficient and the second coefficient constitute the camera-side radial model parameters.

9. The method according to claim 7 or 8, characterized in that, The projector-side radial error model is defined as a linear combination of the projector's first radial characteristic term and the projector's second radial characteristic term, including: The radial error component of the projector side is represented as the product of the first radial characteristic term and the third coefficient of the projector, plus the product of the second radial characteristic term and the fourth coefficient of the projector. The third coefficient and the fourth coefficient constitute the projector-side radial model parameters; The linear combination does not contain a constant term.

10. A structured light distortion correction system based on volume error decomposition, characterized in that, include: processor; A storage device having a computer program stored thereon, which, when executed by the processor, causes the processor to implement the structured light distortion correction method based on volume error decomposition as described in any one of claims 1-9.

Images