Method and system for correcting distortion residual error of projector lens in fringe projection contour

Abstract

The embodiment of the invention discloses a projector lens distortion residual error correction method and system in a fringe projection contour, and the method comprises the steps: firstly confirming a lens distortion model of a projector, and carrying out the preliminary correction of an original distortion projector coordinate, and obtaining a preliminary correction projector coordinate with a distortion residual error; measuring a plane object by using a calibrated fringe projection contour measurement system, calculating a pixel point weight based on a three-dimensional coordinate point corresponding to a projector coordinate subjected to preliminary correction, and fitting an ideal plane by using a weighted least square method to obtain an ideal plane parameter; three-dimensional coordinate points on the ideal plane and corresponding ideal projector image coordinates are determined according to the ideal plane parameters and the projector projection relation, distortion residual errors are calculated, and a distortion residual error distribution diagram is generated; and finally, in combination with the distortion residual distribution diagram and a target compensation strategy, performing compensation processing on the distortion residual of the projector lens, generating corrected projector coordinates, and realizing high-precision lens distortion correction.

Description

Technical Field

[0001] This invention relates to the field of optical correction technology, specifically to a method and system for correcting lens distortion residuals in fringe projection contours. Background Technology

[0002] Fringe projection profilometry (FPP), as an active 3D measurement technique, has been widely used in numerous fields such as industrial inspection, point cloud analysis, and biomedical analysis due to its full-range coverage, high precision, and high efficiency. In a typical FPP system, a projector projects an coded fringe pattern onto an object, while a synchronous camera captures the deformed pattern from different angles. The 3D surface of the object is then reconstructed through phase analysis and spatial mapping. However, due to lens distortion along the optical path (combined with...),... Figure 1 As shown, the projected and captured patterns are distorted, severely affecting the accuracy of the measurement. Unlike cameras, projectors cannot directly capture images, making it difficult to directly obtain the correspondence between two-dimensional pixels. Furthermore, projectors often employ off-axis designs, making it difficult for traditional parametric models such as polynomial models to accurately describe their distortion. While traditional projection lens distortion parametric models are simple to implement, their descriptive capabilities are limited, leading to significant distortion residuals (RD). Existing methods often ignore coupling errors caused by inaccurate parametric models when estimating RD. Pre-distortion RD compensation strategies accumulate nonlinear errors during back-projection and are poorly adapted to discontinuous fringe patterns. Summary of the Invention

[0003] This invention provides a method and system for correcting lens distortion residuals in striped projection contours.

[0004] In a first aspect, embodiments of the present invention provide a method for correcting lens distortion residuals in a fringe projection profile, applied to a lens distortion residual correction system in a fringe projection profile, the method comprising:

[0005] The lens distortion model of the projector is confirmed, and the original distorted projector coordinates are initially corrected using the lens distortion model to generate initially corrected projector coordinates. The initially corrected projector coordinates contain residual distortion error, which is obtained by fitting the ideal projector coordinates.

[0006] The planar object covering the projector's field of view is measured using a calibrated striped projection profile measurement system. The weight of each pixel is calculated based on the three-dimensional coordinate points corresponding to the pre-corrected projector coordinates. The ideal plane parameters are obtained by fitting the ideal plane using the weighted least squares method.

[0007] According to the ideal plane parameter and the projection relationship of the projector, a three-dimensional coordinate point on the ideal plane is determined, and a corresponding ideal projector image coordinate is obtained in combination with the three-dimensional coordinate point;

[0008] According to the preliminary corrected projector coordinate and the ideal projector image coordinate, a distortion residual error is calculated, and a distortion residual error distribution map is generated based on the distortion residual error;

[0009] In combination with the distortion residual error distribution map and a target compensation strategy, a lens distortion residual error of the projector is compensated to generate a corrected projector coordinate.

[0010] In a second aspect, an embodiment of the present application provides a system for correcting a lens distortion residual error of a projector in a fringe projection profile, comprising:

[0011] a processor;

[0012] a storage device having a computer program stored thereon,

[0013] When the computer program is executed by the processor, the processor implements any of the methods for correcting a lens distortion residual error of a projector in a fringe projection profile.

[0014] An embodiment of the present application provides a readable storage medium having a program or instruction stored thereon, and the program or instruction is executed by a processor to implement the steps of the method for correcting a lens distortion residual error of a projector in a fringe projection profile.

[0015] The embodiment of the present application realizes effective correction of the residual error of the lens distortion of the projector, and significantly improves the measurement accuracy and reliability of the fringe projection profilometry system. First, the lens distortion model of the projector is confirmed, and the original distorted projector coordinates are preliminarily corrected. Although the preliminarily corrected projector coordinates have residual error of distortion, the error is obtained by fitting the ideal projector coordinates. Then, the fringe projection profilometry system is used to measure a planar object, the pixel point weight is calculated based on the three-dimensional coordinate points corresponding to the preliminarily corrected projector coordinates, the ideal plane is fitted by using the weighted least square method, and the ideal plane parameters are obtained. In this process, the coupling error caused by the inaccurate parameter model is effectively suppressed by the weighted fitting. Then, the three-dimensional coordinate points on the ideal plane and the corresponding ideal projector image coordinates are determined according to the ideal plane parameters and the projector projection relationship, the residual error of distortion is accurately calculated, and the residual error distribution diagram of distortion is generated, thereby providing intuitive data support for the compensation strategy. Finally, the residual error of lens distortion is compensated by combining the residual error distribution diagram of distortion and the target compensation strategy, and the corrected projector coordinates are generated. Compared with the traditional method, the embodiment of the present application does not need to increase the calibration complexity or rely on additional hardware, avoids the accumulation of nonlinear errors in the back projection process, realizes high-precision and robust lens distortion correction of the projector under the premise of not changing the system hardware and the calibration process, and significantly improves the accuracy and adaptability of the fringe projection profilometry in the fields of industrial detection, point cloud analysis and biomedical analysis. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A residual error diagram of lens distortion of a projector provided by the embodiment of the present application.

[0017] Figure 2 A flowchart of a lens distortion residual error correction method of a fringe projection profilometry provided by the embodiment of the present application.

[0018] Figure 3 A technical roadmap of a lens distortion residual error correction method of a fringe projection profilometry provided by the embodiment of the present application.

[0019] Figure 4 A schematic diagram of the basic structure of a lens distortion residual error correction system of a fringe projection profilometry provided by the embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the embodiment of the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0021] Please refer to Figure 2 and Figure 3 The method can be applied to a lens distortion residual error correction system of a fringe projection profilometry. As shown in FIG. 1, the method comprises the following steps.Figure 2 As shown, the method can include steps 110-150.

[0022] Step 110: Confirm the lens distortion model of the projector, and use the lens distortion model to perform preliminary correction processing on the original distorted projector coordinates to generate preliminary corrected projector coordinates, wherein there is a distortion residual error in the preliminary corrected projector coordinates, and the distortion residual error is obtained by fitting ideal projector coordinates.

[0023] In a fringe projection profilometry system, in order to correct the distortion residual error of the projector lens, the lens distortion model of the projector needs to be first confirmed (because the projector lens has distortion, which will cause the projected pattern to be deformed, affecting the accuracy of measurement). By using a suitable lens distortion model to perform preliminary correction processing on the original distorted projector coordinates, preliminary corrected projector coordinates can be obtained. However, due to the limitations of the model, there is still a distortion residual error in the preliminary corrected projector coordinates, and this error needs to be obtained by fitting ideal projector coordinates.

[0024] In an optional embodiment, the confirmation of the lens distortion model of the projector includes:

[0025] Step 111: Obtain the intrinsic matrix and extrinsic matrix of the projector through the calibration process of the fringe projection profilometry system.

[0026] When performing fringe projection profilometry, the intrinsic matrix and extrinsic matrix of the projector can be obtained by calibrating the fringe projection profilometry system. The intrinsic matrix contains the internal parameters of the projector, such as focal length, principal point position, etc., which describe the optical characteristics of the projector itself. The extrinsic matrix represents the position and attitude of the projector in the world coordinate system, which describes the relative relationship between the projector and the external environment. Through an accurate calibration process, these matrices can be accurately obtained, providing a basis for distortion correction.

[0027] Step 112: Based on the intrinsic matrix and extrinsic matrix, determine to use a polynomial model as the lens distortion model of the projector; the polynomial model is used to describe the radial distortion and tangential distortion characteristics of the projector lens.

[0028] After the intrinsic matrix and the extrinsic matrix of the projector are obtained, a suitable lens distortion model needs to be selected. Through analysis and experiment, it is determined to use a polynomial model as the lens distortion model of the projector. The polynomial model has certain advantages, which can effectively describe the radial distortion and tangential distortion characteristics of the projector lens. The radial distortion refers to the deformation of the image from the center to the outside due to the uneven curvature of the lens, while the tangential distortion refers to the tilting deformation of the image due to inaccurate lens installation and other reasons. The polynomial model can accurately describe these distortion characteristics through a series of coefficients, thereby providing an accurate model for correction.

[0029] Step 113: performing preliminary correction processing on the original distorted projector coordinates through the polynomial model to generate preliminary corrected projector coordinates; the preliminary corrected projector coordinates contain residual distortion errors, which are uncorrected error components due to the limited description ability of the polynomial model.

[0030] Using the determined polynomial model, the original distorted projector coordinates are subjected to preliminary correction processing. In this process, the polynomial model adjusts the original distorted projector coordinates according to its coefficients, thereby generating preliminary corrected projector coordinates. However, due to the limited description ability of the polynomial model, it cannot completely eliminate all distortion errors. Therefore, there are still residual distortion errors in the preliminary corrected projector coordinates, which are caused by the part of the distortion that the polynomial model cannot accurately describe, and are uncorrected error components.

[0031] In another optional embodiment, the preliminary correction processing on the original distorted projector coordinates using the lens distortion model to generate preliminary corrected projector coordinates comprises:

[0032] Step 114: obtaining original distorted projector coordinates, which are actual imaging coordinates under the action of the projector lens distortion.

[0033] Before the correction processing, the original distorted projector coordinates need to be obtained, which are actual imaging coordinates under the action of the projector lens distortion, and they reflect the real imaging situation of the projector before correction. Through analysis and processing of the projection image, these original distorted projector coordinates can be accurately obtained, providing a data basis for correction.

[0034] Step 115: inputting the original distorted projector coordinates into the polynomial model and performing coordinate conversion on the original distorted projector coordinates through the distortion correction formula in the polynomial model; the coordinate conversion process includes radial distortion correction and tangential distortion correction on the original distorted projector coordinates.

[0035] The obtained original distorted projector coordinates are input into the polynomial model. The polynomial model contains distortion correction formulas, which will perform coordinate conversion on the original distorted projector coordinates according to the coefficients of the model. In this process, radial distortion correction and tangential distortion correction will be performed on the original distorted projector coordinates respectively. For radial distortion, the formula will adjust according to the distance of the image point to the center to eliminate the deformation from the center to the outside; for tangential distortion, the formula will adjust according to the inclination of the image to correct the inclination deformation of the image. Through such coordinate conversion, the influence of distortion can be reduced to a certain extent.

[0036] Step 116: generating preliminary corrected projector coordinates, which are approximate ideal coordinates obtained after correction by the polynomial model.

[0037] After the coordinate conversion of the original distorted projector coordinates by the distortion correction formulas of the polynomial model, preliminary corrected projector coordinates are generated, which are approximate ideal coordinates obtained after correction by the model. They reduce the influence of distortion to a certain extent, but due to the limitations of the model, they are not completely ideal coordinates. The preliminary corrected projector coordinates provide a basis for further processing.

[0038] Step 117: calculating the difference between the preliminary corrected projector coordinates and the fitted ideal projector coordinates to obtain distortion residual errors, which include residual errors in the horizontal coordinate direction and residual errors in the vertical coordinate direction.

[0039] In order to obtain the distortion residual errors, the difference between the preliminary corrected projector coordinates and the fitted ideal projector coordinates needs to be calculated. The fitted ideal projector coordinates are ideal imaging coordinates when the projector has no distortion, and these coordinates can be obtained by fitting methods. When calculating the difference, the differences in the horizontal coordinate direction and the vertical coordinate direction are calculated respectively, thereby obtaining the distortion residual errors. The residual errors in the horizontal coordinate direction reflect the uncorrected errors in the horizontal direction, and the residual errors in the vertical coordinate direction reflect the uncorrected errors in the vertical direction, which will be used for further compensation processing.

[0040] Step 120: measuring a planar object covering the field of view of the projector using a completed calibration stripe projection profilometry system, calculating the weight of each pixel point based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, and fitting an ideal plane to obtain ideal plane parameters using the weighted least squares method.

[0041] After the preliminary corrected projector coordinates are obtained, a stripe projection profilometry system with completed calibration is used to measure a planar object covering the field of view of the projector. The purpose of this is to obtain more accurate three-dimensional information for processing. Based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, the weight of each pixel point is calculated. The weight is determined according to the distortion degree of the pixel point. The more serious the distortion degree of the region, the smaller the weight of the corresponding pixel point. Then, the weighted least squares method is used to fit the ideal plane to obtain the ideal plane parameters. The ideal plane parameters can more accurately describe the real state of the planar object.

[0042] In an alternative embodiment, the use of a stripe projection profilometry system with completed calibration to measure a planar object covering the field of view of the projector, the calculation of the weight of each pixel point based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, and the use of the weighted least squares method to fit the ideal plane to obtain the ideal plane parameters include:

[0043] Step 121: Control the stripe projection profilometry system with completed calibration to project the coded fringe pattern to the planar object covering the field of view of the projector.

[0044] The stripe projection profilometry system with completed calibration has the ability to accurately measure. The system is controlled to project the coded fringe pattern to the planar object covering the field of view of the projector. The coded fringe pattern is a specially designed pattern that contains set coded information. By analyzing these information, the three-dimensional information of the object can be obtained. The projection of the coded fringe pattern is to let the planar object reflect these patterns so that the camera can capture and analyze them.

[0045] Step 122: Capture the deformed fringe pattern reflected by the planar object through the camera, and reconstruct the original three-dimensional point cloud of the planar object based on the phase analysis and spatial mapping method.

[0046] The camera captures the deformed fringe pattern reflected by the planar object. Due to factors such as the surface shape of the planar object and the distortion of the projector lens, the reflected fringe pattern will be deformed. By processing and analyzing these deformed fringe patterns through the phase analysis and spatial mapping method, the original three-dimensional point cloud of the planar object can be reconstructed. The original three-dimensional point cloud contains the three-dimensional coordinate information of each point on the surface of the planar object, which reflects the true shape and position of the planar object.

[0047] Step 123: Perform preliminary correction processing on the original three-dimensional point cloud to obtain the preliminary corrected three-dimensional coordinate points, which are one-to-one corresponding to the preliminary corrected projector coordinates.

[0048] The original three-dimensional point cloud obtained by reconstruction is subjected to preliminary correction processing. In this process, the original three-dimensional point cloud is adjusted according to the preliminary corrected projector coordinates obtained previously, so as to obtain preliminary corrected three-dimensional coordinate points, which are one-to-one corresponding to the preliminary corrected projector coordinates, so as to ensure the accuracy and consistency of subsequent calculation.

[0049] Step 124: Calculate the weight value of each pixel point according to the distortion degree of the preliminary corrected three-dimensional coordinate points. The more serious the distortion degree of the region, the smaller the weight value of the corresponding pixel point.

[0050] The weight value of each pixel point is calculated according to the distortion degree of the preliminary corrected three-dimensional coordinate points. The distortion degree can be measured by some indicators, such as the degree of deviation from the ideal plane. For the more serious distortion degree of the region, the smaller the weight value of the corresponding pixel point, because in these regions, the measurement error may be larger, in order to reduce the influence of these errors on subsequent fitting, reduce its weight. By reasonably calculating the weight value, the accuracy of subsequent fitting can be improved.

[0051] Step 125: Input the preliminary corrected three-dimensional coordinate points and the corresponding weight values into the weighted least squares fitting model.

[0052] The preliminary corrected three-dimensional coordinate points and the corresponding weight values are input into the weighted least squares fitting model. The weighted least squares fitting model is a commonly used fitting method, which considers the weight value of each data point, and can more accurately fit the data. By inputting these data and weight values, the model can calculate and fit according to these information to obtain more accurate results.

[0053] Step 126: Fit the preliminary corrected three-dimensional coordinate points to an ideal plane by the weighted least squares fitting model to obtain ideal plane parameters, including coefficients in the plane equation.

[0054] The weighted least squares fitting model fits the preliminary corrected three-dimensional coordinate points to an ideal plane. In this process, the model will fit an ideal plane by minimizing the sum of squares of errors according to the input data and weight values. The ideal plane obtained by fitting can be represented by a plane equation, and the ideal plane parameters are the coefficients in the plane equation, which accurately describe the position and attitude of the ideal plane.

[0055] Step 130: Determine the three-dimensional coordinate points on the ideal plane according to the ideal plane parameters and the projection relationship of the projector, and obtain the corresponding ideal projector image coordinates combined with the three-dimensional coordinate points.

[0056] After the ideal plane parameters are obtained, the three-dimensional coordinate points on the ideal plane are determined according to the ideal plane parameters and the projection relationship of the projector. The projection relationship of the projector describes how the projector projects a point in three-dimensional space onto a two-dimensional image plane. Through this projection relationship and the ideal plane parameters, the three-dimensional coordinates of each point on the ideal plane can be accurately calculated. Then, combined with these three-dimensional coordinate points, the corresponding ideal projector image coordinates are obtained through projection transformation and other methods. The ideal projector image coordinates are the ideal imaging coordinates when the projector has no distortion, and comparing them with the preliminary corrected projector coordinates can calculate the distortion residual error.

[0057] As an implementation manner, the determining the three-dimensional coordinate points on the ideal plane according to the ideal plane parameters and the projection relationship of the projector, and obtaining the corresponding ideal projector image coordinates combined with the three-dimensional coordinate points comprises:

[0058] Step 131: Obtain the projection matrix of the camera and the projection matrix of the projector, which are obtained through the calibration of the stripe projection profilometry system.

[0059] Obtaining the projection matrix of the camera and the projection matrix of the projector is a very important step. These projection matrices are obtained through the calibration process of the stripe projection profilometry system. The projection matrix of the camera describes how the camera projects a point in three-dimensional space onto a two-dimensional image plane, while the projection matrix of the projector describes the projection process of the projector. Accurate projection matrices are crucial for calculation and analysis, and they provide the basis for determining the three-dimensional coordinate points on the ideal plane and the corresponding ideal projector image coordinates.

[0060] Step 132: Establish the corresponding relationship between the three-dimensional coordinate points on the ideal plane and the camera image coordinates based on the ideal plane parameters and the projection matrix of the camera.

[0061] Based on the ideal plane parameters and the projection matrix of the camera, the corresponding relationship between the three-dimensional coordinate points on the ideal plane and the camera image coordinates can be established. The ideal plane parameters determine the position and pose of the ideal plane, while the projection matrix of the camera describes the projection process of the camera. By combining these two pieces of information, a mathematical relationship between the three-dimensional coordinate points and the camera image coordinates can be established, so that the three-dimensional coordinate points on the ideal plane can be determined according to the camera image coordinates, or the corresponding camera image coordinates can be calculated according to the three-dimensional coordinate points.

[0062] Step 133: The three-dimensional coordinate points on the ideal plane are obtained by solving a linear equation system, which is constructed based on the camera projection matrix and the ideal plane parameters.

[0063] According to the established correspondence, a linear equation set is constructed. The linear equation set is constructed based on the camera projection matrix and the ideal plane parameters, and contains the relationship between the three-dimensional coordinate points on the ideal plane and the camera image coordinates. By solving the linear equation set, the three-dimensional coordinate points on the ideal plane can be obtained. Some numerical calculation methods, such as Gaussian elimination method, can be used to solve the linear equation set to ensure the accuracy and efficiency of the solution.

[0064] Step 134: input the three-dimensional coordinate points on the ideal plane into the projection matrix of the projector to calculate the corresponding ideal projector image coordinates through projection transformation; the ideal projector image coordinates are ideal imaging coordinates when the projector is free of distortion, and are used to compare with the preliminary corrected projector coordinates to calculate the distortion residual error.

[0065] The three-dimensional coordinate points on the ideal plane solved are input into the projection matrix of the projector. The projection matrix of the projector describes the projection process of the projector, and through projection transformation, the three-dimensional coordinate points can be converted into corresponding ideal projector image coordinates. The ideal projector image coordinates are ideal imaging coordinates when the projector is free of distortion, and by comparing them with the preliminary corrected projector coordinates, the distortion residual error can be calculated. Through such comparison, the distortion of the projector lens can be accurately understood.

[0066] Step 140: calculate the distortion residual error according to the preliminary corrected projector coordinates and the ideal projector image coordinates, and generate a distortion residual error distribution map based on the distortion residual error.

[0067] According to the preliminary corrected projector coordinates and the ideal projector image coordinates, the distortion residual error can be calculated. By comparing the preliminary corrected projector coordinates with the ideal projector image coordinates, the difference between them is calculated to obtain the distortion residual error. Then, based on these distortion residual errors, a distortion residual error distribution map is generated. The distortion residual error distribution map can directly show the residual error distribution of each pixel point on the projector image plane, providing an important reference for compensation processing.

[0068] In one embodiment, the generating of the distortion residual error distribution map based on the distortion residual error comprises:

[0069] Step 141: generate a first distortion residual error distribution map for the pre-distortion preliminary correction strategy and a second distortion residual error distribution map for the linear grid preliminary correction strategy based on the distortion residual error.

[0070] Based on the calculated distortion residual error, a first distortion residual error distribution map for the pre-distortion preliminary correction strategy and a second distortion residual error distribution map for the linear grid preliminary correction strategy are generated respectively. The pre-distortion preliminary correction strategy and the linear grid preliminary correction strategy are two different correction strategies, and their handling of distortion is different. Therefore, generating the corresponding distortion residual error distribution maps can more accurately reflect the residual error distribution under different correction strategies, providing more targeted reference for compensation processing.

[0071] In the following steps, the generation of the first distortion residual error distribution map for the pre-distortion preliminary correction strategy and the second distortion residual error distribution map for the linear grid preliminary correction strategy based on the distortion residual error includes:

[0072] Step 1411: Process all imaging points of the projector to obtain the distortion residual error corresponding to each imaging point.

[0073] Processing all imaging points of the projector, by comparing the preliminary corrected projector coordinates of each imaging point and the ideal projector image coordinates, the distortion residual error corresponding to each imaging point is obtained. This process needs to traverse the entire image plane of the projector to ensure that the distortion residual error of all imaging points is obtained. The distortion residual error of each imaging point reflects the distortion that still exists after preliminary correction, providing detailed data for processing.

[0074] Step 1412: Scatter data interpolation processing is performed on the distortion residual error to convert discrete distortion residual error data into continuously distributed residual data.

[0075] The obtained distortion residual error is discrete data, in order to better analyze and process, the scatter data interpolation processing is needed. Scatter data interpolation processing can convert discrete distortion residual error data into continuously distributed residual data. Through interpolation method, the residual data of other points can be estimated between discrete data points, so as to obtain a continuous residual distribution, which can more accurately describe the residual error distribution on the projector image plane.

[0076] Step 1413: Spline smoothing processing is performed on the continuously distributed residual data, and based on the smoothed residual data, a first distortion residual error distribution map for the pre-distortion preliminary correction strategy is constructed on the preliminary corrected projector image space, and the first distortion residual error distribution map is used to describe the residual error distribution of each pixel point on the projector image plane in the pre-distortion compensation scenario.

[0077] The residual error data is smoothed by spline smoothing. Spline smoothing can reduce noise and fluctuations in the data, making the residual error data smoother. Based on the smoothed residual error data, a first distortion residual error distribution map for the pre-distortion preliminary correction strategy is constructed on the preliminary corrected projector image space. The first distortion residual error distribution map can intuitively show the residual error distribution of each pixel point in the projector image plane in the pre-distortion compensation scenario. Through the distribution map, it can be clearly understood which area has larger residual error, so as to provide the area of focus for compensation processing.

[0078] Step 1414: Based on the same set of smoothed residual error data, a second distortion residual error distribution map for the linear grid preliminary correction strategy is constructed on the preliminary corrected projector image space, which is used to describe the residual error distribution of each pixel point in the projector image plane in the linear grid compensation scenario; wherein the spatial resolution of the first distortion residual error distribution map and the second distortion residual error distribution map is consistent with the projector image resolution, so as to maintain the compensation accuracy.

[0079] Based on the same set of smoothed residual error data, a second distortion residual error distribution map for the linear grid preliminary correction strategy is constructed on the preliminary corrected projector image space. The second distortion residual error distribution map is used to describe the residual error distribution of each pixel point in the projector image plane in the linear grid compensation scenario. In order to maintain the compensation accuracy, the spatial resolution of the first distortion residual error distribution map and the second distortion residual error distribution map is consistent with the projector image resolution, so as to ensure that the compensation can be accurately performed according to the distribution map in the compensation processing, and the accuracy of the compensation is improved.

[0080] Step 150: Combining the distortion residual error distribution map and the target compensation strategy, the lens distortion residual error of the projector is compensated to generate the corrected projector coordinates.

[0081] The lens distortion residual error of the projector is compensated by combining the distortion residual error distribution map and the target compensation strategy. The distortion residual error distribution map provides the residual error distribution of each pixel point in the projector image plane, and the target compensation strategy specifies how to compensate according to these residual errors. By combining the two, the lens distortion residual error of the projector can be effectively compensated, thereby generating corrected projector coordinates. The corrected projector coordinates are closer to the ideal imaging coordinates, improving the accuracy of the projected image.

[0082] In one example, the target compensation strategy includes the pre-distortion preliminary correction strategy and the preliminary correction strategy of the linear grid. Based on this, the lens distortion residual error of the projector is compensated based on the distortion residual error distribution map and the target compensation strategy, to generate corrected projector coordinates, including step 151 or step 152:

[0083] Step 151: Compensate the lens distortion residual error of the projector based on the pre-distortion preliminary correction strategy and the first lookup table generated using the first distortion residual error distribution map, to generate corrected first projector coordinates; wherein the first lookup table includes two residual error lookup tables in the horizontal and vertical coordinate directions.

[0084] Compensate the lens distortion residual error of the projector based on the pre-distortion preliminary correction strategy and the first lookup table generated using the first distortion residual error distribution map. The first lookup table includes two residual error lookup tables in the horizontal and vertical coordinate directions, which record the residual error information at each position in the pre-distortion compensation scenario. By querying the first lookup table, the residual error corresponding to each pixel point can be obtained, and then the preliminary corrected projector coordinates are adjusted according to these errors, to generate corrected first projector coordinates. The corrected first projector coordinates are closer to the ideal imaging coordinates, reducing the residual error that still exists after pre-distortion compensation.

[0085] Step 151: Compensate the lens distortion residual error of the projector, to generate corrected first projector coordinates, including:

[0086] Step 1511: Obtain a first index from the preliminary corrected projector coordinates, and query the first lookup table according to the first index to compensate the distortion residual error.

[0087] Obtain a first index from the preliminary corrected projector coordinates. The first index is an identifier used to query the first lookup table, which is generated according to the position information of the preliminary corrected projector coordinates. According to the first index, the corresponding horizontal coordinate direction residual error and vertical coordinate direction residual error can be obtained by querying the first lookup table. Then, these residual errors are superimposed with the preliminary corrected projector coordinates, to realize compensation of the distortion residual error, and finally generate corrected first projector coordinates.

[0088] Step 152: Compensate the lens distortion error of the projector containing residual error based on the linear parameter lookup table generated based on the preliminary correction strategy of the linear grid and the second lookup table optimized using the second distortion residual error distribution map, to generate corrected second projector coordinates; wherein the second lookup table contains slope and offset parameters in the horizontal and vertical coordinate directions, and the second lookup table includes four linear parameter lookup tables.

[0089] A linear parameter lookup table generated by a preliminary correction strategy based on linear grids, and a second lookup table optimized using a second distortion residual distribution map, are used to compensate for distortion errors in the projector lens that include residuals. The linear parameter lookup table records linear parameters such as the slope coefficient and offset term for each grid region. The second lookup table, optimized based on the second distortion residual distribution map, includes slope and offset parameters in the horizontal and vertical axes, resulting in four linear parameter lookup tables. By querying the second lookup table, the linear parameters corresponding to each pixel can be obtained. Then, linear operations are performed on the original distorted projector coordinates based on these parameters to generate the corrected second projector coordinates. The corrected second projector coordinates eliminate the residual errors that still exist after linear grid compensation, improving the accuracy of the projected image.

[0090] The step 152, which involves compensating for the distortion error of the projector lens containing residuals and generating corrected second projector coordinates, includes: obtaining a second index through the original distorted projector coordinates; querying a linear parameter from the second lookup table based on the second index; and performing linear operations based on the linear parameter to correct the distortion error containing residuals.

[0091] The second index is obtained using the original distorted projector coordinates. This second index, generated based on the position information of the original distorted projector coordinates, serves as an identifier for lookup in the second lookup table. The corresponding slope and offset parameters are then retrieved from the second lookup table using this index. Next, linear operations are performed on the original distorted projector coordinates using these linear parameters, such as multiplying the slope parameters by the original distorted projector coordinates and adding the offset parameters, to correct the distortion error, including the residuals, ultimately generating the corrected second projector coordinates.

[0092] In another optional embodiment, the step of compensating for the lens distortion residuals of the projector based on the pre-distortion preliminary correction strategy and a first lookup table generated using the first distortion residual distribution map, to generate corrected first projector coordinates, includes:

[0093] Step 210: Based on the first distortion residual distribution map, create residual lookup tables in the horizontal and vertical axes respectively to obtain the first lookup table. The size of the first lookup table is the same as the resolution of the projector image.

[0094] Based on the first distortion residual distribution map, a residual look-up table is created in the horizontal coordinate axis direction and the vertical coordinate axis direction, thereby obtaining a first look-up table. The size of the first look-up table is the same as the resolution of the projector image, so as to ensure that the look-up table can cover the entire range of the projection image. In the process of creating the look-up table, the residual error information in the first distortion residual distribution map is stored in the look-up table according to the coordinate position, so as to be queried and used.

[0095] Step 220: Obtain the preliminary corrected projector coordinates, and convert the preliminary corrected projector coordinates into a first index, which is used for querying in the first look-up table.

[0096] The preliminary corrected projector coordinates are obtained, and then the coordinates are converted into a first index. The first index is generated according to the position information of the preliminary corrected projector coordinates, and is a key identification for querying in the first look-up table. By converting the preliminary corrected projector coordinates into the first index, the corresponding residual error information can be conveniently found in the first look-up table.

[0097] Step 230: Query the corresponding horizontal coordinate direction residual error and vertical coordinate direction residual error in the first look-up table according to the first index.

[0098] According to the generated first index, the first look-up table is queried. Through the query, the corresponding horizontal coordinate direction residual error and vertical coordinate direction residual error are obtained, which accurately reflect the residual distortion error of the pixel point in the pre-distortion compensation scene, and provide specific data for compensation.

[0099] Step 240: Superimpose the preliminary corrected projector coordinates and the queried horizontal coordinate direction residual error and vertical coordinate direction residual error to obtain corrected first projector coordinates; the corrected first projector coordinates are ideal projector coordinates eliminating the residual error after pre-distortion compensation.

[0100] The preliminary corrected projector coordinates are superimposed with the queried horizontal coordinate direction residual error and vertical coordinate direction residual error. In the superimposition process, the error information is added to the preliminary corrected projector coordinates, so as to adjust the coordinates. After superimposition, the corrected first projector coordinates are obtained. The corrected first projector coordinates eliminate the residual error still existing after pre-distortion compensation, and are closer to the ideal imaging coordinates, thereby improving the accuracy of the projection image.

[0101] In yet another optional embodiment, the distortion error containing residual error of the projector lens is compensated based on a linear parameter lookup table generated by a preliminary correction strategy based on the linear grid and a second lookup table generated by optimization using the second distortion residual distribution map, to generate a corrected second projector coordinate, comprising:

[0102] Step 310: dividing the projector image plane into equidistant small grid areas, and describing the distortion error in each grid area by a linear model.

[0103] The projector image plane is divided into equidistant small grid areas, which can refine the entire projection image plane to more accurately describe the distortion of each area. The distortion error in each grid area is described by a linear model. The linear model has the characteristics of simplicity and efficiency, which can accurately describe the change rule of the distortion error in the grid area with slope coefficient and bias term.

[0104] Step 320: selecting multiple sample points in each grid area, and calculating the distortion error of the sample points by a polynomial model.

[0105] Multiple sample points are selected in each grid area, and these sample points are representative and can reflect the distortion of the entire grid area. The distortion error of the sample points is calculated by a polynomial model. The polynomial model has been used for preliminary correction, and it can also be used to calculate the distortion error of the sample points. By calculating the distortion error of the sample points, the specific distortion information in each grid area can be obtained, providing a data basis for linear fitting.

[0106] Step 330: linear fitting of the distortion error of the sample points by least squares method to determine the slope coefficient and bias term of each grid area, and generating a linear parameter lookup table.

[0107] The distortion error of the sample points is linearly fitted by least squares method. Least squares method is a commonly used fitting method, which finds the most suitable linear model parameters by minimizing the sum of squares of errors. In this process, the slope coefficient and bias term of each grid area are determined, which accurately describe the change rule of the distortion error in each grid area. These parameters are stored in a lookup table, generating a linear parameter lookup table. The linear parameter lookup table records the linear model parameter information of each grid area, providing an important basis for compensation.

[0108] Step 340: optimizing the slope coefficient and bias term in the linear parameter lookup table based on the second distortion residual distribution map to obtain a second lookup table, which contains slope parameter lookup table and bias parameter lookup table in the horizontal and vertical coordinate axis directions.

[0109] Based on the second distortion residual distribution map, the slope coefficient and the bias term in the linear parameter lookup table are optimized. The second distortion residual distribution map provides more detailed residual error distribution information. By referring to these information, the parameters in the linear parameter lookup table can be adjusted and optimized so that they can more accurately describe the distortion error situation. After optimization, the second lookup table is obtained. The second lookup table includes slope parameter lookup tables and bias parameter lookup tables in the horizontal coordinate axis direction and the vertical coordinate axis direction, which record the optimized linear model parameter information and provide more accurate basis for compensation processing.

[0110] Step 350: Obtain original distortion projector coordinates, and convert the original distortion projector coordinates into second indexes, which are used for querying in the second lookup table.

[0111] The original distortion projector coordinates are obtained, and then the coordinates are converted into second indexes. The second indexes are generated according to the position information of the original distortion projector coordinates, and are the key identifiers for querying in the second lookup table. By converting the original distortion projector coordinates into second indexes, the corresponding linear model parameter information can be easily found in the second lookup table.

[0112] Step 360: Query the corresponding slope parameter and bias parameter in the second lookup table according to the second index.

[0113] According to the generated second index, the second lookup table is queried. Through the query, the corresponding slope parameter and bias parameter can be obtained, which accurately describe the distortion error variation law of the grid area where the position is located, and provide specific data for linear operation.

[0114] Step 370: Combine the original distortion projector coordinates with the queried slope parameter and bias parameter through linear operation to calculate corrected second projector coordinates, which are ideal projector coordinates eliminating residual errors after linear grid compensation.

[0115] The original distortion projector coordinates are combined with the queried slope parameter and bias parameter through linear operation. In the linear operation process, the slope parameter is multiplied by the original distortion projector coordinates, and then the bias parameter is added, so as to adjust the original distortion projector coordinates. After linear operation, the corrected second projector coordinates are obtained. The corrected second projector coordinates eliminate the residual errors still existing after linear grid compensation, and are closer to the ideal imaging coordinates, thereby improving the accuracy of the projection image.

[0116] In another implementation, in a fringe projection profilometry system, the distortion of the projector lens can adversely affect the measurement results, and in order to improve the accuracy of the measurement, the distortion residual of the projector lens needs to be corrected. First, through the calibration process of the fringe projection profilometry system, the intrinsic matrix and extrinsic matrix of the projector are obtained, and based on these matrices, a polynomial model is determined as the lens distortion model of the projector. This polynomial model can describe the radial distortion and tangential distortion characteristics of the projector lens. Using this model, the original distorted projector coordinates are preliminarily corrected, that is, the original distorted projector coordinates are input into the polynomial model, and the coordinates are converted through the distortion correction formula in it, including radial distortion correction and tangential distortion correction, thereby generating preliminarily corrected projector coordinates. However, due to the limited description ability of the polynomial model, there is a distortion residual error in the preliminarily corrected projector coordinates, which is obtained by calculating the difference between the preliminarily corrected projector coordinates and the fitted ideal projector coordinates, and contains residual errors in the horizontal and vertical directions.

[0117] Then, using the calibrated fringe projection profilometry system, a planar object covering the field of view of the projector is measured. The system is controlled to project an encoded fringe pattern onto the planar object, and the camera captures the deformed fringe pattern reflected by the planar object. Based on the phase analysis and spatial mapping method, the original three-dimensional point cloud of the planar object is reconstructed. The original three-dimensional point cloud is preliminarily corrected to obtain preliminarily corrected three-dimensional coordinate points corresponding to the preliminarily corrected projector coordinates. According to the distortion degree of these three-dimensional coordinate points, the weight value of each pixel point is calculated, and the pixel point weight value corresponding to the area with more serious distortion is smaller. The preliminarily corrected three-dimensional coordinate points and the corresponding weight values are input into the weighted least squares fitting model to fit the ideal plane, and the ideal plane parameters are obtained, which include the coefficients in the plane equation.

[0118] Then, according to the ideal plane parameters and the projection relationship of the projector, the three-dimensional coordinate points on the ideal plane are determined, and the corresponding ideal projector image coordinates are obtained. Specifically, the projection matrix of the camera and the projector is obtained first, which is obtained through system calibration. Based on the ideal plane parameters and the projection matrix of the camera, the correspondence between the three-dimensional coordinate points on the ideal plane and the camera image coordinates is established, and then the three-dimensional coordinate points on the ideal plane are obtained by solving the linear equation system. These three-dimensional coordinate points are input into the projection matrix of the projector, and the ideal projector image coordinates are calculated through projection transformation, which are used to compare with the preliminarily corrected projector coordinates to calculate the distortion residual error.

[0119] After that, the distortion residual error is calculated according to the preliminary corrected projector coordinates and the ideal projector image coordinates, and a distortion residual error distribution map is generated based on the same. All imaging points of the projector are processed to obtain the distortion residual error corresponding to each imaging point. The discrete error data is processed by scatter data interpolation to convert it into continuous distribution residual data, and then spline smoothing is performed. Based on the smoothed residual data, a first distortion residual error distribution map for the pre-distortion preliminary correction strategy and a second distortion residual error distribution map for the linear grid preliminary correction strategy are respectively constructed on the preliminary corrected projector image space. The spatial resolution of the two distribution maps is consistent with the projector image resolution to maintain the compensation accuracy.

[0120] Finally, the lens distortion residual error of the projector is compensated by combining the distortion residual error distribution map and the target compensation strategy to generate the corrected projector coordinates. The target compensation strategy includes the pre-distortion preliminary correction strategy and the linear grid preliminary correction strategy. For the pre-distortion preliminary correction strategy, the first distortion residual error distribution map is used to generate a first lookup table containing two residual lookup tables in the horizontal and vertical coordinate directions. The preliminary corrected projector coordinates are converted into a first index, and the residual error is obtained by querying the first lookup table through the index. The residual error is added to the preliminary corrected projector coordinates to obtain the corrected first projector coordinates. For the linear grid preliminary correction strategy, the projector image plane is divided into equidistant grid regions, and multiple sample points are selected in each region. The distortion error of the sample points is calculated using a polynomial model, and the least squares method is used for linear fitting to determine the slope coefficient and the bias term to generate a linear parameter lookup table. Based on the second distortion residual error distribution map, the lookup table is optimized to obtain a second lookup table containing slope parameter lookup tables and bias parameter lookup tables in the horizontal and vertical coordinate directions. The original distortion projector coordinates are converted into a second index, and the linear parameters are obtained by querying the second lookup table through the index. The corrected second projector coordinates are obtained by linear operation.

[0121] The embodiment of the present application realizes effective correction of the residual error of the lens distortion of the projector, and significantly improves the measurement accuracy and reliability of the fringe projection profilometry system. First, the lens distortion model of the projector is confirmed, and the original distorted projector coordinates are preliminarily corrected. Although the preliminarily corrected projector coordinates have residual error of distortion, the error is obtained by fitting the ideal projector coordinates. Then, the fringe projection profilometry system is used to measure a planar object, the pixel point weight is calculated based on the three-dimensional coordinate points corresponding to the preliminarily corrected projector coordinates, the ideal plane parameters are obtained by fitting the ideal plane by using the weighted least square method, and the coupling error caused by the inaccurate parameter model is effectively suppressed through the weighted fitting. Then, the three-dimensional coordinate points on the ideal plane and the corresponding ideal projector image coordinates are determined according to the ideal plane parameters and the projection relationship of the projector, the residual error of distortion is accurately calculated, and the residual error distribution map of distortion is generated, thereby providing intuitive data support for the compensation strategy. Finally, the residual error of lens distortion is compensated by combining the residual error distribution map of distortion and the target compensation strategy, and the corrected projector coordinates are generated. Compared with the traditional method, the embodiment of the present application does not need to increase the calibration complexity or rely on additional hardware, avoids the accumulation of nonlinear errors in the back projection process, realizes high-precision and robust lens distortion correction of the projector without changing the system hardware and the calibration process, and significantly improves the accuracy and adaptability of the fringe projection profilometry in the fields of industrial detection, point cloud analysis and biomedical analysis.

[0122] Referring to FIG. Figure 4 The fringe projection profilometry lens distortion residual error correction system 200 provided by the embodiment of the present application includes:

[0123] a processor 201;

[0124] a storage device 202, in which a computer program 2020 is stored;

[0125] When the computer program 2020 is executed by the processor 201, the processor 201 realizes the fringe projection profilometry lens distortion residual error correction method.

[0126] On the basis described above, a readable storage medium is provided, in which a program or instruction is stored, and the program or instruction is executed by a processor to realize the steps of the above method.

[0127] It should be noted that the various embodiments described in the specification are intended to be exemplary only and that the scope of the application is not intended to be limited to the embodiments described in the specification.

Claims

1. A method for correcting residual errors of a projector lens distortion in fringe projection profilometry, characterized in that, The method comprises the following steps: Confirming a lens distortion model of the projector, using the lens distortion model to perform preliminary correction processing on original distorted projector coordinates, and generating preliminary corrected projector coordinates, wherein residual distortion errors exist in the preliminary corrected projector coordinates, and the residual distortion errors are obtained by fitting ideal projector coordinates; Measuring a planar object covering a field of view of the projector using a fringe projection profilometry system that has completed calibration, calculating a weight value of each pixel point based on a three-dimensional coordinate point corresponding to the preliminary corrected projector coordinates, and fitting an ideal plane using a weighted least squares method to obtain ideal plane parameters; Determining three-dimensional coordinate points on the ideal plane according to the ideal plane parameters and a projection relationship of the projector, and combining the three-dimensional coordinate points to obtain corresponding ideal projector image coordinates; Calculating residual distortion errors according to the preliminary corrected projector coordinates and the ideal projector image coordinates, and generating a residual distortion distribution map based on the residual distortion errors; Combining the residual distortion distribution map and a target compensation strategy to perform compensation processing on lens distortion residuals of the projector, and generating corrected projector coordinates.

2. The method of claim 1, wherein, The method of generating the residual distortion distribution map based on the residual distortion errors comprises the following steps: Generating a first residual distortion distribution map for a pre-distortion preliminary correction strategy and a second residual distortion distribution map for a linear grid preliminary correction strategy based on the residual distortion errors.

3. The method of claim 2, wherein, The target compensation strategy comprises the pre-distortion preliminary correction strategy and the linear grid preliminary correction strategy. The method of combining the residual distortion distribution map and the target compensation strategy to perform compensation processing on the lens distortion residuals of the projector to generate the corrected projector coordinates comprises the following steps: Performing compensation processing on the lens distortion residuals of the projector based on the pre-distortion preliminary correction strategy and a first lookup table generated using the first residual distortion distribution map to generate corrected first projector coordinates, wherein the first lookup table comprises two residual lookup tables in the horizontal and vertical coordinate directions. The method of performing compensation processing on the lens distortion residuals of the projector to generate the corrected first projector coordinates comprises the following steps: obtaining a first index from the preliminary corrected projector coordinates, querying the first lookup table according to the first index to compensate for the residual distortion errors, or Combining a linear parameter lookup table generated based on the linear grid preliminary correction strategy and a second lookup table generated by optimizing the second residual distortion distribution map to perform compensation processing on the distortion errors of the lens residuals of the projector to generate corrected second projector coordinates, wherein the second lookup table comprises slope and offset parameters in the horizontal and vertical coordinate directions, and the second lookup table comprises four linear parameter lookup tables. The method of performing compensation processing on the distortion errors of the lens residuals of the projector to generate the corrected second projector coordinates comprises the following steps: obtaining a second index from the original distorted projector coordinates, querying linear parameters from the second lookup table according to the second index, and performing linear operations according to the linear parameters to correct the distortion errors of the lens residuals. ​ 4. The method of claim 2, wherein, The generating a first distortion residual error distribution map for a pre-distortion preliminary correction strategy and a second distortion residual error distribution map for a linear grid preliminary correction strategy based on the distortion residual error comprises: Processing all imaging points of the projector to obtain a distortion residual error corresponding to each imaging point; Performing scatter data interpolation processing on the distortion residual error to convert discrete distortion residual error data into continuous distribution residual error data; Performing spline smoothing processing on the continuous distribution residual error data, and constructing a first distortion residual error distribution map for a pre-distortion preliminary correction strategy on a preliminary corrected projector image space based on the smoothed residual error data, wherein the first distortion residual error distribution map is used to describe the residual error distribution of each pixel point on the projector image plane in a pre-distortion compensation scenario; Constructing a second distortion residual error distribution map for a linear grid preliminary correction strategy on the preliminary corrected projector image space based on the same set of smoothed residual error data, wherein the second distortion residual error distribution map is used to describe the residual error distribution of each pixel point on the projector image plane in a linear grid compensation scenario; and wherein the spatial resolution of the first distortion residual error distribution map and the second distortion residual error distribution map is consistent with the projector image resolution to maintain the compensation accuracy.

5. The method of claim 1, wherein, The confirming the lens distortion model of the projector comprises: Obtaining an intrinsic matrix and an extrinsic matrix of the projector through a calibration process of a fringe projection profilometry system; Determining to use a polynomial model as the lens distortion model of the projector based on the intrinsic matrix and the extrinsic matrix, wherein the polynomial model is used to describe the radial distortion and tangential distortion characteristics of the projector lens; Performing preliminary correction processing on the original distorted projector coordinates through the polynomial model to generate preliminary corrected projector coordinates, wherein the preliminary corrected projector coordinates contain distortion residual errors, and the distortion residual errors are uncorrected error components caused by the limited description capability of the polynomial model.

6. The method of claim 5, wherein, The preliminary correction processing on the original distorted projector coordinates through the lens distortion model to generate preliminary corrected projector coordinates comprises: Obtaining original distorted projector coordinates, wherein the original distorted projector coordinates are actual imaging coordinates under the action of the projector lens distortion; Inputting the original distorted projector coordinates into the polynomial model to perform coordinate conversion on the original distorted projector coordinates through a distortion correction formula in the polynomial model, wherein the coordinate conversion process comprises radial distortion correction and tangential distortion correction on the original distorted projector coordinates; Generating preliminary corrected projector coordinates, wherein the preliminary corrected projector coordinates are approximate ideal coordinates obtained after correction by the polynomial model; Calculating the difference between the preliminary corrected projector coordinates and the fitted ideal projector coordinates to obtain distortion residual errors, wherein the distortion residual errors include residual errors in the horizontal coordinate direction and residual errors in the vertical coordinate direction.

7. The method of claim 1, wherein, The calibrated fringe projection profilometry system is used to measure a planar object covering the field of view of the projector, the weight of each pixel point is calculated based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, and the ideal plane parameters are obtained by fitting an ideal plane using a weighted least squares method, including: The calibrated fringe projection profilometry system is used to measure a planar object covering the field of view of the projector, the weight of each pixel point is calculated based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, and the ideal plane parameters are obtained by fitting an ideal plane using a weighted least squares method, including: The calibrated fringe projection profilometry system is used to measure a planar object covering the field of view of the projector, the weight of each pixel point is calculated based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, and the ideal plane parameters are obtained by fitting an ideal plane using a weighted least squares method, including: The calibrated fringe projection profilometry system is used to measure a planar object covering the field of view of the projector, the weight of each pixel point is calculated based on the three-dimensional coordinate points corresponding to the preliminary corrected projector coordinates, and the ideal plane parameters are obtained by fitting an ideal plane using a weighted least squares method, including: The preliminary corrected three-dimensional coordinate points and the corresponding weight are input into a weighted least squares fitting model. The preliminary corrected three-dimensional coordinate points are fitted to an ideal plane by the weighted least squares fitting model to obtain ideal plane parameters, including the coefficients in the plane equation. The three-dimensional coordinate points on the ideal plane are determined based on the ideal plane parameters and the projection relationship of the projector, and the corresponding ideal projector image coordinates are obtained in combination with the three-dimensional coordinate points, including:

8. The method of claim 1, wherein, The projection matrix of the camera and the projection matrix of the projector are obtained, and the projection matrix is obtained by fringe projection profilometry system calibration; The correspondence between the three-dimensional coordinate points on the ideal plane and the camera image coordinates is established based on the ideal plane parameters and the projection matrix of the camera; The three-dimensional coordinate points on the ideal plane are obtained by solving a linear equation set, and the linear equation set is constructed based on the camera projection matrix and the ideal plane parameters; The three-dimensional coordinate points on the ideal plane are input into the projection matrix of the projector, and the corresponding ideal projector image coordinates are calculated by projection transformation; the ideal projector image coordinates are ideal imaging coordinates when the projector has no distortion, and are used to compare with the preliminary corrected projector coordinates to calculate the distortion residual error. The lens distortion residual error of the projector is compensated based on the pre-distortion preliminary correction strategy and the first lookup table generated by the first distortion residual distribution map to generate corrected first projector coordinates, including:

9. The method of claim 3, wherein, Based on the first distortion residual distribution map, residual lookup tables are created in the horizontal coordinate axis direction and the vertical coordinate axis direction respectively to obtain a first lookup table, and the size of the first lookup table is the same as the projector image resolution; The preliminary corrected projector coordinates are obtained, and the preliminary corrected projector coordinates are converted into a first index, which is used for querying in the first lookup table; The corresponding horizontal coordinate direction residual error and vertical coordinate direction residual error are obtained by querying the first index in the first lookup table. ​ Superimpose the preliminary corrected projector coordinates with the query obtained horizontal coordinate direction residual error and vertical coordinate direction residual error to obtain the corrected first projector coordinates; the corrected first projector coordinates eliminate the ideal projector coordinates of the pre-distortion compensation residual error; The linear parameter lookup table generated based on the preliminary correction strategy of the linear grid and the second lookup table optimized and generated by using the second distortion residual distribution diagram are combined to compensate the distortion error of the projector lens containing residual error to generate the corrected second projector coordinates, including: Divide the projector image plane into equidistant small grid areas, and the distortion error in each grid area is described by a linear model; Select multiple sample points in each grid area, and calculate the distortion error of the sample points by using a polynomial model; Determine the slope coefficient and the bias term of each grid area by linear fitting the distortion error of the sample points by using the least square method to generate a linear parameter lookup table; Based on the second distortion residual distribution diagram, optimize the slope coefficient and the bias term in the linear parameter lookup table to obtain a second lookup table, and the second lookup table contains a slope parameter lookup table and a bias parameter lookup table in the horizontal coordinate axis direction and the vertical coordinate axis direction; Obtain the original distortion projector coordinates, and convert the original distortion projector coordinates into a second index, which is used for querying in the second lookup table; According to the second index, query the corresponding slope parameter and bias parameter in the second lookup table; Combine the original distortion projector coordinates with the query obtained slope parameter and bias parameter by linear operation to calculate the corrected second projector coordinates, and the corrected second projector coordinates eliminate the ideal projector coordinates of the linear grid compensation residual error.

10. A system for correcting lens distortion residuals in a striped projection profile, characterized in that, It includes: A processor; A storage device having a computer program stored thereon, when the computer program is executed by the processor, the processor implements the stripe projection profile projector lens distortion residual correction method according to any one of claims 1-9.

Images