Projector curved surface automatic correction method and system based on double 3D TOF and medium

Abstract

The invention provides a double 3D TOF-based projector curved surface automatic correction method and system and a medium, and the method comprises the steps: synchronously collecting the point cloud data of a to-be-projected curved surface based on a double 3D TOF sensor, and obtaining the three-dimensional point cloud data; removing outer points based on an RANSAC (Random Sample Consensus) algorithm, and performing point cloud scale unification by adopting a weighted least square method; based on an improved ICP algorithm, depth and normal features of the complete curved surface three-dimensional point cloud are extracted, and curvature and concave-convex distribution key parameters of the curved surface are calculated; constructing a global pixel-three-dimensional coordinate mapping table, and optimizing edge compensation parameters by adopting weighted B-spline surface fitting; according to the global pixel-three-dimensional coordinate mapping table and the optimized edge compensation parameters, a projection picture is driven to be adjusted in real time, and a curved surface projection correction result is obtained; through automatic point cloud acquisition, modeling and correction, point cloud scale unification and curved surface fitting, curved surface edge compensation is realized, and correction precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of projector correction, in particular to a double 3D TOF-based projector curved surface automatic correction method, system and medium. BACKGROUND

[0002] With the diversification of projector application scenarios, there is an increasing demand for curved surface projection such as arc-shaped screen, irregular wall surface and cylindrical surface. The existing projector curved surface correction technology mainly has the following defects: 1. Ordinary TOF relies on two-dimensional ranging data to calculate three-dimensional information, and the traditional correction algorithm does not fully utilize 3D features, has poor adaptability to irregular curved surfaces, and has limited modeling accuracy; 2. Single 3D TOF has point cloud blind area, and complex curved surface modeling is incomplete; 3. Camera vision correction scheme: relies on image feature recognition, is easily affected by environmental light and curved surface texture, has weak anti-interference ability, and fails in strong light or pure color curved surface scenes; 4. Fixed parameter correction scheme: needs to manually preset the curved surface type (such as arc-shaped, cylindrical surface), cannot adapt to curved surfaces with different curvatures, and the overall adaptation strategy may cause residual distortion at the edge of the picture, affecting the display effect; 5. High operation complexity: some schemes require users to manually mark correction points, have poor usability, and cannot meet the demand for rapid automatic correction; Therefore, there is an urgent need for a high-precision, self-adaptive and strong anti-interference curved surface correction scheme to adapt to various curved surface projection scenarios and improve user experience. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a double 3D TOF-based projector curved surface automatic correction method, system and medium, which realizes curved surface edge compensation and improves correction accuracy through automatic point cloud acquisition, modeling and correction, point cloud scale unification and curved surface fitting.

[0004] The embodiments of the present application also provide a double 3D TOF-based projector curved surface automatic correction method, which comprises: Synchronously acquiring point cloud data of a curved surface to be projected based on a double 3D TOF sensor, obtaining three-dimensional information of the curved surface including depth, coordinates and intensity, and obtaining three-dimensional point cloud data; Preprocessing the three-dimensional point cloud data, removing outliers based on a RANSAC algorithm, and uniformly unifying the scale of the preprocessed three-dimensional point cloud data by using a weighted least squares method; Registering the three-dimensional point cloud data after scale unification based on an improved ICP algorithm, and generating complete curved surface three-dimensional point cloud without blind area; The depth and normal features of the complete surface 3D point cloud are extracted, and a dynamic surface model is generated by fitting a quadratic surface model. The curvature and concavity / convexity distribution key parameters of the surface are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed, and the edge compensation parameters are optimized by weighted B-spline surface fitting. Based on the global pixel-3D coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time to obtain the curved surface projection correction result.

[0005] Optionally, in the projector surface automatic correction method based on dual 3D TOF described in the embodiments of this application, point cloud data of the surface to be projected is simultaneously acquired based on dual 3D TOF sensors to obtain the three-dimensional information of the surface's depth, coordinates, and intensity, thereby obtaining three-dimensional point cloud data, specifically including: Two 3D TOF sensors are deployed symmetrically or complementaryly at a preset spacing; The hardware synchronous triggering method is used to control two 3D TOF sensors to start acquisition at the same time stamp to obtain point cloud data of the surface to be projected; The two sensors simultaneously emit modulated infrared light signals. The light is reflected by the surface to be projected and then received by the sensors to obtain the depth information of the surface. Based on the sensor's internal calibration parameters, the image coordinates of each pixel are converted into three-dimensional coordinates in the sensor's local coordinate system to obtain coordinate information. When the sensor receives the reflected light signal, it simultaneously acquires the intensity signal of the reflected light, performs analog-to-digital conversion and noise reduction on the intensity signal, and outputs the quantized value to obtain the intensity information; The depth, coordinate, and intensity information are stitched together and optimized to obtain three-dimensional data.

[0006] Optionally, in the automatic correction method for projector curved surfaces based on dual 3D TOF described in the embodiments of this application, the three-dimensional point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed three-dimensional point cloud data, specifically including: Acquire 3D point cloud data output from dual 3D TOF sensors, convert the two 3D point cloud data into a world coordinate system with the optical center of the projector as the origin, and synchronously initialize the data sampling interval; Multiple non-collinear points are randomly selected from single-path point cloud data as initial samples. The least squares method is used to fit the local surface model, and the Euclidean distance from all point cloud data to the fitted model is calculated. Set an interior point distance threshold, compare the Euclidean distance with the interior point distance threshold, filter out point cloud data that are greater than the interior point distance threshold, and record them as exterior points. Remove exterior points based on the RANSAC algorithm. Extract the overlapping acquisition area of ​​the two point cloud data after removing outliers, construct a matching sample set, match weight values ​​based on the difference in ranging accuracy of the two sensors, and optimize the error of the two point cloud data based on the weight values. The point cloud scale is unified using the 3D point cloud data after error optimization by weighted least squares method.

[0007] Optionally, in the projector surface automatic correction method based on dual 3D TOF described in the embodiments of this application, the scale-unified 3D point cloud data is registered based on the improved ICP algorithm to generate a complete surface 3D point cloud without blind spots, specifically including: Based on the preset installation parameters of the dual 3D TOF sensors, the initial spatial position relationship of the two point clouds is calculated, an initial transformation matrix is ​​generated, and the two point clouds are initially aligned based on the initial transformation matrix. Calculate the normal vector of each point in the two point clouds to obtain candidate matching point pairs; Set an initial distance threshold, and remove candidate matching point pairs whose Euclidean distance is greater than the set threshold to obtain valid matching point pairs; Based on the effective matching point pairs, the non-overlapping regions of the two point clouds are identified. Based on the depth and normal features of the surrounding effective point clouds, an interpolation algorithm is used to fill in the blind area data. By removing duplicate points from the filled blind zone data, a complete 3D point cloud of the curved surface without blind zones is obtained.

[0008] Optionally, in the automatic surface correction method for projectors based on dual 3D TOF described in this application embodiment, the depth and normal features of the complete surface 3D point cloud are extracted, a dynamic surface model is generated by fitting a quadratic surface model, and the curvature and concavity / convexity distribution key parameters of the surface are calculated, specifically including: Obtain a complete 3D point cloud of the curved surface without blind spots, remove residual noise based on the Gaussian filtering algorithm, and then construct sub-blocks according to a fixed mesh. Extract the features of each sub-block to obtain the point cloud features of each sub-block; Calculate the mean depth, variance, and gradient data for each sub-block, identify surface edges or concave / convex boundaries, and obtain depth and normal features; The parameters of the quadratic surface model are initialized based on point cloud features, and the optimal parameters are solved using the weighted least squares method to obtain the dynamic surface model. The key parameters of curvature and concavity / convexity distribution of a surface are solved based on a dynamic surface model.

[0009] Optionally, in the projector surface automatic correction method based on dual 3D TOF described in the embodiments of this application, a global pixel-3D coordinate mapping table is constructed based on the dynamic surface model and key surface parameters, and weighted B-spline surface fitting is used to optimize the edge compensation parameters, specifically including: Initialize the projector hardware parameters, partition the dynamic surface model according to the surface curvature, and establish the initial association between the pixel coordinate system and the world coordinate system; Iterate through all projected pixels to generate a projection ray, calculate the intersection of the ray with the dynamic surface model, supplement the interpolated coordinates of pixels without intersections, and form a global pixel-3D coordinate mapping table. Locate the edges and high curvature regions of the curved surface, and select the point cloud of the edge and transition regions as samples; We use weighted B-spline surface fitting to solve for the optimal compensation parameters, correct the coordinates of pixels in the edge region of the mapping table, and fit and optimize the edge compensation parameters.

[0010] Secondly, embodiments of this application provide an automatic projector surface correction system based on dual 3D TOF. The system includes a memory and a processor. The memory includes a program for an automatic projector surface correction method based on dual 3D TOF. When the program for the automatic projector surface correction method based on dual 3D TOF is executed by the processor, it performs the following steps: Based on the simultaneous acquisition of point cloud data of the surface to be projected by dual 3D TOF sensors, the depth, coordinates and intensity of the surface are obtained to obtain three-dimensional point cloud data. The 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed 3D point cloud data. Based on the improved ICP algorithm, the scale-unified 3D point cloud data is registered to generate a complete surface 3D point cloud without blind spots. The depth and normal features of the complete surface 3D point cloud are extracted, and a dynamic surface model is generated by fitting a quadratic surface model. The curvature and concavity / convexity distribution key parameters of the surface are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed, and the edge compensation parameters are optimized by weighted B-spline surface fitting. Based on the global pixel-3D coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time to obtain the curved surface projection correction result.

[0011] Optionally, in the projector surface automatic correction system based on dual 3D TOF described in this application embodiment, point cloud data of the surface to be projected is simultaneously acquired using dual 3D TOF sensors to obtain the surface's depth, coordinates, and intensity three-dimensional information, resulting in three-dimensional point cloud data. Specifically, this includes: Two 3D TOF sensors are deployed symmetrically or complementaryly at a preset spacing; The hardware synchronous triggering method is used to control two 3D TOF sensors to start acquisition at the same time stamp to obtain point cloud data of the surface to be projected; The two sensors simultaneously emit modulated infrared light signals. The light is reflected by the surface to be projected and then received by the sensors to obtain the depth information of the surface. Based on the sensor's internal calibration parameters, the image coordinates of each pixel are converted into three-dimensional coordinates in the sensor's local coordinate system to obtain coordinate information. When the sensor receives the reflected light signal, it simultaneously acquires the intensity signal of the reflected light, performs analog-to-digital conversion and noise reduction on the intensity signal, and outputs the quantized value to obtain the intensity information; The depth, coordinate, and intensity information are stitched together and optimized to obtain three-dimensional data.

[0012] Optionally, in the projector surface automatic correction system based on dual 3D TOF described in this application embodiment, the 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the preprocessed 3D point cloud data is scaled using the weighted least squares method, specifically including: Acquire 3D point cloud data output from dual 3D TOF sensors, convert the two 3D point cloud data into a world coordinate system with the optical center of the projector as the origin, and synchronously initialize the data sampling interval; Multiple non-collinear points are randomly selected from single-path point cloud data as initial samples. The least squares method is used to fit the local surface model, and the Euclidean distance from all point cloud data to the fitted model is calculated. Set an interior point distance threshold, compare the Euclidean distance with the interior point distance threshold, filter out point cloud data that are greater than the interior point distance threshold, and record them as exterior points. Remove exterior points based on the RANSAC algorithm. Extract the overlapping acquisition area of ​​the two point cloud data after removing outliers, construct a matching sample set, match weight values ​​based on the difference in ranging accuracy of the two sensors, and optimize the error of the two point cloud data based on the weight values. The point cloud scale is unified using the 3D point cloud data after error optimization by weighted least squares method.

[0013] Thirdly, embodiments of this application also provide a computer-readable storage medium, which includes a projector surface automatic correction method program based on dual 3D TOF. When the projector surface automatic correction method program based on dual 3D TOF is executed by a processor, it implements the steps of the projector surface automatic correction method based on dual 3D TOF as described in any of the above claims.

[0014] As can be seen from the above, the projector curved surface automatic correction method, system, and medium based on dual 3D TOF provided in the embodiments of this application, through dual 3D... The TOF sensor synchronously acquires point cloud data of the surface to be projected, obtaining three-dimensional information of the surface's depth, coordinates, and intensity, thus obtaining three-dimensional point cloud data. The three-dimensional point cloud data is preprocessed, with outliers removed using the RANSAC algorithm, and the scale of the preprocessed three-dimensional point cloud data is unified using the weighted least squares method. The scale-unified three-dimensional point cloud data is then registered using an improved ICP algorithm to generate a complete surface three-dimensional point cloud without blind spots. The depth and normal features of the complete surface three-dimensional point cloud are extracted, and a dynamic surface model is generated through quadratic surface model fitting. Key parameters of the surface curvature and concavity / convexity distribution are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-three-dimensional coordinate mapping table is constructed, and edge compensation parameters are optimized using weighted B-spline surface fitting. Based on the global pixel-three-dimensional coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time, obtaining the surface projection correction result. Through automatic point cloud acquisition, modeling, and correction, point cloud scale unification and surface fitting, surface edge compensation is achieved, improving correction accuracy. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart of an automatic surface correction method for projectors based on dual 3D TOF provided in this application embodiment; Figure 2 A flowchart of a three-dimensional point cloud data acquisition method for an automatic surface correction method for projectors based on dual 3D TOF provided in an embodiment of this application; Figure 3 A flowchart of the out-point removal method for the automatic correction method of projector curved surfaces based on dual 3D TOF provided in the embodiments of this application; Figure 4 The surface correction flowchart of the projector surface automatic correction method based on dual 3D TOF provided in this application embodiment. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0018] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0019] Please refer to Figures 1-4 As shown, the automatic projector surface correction method is used in terminal devices. This automatic projector surface correction method based on dual 3D TOF includes the following steps: S101, based on dual 3D TOF sensors, synchronously acquires point cloud data of the surface to be projected, obtains the three-dimensional information of the surface's depth, coordinates, and intensity, and obtains three-dimensional point cloud data; S102, preprocess the 3D point cloud data, remove outliers based on the RANSAC algorithm, and use the weighted least squares method to unify the point cloud scale of the preprocessed 3D point cloud data. S103, based on the improved ICP algorithm, registers the scale-unified 3D point cloud data to generate a complete curved 3D point cloud without blind spots; S104: Extract the depth and normal features of the complete surface 3D point cloud, generate a dynamic surface model by fitting a quadratic surface model, and calculate the key parameters of the surface curvature and concavity / convexity distribution. S105, based on the dynamic surface model and key surface parameters, constructs a global pixel-3D coordinate mapping table, and uses weighted B-spline surface fitting to optimize edge compensation parameters; S106, based on the global pixel-3D coordinate mapping table and the optimized edge compensation parameters, drives the projection screen to adjust in real time to obtain the curved surface projection correction result.

[0020] It should be noted that the dual 3D TOF sensors are symmetrically arranged and synchronously output high-density 3D point clouds. By improving the ICP algorithm, the scale of the two point clouds is unified and accurately registered. The two point clouds are fused to generate a curved 3D model without blind spots, breaking through the limitations of ordinary TOF 2D ranging. The 3D point cloud directly provides depth and coordinate 3D information. After the dual sensor registration, the modeling blind spots of irregular curved surfaces (such as wavy or irregular curved surfaces) are eliminated. The point cloud density is increased to 100k points / frame, and the modeling error is smaller.

[0021] According to an embodiment of the present invention, point cloud data of a surface to be projected is simultaneously acquired using dual 3D TOF sensors to obtain three-dimensional information of the surface's depth, coordinates, and intensity, thereby obtaining three-dimensional point cloud data. Specifically, this includes: S201, two 3D TOF sensors are deployed symmetrically or complementaryly at a preset spacing; S202 uses a hardware synchronous triggering method to control two 3D TOF sensors to start acquisition at the same time stamp to obtain point cloud data of the surface to be projected; S203: Dual sensors simultaneously emit modulated infrared light signals. The light is reflected by the surface to be projected and then received by the sensors to obtain the depth information of the surface. S204, based on the sensor's internal calibration parameters, converts the image coordinates of each pixel into three-dimensional coordinates in the sensor's local coordinate system to obtain coordinate information; S205: When the sensor receives the reflected light signal, it simultaneously acquires the intensity signal of the reflected light, performs analog-to-digital conversion and noise reduction on the intensity signal, outputs the quantized value, and obtains the intensity information. S206 involves stitching and optimizing the point cloud data based on depth, coordinate, and intensity information to obtain 3D data.

[0022] According to an embodiment of the present invention, the three-dimensional point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed three-dimensional point cloud data, specifically including: S301 acquires 3D point cloud data output from dual 3D TOF sensors, converts the two 3D point cloud data into a world coordinate system with the optical center of the projector as the origin, and synchronously initializes the data sampling interval. S302: Randomly select multiple non-collinear points from single-path point cloud data as initial samples, use the least squares method to fit the local surface model, and calculate the Euclidean distance from all point cloud data to the fitted model. S303, set an interior point distance threshold, compare the Euclidean distance with the interior point distance threshold, filter out point cloud data that are greater than the interior point distance threshold, record them as exterior points, and remove exterior points based on the RANSAC algorithm; S304: Extract the overlapping acquisition area of ​​the two point cloud data after removing outliers, construct a matching sample set, match weight values ​​based on the difference in ranging accuracy of the two sensors, and optimize the error of the two point cloud data based on the weight values. S305 uses weighted least squares error optimization to unify the scale of 3D point cloud data.

[0023] It should be noted that the pre-processing preparations are as follows: Data loading and coordinate system one: Read the raw 3D point cloud data (including depth, coordinates and intensity information) output by the dual 3D TOF sensors, convert the two point clouds to the world coordinate system with the optical center of the projection lens as the origin, and synchronously initialize the data sampling interval (e.g. 0.5mm) to ensure that the resolution of the two data streams is consistent.

[0024] Initial data filtering: Based on the sensor ranging range (1.0-3.5m), invalid point clouds outside this range are removed to initially reduce data redundancy.

[0025] RANSAC algorithm outlier removal (decentralized execution, consistent process in both paths): Algorithm parameter configuration: Combining point cloud density (100k points / frame) and ranging accuracy (±1mm), the core parameters are dynamically set, with 800-1200 iterations (adaptively adjusted according to the total amount of point cloud), an interior point distance threshold of 2mm, and a confidence level of 99%, balancing the elimination accuracy and efficiency.

[0026] Initial model fitting: Eight non-collinear points are randomly selected from the single-path point cloud as initial samples, and the least squares method is used to fit the local surface model (adapting to the non-planar features of the surface projection scene).

[0027] Inner / outer point determination: Calculate the Euclidean distance from all point clouds of the path to the fitted model. Points with a distance ≤ the threshold are marked as inner points (valid feature points), and points with a distance > the threshold are marked as candidate outer points (which may be noise or environmental interference points).

[0028] Model Iteration Optimization: The process of selecting repeated samples, fitting the model, and determining the inliers continues for a preset number of iterations. The model with the most inliers and the smallest variance in the inlier distribution is selected as the optimal fitted model.

[0029] Accurate removal of outliers: retain the interior points corresponding to the optimal model, and at the same time, combine the point cloud intensity and phase information to verify the candidate outliers a second time, so as to avoid misjudging the inherent concave and convex feature points of the surface as outliers, thus completing the single-path denoising; repeat the above steps to process the other path of point cloud.

[0030] Weighted least squares method for achieving scale uniformity of dual-path point clouds: Overlapping region extraction: By comparing coordinate ranges, the overlapping acquisition regions of the two denoised point clouds (accounting for ≥25% of the total point cloud in a single channel) are extracted, and a matching sample set is constructed to provide an accuracy basis for scale uniformity.

[0031] Weighting coefficient allocation: Weights are assigned based on the difference in ranging accuracy between the two sensors. The point cloud collected by the sensor with smaller ranging error is assigned a weight of 0.6, and the other is assigned a weight of 0.4, which strengthens the dominant role of high-precision data in scale matching.

[0032] According to an embodiment of the present invention, the scale-unified 3D point cloud data is registered based on an improved ICP algorithm to generate a complete curved surface 3D point cloud without blind spots, specifically including: Based on the preset installation parameters of the dual 3D TOF sensors, the initial spatial position relationship of the two point clouds is calculated, an initial transformation matrix is ​​generated, and the two point clouds are initially aligned based on the initial transformation matrix. Calculate the normal vector of each point in the two point clouds to obtain candidate matching point pairs; Set an initial distance threshold, and remove candidate matching point pairs whose Euclidean distance is greater than the set threshold to obtain valid matching point pairs; Based on the effective matching point pairs, the non-overlapping regions of the two point clouds are identified. Based on the depth and normal features of the surrounding effective point clouds, an interpolation algorithm is used to fill in the blind area data. By removing duplicate points from the filled blind zone data, a complete 3D point cloud of the curved surface without blind zones is obtained.

[0033] It should be noted that the following preparations are required before registration: Load the preprocessed (outer point removal + scale unification) dual-channel 3D point cloud data, and verify the scale consistency (average distance error of overlapping area ≤ 1mm) and data integrity (inner point retention rate of single channel point cloud ≥ 92%) of the two channel point clouds to ensure that the registration input requirements are met.

[0034] Initial registration and coarse positioning: Based on the preset installation parameters of the dual 3D TOF sensors (horizontal distance of 7-10cm from the projection lens, symmetrical arrangement), the initial spatial position relationship of the two point clouds is calculated, and an initial transformation matrix (including translation and rotation parameters) is generated to initially align the two point clouds and narrow the search range for subsequent fine registration.

[0035] Improved ICP algorithm for precise registration core process: Matching point pairs for filtering; Based on point cloud normal feature constraints: calculate the normal vector of each point in the two point clouds (using the k-nearest neighbor algorithm, with k set to 10-15), and retain only point pairs with an angle between normal vectors ≤15° as candidate matching point pairs, while eliminating invalid matches.

[0036] Distance threshold filtering: Set an initial distance threshold (3mm) to remove candidate matching point pairs whose Euclidean distance is greater than the threshold, thereby reducing the interference of abnormal matching pairs.

[0037] Solving for the transformation matrix; Based on the filtered valid matching point pairs, a least-squares optimization objective function is constructed. Local alignment of two point clouds is achieved by solving for the optimal rotation matrix R and translation matrix T using singular value decomposition (SVD).

[0038] Dynamic iterative optimization; Calculate the average distance error of all matching point pairs after iteration. If the error is greater than the preset convergence threshold (0.5mm), dynamically adjust the distance threshold (reducing it by 0.2mm in each iteration) and the iteration step size (the larger the error, the smaller the step size), and repeat the "matching point pair screening - transformation matrix solution" process.

[0039] Iteration termination condition: average distance error ≤ 0.5mm or the number of iterations reaches the maximum value (50 times), ensuring that the registration converges to the global optimum.

[0040] Blind spot filling and point cloud fusion; Identify the non-overlapping regions of the two point clouds (i.e., blind spots acquired by a single sensor), and use interpolation algorithms (such as Kriging interpolation) to fill in the blind spot data based on the depth and normal features of the surrounding effective point clouds, ensuring the continuity of the surface point cloud.

[0041] Merge the registered dual-path point cloud with the filled blind zone data, remove duplicate points (points with a distance ≤0.3mm are considered duplicate points, and points with better intensity information are retained) to generate a complete curved 3D point cloud without blind zones and with high density (point cloud density ≥100k points / frame).

[0042] Registration result verification: Accuracy verification: Calculate the average registration error of the complete point cloud after registration. It must be ≤0.8mm to ensure that the registration accuracy meets the requirements of subsequent surface modeling.

[0043] Integrity verification: Check if there are any unfilled blind spots in the point cloud (blind spot area ratio ≤ 0.5%) to ensure complete surface coverage.

[0044] Consistency verification: Verify the spatial continuity of point cloud data. The depth change gradient between adjacent point clouds should be ≤2mm / point to avoid data breaks or distortions after registration.

[0045] According to an embodiment of the present invention, the depth and normal features of the complete three-dimensional point cloud of the surface are extracted, a dynamic surface model is generated by fitting a quadratic surface model, and the curvature and concavity / convexity distribution key parameters of the surface are calculated, specifically including: Obtain a complete 3D point cloud of the curved surface without blind spots, remove residual noise based on the Gaussian filtering algorithm, and then construct sub-blocks according to a fixed mesh. Extract the features of each sub-block to obtain the point cloud features of each sub-block; Calculate the mean depth, variance, and gradient data for each sub-block, identify surface edges or concave / convex boundaries, and obtain depth and normal features; The parameters of the quadratic surface model are initialized based on point cloud features, and the optimal parameters are solved using the weighted least squares method to obtain the dynamic surface model. The key parameters of curvature and concavity / convexity distribution of a surface are solved based on a dynamic surface model.

[0046] According to an embodiment of the present invention, based on a dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed, and weighted B-spline surface fitting is used to optimize edge compensation parameters, specifically including: Initialize the projector hardware parameters, partition the dynamic surface model according to the surface curvature, and establish the initial association between the pixel coordinate system and the world coordinate system; Iterate through all projected pixels to generate a projection ray, calculate the intersection of the ray with the dynamic surface model, supplement the interpolated coordinates of pixels without intersections, and form a global pixel-3D coordinate mapping table. Locate the edges and high curvature regions of the curved surface, and select the point cloud of the edge and transition regions as samples; We use weighted B-spline surface fitting to solve for the optimal compensation parameters, correct the coordinates of pixels in the edge region of the mapping table, and fit and optimize the edge compensation parameters.

[0047] It should be noted that the following preparations are required before building the mapping table: Projection parameter initialization: Read the projector hardware parameters (resolution, projection focal length, throw ratio) and the global coordinate range of the dynamic surface model, and establish the initial association between the pixel coordinate system (u, v) of the projected image and the world coordinate system (X, Y, Z).

[0048] Surface partitioning adaptation: Based on the surface curvature distribution parameters, the dynamic surface model is divided into high curvature regions (curvature > 5mm). -1 ), medium curvature region (1mm) -1 ≤curvature≤5mm -1 and flat areas (curvature < 1 mm) -1 Different regions employ differentiated mapping strategies.

[0049] Solving for the 3D coordinates of a pixel: Traverse all pixels (u, v) on the projected screen and generate rays emanating from the pixels based on the perspective projection principle of the projector. Calculate the coordinates (X, Y, Z) of the intersection point between the ray and the dynamic surface model. This intersection point is the target projection position of the pixel on the surface. If the ray and the surface do not intersect (such as in the blind area at the edge of the surface), the three-dimensional coordinates of adjacent pixels are used for interpolation to ensure the integrity of the mapping table.

[0050] Mapping table encapsulation: Store the mapping relationship between all pixels (u, v) and their corresponding 3D coordinates (X, Y, Z) as a two-dimensional array to generate a global pixel-3D coordinate mapping table, which supports fast index lookup (lookup time ≤ 1μs / pixel).

[0051] Weighted B-spline surface fitting optimization of edge compensation parameters: Edge region localization: Based on the surface concavity and convexity distribution and depth gradient data, identify surface edge regions (regions with abrupt depth changes ≥ 2 mm / point) and high curvature edges (curvature > 5 mm). -1 The edges of the area are marked as key areas for compensation.

[0052] B-spline surface model initialization: Point cloud data of the edge region and adjacent transition region are selected as fitting samples. The order of the B-spline surface is set to 3 (to ensure curve smoothness). The number of control vertices is adaptively adjusted according to the edge length (1 control vertex is set for every 10mm).

[0053] Weighted fitting optimization: Weights are assigned to the fitted samples: the weights of edge contour points are set to 0.9-1.0, and the weights of transition region points are set to 0.5-0.8, to enhance the influence of edge features on the compensation parameters; Construct a weighted B-spline fitting objective function, minimize the distance error from the sample points to the fitted surface, and solve for the optimal control vertex parameters; Based on the optimized B-spline surface, the compensation amount (including translation compensation and rotation compensation) of each pixel in the edge region is calculated, and the corresponding entries in the global pixel-3D coordinate mapping table are corrected.

[0054] Compensation parameter verification: Calculate the distortion rate of the optimized edge region, which must be ≤3% to ensure that there is no residual distortion in the edge display.

[0055] Mapping table updated dynamically: The optimized global pixel-3D coordinate mapping table is transmitted to the projection screen correction unit. Every 50ms, the mapping table is re-optimized in conjunction with the updated dynamic surface model to ensure that the compensation parameters are synchronized with the surface changes in real time, with a correction delay of ≤100ms.

[0056] Secondly, embodiments of this application provide an automatic projector surface correction system based on dual 3D TOF. The system includes a memory and a processor. The memory includes a program for an automatic projector surface correction method based on dual 3D TOF. When the program for the automatic projector surface correction method based on dual 3D TOF is executed by the processor, it implements the following steps: Based on the simultaneous acquisition of point cloud data of the surface to be projected by dual 3D TOF sensors, the depth, coordinates and intensity of the surface are obtained to obtain three-dimensional point cloud data. The 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed 3D point cloud data. Based on the improved ICP algorithm, the scale-unified 3D point cloud data is registered to generate a complete surface 3D point cloud without blind spots. The depth and normal features of the complete surface 3D point cloud are extracted, and a dynamic surface model is generated by fitting a quadratic surface model. The curvature and concavity / convexity distribution of the surface are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed, and the edge compensation parameters are optimized by weighted B-spline surface fitting. Based on the global pixel-3D coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time to obtain the curved surface projection correction result.

[0057] According to an embodiment of the present invention, point cloud data of a surface to be projected is simultaneously acquired using dual 3D TOF sensors to obtain three-dimensional information of the surface's depth, coordinates, and intensity, thereby obtaining three-dimensional point cloud data. Specifically, this includes: Two 3D TOF sensors are deployed symmetrically or complementaryly at a preset spacing; The hardware synchronous triggering method is used to control two 3D TOF sensors to start acquisition at the same time stamp to obtain point cloud data of the surface to be projected; The two sensors simultaneously emit modulated infrared light signals. The light is reflected by the surface to be projected and then received by the sensors to obtain the depth information of the surface. Based on the sensor's internal calibration parameters, the image coordinates of each pixel are converted into three-dimensional coordinates in the sensor's local coordinate system to obtain coordinate information. When the sensor receives the reflected light signal, it simultaneously acquires the intensity signal of the reflected light, performs analog-to-digital conversion and noise reduction on the intensity signal, and outputs the quantized value to obtain the intensity information; The depth, coordinate, and intensity information are stitched together and optimized to obtain three-dimensional data.

[0058] According to an embodiment of the present invention, the three-dimensional point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed three-dimensional point cloud data, specifically including: Acquire 3D point cloud data output from dual 3D TOF sensors, convert the two 3D point cloud data into a world coordinate system with the optical center of the projector as the origin, and synchronously initialize the data sampling interval; Multiple non-collinear points are randomly selected from single-path point cloud data as initial samples. The least squares method is used to fit the local surface model, and the Euclidean distance from all point cloud data to the fitted model is calculated. Set an interior point distance threshold, compare the Euclidean distance with the interior point distance threshold, filter out point cloud data that are greater than the interior point distance threshold, and record them as exterior points. Remove exterior points based on the RANSAC algorithm. Extract the overlapping acquisition area of ​​the two point cloud data after removing outliers, construct a matching sample set, match weight values ​​based on the difference in ranging accuracy of the two sensors, and optimize the error of the two point cloud data based on the weight values. The point cloud scale is unified using the 3D point cloud data after error optimization by weighted least squares method.

[0059] A third aspect of the present invention provides a computer-readable storage medium including a projector surface automatic correction method program based on dual 3D TOF, wherein when the projector surface automatic correction method program based on dual 3D TOF is executed by a processor, the steps of the projector surface automatic correction method based on dual 3D TOF as described in any of the above claims are implemented.

[0060] This invention discloses a method, system, and medium for automatic correction of projector curved surfaces based on dual 3D Time-of-Flight (TOF). The TOF sensor synchronously acquires point cloud data of the surface to be projected, obtaining three-dimensional information of the surface's depth, coordinates, and intensity, thus obtaining three-dimensional point cloud data. The three-dimensional point cloud data is preprocessed, with outliers removed using the RANSAC algorithm, and the scale of the preprocessed three-dimensional point cloud data is unified using the weighted least squares method. The scale-unified three-dimensional point cloud data is then registered using an improved ICP algorithm to generate a complete surface three-dimensional point cloud without blind spots. The depth and normal features of the complete surface three-dimensional point cloud are extracted, and a dynamic surface model is generated through quadratic surface model fitting. Key parameters of the surface curvature and concavity / convexity distribution are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-three-dimensional coordinate mapping table is constructed, and edge compensation parameters are optimized using weighted B-spline surface fitting. Based on the global pixel-three-dimensional coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time, obtaining the surface projection correction result. Through automatic point cloud acquisition, modeling, and correction, point cloud scale unification and surface fitting, surface edge compensation is achieved, improving correction accuracy.

[0061] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0062] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0063] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0064] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for automatic surface correction of a projector based on dual 3D TOF, characterized in that, include: Based on the simultaneous acquisition of point cloud data of the surface to be projected by dual 3D TOF sensors, the depth, coordinates and intensity of the surface are obtained to obtain three-dimensional point cloud data. The 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed 3D point cloud data. Based on the improved ICP algorithm, the scale-unified 3D point cloud data is registered to generate a complete surface 3D point cloud without blind spots. The depth and normal features of the complete surface 3D point cloud are extracted, and a dynamic surface model is generated by fitting a quadratic surface model. The curvature and concavity / convexity distribution key parameters of the surface are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed, and the edge compensation parameters are optimized by weighted B-spline surface fitting. Based on the global pixel-3D coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time to obtain the curved surface projection correction result.

2. The automatic surface correction method for projectors based on dual 3D TOF as described in claim 1, characterized in that, By synchronously acquiring point cloud data of the surface to be projected using dual 3D Time-of-Flight (TOF) sensors, the depth, coordinates, and intensity of the surface are obtained, resulting in 3D point cloud data, specifically including: Two 3D TOF sensors are deployed symmetrically or complementaryly at a preset spacing; The hardware synchronous triggering method is used to control two 3D TOF sensors to start acquisition at the same time stamp to obtain point cloud data of the surface to be projected; The two sensors simultaneously emit modulated infrared light signals. The light is reflected by the surface to be projected and then received by the sensors to obtain the depth information of the surface. Based on the sensor's internal calibration parameters, the image coordinates of each pixel are converted into three-dimensional coordinates in the sensor's local coordinate system to obtain coordinate information. When the sensor receives the reflected light signal, it simultaneously acquires the intensity signal of the reflected light, performs analog-to-digital conversion and noise reduction on the intensity signal, and outputs the quantized value to obtain the intensity information. The depth, coordinate, and intensity information are stitched together and optimized to obtain three-dimensional data.

3. The automatic surface correction method for projectors based on dual 3D TOF as described in claim 2, characterized in that, The 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed 3D point cloud data. Specifically, this includes: Acquire 3D point cloud data output from dual 3D TOF sensors, convert the two 3D point cloud data into a world coordinate system with the optical center of the projector as the origin, and synchronously initialize the data sampling interval; Multiple non-collinear points are randomly selected from single-path point cloud data as initial samples. The least squares method is used to fit the local surface model, and the Euclidean distance from all point cloud data to the fitted model is calculated. Set an interior point distance threshold, compare the Euclidean distance with the interior point distance threshold, filter out point cloud data that are greater than the interior point distance threshold, and record them as exterior points. Remove exterior points based on the RANSAC algorithm. Extract the overlapping acquisition area of ​​the two point cloud data after removing outliers, construct a matching sample set, match weight values ​​based on the difference in ranging accuracy of the two sensors, and optimize the error of the two point cloud data based on the weight values. The point cloud scale is unified using the 3D point cloud data after error optimization by weighted least squares method.

4. The automatic surface correction method for projectors based on dual 3D TOF as described in claim 3, characterized in that, Based on the improved ICP algorithm, the scale-unified 3D point cloud data is registered to generate a complete surface 3D point cloud without blind spots, specifically including: Based on the preset installation parameters of the dual 3D TOF sensors, the initial spatial position relationship of the two point clouds is calculated, an initial transformation matrix is ​​generated, and the two point clouds are initially aligned based on the initial transformation matrix. Calculate the normal vector of each point in the two point clouds to obtain candidate matching point pairs; Set an initial distance threshold, and remove candidate matching point pairs whose Euclidean distance is greater than the set threshold to obtain valid matching point pairs; Based on the effective matching point pairs, the non-overlapping regions of the two point clouds are identified. Based on the depth and normal features of the surrounding effective point clouds, an interpolation algorithm is used to fill in the blind area data. By removing duplicate points from the filled blind zone data, a complete 3D point cloud of the curved surface without blind zones is obtained.

5. The automatic surface correction method for projectors based on dual 3D TOF according to claim 4, characterized in that, The depth and normal features of the complete 3D point cloud of the surface are extracted, and a dynamic surface model is generated by fitting a quadratic surface model. The curvature and key parameters of the surface's concavity / convexity distribution are then calculated, specifically including: Obtain a complete 3D point cloud of the curved surface without blind spots, remove residual noise based on the Gaussian filtering algorithm, and then construct sub-blocks according to a fixed mesh. Extract the features of each sub-block to obtain the point cloud features of each sub-block; Calculate the mean depth, variance, and gradient data for each sub-block, identify surface edges or concave / convex boundaries, and obtain depth and normal features; The parameters of the quadratic surface model are initialized based on point cloud features, and the optimal parameters are solved using the weighted least squares method to obtain the dynamic surface model. The key parameters of curvature and concavity / convexity distribution of the surface are solved based on the dynamic surface model.

6. The automatic surface correction method for projectors based on dual 3D TOF as described in claim 5, characterized in that, Based on the dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed. Weighted B-spline surface fitting is used to optimize edge compensation parameters, specifically including: Initialize the projector hardware parameters, partition the dynamic surface model according to the surface curvature, and establish the initial association between the pixel coordinate system and the world coordinate system; Iterate through all projected pixels to generate a projection ray, calculate the intersection of the ray with the dynamic surface model, supplement the interpolated coordinates of pixels without intersections, and form a global pixel-3D coordinate mapping table. Locate the edges and high curvature regions of the curved surface, and select the point cloud of the edge and transition regions as samples; We use weighted B-spline surface fitting to solve for the optimal compensation parameters, correct the coordinates of pixels in the edge region of the mapping table, and fit and optimize the edge compensation parameters.

7. A projector surface automatic correction system based on dual 3D TOF, characterized in that, The system includes a memory and a processor. The memory contains a program for an automatic projector surface correction method based on dual 3D Time-of-Flight (TOF). When the processor executes the program for the automatic projector surface correction method based on dual 3D TOF, it performs the following steps: Based on the simultaneous acquisition of point cloud data of the surface to be projected by dual 3D TOF sensors, the depth, coordinates and intensity of the surface are obtained to obtain three-dimensional point cloud data. The 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed 3D point cloud data. Based on the improved ICP algorithm, the scale-unified 3D point cloud data is registered to generate a complete surface 3D point cloud without blind spots. The depth and normal features of the complete surface 3D point cloud are extracted, and a dynamic surface model is generated by fitting a quadratic surface model. The curvature and concavity / convexity distribution key parameters of the surface are calculated. Based on the dynamic surface model and key surface parameters, a global pixel-3D coordinate mapping table is constructed, and the edge compensation parameters are optimized by weighted B-spline surface fitting. Based on the global pixel-3D coordinate mapping table and the optimized edge compensation parameters, the projected image is driven to adjust in real time to obtain the curved surface projection correction result.

8. The projector surface automatic correction system based on dual 3D TOF according to claim 7, characterized in that, By synchronously acquiring point cloud data of the surface to be projected using dual 3D Time-of-Flight (TOF) sensors, the depth, coordinates, and intensity of the surface are obtained, resulting in 3D point cloud data, specifically including: Two 3D TOF sensors are deployed symmetrically or complementaryly at a preset spacing; The hardware synchronous triggering method is used to control two 3D TOF sensors to start acquisition at the same time stamp to obtain point cloud data of the surface to be projected; The two sensors simultaneously emit modulated infrared light signals. The light is reflected by the surface to be projected and then received by the sensors to obtain the depth information of the surface. Based on the sensor's internal calibration parameters, the image coordinates of each pixel are converted into three-dimensional coordinates in the sensor's local coordinate system to obtain coordinate information. When the sensor receives the reflected light signal, it simultaneously acquires the intensity signal of the reflected light, performs analog-to-digital conversion and noise reduction on the intensity signal, and outputs the quantized value to obtain the intensity information. The depth, coordinate, and intensity information are stitched together and optimized to obtain three-dimensional data.

9. The projector surface automatic correction system based on dual 3D TOF according to claim 8, characterized in that, The 3D point cloud data is preprocessed, outliers are removed based on the RANSAC algorithm, and the weighted least squares method is used to unify the point cloud scale of the preprocessed 3D point cloud data. Specifically, this includes: Acquire 3D point cloud data output from dual 3D TOF sensors, convert the two 3D point cloud data into a world coordinate system with the optical center of the projector as the origin, and synchronously initialize the data sampling interval; Multiple non-collinear points are randomly selected from single-path point cloud data as initial samples. The least squares method is used to fit the local surface model, and the Euclidean distance from all point cloud data to the fitted model is calculated. Set an interior point distance threshold, compare the Euclidean distance with the interior point distance threshold, filter out point cloud data that are greater than the interior point distance threshold, and record them as exterior points. Remove exterior points based on the RANSAC algorithm. Extract the overlapping acquisition area of ​​the two point cloud data after removing outliers, construct a matching sample set, match weight values ​​based on the difference in ranging accuracy of the two sensors, and optimize the error of the two point cloud data based on the weight values. The point cloud scale is unified using the 3D point cloud data after error optimization by weighted least squares method.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a projector surface automatic correction method program based on dual 3D TOF, which, when executed by a processor, implements the steps of the projector surface automatic correction method based on dual 3D TOF as described in any one of claims 1 to 6.

Images